Mail/MTA/Synchro/ImapSync

De TartareFR
Aller à la navigation Aller à la recherche
Logo ImapSync

IMAPSYNC

Introduction

imapsync - IMAP synchronisation, sync, copy or migration tool. Synchronise mailboxes between two imap servers. Good at IMAP migration. More than 44 different IMAP server softwares supported with success.

The command imapsync is a tool allowing incremental and recursive imap transfer from one mailbox to another.

By default all folders are transferred, recursively.

We sometimes need to transfer mailboxes from one imap server to another. This is called migration.

imapsync is a good tool because it reduces the amount of data transferred by not transferring a given message if it is already on both sides. Same headers and the transfer is done only once. All flags are preserved, unread will stay unread, read will stay read, deleted will stay deleted. You can stop the transfer at any time and restart it later, imapsync works well with bad connections.

You can decide to delete the messages from the source mailbox after a successful transfer, it can be a good feature when migrating live mailboxes since messages will be only one side. In that case, use the --delete option. Option --delete implies also option --expunge so all messages marked deleted on host1 will be really deleted. (you can use --noexpunge to avoid this but I don't see any good real world scenario for the combinaison --delete --noexpunge).

You can also just synchronize a mailbox B from another mailbox A in case you just want to keep a live copy of A in B. In that case --delete2 can be used, it deletes messages in host2 folder B that are not in host1 folder A.

imapsync is not adequate for maintaining two active imap accounts in synchronization where the user plays independently on both sides. Use offlineimap (written by John Goerzen) or mbsync (written by Michael R. Elkins) for 2 ways synchronizations.

imapsync is free, open, public but not always gratis software cover by the NOLIMIT Public License. See the LICENSE file included in the distribution or just read this: No limit to do anything with this work and this license.  

imapsync was wroten by Gilles LAMIRAL <lamiral@linux-france.org>

Feel free to hack imapsync as the NOLIMIT license permits it.

Entries for imapsync:   http://www.imap.org/products/showall.php 

imapsync can be found on github

Usage

imapsync [options]

The option list: 

imapsync [--host1 server1] [--port1 <num>]
         [--user1 <string>] [--passfile1 <string>]
         [--host2 server2] [--port2 <num>]
         [--user2 <string>] [--passfile2 <string>]
         [--ssl1] [--ssl2]
         [--tls1] [--tls2]
         [--authmech1 <string>] [--authmech2 <string>]
         [--proxyauth1] [--proxyauth2]
         [--domain1] [--domain2] 
         [--authmd51] [--authmd52]
         [--folder <string> --folder <string> ...]
         [--folderrec <string> --folderrec <string> ...]
         [--include <regex>] [--exclude <regex>]
         [--prefix2 <string>] [--prefix1 <string>] 
         [--regextrans2 <regex> --regextrans2 <regex> ...]
         [--sep1 <char>]
         [--sep2 <char>]
         [--justfolders] [--justfoldersizes] [--justconnect] [--justbanner]
         [--syncinternaldates]
         [--idatefromheader]
         [--syncacls]
         [--regexmess <regex>] [--regexmess <regex>]
         [--maxsize <int>]
         [--minsize <int>]
         [--maxage <int>]
         [--minage <int>]
         [--skipheader <regex>]
         [--useheader <string>] [--useheader <string>]
         [--nouid1] [--nouid2] 
         [--usecache]
         [--skipsize] [--allowsizemismatch]
         [--delete] [--delete2]
         [--expunge] [--expunge1] [--expunge2] [--uidexpunge2]
         [--delete2folders] [--delete2foldersonly] [--delete2foldersbutnot]
         [--subscribed] [--subscribe] [--subscribe_all] 
         [--nofoldersizes] [--nofoldersizesatend] 
         [--dry]
         [--debug] [--debugimap][--debugimap1][--debugimap2]
         [--timeout <int>] [--fast]
         [--split1] [--split2] 
         [--reconnectretry1 <int>] [--reconnectretry2 <int>]
         [--noreleasecheck]
         [--pidfile <filepath>]
         [--tmpdir <dirpath>]
         [--version] [--help]
         [--tests] [--tests_debug]

Options

Description of each option

--dry Makes imapsync doing nothing, just print what would be done without --dry.
--host1 <string> Source or "from" imap server. Mandatory.
--port1 <int> Port to connect on host1. Default is 143.
--user1 <string> User to login on host1. Mandatory.
--showpasswords Shows passwords on output instead of "MASKED". Useful to restart a complete run by just reading a log.
--password1 <string> Password for the user1.
--host2 <string> "destination" imap server. Mandatory.
--port2 <int> Port to connect on host2. Default is 143.
--user2 <string> User to login on host2. Mandatory.
--password2 <string> Password for the user2.
--passfile1 <string> Password file for the user1. It must contain the password on the first line. This option avoids to show the password on the command line like --password1 does.
--passfile2 <string> Password file for the user2. Contains the password.
--domain1 <string> Domain on host1 (NTLM authentication).
--domain2 <string> Domain on host2 (NTLM authentication).
--authuser1 <string> User to auth with on host1 (admin user). Avoid using --authmech1 SOMETHING with --authuser1.
--authuser2 <string> User to auth with on host2 (admin user).
--proxyauth1 Use proxyauth on host1. Requires --authuser1. Required by Sun/iPlanet/Netscape IMAP servers to be able to use an administrative user.
--proxyauth2 Use proxyauth on host2. Requires --authuser2. Required by Sun/iPlanet/Netscape IMAP servers to be able to use an administrative user
--authmd51 Use MD5 authentification for host1.
--authmd52 Use MD5 authentification for host2.
--authmech1 <string> Auth mechanism to use with host1 PLAIN, LOGIN, CRAM-MD5 etc. Use UPPERCASE.
--authmech2 <string> Auth mechanism to use with host2. See --authmech1
--ssl1 Use an SSL connection on host1.
--ssl2 Use an SSL connection on host2.
--tls1 Use an TLS connection on host1.
--tls2 Use an TLS connection on host2.
--folder <string> Sync this folder.
--folder <string> and this one, etc.
--folderrec <string> Sync this folder recursively.
--folderrec <string> and this one, etc.
--include <regex> Sync folders matching this regular expression
--include <regex> or this one, etc. in case both --include --exclude options are use, include is done before.
--exclude <regex> fold2|f3' skips fold1, fold2 and f3.
--exclude <regex> or this one, etc.
--regextrans2 <regex> Apply the whole regex to each destination folders.
--regextrans2 <regex> and this one. etc. When you play with the --regextrans2 option, first add also the safe options --dry --justfolders Then, when happy, remove --dry, remove --justfolders. Have in mind that --regextrans2 is applied after prefix and separator inversion.
--tmpdir <string> Where to store temporary files and subdirectories. Will be created if it doesn't exist. Default is system specific, Unix is /tmp but it's often small and deleted at reboot. --tmpdir /var/tmp should be better.
--pidfile <string> The file where imapsync pid is written.
--pidfilelocking Abort if pidfile already exists. Usefull to avoid concurrent transfers on the same mailbox.
--prefix1 <string> Remove prefix to all destination folders (usually INBOX. or INBOX/ or an empty string "") you have to use --prefix1 if host1 imap server does not have NAMESPACE capability, all other cases are bad.
--prefix2 <string> Add prefix to all host2 folders. See --prefix1
--sep1 <string> Host1 separator in case NAMESPACE is not supported.
--sep2 <string> Host2 separator in case NAMESPACE is not supported.
--regexmess <regex> Apply the whole regex to each message before transfer. Example:'s/\000/ /g' # to replace null by space.
--regexmess <regex> and this one.
--regexmess <regex> and this one, etc.
--regexflag <regex> Apply the whole regex to each flags list. Example's/"Junk"//g' # to remove "Junk" flag.
--regexflag <regex> and this one, etc.
--delete Deletes messages on host1 server after a successful transfer. Option --delete has the following behavior: it marks messages as deleted with the IMAP flag \Deleted, then messages are really deleted with an EXPUNGE IMAP command.
--delete2 Delete messages in host2 that are not in host1 server. Useful for backup or pre-sync.
--delete2duplicates Delete messages in host2 that are duplicates. Works only without --useuid since duplicates are detected with header part of each message.
--delete2folders Delete folders in host2 that are not in host1 server. For safety, first try it like this (it is safe --delete2folders --dry --justfolders --nofoldersizes )
--delete2foldersonly <regex> Deleted only folders matching regex.
--delete2foldersbutnot <regex> Contacts|Foo/"
--noexpunge Do not expunge messages on host1. Expunge really deletes messages marked deleted. Expunge is made at the beginning, on host1 only. Newly transferred messages are also expunged if option --delete is given. No expunge is done on host2 account (unless --expunge2)
--expunge1 Expunge messages on host1 after messages transfer.
--expunge2 Expunge messages on host2 after messages transfer.
--uidexpunge2 uidexpunge messages on the host2 account that are not on the host1 account, requires --delete2
--syncinternaldates Sets the internal dates on host2 same as host1. Turned on by default. Internal date is the date a message arrived on a host (mtime).
--idatefromheader Sets the internal dates on host2 same as the "Date:" headers.
--maxsize <int> Skip messages larger (or equal) than <int>bytes
--minsize <int> Skip messages smaller (or equal) than <int> bytes
--maxage <int> Skip messages older than <int> days. final stats (skipped) don't count older messages see also --minage
--minage <int> Skip messages newer than <int> days. final stats (skipped) don't count newer messages. You can do (+ are the messages selected) 
past|----maxage+++++++++++++++>now
past|+++++++++++++++minage---->now
past|----maxage+++++minage---->now (intersection)
past|++++minage-----maxage++++>now (union)
--search <string> Select messages returned by this IMAP SEARCH command.
--exitwhenover <int> Stop syncing when total bytes transferred reached. Gmail per day allows 2500000000 down 500000000 upload.
--useheader <string> Use this header to compare messages on both sides. Ex: Message-ID or Subject or Date.
--useheader <string> and this one, etc.
--subscribed Transfers subscribed folders.
--subscribe Subscribe to the folders transferred on the host2 that are subscribed on host1. On by default.
--subscribe_all Subscribe to the folders transferred on the host2 even if they are not subscribed on host1.
--nofoldersizes Do not calculate the size of each folder in bytes and message counts. Default is to calculate them.
--nofoldersizesatend Do not calculate the size of each folder in bytes and message counts at the end. Default is on.
--justfoldersizes Exit after having printed the folder sizes.
--syncacls Synchronises acls (Access Control Lists).
--nosyncacls Does not synchronize acls. This is the default. Acls in IMAP are not standardized, be careful.
--usecache Use cache to speedup.
--nousecache Do not use cache. Caveat: --useuid --nousecache creates duplicates on multiple runs.
--useuid Use uid instead of header as a criterium to recognize messages. Option --usecache is then implied unless --nousecache is used.
--debug Debug mode.
--debugcontent Debug content of the messages transfered.
--debugflags Debug flags.
--debugimap1 IMAP debug mode for host1. imap debug is very verbose.
--debugimap2 IMAP debug mode for host2.
--debugimap IMAP debug mode for host1 and host2.
--version Print software version.
--noreleasecheck Do not check for new imapsync release (a http request).
--justconnect Just connect to both servers and print useful information. Need only --host1 and --host2 options.
--justlogin Just login to both host1 and host2 with users credentials, then exit.
--justfolders Do only things about folders (ignore messages).
--help print this help.

Example

While working on imapsync parameters please run imapsync in dry mode (no modification induced) with the --dry option. Nothing bad can be done this way.

To synchronize the imap account ``buddy (with password ``secret1) on host ``imap.src.fr to the imap account ``max (with password ``secret2) on host ``imap.dest.fr: 

 imapsync --host1 imap.src.fr --user1 buddy --password1 secret1 \
 --host2 imap.dest.fr --user2 max --password2 secret2

Then you will have max's mailbox updated from buddy's mailbox.  

Security

You can use --passfile1 instead of --password1 to give the password since it is safer. With --password1 option any user on your host can see the password by using the <app>ps aux</app>command. Using a variable (like $PASSWORD1) is also dangerous because of the same reasons. So, saving the password in a well protected file (600 or rw----) is the best solution.

imasync is not totally protected against sniffers on the network since passwords may be transferred in plain text if CRAM-MD5 is not supported by your imap servers. Use --ssl1 (or --tls1) and --ssl2 (or --tls2) to enable encryption on host1 and host2.

You may authenticate as one user (typically an admin user), but be authorized as someone else, which means you don't need to know every user's personal password. Specify --authuser1 adminuser to enable this on host1. In this case, --authmech1 PLAIN will be used by default since it is the only way to go for now. So don't use --authmech1 SOMETHING with --authuser1 adminuser, it will not work. Same behavior with the --authuser2 option.

When working on Sun/iPlanet/Netscape IMAP servers you must use --proxyauth1 to enable administrative user to masquerade as another user. Can also be used on destination server with --proxyauth2

Exit Status

imapsync will exit with a 0 status (return code) if everything went good. Otherwise, it exits with a non-zero status.

So if you have an unreliable internet connection, you can use this loop in a Bourne shell: 

 while ! imapsync ...; do 
 echo imapsync not complete
 done

Huge Migration

Pay special attention to options --subscribed --subscribe --delete --delete2 --delete2folders --maxage --minage --maxsize --useuid --usecache

If you have many mailboxes to migrate think about a little shell program. Write a file called file.txt (for example) containing users and passwords. The separator used in this example is ';'

The file.txt file contains: 

user001_1;password001_1;user001_2;password001_2
user002_1;password002_1;user002_2;password002_2
user003_1;password003_1;user003_2;password003_2 
user004_1;password004_1;user004_2;password004_2
user005_1;password005_1;user005_2;password005_2
...

On Unix the shell program can be: 

 { while IFS=';' read u1 p1 u2 p2; do 
 imapsync --host1 imap.side1.org --user1 "$u1" --password1 "$p1" \
 --host2 imap.side2.org --user2 "$u2" --password2 "$p2" ...
 done ; } < file.txt

On Windows the batch program can be: 

 FOR /F "tokens=1,2,3,4 delims=; eol=#" %%G IN (file.txt) DO imapsync ^
 --host1 imap.side1.org --user1 %%G --password1 %%H ^
 --host2 imap.side2.org --user2 %%I --password2 %%J ...

The ... have to be replaced by nothing or any imapsync option.

Welcome in shell programming !

Similar Softwares

Récupérée de « https://wikiold.home.tartarefr.eu/index.php?title=Mail/MTA/Synchro/ImapSync&oldid=1693 »