Fetchmail - Revision history

imported>Huston at 20:15, 24 August 2020

2020-08-24T20:15:13Z

← Older revision		Revision as of 20:15, 24 August 2020
Line 1:		Line 1:
	Fetchmail is a wonderful program which might just solve a lot of issues that people have when they want to copy data from the IMAP server, whether it be because they're leaving the department and want a copy of their mail, or if you wish to use a mail client which can't access the IMAP server directly (though that is an unsupported path, so you're on your own).		Fetchmail is a wonderful program which might just solve a lot of issues that people have when they want to copy data from the IMAP server, whether it be because they're leaving the department and want a copy of their mail, or if you wish to use a mail client which can't access the IMAP server directly (though that is an unsupported path, so you're on your own).

			{{update\|Is this still relevant or should it be removed entirely?}}


	== Running Fetchmail ==		== Running Fetchmail ==
	Fetchmail is installed on ~~a few of the~~ linux machines in the department ~~(Cygnus is one example)~~. Before you run it, however, be sure to read and understand all the information here. Running it improperly can cause mail loops, deletion of all mail on the server, and many other Bad Things (some of which are a minor nuisance, others a major problem and unrecoverable).		Fetchmail is not installed on linux machines in the department by default, so you would have to [[Requesting assistance\|request]] it be installed first. Before you run it, however, be sure to read and understand all the information here. Running it improperly can cause mail loops, deletion of all mail on the server, and many other Bad Things (some of which are a minor nuisance, others a major problem and unrecoverable).

	'''NOTE:'''		'''NOTE:'''

imported>Lkoven: s/Squirrelmail/Roundcube

2013-10-30T18:03:19Z

s/Squirrelmail/Roundcube

← Older revision		Revision as of 18:03, 30 October 2013
Line 45:		Line 45:

	=== Download & delete ===		=== Download & delete ===
	If you wish to be brave, and remove all your mail from the server keeping it only in a file, then remove the 'keep' directive from the .fetchmailrc file. This tells fetchmail to delete the messages as it downloads them. If you still have the 'fetchall' keyword there, then mail that you've read before on the server (through an IMAP client, such as Pine or ~~Squirrelmail~~) will also be downloaded; if you remove 'fetchall', then only mail that is unread will be pulled down and deleted from the server.		If you wish to be brave, and remove all your mail from the server keeping it only in a file, then remove the 'keep' directive from the .fetchmailrc file. This tells fetchmail to delete the messages as it downloads them. If you still have the 'fetchall' keyword there, then mail that you've read before on the server (through an IMAP client, such as Pine or Roundcube) will also be downloaded; if you remove 'fetchall', then only mail that is unread will be pulled down and deleted from the server.

imported>Huston: Merged my.pmrc into Procmail

2007-05-21T18:47:35Z

Merged my.pmrc into Procmail

← Older revision		Revision as of 18:47, 21 May 2007
Line 32:		Line 32:

	=== my.pmrc ===		=== my.pmrc ===
	~~{{mergeto\|Procmail\|Talk:Fetchmail#Procmail recipes}}~~		The my.pmrc file is listed [[Procmail#my.pmrc\|here]]. You can also see other examples of [[Procmail]] recipes you might use with Fetchmail.
	~~Once you have the .fetchmailrc file, you now need to configure procmail to handle the messages once they're downloaded.~~ The ~~simplest way to do this is with the following '~~my.pmrc' file ~~(which~~ is ~~what you should name it, since that's what you told fetchmail to pass on to procmail in the above file):~~

	~~MAILDIR=$HOME/mail~~
	~~DEFAULT=$MAILDIR/INBOX~~
	~~LOGFILE=$MAILDIR/delivery-log~~
	~~LOCKFILE=$MAILDIR/~~.~~default.lock~~
	~~:0 fhw~~
	\|~~formail~~
	#
	~~# Any other rules the user wishes to either include with INCLUDERC,~~
	~~# or hardcode into this file, would go~~ here.
	~~# --------------------------------------------------------------------------~~
	~~# If we're here, the mail didn't match any~~ other ~~rules, so deliver normally.~~
	~~:0:~~
	~~$DEFAULT~~
	~~# If that fails, report an error and throw the mail away.~~
	~~EXITCODE=75~~
	:0
	~~/dev/null~~

	~~These lines mean:~~
	* Lines 1-4 set some environment variables, making the default destination ~/mail/INBOX, setting a log destination and a lock file
	*: '''NOTE:'''
	*:: If the file exists, and is a mailbox file, all downloaded messages will be appended to it. Also, if you want the logging to be more verbose (and much larger), specify "<tt>VERBOSE=ON</tt>" in the file.
	* <tt>:0 fhw</tt> starts a Procmail recipe
	* <tt>\|formail</tt> tells Procmail to pipe every message through 'formail'
	*: This adds a "From " header to the top of each message, required for some mail programs to determine where one mail stops and the next one starts. This header is different than the standard "From:" header (note the space, and lack of colon). Do not remove this line unless you know what you're doing, or if you don't need the emails in standard Unix mbox format. Since most programs that don't need this line will ignore it if it exists, it's safe to have it added to the messages.
	* A few comments, which serve as placeholders.
	*: If you want to get fancy and insert normal Procmail recipes here, go right ahead. One example would be if you told Fetchmail to gather messages from every folder on the IMAP server, and want it to filter the messages into multiple files, you would put the proper procmail recipes there. For more information on Procmail recipes, see the [[Procmail]] ~~article. Do note that forwarding messages from Procmail at this point is a bad idea; it would be much better to do this directly on the mail server instead with [[Websieve]].~~
	* <tt>:0:</tt> sets up another recipe, this time locking the file (the trailing colon).
	* <tt>$DEFAULT</tt> tells procmail to write the message to the file named in $DEFAULT (which you ~~defined above).~~
	* Should the write above fail for some reason, the last few lines in the script take care of it:
	** Set the EXITCODE to 75
	*: When Procmail exits it will be with a non-zero status. Exit code 75 specifically tells the parent process that the message could not be delivered, and that it should try again some other time. This will pass the information on to fetchmail that it should not* delete the mail from the server (if it's configured to do so), or mark the mail as read.
	** Throw the message away to /dev/null since it couldn't deliver it.

	Now that you have your .fetchmailrc file, and a basic my.pmrc file setup, run the 'fetchmail' command. It will ask for your password on the server; once you give it, the program will start to collect your mail from the server and deposit it into a file for you.

imported>Huston at 17:22, 21 May 2007

2007-05-21T17:22:11Z

← Older revision		Revision as of 17:22, 21 May 2007
Line 6:		Line 6:

	'''NOTE:'''		'''NOTE:'''
	: This is important, and bears repeating. If you do not understand what you're doing with Fetchmail, you '''''CAN''''' and most likely '''''WILL''''' cause problems ranging from losing your email to infinite loops which can crash a mail server. If you're not sure, ask ~~help@astro~~ to have a look at your setup. While Fetchmail isn't actively supported here, we'd rather you wait to get an answer from us than blindly run the program and then ask us to restore your entire mailbox.		: This is important, and bears repeating. If you do not understand what you're doing with Fetchmail, you '''''CAN''''' and most likely '''''WILL''''' cause problems ranging from losing your email to infinite loops which can crash a mail server. If you're not sure, [[Requesting assistance\|ask us]] to have a look at your setup. While Fetchmail isn't actively supported here, we'd rather you wait to get an answer from us than blindly run the program and then ask us to restore your entire mailbox.

imported>Huston at 15:00, 11 May 2007

2007-05-11T15:00:28Z

← Older revision		Revision as of 15:00, 11 May 2007
Line 113:		Line 113:
	[[Category:Email]]		[[Category:Email]]
	[[Category:Software]]		[[Category:Software]]
	[[Category:Unix Commands]]		[[Category:Unix Commands\|fetchmail]]

imported>Huston: Categorizing

2007-05-09T15:47:54Z

Categorizing

← Older revision		Revision as of 15:47, 9 May 2007
Line 113:		Line 113:
	[[Category:Email]]		[[Category:Email]]
	[[Category:Software]]		[[Category:Software]]
			[[Category:Unix Commands]]

imported>Huston: Categorizing

2007-05-09T15:41:46Z

Categorizing

← Older revision		Revision as of 15:41, 9 May 2007
Line 112:		Line 112:

	[[Category:Email]]		[[Category:Email]]
			[[Category:Software]]

imported>Huston: Finished (?) cleanup

2007-05-08T20:35:49Z

Finished (?) cleanup

← Older revision		Revision as of 20:35, 8 May 2007
Line 1:		Line 1:
	~~{{inuse\|a little while -- [[User:Huston\|huston]] 15:33, 8 May 2007 (EDT)}}~~
	~~{{oldfaq\|NUM=118}}~~
	Fetchmail is a wonderful program which might just solve a lot of issues that people have when they want to copy data from the IMAP server, whether it be because they're leaving the department and want a copy of their mail, or if you wish to use a mail client which can't access the IMAP server directly (though that is an unsupported path, so you're on your own).		Fetchmail is a wonderful program which might just solve a lot of issues that people have when they want to copy data from the IMAP server, whether it be because they're leaving the department and want a copy of their mail, or if you wish to use a mail client which can't access the IMAP server directly (though that is an unsupported path, so you're on your own).

Line 34:		Line 32:

	=== my.pmrc ===		=== my.pmrc ===
			{{mergeto\|Procmail\|Talk:Fetchmail#Procmail recipes}}
	Once you have the .fetchmailrc file, you now need to configure procmail to handle the messages once they're downloaded. The simplest way to do this is with the following 'my.pmrc' file (which is what you should name it, since that's what you told fetchmail to pass on to procmail in the above file):		Once you have the .fetchmailrc file, you now need to configure procmail to handle the messages once they're downloaded. The simplest way to do this is with the following 'my.pmrc' file (which is what you should name it, since that's what you told fetchmail to pass on to procmail in the above file):

Line 73:		Line 72:


	== ~~ADVANCED TOPICS~~ ==		== Advanced topics ==
	~~Some more information on Fetchmail, if you~~'~~re so inclined, follows.~~ Make sure you understand the implications of what you're doing; deleting all your mail off the server and asking for it to be replaced may be impossible, so always test ~~the~~ script with all destructive options off first (namely, use the flag '-k' to fetchmail to force it to keep all mail, regardless of what you said in the .fetchmailrc script).		'''NOTE:'''
			: Make sure you understand the implications of what you're doing; deleting all your mail off the server and asking for it to be replaced may be impossible, so always test your script with all destructive options off first (namely, use the flag '-k' to fetchmail to force it to keep all mail, regardless of what you said in the .fetchmailrc script).


			=== Only fetch unread mail ===
			If you wish to keep your mail on the server but pull unread messages from it (and flag them as read on the server), remove the 'fetchall' keyword from the .fetchmailrc file. This tells fetchmail to only download unread mail, but mark it as read when it does. In keeping with this, you can then run 'fetchmail -c' to only check for unread mail on the server, and not try to download it.

	One trick, if for example you wish to keep your mail on the server but pull unread messages from it (and flag them as read on the server), is to remove the 'fetchall' keyword from the .fetchmailrc file. This tells fetchmail to only download unread mail, but mark it as read when it does. In keeping with this, you can then run 'fetchmail -c' to only check for unread mail on the server, and not try to download it.

	If you wish to be brave, and remove all your mail from the server keeping it only in a file, then remove the 'keep' directive from the .fetchmailrc file. This tells fetchmail to delete the messages as it downloads them. If you still have the 'fetchall' keyword there, then mail that you've read before on the server (through an IMAP client, such as Pine or Squirrelmail) will also be downloaded; if you remove 'fetchall', then only mail that is unread will be pulled down and deleted from the server.		=== Download & delete ===
			If you wish to be brave, and remove all your mail from the server keeping it only in a file, then remove the 'keep' directive from the .fetchmailrc file. This tells fetchmail to delete the messages as it downloads them. If you still have the 'fetchall' keyword there, then mail that you've read before on the server (through an IMAP client, such as Pine or Squirrelmail) will also be downloaded; if you remove 'fetchall', then only mail that is unread will be pulled down and deleted from the server.

	You can tell fetchmail to get your email from other sources as well, with using other 'poll' directives naming other servers. For example you could tell it to get your @princeton.edu mail from OIT's IMAP server. 'man fetchmail' should give you all the information you need to figure this out.

	You can ~~also~~ tell fetchmail ~~that there's a preconnect condition that must be met before it can connect~~ to ~~the named server. Why would you do this? Let's say you want to pull all~~ your ~~unread mail~~ from ~~our server~~, ~~while at home. But you don~~'~~t want to use SSL to do it, you~~'~~d rather use SSH to tunnel the connection~~. ~~Then~~ you ~~can~~ tell ~~fetchmail that instead of polling the server 'mail.astro.princeton.edu',~~ it ~~has~~ to ~~poll "localhost port 9000". Set a preconnect condition to do "ssh -L 9000:mail.astro.princeton.edu:143 coma.~~princeton.edu '~~sleep 60'"~~. ~~This tells SSH to connect to Coma, and run the command~~ '~~sleep 60~~'~~, which keeps the pipe open just enough for~~ you ~~to tunnel through it. Next, it will forward the local port 9000, through the SSH tunnel to coma, and from there connect to port 143 on~~ the mail server (which is the IMAP port). Now your username and password will be encrypted from the local machine to coma, and will only travel in cleartext from coma to the mail server (which being on the same network means it's pretty safe). This is a great trick to use if you ~~have~~ to ~~get mail with POP3, which is not usually SSL-aware (so it's an unencrypted connection), but you have a shell account on the POP3 server or another machine on its local network~~.		=== Use other sources ===
			You can tell fetchmail to get your email from other sources as well, by using other 'poll' directives naming other servers. For example you could tell it to get your @princeton.edu mail from OIT's IMAP server. 'man fetchmail' should give you all the information you need to figure this out.

	If you want to pull mail from more than one folder on the IMAP server, use the 'folders' keyword in the .fetchmailrc file. The keyword tells the program to get mail from the folders listed. Then list the folders you want to retrieve after the keyword, separated with commas. One thing to remember is that Cyrus IMAP keeps all folders as subfolders of your inbox, so if you want to get your inbox and two folders (we'll call them foo and bar), you need to specify:

	~~folders INBOX~~, ~~INBOX~~.~~foo~~, ~~INBOX~~.~~bar~~		=== Preconnect condition ===
			You can tell fetchmail that there's a preconnect condition that must be met before it can connect to the named server. Why would you do this? Let's say you want to pull all your unread mail from our server, while at home. But you don't want to use SSL to do it, you'd rather use SSH to tunnel the connection:
			# Tell fetchmail that instead of polling the server 'mail.astro.princeton.edu', it has to poll "localhost port 9000".
			# Set a preconnect condition to do "ssh -L 9000:mail.astro.princeton.edu:143 coma.astro.princeton.edu 'sleep 60'".
			#: This tells SSH to connect to coma, and run the command 'sleep 60', which keeps the pipe open just enough for you to tunnel through it. Next, it will forward the local port 9000, through the SSH tunnel to coma, and from there connect to port 143 on the mail server (which is the IMAP port).

	If you want to get really tricky, then tell procmail to deliver normally to $MAILDIR/$FOLDER. Edit the 'mda' line in your .fetchmailrc and add the '-p' flag to the call to procmail (this preserves the environment when you call the program). Then do this with a bash script (untested, use at your own risk!):		Now your username and password will be encrypted from the local machine to coma, and will only travel in cleartext from coma to the mail server (which being on the same network means it's pretty safe). This is a great trick to use if you have to get mail with POP3, which is not usually SSL-aware (so it's an unencrypted connection), but you have a shell account on the POP3 server or another machine on its local network.


			=== Not just for inboxes ===
			If you want to pull mail from more than one folder on the IMAP server, use the 'folders' keyword in the .fetchmailrc file. The keyword tells the program to get mail from the folders listed. Then list the folders you want to retrieve after the keyword, separated with commas. One thing to remember is that Cyrus IMAP keeps all folders as subfolders of your inbox, so if you want to get your inbox and two folders (we'll call them foo and bar), you need to specify "<tt>folders INBOX, INBOX.foo, INBOX.bar</tt>".

			If you want to get really tricky, then tell procmail to deliver normally to $MAILDIR/$FOLDER. Edit the 'mda' line in your .fetchmailrc and add the '-p' flag to the call to procmail (this preserves the environment when you call the program). Then do this with a bash script (untested, use at your own risk!):

	for F in inbox inbox.foo inbox.bar ; do		for F in inbox inbox.foo inbox.bar ; do
Line 95:		Line 108:
	done		done

	This will set $FOLDER to inbox and run fetchmail, gathering your inbox and putting the messages into ~/mail/inbox ; then it will run again, getting mail from INBOX.foo, and putting it into the file ~/mail/inbox.foo, and then again with bar, etc. You could do this listing all your mail folders in the "for" line, and it will pull down all your mail from the entire server. It will ask for your password every time, but there is a way around that in the .fetchmailrc file (an exercise left for the reader, since it's very insecure to have your password in a file like that). Also be very sure that you know what you're doing if you do this; if you have the "fetchall" keyword in your .fetchmailrc file, and no "keep", you'll not only be copying all your messages to local files but also deleting them from the server, which is not ~~reversable~~.		This will set $FOLDER to inbox and run fetchmail, gathering your inbox and putting the messages into ~/mail/inbox; then it will run again, getting mail from INBOX.foo, and putting it into the file ~/mail/inbox.foo, and then again with bar, etc. You could do this listing all your mail folders in the "for" line, and it will pull down all your mail from the entire server. It will ask for your password every time, but there is a way around that in the .fetchmailrc file (an exercise left for the reader, since it's very insecure to have your password in a file like that). Also be very sure that you know what you're doing if you do this; if you have the "fetchall" keyword in your .fetchmailrc file, and no "keep", you'll not only be copying all your messages to local files but also deleting them from the server, which is not reversible.


	[[Category:Email]]		[[Category:Email]]

imported>Huston: Beginning cleanup

2007-05-08T20:03:01Z

Beginning cleanup

Show changes

imported>Huston: Tagging inuse

2007-05-08T19:33:38Z

Tagging inuse

← Older revision		Revision as of 19:33, 8 May 2007
Line 1:		Line 1:
			{{inuse\|a little while -- [[User:Huston\|huston]] 15:33, 8 May 2007 (EDT)}}
	{{oldfaq\|NUM=118}}		{{oldfaq\|NUM=118}}