I use the Emacs package Gnus (included in Emacs) to manage my email, and have used it this way for years. There are many reasons for managing your email locally, and even more for managing them in Emacs, but it took me years to make it through the obstacle course of actually figuring out how to do it. It requireed learning many new concepts that our day of webmail and Exchange-programs generally glosses over. So, without further ado, here is the way I got it to work. Some notes to characterize my email situation, first:

• I’m using Linux locally
• I have, depending on the day, 8-12 external email inboxes I need to gather my mail from
• I want to use POP to get the mail, rather than IMAP, because half my point was to have a local setup that would just work without regards for internet connection sometimes.

Ok, those describe my position. Now the solution. Things we’ll be thinking about for this solution:

• Downloading and managing my messages are two separate processes.
  • I use getmail to download the messages. I used to use fetchmail, but getmail is a better and robust piece of software. This results in my messages being in my local spool, which is where emacs gnus will look for them.
  • Gnus is what renders them, sorts them into my specified folders, threads or dedups them.
  • I use the system tool notmuch to quickly search and index my messages. And, by “quick”, I mean faster than Google. Notmuch has some good interfaces with gnus and emacs.
• Sending messages requires complementary setup in gnus and in my Linux system’s Authinfo (actual code samples below). In general, sending mail is a very different process than retrieving mail. I’ll cover both below.

Getting Mail – a POP method

Some users like staying closely in touch with their webmail setup and also minimizing their local footprint, and for those users, IMAP is a good solution. I, however, want all my messages locally and I don’t much use webmail at all; it is only my redundancy fallback. So, to obtain my mail, there are two requirements: fetching the mail FROM the web server TO my local spool, and then FROM my local spool into Emacs Gnus.

Downloading my mail to be offline: setting up getmail

As I mentioned, I am reading close to a dozen email inboxes from my single emacs session. If I were using webmail interfaces and a browser, this means I would have to keep tabs on a dozen different sites (which would be a pain). Here’s an idea of my getmail setup to make this happen:

First, install getmail with the instructions on their site: https://pyropus.ca/software/getmail/configuration.html . For me, it was in my distro repository.

Getmail conf files for every remote mailbox

Next, make getmail configurations for each of your email addresses you are collecting mail from. I store all of these in my ~/.getmail directory, which looks as follows (omitting the files getmail itself will generate to keep track of what’s been downloaded already):

<torysa@endlessinside> ~/ 08:38$ tree ~/.getmail
/home/torysa/.getmail
├── getallmail.sh
├── mail@mymailsite.com
├── mail@mymailsite.com.log
├── church@gmail.com
├── church@gmail.com.log
├── sent@mymailsite.com
├── sent@mymailsite.com.log
├── social@mymailsite.com
├── social@mymailsite.com.log
├── personalmail@gmail.com
├── personalmail@gmail.com.log
├── personalmail@yahoo.com
├── personalmail@yahoo.com.log
├── me@mymailsite.com
├── me@mymailsite.com.log
├── job@mymailsite.com
├── job@mymailsite.com.log
├── me@yahoo.com
└── me@yahoo.com.log

Here’s an example of just one, the church@gmail.com file (yeah, it’s weird that they don’t have a regular extension, right?). All of those .com files above look like this, with the obvious alterations for their credentials:

[retriever]
type = SimplePOP3SSLRetriever
server = pop.gmail.com
username = recent:church@gmail.com
password = mysecretpassword

[destination]
type = Mboxrd
path = /var/spool/mail/<my-user>

[options]
read_all = false
verbose = 2
message_log-verbose = true
message_log = ~/.getmail/church@gmail.com.log

Notice the path = portion specifies where on my local machine to put the email: it’s identical for each of my email setups, because they are all coming to the same inbox and my gnus setup will sort them out from there.

Having those files containing instructions and credentials, now it’s time to actually set up a routine to USE those instructions.

Automating Mail Retrieval

First, I wrote the shell file you saw in the directory listing above, getallmail.sh. It’s a simple little thing that just tells getmail to read all my getmail files and download messages to my local spool:

#!/bin/bash
# ~/.getmail/getallmail.sh

connected () {
# determine whether we have any internet connection at all; otherwise don't try to get messages. If you don't do this, your local daemon might spam you with "connection failed" emails when you're offline.
    CON="$(curl -s --max-time 2 -I https://google.com | sed 's/^[^ ]*  *\([0-9]\).*/\1/; 1q')"
if [ $CON == 2 ] || [ $CON == 3 ]
   then 
       return 0
else
    return 1
fi
}

if [ connected ]
then
    getmail -rsocial@mymailsite.com -rpersonalmail@yahoo.com -rjob@mymailsite.com -rmail@mymailsite.com -rchurch@gmail.com -rsent@mymailsite.com -rpersonalmail@gmail.com -rme@mymailsite.com -rme@yahoo.com -q
fi

This getmail command just downloads my emails, as per my earlier config files, on demand. Of course, I want this to be automatic so I need something on my local path to trigger this shell script.

#!/bin/bash
# File at ~/bin/get-mail just runs the above script when I run "get-mail". 
maildir=/home/torysa/.getmail
cd $maildir
./getallmail.sh 2> /dev/null
cd - > /dev/null

Finally, with all these pieces in place, I set up a chronjob to download my mail every 5 minutes. I’ve sometimes considered just doing this hourly to minimize distractions, but I have too many people who stop by my office saying, “I just sent you a message about… What do you think?” So, the cronjob looks something like this (incidentally, using emacs crontab-get after you have created a localhost crontab):

*/5 * * * * /home/torysa/bin/get-mail

Manual Mail Retrieval

For those occasions where I’m looking for a confirmation email or a receipt or a login email, sometimes it’s necessary to get the mail on demand. With the above setup this is trivial; just bind a key to run that same file that your cronjob is running. Back when I was using KDE I would just set a KDE global keyboard shortcut to execute my get-mail bin script. Since I’m now using EXWM, my globabl keybinding is right in emacs and looks like this:

(exwm-input-set-key (kbd "s-<f9>") (lambda () (interactive) (shell-command "~/bin/get-mail &" nil nil)))

Setting up Gnus for displaying mail

Okay; except for the exwm bit, everything above knows nothing about emacs. Now for the part that might be especially precious information: how to get Gnus set up to my liking so that it can work with those downloaded messages. For me, I have a few small passages in my emacs.el, and the rest in my gnus file (simply named “gnus”, no file extension).

(setq nnmail-crosspost nil) ;; stop cross-posting
(setq gnus-show-threads nil) ;; no threads
(setq mail-source-movemail-program "movemail") ;; default for actually getting mail from local spool to the place gnus wants it

(setq
 nnmail-spool-file "/var/mail/torysa" ;; where my localhost is
 display-time-mail-file "/var/mail/torysa"  ;; I will get a biff icon in emacs whenever mail is found uncollected here
 message-default-charset `utf-8
 nnml-directory "~/Mail/" ;; where it puts the mail once it collects it
 gnus-select-method '(nnml "")
 gnus-prompt-before-saving t
 nnml-get-new-mail t ;; get new mail when I appear
 gnus-activate-level 5 ;; gnus levels are 1-7; normal mail groups will be 5.
)

Now, let’s format what lines will look like in my message list. This is per *

(setq gnus-summary-line-format "%U%R%([%-30,30f]:%) %-50,40s(%&user-date;)\n")

;;;; default: 
;; "%U%R%z%I%(%[%4L: %-23,23f%]%) %s"

Now I specify the addresses I want to go to my bulk-mail folder (not always spam, because I may want to review them):

(setq my-gnus-bulk-from-address-list '("@maillist.codeproject.com"
                                   "@pizzahutoffers.com"
                                   "@papajohns-specials.com"
                                   "@qomail.quikorder.com"
                                   "@linkedin.com"
                                   "@facebookmail.com"
                                   "@plus.google.com"
                                   "@twitter.com"
                                   "@youtube.com"
                                   "@linguistlist.org"
                                   "@diigo.com"
                                   "@sportsauthority.com"
                                   "@bookbub.com"
                                   "@getpocket.com"
                                   "@a.narrativemagazine.com"
                                   "@emails.deseretbook.com"
                                   "@javacodegeeks.com"
                                   "@bencarson.com"
                                   "@yelp.com"
                                   "@flickr.com"
                                   "@mail.goodreads.com"
                                   "@mailer.netflix.com"
                                   "@imdb.com"
                                   "@explore.pinterest.com"
                                   "@pastebin.com"
                                   "@ldsplanet.com"
                                   "@pdfbox.apache.org"
                                   "@quora.com"
                                   )) ;; list of bulkmail addresses
(setq my-bulk-from (concat "^From:.*" (regexp-opt my-gnus-bulk-from-address-list))) ;; compile them for GNUS consumption

Now I specify the actual GNUs folders that will receive my mail, and the rules that they’ll be sorted by.

(setq nnmail-split-methods
      `(("archive.sent" ,my-from-address)
    ("mail.amazon" "^From:.*amazon.*")
    ("mail.basecamp" "^From:.*basecamp.*")
    ("mail.social" "^To:.*my.social@email.com.*")
    ("mail.church" "\\(^From\\|^Cc\\|^To\\):.*\\(church@gmail.com\\)")
    ("mail.bulk" ,my-bulk-from) ;; those bulkmail addresses
    ("mail.git" "^\\(^From\\|^Cc\\|^To\\):.*\\(@github.com\\|@gitlab.com\\|@pdfbox.apache.org\\)")
    ;; ... a few others elided
    ("mail.misc" "") ;; default folder for everything else
    ))

Now, for saving messages I send: because I use multiple computers and want “sent” to go to all of them, I don’t use emacs to save archives. Instead I BCC all outgoing messages to an email address used for nothing else but sent mail, and collect that from every computer.

;; Don't save anything "sent" locally (I use a catch-all BCC)
(setq gnus-message-archive-group nil)

Clock and Biff

Whenever mail exists tht I haven’t collected yet, I want an icon to appear on my emacs mode line to let me know. How about a red mail envelope?

(setq
 display-time-24hr-format t
 display-time-use-mail-icon t
 display-time-mail-face 'display-time-mail-face
 display-time-day-and-date t)
(display-time-mode 1)

Sending Mail

Now for sending mail in emacs. Aagin we are in my gnus config file. First, I specify smtp settings

(setq
 smtpmail-stream-type 'ssl
 smtpmail-stream-type 'plain
 smtpmail-smtp-server "smtp.gmail.com" ;; default, but not always used
 smtpmail-smtp-service 465
 mail-from-style 'parens
 smtpmail-debug-info t
 smtpmail-debug-verb t)

Now, I create a list of my “from” addresses I use. Note this will interact with my system ~/.authinfo.gpg file. In particular, because authinfo is used and it links username and credentials to server names, it is necessary to spoof servernames to use multiple emails on the same server. Hence the “yahoo2”, “mysite3”, etc.

(setq smtp-accounts ; setq instead of defvar
  '( ;; Farther down the list means earlier in the candidates
    ("social@mysite.com" "social.mysite.com" 465 ssl)
    ("webdev@mysite.com" "mail.mysite2.com" 465 ssl) ;; Web development
    ("mail@mysite.com" "mail.mysite3.com" 465 ssl);; Professional
    ("torys@mysite.com" "mail.toryanderson3.com" 465 ssl);; Personal
    ("me@gatech.edu" "mail.toryanderson3.com" 465 ssl) ;; School
    ("church@gmail.com" "eldersgmail" 465 ssl) ; Murray 1st Elders Quorum
    ("lingua.infinitum@gmail.com" "smtp.gmail2.com" 465 ssl) ;; Linguistics
    ("me@gmail.com" "smtp.gmail.com" 465 ssl) ;; Public
    ("me@fiu.edu" "smtp.gmail3.com" 465 ssl)
    ("me@byu.edu" "gateway.byu.edu" 25 plain)
    ("mine@yahoo.com" "smtp.mail.yahoo.com" 465 ssl)
    ("mineother@yahoo.com" "smtp.mail.yahoo2.com" 465 ssl)))

These are used by a function that automatically sets my “from” and outgoing settings according to the message I’m replying with, so that whoever I’ve specified my “from” to be will get respected:

(defun set-smtp (server port user stream-type)
  "Set related SMTP variables for supplied parameters. String `user' will not be evaluated."
  (message "set-smtp called with `%s' (`%s') :: `%s' (`%s') :: `%s' (`%s') :: `%s' (`%s')"
       server (type-of server) port (type-of port) user (type-of user) stream-type (type-of stream-type))
  (setq smtpmail-smtp-server server smtpmail-smtp-service port smtpmail-stream-type stream-type)
  (message "Setting SMTP server to `%s:%s' for user `%s' with stream-type `%s.'"
       server port user smtpmail-stream-type))

(defun change-smtp () 
  "Change the SMTP server according to the current from line."
  (save-excursion
    (cl-loop with from = (save-restriction
                       (message-narrow-to-headers)
                       (message-fetch-field "from"))
         for (address server port stream-type) in smtp-accounts
         do (if (string-match address from)
                (return (funcall 'set-smtp server port address stream-type))
              (message "Failed to match %s with %s" address from))
         finally (error "Cannot infer SMTP information."))))

(defun my-change-and-send ()
  "Change the SMTP settings (before opening connection) 
and then send with `message-send-and-exit'"
  (interactive)
  (change-smtp)
  (message-send-and-exit))

(add-hook 'message-mode-hook
      (lambda ()
        (local-set-key (kbd "C-c C-c") 'my-change-and-send)))

 ;; Reply-to with same address an email was sent to
(defun tsa/generate-posting-styles-from-smtp-accounts (smtp-accounts)
  "Given the smtp-accounts variable where the first value of each item is an email address, create posting styles to reply with the same addresses"
    (let ((res ()))
      (dolist (acct smtp-accounts)
    (dolist (hdr '("to" "cc" "bcc"))
      (push `((header ,hdr ,(car acct))
              (address ,(car acct))) res)))
      res
      ))

(setq gnus-posting-styles (tsa/generate-posting-styles-from-smtp-accounts smtp-accounts))
(add-to-list 'gnus-posting-styles '(".*" (bcc "sent@toryanderson.com")) t ) ;; append: always send to bcc address
(add-to-list 'gnus-posting-styles '(".*" ("Content-Transfer-Encoding" "quoted-printable")) t )

Authinfo

Finally, I need to make my configuration in our ~/.authinfo file (mine is encrypted as authinfo.gpg). My ~/.authinfo.gpg file looks like this:

machine smtp.mail.yahoo.com login "me@yahoo.com" port 465 password "<mypassword>"
machine smtp.mail.yahoo2.com login "me@yahoo.com" port 465 password "<mypassword>"
machine irc.freenode.net login "me@irc" password "<mypassword>"
machine mail.mysite3.com login mail@mysite.com port 465 password "<mypassword>"
machine mail.mysite2.com login other@mysite.com port 465 password "<mypassword>"
machine mail.mysite.com login mes@mysite.com port 465 password "<mypassword>"
machine smtp.gmail.com login myusername port 465 password "<mypassword>"
machine smtp.gmail2.com login otherlogin port 465 password "<mypassword>"
machine smtp.gmail3.com login otherother  port 465 password "<mypassword>"
machine eldersgmail login "church"  port 465 password "<mypassword>"
machine mail.gatech.edu login me@gatech.edu port 465 password "<mypassword>"
machine mysite.com login toryande password "<mypassword>"
machine research.iac.gatech.edu login tanderson49 password "<mypassword>"
machine torycanderson.com login toryande password "<mypassword>"
machine localhost.localdomain login root password "<mypassword>"
machine social.mysite.com login social@mysite.com password "<mypassword>"

There are two things going on in this file. First, it’s the username and password (and port, if necessary) expected by the web service. Second, this is the only way I know of to allow the .gnus configuration to associate two different configurations with the same server setup.

Searching Inbox

Once inside a message group GNUS provides several search commands all starting with the / key. This is fine for finding messages amidst a small group, but it is essentiallhy just searching the text iteratively, so is horrible for doing a complete search over everything. It turns out notmuch is perfect for this and is supported by emacs out of the box. First install notmuch, which might be in your distro repos. Then get make some config settings:

(setq gnus-secondary-select-methods
    '((nnml ""
            (nnimap-address "localhost")
            (nnir-search-engine notmuch))))

(setq nnir-method-default-engines '((nnml . notmuch)
                                  (nnimap . imap)
                                  (nntp . gmane)))

Now, whenever in message mode I try “G G”, it will start super-fast notmuch searches. Having not much on my command path, now I ask emacs to update the notmuch index command whenever I run the command that start my email:

(defun tsa/quick-gnus (&optional new-frame)
  (interactive "P")
  (if new-frame (gnus-other-frame)
    (gnus))
  (async-shell-command "notmuch new" nil nil))

(global-set-key [f9] 'tsa/quick-gnus)

I am also now trying helm-notmuch which requires nothng more than to install the package to start rapidly searching your messages from anywhere the key is defined (below command using use-package, which I recommend to anyone for emacs package management).

(use-package helm-notmuch :ensure t)