Recently in email Category

Most electronic mail clients support storing mail in folders; all of these which I used support filtering new mails into appropriate folders. When mail is divided into several folders, e.g. for some mailing lists to which I’m subscribed or for some automatically sent messages, it is easier to manage it. I prefer to read more important mail before the less important one (or less important first?) and these folders make it nicer.

It’s fine with one computer running a mail client, but the situation becomes more complicated with two computers. Then IMAP is necessary (excluding the use of webmails, but they usually use only a single computer). It is the protocol which stores mail on the server, is fast and allows realtime notifications for new messages (this feature is not currently supported by Kmail, but is planned).

I use IMAP since I first configured my mail server. Initially because most free software webmails require IMAP, but about a year later I began to access my mail from two computers. The filters were done by only one of the mail clients, so they became less useful. I could have configured them on both of my client computers, but it would require sharing the settings.

I have only one mail server, so I decided to filter the mail on the server. Quickly I found the well-known Procmail program used for that. Since I use Gentoo GNU/Linux operating system, I installed the mail-filter/procmail package. To my Postfix SMTP server configuration file /etc/postfix/main.cf I added

mailbox_command = /usr/bin/procmail

just as the comment in the file shows (as stated by the comment, I forward root’s mail to my normal user). Then the command /etc/init.d/postfix reload requested Postfix to reload the configuration file.

In my home directory I put a file .procmailrc instructing Procmail what to do with my mail. It begins with

MAILDIR=$HOME/.maildir/
LOGFILE=$HOME/.procmaillog
LOGABSTRACT=no
VERBOSE=off

which contains generic parameters used by Procmail. I use maildirs with the default Gentoo path of $HOME/.maildir. For all maildirs the trailing slash should be used in Procmail configuration, since it specifies the file format. I learned it by finding an mbox file when the slash was omitted.

Then specific recipes are stated. One of the ones which I use is

:0
* ^From: *forum-mods@gentoo\.org$
$MAILDIR/.gentoo-forums/

The first line begins with :0 and may contain specific flags which I don’t use in this recipe. The lines beginning with an asterisk contain an extended regular expression which is by default matched in the mail headers (in this case the sender address). If all conditions are satisfied, then the action specified in the last line is performed. Here it moves the mail to the gentoo-forums folder in my mailbox.

I don’t use more complicated recipes in my .procmailrc file. See the procmailrc(5) man page for the complete syntax of these files. Features explicitly stated in the man page look similar to filtering features of the Kmail mail client, but there are much larger possibilities.

Today many people use many online services. Each service wants to identify the user. Therefore they need to check if a human uses them, and which human it is. But this checks aren’t always correct.

Many real world security systems are designed to be seen by humans who pay for them. Clearly, this criterion prefers solutions difficult to humans over solutions difficult for bots, since these services may be simpler to distinguish by a human. A nice example of this is a CAPTCHA. It is clearly a problem for humans, I usually need three tries to correctly read text from a CAPTCHA.

For bots CAPTCHAs are not always difficult. Some are designed to be difficult to read by humans, since this may be easily considered ‘secure’, but also easily readable by bots. The reCAPTCHA sites list several examples of such snake oil CAPTCHAs, I have seen one of them at a site of one of the most well-known technical universities in Poland. It wasted human time, but not very much – sometimes it was the same as previously. Clearly, this wasn’t useful.

A CAPTCHA could be necessary on that site, since they generated easy to guess passwords and usernames. On everything else I use long passphrases or the output of head -c6 /dev/random | base64 which produces clearly better passwords than five lowercase letters generated by the technical university. Of course, even five lowercase letters password is more secure than five lowercase letters password sent in an unencrypted email. It is best when the user may write any username and password, just like many other universities allow them to do.

When the user has a password, they may forget it (or forget where they had written this password). Then there are several solutions. Some services allow them to answer questions which they have written previously. These question may be trivial to answer, so I use separate outputs of head -c12 /dev/random | base64 as the question and answer (16 random characters question answered by another 16 random characters). Other services send emails with an URL allowing changing the password. This is not completely secure, since email is insecure, but it may be improbable that someone else will read this email before the URL will be used by the correct user.

The Jacob’s Law of the Internet User Experience stating that ‘users spend most of their time on other websites’ leads to a clear conclusion in this case. The popular ‘solutions’ will be still popular, since people know them. But avoiding the mistakes described in this post should not be a problem for usability – a better CAPTCHA or none is easier to use than a bad one, people usually enter passwords and use emails to reset passwords (although these email are probably not read, since usually they work as expected). It is nice that an organization valuing security or usability may by one decision improve both security and usability.

Update: in December 2009 I decided to use Gmail having instead of all these problems with using my own mail server. The explanation of these problems (excluding power use of the server) still might be useful.

Today Google again decided to block my IP address from sending mail to Gmail users.

In elder days Internet users trusted each other. It is well-known especially from texts about ITS (e.g. OS and JEDGAR from Jargon File). In these times SMTP, the protocol on which Internet mail is based, was developed.

This protocol made spam (named after the Monty Python sketch) very easy – nearly all cost of email was on the receiver side who had to store it or relay to other hosts.

It is clearly visible how harmful is spam. So hosts sending spam were blocked by some receivers. Then spammers used open relays – SMTP servers sending emails from anyone to anyone. Therefore open relays were blocked.

Now spammers use botnets to send spam. Large numbers of hosts with insecure software are used for this. This makes direct fight with spammers impossible. The same botnets have also other uses, some very political.

There is no general way of finding who controls botnets. It would be also too inconvenient to block millions of users.

Since most users do not have their own SMTP servers and their ISPs prefer them to relay mail through their SMTP servers, it is possible to block mail sent directly from spam-sending hosts. So called ‘real time blackhole lists’ (RBLs) are used for that.

There are three main problems with RBLs – organizations controlling them are not entirely neutral, it is difficult to be removed from an RBL, and the same IP addresses are used for both spam and ham. The reasons why RBLs should not be used are nicely explained by Samuel Hart.

Some RBLs trust only users of very expensive connections. Many do not allow any mail from dynamic IP addresses. In many places there are no cheap static IP services. Supporting only these who are willing to pay more is fundamentally wrong (except for paid services).

Since I have my own SMTP server (privacy and the need to learn by doing are my main reasons for this) and I use an affordable Internet connection, an RBL decided to block my current IP address. Therefore technically I cannot send mails to Gmail users.

There are good methods of fighting with spam – use a secure, free operating system, recommend it to your friends and oppose RBLs. And Bayesian spam filtering by the recipient only may help if someone sends spam to you.