Getting email around spam filters

Sending email programmatically can be tricky, especially if you’re sending it in bulk.

Unless you’re careful it tends to get marked as spam, and the problem is exacerbated by the fact you’re unlikely to know about it – users rarely trawl through their spam folder – and rarer still let you know your email has been caught there.

Solving this problem is a bit of a dark art, like SEO, since spam filters tend to be a bit arbitrary but here are a few tips that will help:

Send from a static IP address:
Email providers often blacklist blocks of dynamic IP addresses since these are often used by spammers. So, make sure you’re sending it from a static IP.

Don’t include images
This is a contentious one, but I’ve seen a fair amount of legitimate corporate email sent to the spam folder because of the inclusion of a company logo. I prefer to play it safe and, if possible, only send plain text emails.

SPF records
The Sender Policy Framework (SPF) was introduced to combat fake sender addresses, which nearly all abusive e-mail messages carry.

I’ve gone to my inbox before, only to find thousands of autoresponders and failed delivery messages. Initially I thought my webserver had been hacked, but instead the spammers were just setting the sender address as my one – to try and make them look more legitimate.

SPF is designed to combat this and allows the owner of a domain to specify their mail sending policy, e.g. which mail servers they use to send mail from their domain. You need to set a SPF record on your domain’s DNS record so receiving servers can check whether the message complies with the domain’s stated policy. You can see SPF at work if you look at an email’s header, you can do this in GMail like this:

Here we can see that Twitter hasn’t configured SPF correctly:
Received-SPF: softfail (google.com: domain of transitioning noreply@twitter.com does not designate 66.7.206.23 as permitted sender) client-ip=66.7.206.23;

There are lots of good articles out there, I recommend the easyDNS one, but you should be able to find one specific to your DNS provider.

Reverse DNS Record
Reverse DNS is a way of associating an IP address with its domain name. Email providers often check that the reverse DNS record of the IP that sent an email is the same as the sender’s domain to determine whether the email’s spam (although they generally don’t require that these are the same).

Reverse DNS is setup by configuring PTR records (Pointer Records) in your DNS server. To make changed to your Reverse DNS PTR records, you must contact the company where you get your IP addresses from, usually a hosting facility or an ISP.

Try running nslookup on your domain name’s IP – that will show you the Reverse DNS Record.
Here’s the result from 75.101.158.59 (aireohq.com)

  nslookup 75.101.158.59
  Server:		62.140.195.84
  Address:	62.140.195.84#53

  Non-authoritative answer:
  59.158.101.75.in-addr.arpa	name = ec2-75-101-158-59.compute-1.amazonaws.com.

This is another problem with sending email from EC2 – the Reverse DNS Record doesn’t match the sender’s domain, and you can’t change the PTR Records to rectify the problem.

MX Records
This one’s really important – most email providers will check the sender domain’s MX Record to see if the server that sent the email is included. You can check the MX Records for your domain like this:
dig -t mx madebymany.co.uk

Delegation
Well, the easiest cop out solution is to delegate your email sending to a SMTP relay like AuthSMTP or easydns. These guys usually send so much email that they have agreements with the major ISPs so their email doesn’t get marked as spam. Don’t forget to configure your SPF record to authorize the relay to send email on your behalf. Paul Dowman has a good tutorial on how to relay mail from an EC2 instance to AuthSMTP through Postfix.

There are other ways that mail gets marked as spam, but you can’t influence most of them. Keep checking the SMTP logs to see if mail gets rejected, since usually there’s a helpful explanation.

Does anyone have any other tips?