Some of the most common questions that I get asked relate to keeping email lists clean, commonly known as list hygiene.
In the previous blog post, I explained how list hygiene can impact your sender reputation and explained about the different types of “bad” email addresses that you should avoid sending emails to. In this post, I’m going to explain how you can keep your email list clean so that you avoid sending emails to the “bad” addresses I talked about last time. I’ll describe the most common options and methods for keeping your list clean, and explain the pros and cons of each.
As I mentioned in the last post, keeping your email list clean is incredibly important. If you don’t do it, it can have a huge impact on your sending reputation and ultimately cause your emails to end up in the spam folder.
The RACE Model
Firstly, I’ll recap on the four pillars of deliverability that can all affect whether or not the emails that you send will reach the inbox, or whether they’ll be confined to the spam folder. They spell out the word RACE, which stands for:
- Reputation
- Authentication
- Content
- Engagement
Recap on Reputation
In this post, we’re still looking at reputation, which plays a big part in whether your email will be delivered to the inbox. If you send emails to “bad” addresses, it will very quickly drag your reputation down and, as previously mentioned, you’ll find that more of your emails end up in the spam folder.
Hopefully you already know that now, so let’s jump right into the detail.
Options for Keeping Your List Clean
As always, the devil’s in the detail, but let’s keep things as simple as we can. There are four main approaches you can take to keep your list clean:
- Perimeter defence
- Rule-based checking
- Verification and scrubbing
- Monitoring sending
Let’s talk about each of those in more detail.
1. Perimeter Defence
This is one of my favourite approaches, not least because it solved a nasty problem that has affected me personally, and I know it affects many other people.
Consider the problem of spam bots. These are maliciously written pieces of software that scrape legitimate email addresses from the Internet and search the Internet for web forms that allow you to sign up to an email list. They then submit the email addresses that they find to those web forms, with no permission from the owners of the email addresses. This causes what should be a legitimate email to be sent to an email address that looks ok, but typically causes a much higher than usual rate of spam complaints because the recipient of the email never requested it.
I’ve worked with clients who have had their domain sending reputation ruined because of this. They were typically getting spam complaint rates of 0.5%, 1%, or even higher (noting that the industry standard acceptable limit for spam complaints is below 0.1%). This led quite quickly to the other legitimate emails that they were sending ending up in the spam folder.
Luckily, there are a few ways to deal with this. My preference is to set up a perimeter defence solution which is designed to catch spam bots quickly and easily.
This typically works by adding one or more extra fields to your webforms, where the additional fields are called something similar to “confirm email address” which would, to a bot, look like another place for it to submit the email address. Here’s where it gets clever though. You can set the “style” of the form to be hidden, so that a real user will never see it, and therefore never enter anything into that box. A bot isn’t that clever and will still enter the email address into that box.
You can then put code in place to detect that the extra field that has been populated at the time the form is submitted, and hence prevent it from even getting into your email marketing platform in the first place. This is why I call it “perimeter defence” – because the bad data never makes it into your database.
If you’re not sure how to create the code to stop malicious submissions in the first place, you’ve got two options.
Firstly, you can put rules in place to detect and remove the contacts once they’ve been added to your database; I’ll talk more about that in a moment.
Secondly, you can use Mihir Dhandha’s excellent SpamKill software. It’s a true perimeter defence solution that quickly and easily modifies your web forms for you so that bot submissions will never make it into your database. It’s ridiculously cheap, works flawlessly and is excellent value for money. I’m happy to endorse it because it’s not let a single spam bot submission into my database since I started using it. Thanks, Mihir!
2. Rule-based Checking
This is the next stage of the process. As I mentioned above, even if you create a fake email field to capture the bot submissions, unless you can write code or use Spamkill to implement a perimeter defence solution, you’re still going to end up with the fake contacts appearing in your email marketing database.
So this is where you need to create rules to look for signs of malicious submissions.
If you’ve got a fake email address field, it’s easy to create a rule in many marketing automation platforms to say “if the fake email address field isn’t empty, delete this contact”.
You can also look for other signs of malicious submissions. One example is where many spam bots use random strings of letters and numbers when they fill out the “name” field. So you could look for names that start with numbers and flag those for manual review (I wouldn’t recommend deleting them automatically in case someone has submitted a genuine opt-in with a number in the name field).
Luckily there are also specialised software solutions that can make this easier for you.
SpamKill, mentioned above, also has a “rules based” detection option, where it will identify contacts as a high, medium, or low risk of being a spam bot submission.
Another tool that gets an honourable mention is Jordan Hatch’s listcleaner.io solution. Again, it’s very cheap, it’s incredibly easy to set up, and can help you win the battle against spam bot submissions.
3. Verification and Scrubbing
One downside of the approaches that I’ve described so far is that they don’t prevent someone from entering an address that looks ok, but doesn’t actually exist. Most commonly this is because people make typing mistakes when they fill out forms; sometimes it’s because they don’t want to give out their email address and just choose a random address such as “mickey@mouse.com” or similar.
The net result of this is that your email marketing platform is going to try to send an email to that address, which doesn’t exist, which will then generate a bounce error. This isn’t the end of the world, but most email marketing providers still get upset if you send too many emails that bounce. So, if you’re getting a lot of invalid email addresses opting into your list, it could still be a bad thing.
This is where email address validation or verification comes in. The way this works is one of the more closely guarded secrets of the email industry – there are many solutions out there that offer this type of service. None of the ones that I’ve spoken to have ever revealed exactly how they do what they do, but the simplest explanation is that they attempt to start sending an email to the address that they’re being asked to verify. If they can connect to the remote server that manages that email address, and the server doesn’t say “this email address doesn’t exist”, they deem the address to be valid.
Some verification servers go one step further to provide the ultimate solution, known as list scrubbing. This validates the email address as just described, but it goes several steps further, using proprietary technology to identify spam traps, role accounts, seed addresses, catch-all addresses and many other types of “bad” email addresses.
If you’ve ever received that dreaded email from your marketing automation provider, where they warn you that you’re sending emails to spam trap addresses, scrubbing your list is the best solution available to you. Even if you’ve never received such a warning, it definitely makes sense to scrub your list if you’ve never done it. Almost everyone I know who’s done this is surprised just how many spam traps have made their way onto their mailing list. It’s true that sometimes, a small number of “false positive” results are returned, for instance when a customer’s email address that you know is legitimate gets flagged as a spam trap. So you need to be careful and don’t just delete all contacts that get flagged as spam traps, but in most cases it’s safer to make sure you don’t mail those addresses again.
I often get asked, “How often should I scrub my list?”. Most list scrubbing providers will recommend that you scrub your entire email list every three months, and my response to that is often “well, they would say that, wouldn’t they?” 🙂 Of course, they make money every time you scrub your list and as long as you’re doing effective list hygiene in other ways, I’m not convinced that you do need to scrub your list every three months. But it very much depends on how many new contacts you add, how often you mail them and how quickly you remove unengaged and bounced contacts. My suggestion is to wait 6 months after an initial list scrub, then scrub the list again and see how many more spam traps have appeared. If it’s a worryingly high number, consider scrubbing more frequently; if it’s a lower number, maybe leave it a year until the next time you do it.
There are many list scrubbing solutions out there and I’m hesitant to recommend one over another as I’ve never done enough testing myself to establish which one is better or worse. It’s also very subjective, as there’s no way of telling for sure whether an email address is a spam trap or not.
Having said that, of the scrubbing services that I have tested, and having taken on board feedback from my peers in the email deliverability world, I’ve settled on using Klean13 which is by no means the cheapest solution; it’s not the most expensive either, but is reassuringly expensive enough to justify the old adage, “you get what you pay for”.
The way that most scrubbing services work, you download the contacts from your email marketing platform as a CSV file, upload those contacts to the list scrubbing solution, scrub the list, download the results and upload them back into your email marketing platform.
If you use ActiveCampaign, Infusionsoft or Keap, you can also use the free “in-situ” Klean13 scrubbing integration that I’ve built into Deliverability Dashboard which lets you choose the contacts you want to scrub, click a button and then wait while everything is done automatically – much easier than the process I described above! And, as an extra bonus, you can also choose to enable “real time scrubbing”, which will automatically scrub every new contact that gets created in your email marketing platform.
4. Monitoring Sending
This is the final approach and I’d strongly recommend that you do this, even if you don’t adopt any of the other approaches that I’ve described so far.
The good news is that your email marketing platform will do a lot of this for you already. Any emails you send that bounce, or where the recipient clicks the “spam” button, will be automatically dealt with by your email platform so that you can’t accidentally mail them ever again.
It’s important that you send emails to your everyone who’s still engaged on your mailing list on a regular basis, so that any addresses which fall out of use (and hence bounce) will be spotted quickly – this minimses the risk of you continuing to mail an address that could be converted into a recycled spam trap in the future.
And, talking of engagement, this is something else that you really must monitor. Make sure that you stop sending emails to anyone who’s not opened or clicked anything in your emails in the last 90 days. If you continue to send emails to people who haven’t opened anything in longer than 90 days, you run the severe risk of damaging your sending reputation.
Conclusion and Recommendation
Now that you’ve made it this far, you should have a clearer view regarding the different ways that you can keep your list clean.
It’s important that you take at least some of the steps that I’ve explained in this article, because there’s quite a significant risk to your reputation if you don’t protect yourself.
My suggestions, as a minimum would be:
- Scrub your existing list with Klean13
- If you’re getting more than a tiny number of spam bot submissions and are using either Infusionsoft or ActiveCampaign, use SpamKill to set up your perimeter defence.
- Monitor your email sending and remove contacts that have bounced, reported spam, or not opened any of your emails for the last 3 months.
