Hosting your own email server and getting block list related “Undelivered Mail Returned to Sender” errors (e.g. S3150) for Microsoft hosted email domains like hotmail.com? Read on to find out how to reliably work around this issue quickly.
Well, I would bet everyone who operates his own email server once ran into this problem.
Out of nowhere emails addressed to Microsoft hosted email domains (like hotmail.com, hotmail.de, outlook.com, outlook.de, etc.) bounce back with some kind of spam/block list related error message.
In my case it was:
This is the mail system at host somehost.somedomain.tld. I'm sorry to have to inform you that your message could not be delivered to one or more recipients. It's attached below. For further assistance, please send mail to postmaster. If you do so, please include this problem report. You can delete your own text from the attached returned message. The mail system <somerecipient@outlook.de>: host eur.olc.protection.outlook.com[104.47.12.33] said: 550 5.7.1 Unfortunately, messages from [**.***.***.***] weren't sent. Please contact your Internet service provider since part of their network is on our block list (S3150). You can also refer your provider to http://mail.live.com/mail/troubleshooting.aspx#errors. [DB3EUR04FT004.eop-eur04.prod.protection.outlook.com] (in reply to MAIL FROM command)
It may be different errors in your case. But in my case it was just totally unclear what the problem was. I studied the information Microsoft gives under the link and spent hours trying to figure out what I could do to sort the problem out. But in my eyes I already followed their rules. I am also member of their SNDS (Smart Network Data Service) and JMRP (Junk Mail Reporting Program) and spent additional hours to find useful information there. Nothing. I ended up using this link to open a support ticket: https://support.microsoft.com/en-us/supportrequestform/8ad563e3-288e-2a61-8122-3ba03d6b8d75. I did that 6 days ago and the problem still persists. None of my emails are delivered to Microsoft hosted email domains. Correspondence with Microsoft so far was super-slow and not helpful at all.
I have had the exact same problem again a few months after finishing this article and made a sperate post to document it: Microsoft Silently Dropping Emails – a Sad but True Story
For someone who operates an email server it is a nightmare when legitimate emails can’t be delivered. It’s annoying for the sender as well as for the recipient – something Microsoft doesn’t seem to care of although they are harming their own customers. This is true if it doesn’t work for hours. But gets worse if it takes days or even weeks to solve the problem.
To cut a long story short: we need a workaround that helps instantly without depending on Microsoft’s mercy to help us!
As this happened to me many times and always took many days to sort out I found the following, very reliable solution for me.
Will this guide work for you?
This guide will likely work for you, if
- you have email delivery problems because you are listed on a block list
- you have email delivery problems with Microsoft because you trigger one of their secret spam detection rules
- Microsoft silently drops emails sent by your server
- I’m not kidding: I experienced cases where an email was sent, accepted by Microsoft servers for delivery but then just disappeared. No error message to the sender, no information for the recipient, no information to the postmaster
- you send legitimate non-spam emails
This guide will not work for you, if
- you have email delivery problems that are not related to block lists or spam rejects (e.g. receipient’s mailbox is over quota, email address doesn’t exist, etc.)
- you are really sending spam
- your Postfix server is severely misconfigured (e.g. as an open relay, really allowing sending of spam)
Our Goal
We have to relay our emails through an email server that Microsoft already trusts. In this guide I will use Mailjet for this.
Our Environment
This guide is written for a working Postfix setup.
Prerequisites
- You need access to the DNS records of the domain you plan to send emails for because we need to add/modify some TXT records
- You need root access to your Postfix server
For the Pros: This Guide in a Nutshell
If you want to read all details just skip this section and start with Step 1.
But in case you are a professional and know your stuff, this guide maybe far too detailed for you. So here is the short version of it:
- Set up a free Mailjet acoount (step 1)
- Get your SMTP relay credentials and server information from Mailjet (step 2)
- Add DNS record to validate your domain (step 3, step 4)
- Add DNS records (SPF/DKIM) for domain authentication (step 5, step 6)
- Configure Postfix to relay specific recipient domains through Mailjet servers (step 8)
- Test mail delivery (step 9)
Step 1: Setup your Mailjet-Account
Why Mailjet?
I tried different SMTP relay services but finally sticked with Mailjet because they
- have a good free plan (up to 6,000 emails/month)
- allow you to leave at any time
- offer easy and instant registering
- allow to send emails from different domains with a single authentication (a must if you have to relay for many domains!)
- have no hidden fees
- are GDPR certified
- Go to https://app.mailjet.com/signup, enter your email, choose a password, check the “I’m not a robot” checkbox and hit “Sign up!”
- Fill out your profile in the appearing form, and click “Complete Signup”
- Wait for an email of Mailjet and click the activation button
- Clicking the activation link brings you to a welcome page. Click “Get started” in the “Developer” section
- Under “Sending method” choose “SMTP relay” and click “continue”
Step 2: Write Down Credentials and Server Information
- If you followed step 1 you now see a “Configure” section telling you your credentials and server information
- Write those information down, we will need it later
- Username
- Password
- SMTP Server
In this guide I will use the following credentials/server:
Username: | 12121212121212121212121212121212 |
Password: | 34343434343434343434343434343434 |
SMTP Server: | in-v3.mailjet.com |
These credentials of course are only placeholders. If referenced somewhere in this guide please replace those with your own!
Step 3: Start Validating your Domain
To be able to send emails for your domain you have to validate it first. Mailjet will not accept emails for domains that you don’t own or haven’t validated. For this you need access to the DNS records of your domain.
- In your Mailjet account visit your “Account settings”
- In the “Senders & Domains” section click “Add a Sender Domain or Address”
- In the “Domains” section click “+ Add domain”
- Under “any@” enter your domain (I will use yourdomain.xyz in this guide)
- Under “Label” enter a label for your domain (maybe your domain name, too)
- Click “Continue”
Step 4: Add DNS Record for Domain Validation
Note: How to add or change DNS records in your case is beyond the scope of this guide. It completely depends on your registrar or maybe it es even a separate service, e.g. if you are using Cloudflare or something.
But it should be very easy to figure out how to add a TXT record to your domain.
- Completing step 3 you now should see two options for validations
- We pay our attention to “Option 2: Create a DNS record”
- Use the information to add a DNS record to your domain. Screenshot on the right shows how it looks in the GUI of my DNS service provider. It will look different in yours.
- After adding the TXT record click “Check now” in the Mailjet GUI
- It may take some time for your DNS service provider to propagate your changes. So you may have to wait and check again multiple times
- If adding the TXT record was successful you receive a success message
Step 5: Start Authentication Setup
Setting up authentication is essential. Many email servers either reject emails or classify them as spam if we don’t set SPF/DKIM up properly. Good news: it is very simple to do with Mailjet. They tell you exactly what to do and even sign (DKIM) emails that you relay through their servers. Here are the simple steps you have to do:
- Completing step 4 you are now on the screen that gave you the success message
- Click “Authenticate this domain (SPF/DKIM)”
- Wait a few seconds and “Click here to refresh” in the “Set up SPF” section
Step 6: Add/change DNS records for Domain Authentication (SPF/DKIM)
- Completing step 5 you should now see information about two TXT entries you have to add/change in your DNS records
- Do those changes with your DNS service provider (I made a screenshot to show how it looks in my case, please use you own data that Mailjet shows to you)
- If you made your changes click “FORCE REFRESH”
- If everything went well you get two success messages like “Your SPF record looks good!” and “Your DomainKey record looks good!”
Not getting success messages from Mailjet?
If you don’t get the mentioned success messages from Mailjet it means that your DNS records aren’t set up correctly yet. Here are a few tips that might help.
- Your DNS service provider might need a few minutes to process data. You may just have to wait and click “FORCE REFRESH” again.
- Be sure to edit your SPF record to the value Mailjet suggests if you already have an SPF record
- Be sure to add an SPF record to the value Mailjet suggests if you don’t have an SPF record yet
- Unfortunately there are no copy-to-clipboard buttons in this dialog. So be sure you copied the whole content of the TXT boxes (especially the value of the DKIM TXT record is very long)
Step 7: Repeat Steps 3 – 6 for each domain
- If your mailserver is responsible for multiple domains you have to repeat steps 3 – 6 for each of them
Step 8: Configure Postfix to Relay through Mailjet Servers
Login to Your Postfix server
- First of all log in to your Postfix server (maybe via SSH)
Backup Files
- Always make a backup of the config files you are going to change:
sudo cp /etc/postfix/main.cf /etc/postfix/main.cf_bkp_$(date +%y%m%d_%H%M%S)
- If you already have a
/etc/postfix/sasl/sasl_passwd
file, make a backup of it, too:
sudo cp /etc/postfix/sasl/sasl_passwd /etc/postfix/sasl/sasl_passwd_bkp_$(date +%y%m%d_%H%M%S)
- If you already have a
/etc/postfix/transport
file, make a backup of it, too:
sudo cp /etc/postfix/transport /etc/postfix/transport_bkp_$(date +%y%m%d_%H%M%S)
Edit Files
Some tips/warnings for editing Postfix config files
Every Postfix installation is different. So be very careful when editing your config files.
- Be prepared for problems occuring during reload or restart of the Postfix service
- Always have a backup of your last working configuration and know how to restore it
- We will change some parameters in the config files. Always check if those parameters are already set and if these can be replaced or changed without harming your current setup! E.g. in my configuration I already had a line like this in my
main.cf
:
transport_maps = hash:/var/lib/mailman/data/transport-mailman, proxy:mysql:/etc/postfix/mysql-virtual_transports.cf
- Just replacing it with the line described later in this guide would break the current transport logic. So I added the new transport table:
transport_maps = hash:/etc/postfix/transport, hash:/var/lib/mailman/data/transport-mailman, proxy:mysql:/etc/postfix/mysql-virtual_transports.cf
- It may also be a good idea to write a comment and keep the old line for reference. This way you can always reproduce what you did and revert easily. In my config it looks like this now:
# 2020-07-10: Nerd Admin, Added new transport file # transport_maps = hash:/var/lib/mailman/data/transport-mailman, proxy:mysql:/etc/postfix/mysql-virtual_transports.cf transport_maps = hash:/etc/postfix/transport, hash:/var/lib/mailman/data/transport-mailman, proxy:mysql:/etc/postfix/mysql-virtual_transports.cf
- Always do some testing after making changes, e.g. sending and receiving emails to/from different mailboxes
What do the changes and parameters mean?
- The line in
/etc/postfix/sasl/sasl_passwd
tells Postfix which credentials to use when having to authenticate to server in-v3.mailjet.com - The
/etc/postfix/transport
file contains rules which tell Postfix to use Mailjet SMTP servers as next transport hop when emails target specific recipient domains smtp_sasl_auth_enable
parameter tells Postfix to use authentication when acting as a SMTP clientsmtp_sasl_password_maps
parameter tells Postfix where to find the credentials needed to authenticate when acting as a SMTP clienttransport_maps
parameter tells Postfix where to find transport/routing rules
- Edit
/etc/postfix/sasl/sasl_passwd
:
sudo nano /etc/postfix/sasl/sasl_passwd
- Add credentials of your Mailjet account (please don’t copy and paste from here but use your credentials that you wrote down in step 2):
in-v3.mailjet.com 12121212121212121212121212121212:34343434343434343434343434343434
- Save file using
CTRL + X
,Y
,Enter
- Generate map for Postfix:
sudo postmap /etc/postfix/sasl/sasl_passwd
- Edit
/etc/postfix/transport
:
sudo nano /etc/postfix/transport
- Add a line for every domain that doesn’t accept your emails (I just added all Microsoft hosted email domains I am aware of):
hotmail.com smtp:[in-v3.mailjet.com] hotmail.co.uk smtp:[in-v3.mailjet.com] hotmail.fr smtp:[in-v3.mailjet.com] hotmail.de smtp:[in-v3.mailjet.com] outlook.com smtp:[in-v3.mailjet.com] outlook.de smtp:[in-v3.mailjet.com] outlook.fr smtp:[in-v3.mailjet.com] outlook.be smtp:[in-v3.mailjet.com] outlook.in smtp:[in-v3.mailjet.com] live.com smtp:[in-v3.mailjet.com]
- Save file using
CTRL + X
,Y
,Enter
- Generate map for Postfix:
sudo postmap /etc/postfix/transport
- Edit your Postfix
main.cf
:
sudo nano /etc/postfix/main.cf
- Make sure to have the following lines (be careful when changing config files, see tips on top of this section):
[...] smtp_sasl_auth_enable = yes smtp_sasl_password_maps = hash:/etc/postfix/sasl/sasl_passwd transport_maps = hash:/etc/postfix/transport [...]
- Save file using
CTRL + X
,Y
,Enter
- Reload Postfix config:
sudo /etc/init.d/postfix reload
- If Postfix reloads without giving an error message you should be done
Congratulations! Your emails to Microsoft hosted email domains should now be relayed through Mailjet’s servers and reach the inboxes of Microsoft users.
Step 9: Test if Emails are Really Relayed, Delivered and DKIM-Signed
- Send an email from your formerly blocked domain to a Microsoft hosted email address (e.g. something@hotmail.com) that you have access to
- Enjoy the moment not getting an “Undelivered Mail Returned to Sender” error
- Open the email in the recipient’s account
- View the source code of the email (including headers, etc.)
- If your email was relayed by Mailjet you will see lines like this:
Received: from VI1EUR05HT219.eop-eur05.prod.protection.outlook.com (2603:10a6:6:2a::28) by DB8PR03MB5628.eurprd03.prod.outlook.com with HTTPS via DB6PR07CA0066.EURPRD07.PROD.OUTLOOK.COM; Fri, 10 Jul 2020 11:31:12 +0000 Received: from VI1EUR05FT064.eop-eur05.prod.protection.outlook.com (2a01:111:e400:fc12::52) by VI1EUR05HT219.eop-eur05.prod.protection.outlook.com (2a01:111:e400:fc12::232) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3174.21; Fri, 10 Jul 2020 11:31:12 +0000 Authentication-Results: spf=pass (sender IP is 87.253.233.164) smtp.mailfrom=bnc3.mailjet.com; hotmail.com; dkim=pass (signature was verified) header.d=yourdomain.xyz;hotmail.com; dmarc=bestguesspass action=none header.from=yourdomain.xyz;compauth=pass reason=109 Received-SPF: Pass (protection.outlook.com: domain of bnc3.mailjet.com designates 87.253.233.164 as permitted sender) receiver=protection.outlook.com; client-ip=87.253.233.164; helo=o164.p8.mailjet.com; Received: from o164.p8.mailjet.com (87.253.233.164) by VI1EUR05FT064.mail.protection.outlook.com (10.233.243.113) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3174.21 via Frontend Transport; Fri, 10 Jul 2020 11:31:12 +0000 X-IncomingTopHeaderMarker: OriginalChecksum:FF778753C27D023279D1606619ACA388E98A135233AC294969D01225BF889963;UpperCasedChecksum:58E61C48B36FD75CFCC9F24BDE894691AB04158A36A474E480B28D3F0CB42A3B;SizeAsReceived:4968;Count:20 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; q=dns/txt; d=yourdomain.xyz; i=somebody@yourdomain.xyz; s=mailjet; h=message-id:mime-version:from:to:subject:date:list-unsubscribe-post:list-unsubscribe: autocrypt:feedback-id:x-csa-complaints:x-mj-mid:x-mj-smtpguid:x-report-abuse-to: x-virus-scanned:content-type:content-transfer-encoding:content-language; bh=ejeblXPV3PshDlU281l3Kl7YA1WqBqgPvBVNFbTdi08=; b=b/ePYe+dtzB97i/yrc7yehKHmVLXR8bhwvQqOjXEWWwxkBsiDmYtae9oN gkK/dC6Yu2vAJLYomKzlVVIdOGSdGQeevYSmLpfMaaz6RHXEZSb1l+hHrF09 mHnN7au5CO+YbyijGCbWs37r+ctey2BuKrU2FP+Se9o+rh2Hy6X3Uk= Return-Path: 535df2d3.AVYAAAY107kAAAAAAAAAALSfEeUAAYCsO8oAAAAAABRvmgBfCFF_@bnc3.mailjet.com Message-Id: <535df2d3.AVYAAAY107kAAAAAAAAAALSfEeUAAYCsO8oAAAAAABRvmgBfCFF_@mailjet.com>
- Note the references to mailjet servers as well as the added DKIM signature that is automatically added by Mailjet and helps you to pass Microsoft’s sensitive filters
Final Remarks
As email delivery to Microsoft hosted email domains is working now you can lean back and sort those problems out without any pressure. This workaround would even work forever.
Deactivate Relaying
If your delivery problems are solved you may want to deactivate relaying by Mailjet. But as this kind of problem may reappear at any time it may be wise to keep anything in place and just deactivate the transport rules. This way you can easily turn relaying on and off whenever needed.
To turn relay off, just comment out the transport rules:
sudo nano /etc/postfix/transport
#hotmail.com smtp:[in-v3.mailjet.com] #hotmail.co.uk smtp:[in-v3.mailjet.com] #hotmail.fr smtp:[in-v3.mailjet.com] #hotmail.de smtp:[in-v3.mailjet.com] #outlook.com smtp:[in-v3.mailjet.com] #outlook.de smtp:[in-v3.mailjet.com] #outlook.fr smtp:[in-v3.mailjet.com] #outlook.be smtp:[in-v3.mailjet.com] #outlook.in smtp:[in-v3.mailjet.com] #live.com smtp:[in-v3.mailjet.com]
sudo postmap /etc/postfix/transport
sudo /etc/init.d/postfix reload
Done. Relaying for those domains is turned off now.
If you want to reactivate: just do the same but remove the comment symbol #
.
Limitations of Mailjet Free Plan
The free plan of Mailjet allows to send 6,000 mails per month and is limited to 200 mails per day. It is absolutely perfect for smaller servers with low email traffic. And as we only relay to Microsoft hosted email addresses the free plan may be more than enough for you.
But if you are unsure if this plan is big enough for you or you are in doubt and don’t want to risk email service interruption, just upgrade to one of their bigger plans which are very fair priced (e.g. 30,000 mails/month for $9.95). There is no minimum contract duration and you can leave at any time. So it comes without any risk.
Microsoft’s Email Policies
Of course all email service providers have spam fighting policies in place. We know that. And it is very important. We got it.
But: Microsoft is the only email service provider that causes such massive delivery problems for me that I spent a lot of time to find a solution to work around it. If they start to reject or even silently drop your emails for whatever secret reason you are in serious trouble. Because you know: it will take a long time to get this solved.
Why do I know? I operate multiple servers and each of those ran into the same Microsoft related problems some day. It comes without warning. My last affected email server ran for years without having any problems. But all of a sudden I was placed on one of Microsoft’s block lists for unknown reasons.
I wouldn’t complain if there would be really something wrong with my server setup. I wouldn’t complain if they would protect themselves from real spam sent by my server. I wouldn’t complain if they would tell me what the exact problem is. I wouldn’t complain if other email service providers would also block my emails because there are serious issues. But all this is not the case. The by far biggest part of my email delivery problems was Microsoft related.
Googeling the problems shows me that I am not alone and many operators of (especially smaller) email servers feel the same pain.
This finally brought me to write this article and I really hope it helps some of my fellow sufferers.
Although the problem was always solved by Microsoft and my servers finally were taken from their block lists,
- they never told me what the real problem was, so I will never be able to avoid whatever problems next time
- it always took a long time to solve (days/weeks, not hours)
- they in many cases told me there wasn’t any problem when I opened the first ticket. I had to be persistant and write to them again and again, begging them to throw me a bone
Fortunately I’m now prepared for the next email delivery incident!
Was this article helpful?
Consider buying me a coffee to keep my brain fueled 🙂
Even though this solution works like a charm, I really wonder why Microsoft trusts Mailgun, but doesn’t trust our DIY-Postfix servers, even if we do comply with their policies and all Authentication mechanisms. I would really appreciate to know what they do different so that we can improve too. It’s either the fact that Mailgun has many IPs so they can afford to loose one in a while, or there must be some kind of certification or other contract of trust between these kinds of companies.
For now, I refuse to relay my mails over a third party provider and will try to properly resolve the issue. Nonetheless, the idea of only relaying M$FT domains over Mailgun is quite a smart solution! 🙂
Thank you very much for you feedback. You are absolutely right. It would be awesome if Microsoft would give specific informations about what exactly they expect from us small server owners. Unfortunately they don’t do so and don’t care much about the emails of small service providers. I think Mailjet or similar companies are just too big to block. If M$FT would stop delivering their emails they may get in serious trouble soon because many of their customers would complain and bug their support heavily. They may also face legal problems because not delivering emails may cause severe financial damages. But my email traffic is <0,00000000000001% for them. No need to care for it. Almost no risk for them in blocking my servers. It's that simple. They have no reason to care.
Basically I am fully on your side. It is always best to resolve things properly. Unfortunately it seems just impossible. I tried it many times and spent >100 hours for this. Problem was solved many times but then just reappears. And solving it takes a lot of time – anytime it happens. Under https://www.nerd-quickies.net/2020/10/20/microsoft-silently-dropping-emails-a-sad-but-true-story/ I documented the “normal” sequence it needs to get it resolved properly. You may not have the time or nerve to go through it over and over again. To me it just happens again 2 weeks ago. Out of nowhere they just blocked 2 of my servers again. So I reactivated the solution I described in this article again.
And know what? I think, this time I am giving up. It started as a workaround, but now is my final solution to deliver emails to inboxes of Microsoft customers. This a fight I can’t win and my time is just to valuable to waste it for a M$FT blackbox. Fortunately they are still the only email provider I have problems with. Now they get the exact same emails from me like they did before but relayed by a “better” server. If this is what they want and makes them happy, they have achieved their goal…
Ran in the exact same problem, I figured out that although I have a business subscription from my ISP and I have a static IP and I’m allowed to run an email service. Ran mail server for years. I do have dkim, dmarc and SPF. They still feel like blocking me. I think this is because they think my IP range is a dynamic range. Although if you resolve the IP it clearly says xxxxx.static.someisp.tld.
However if relay my e-mail through my secondary mailservers which are in dedicated server range. The same settings, as the main mailserver and yes you’ve guessed it they work.
To me it seems that it’s to much effort to validate all the small fish rather then verifying, they’ve just block them.
It’s a win win for Microsoft. Chances are small businesses will get mailservice hosting from Microsoft just to not run into these types of problems ever again and lets face it a lot of spam comes from those types of ranges. So rather then figuring out who is a valid mailservice they just block everyone.
hello! Just to notify you that I’ve been through almost exactly the same frustration with emails not delivered and I recently applied your medicine (although I had some issues with updating my SPF and DKIM DNS TXT records for mailjet with bind9 – mostly because I had to wait for propagation I guess – but that ended up alright).
And that did the trick! I have even written in french a very similar article to yours, although with some other details (pain of running one’s own mail server in 2021: greylisting, filtering, etc.) – here it is: https://blog.dahanne.net/2021/10/01/auto-hebergement-des-courriels-en-2021-encore-possible-mais/
Thanks a lot, great article – that was the final push I needed before I addressed this issue at last!