Google, your IPv6-related email restrictions suck

17 09 | 2013

Google, your IPv6-related email restrictions suck

Written by Tanguy

Classified in : Homepage, Debian, Grumble

After years of waiting, Google has finally enabled IPv6 for their email service Gmail. And a few weeks ago, they updated their policy, adding one specific rule: reject email from IP addresses with no reverse name:

% nc -Cv gmail-smtp-in.l.google.com. smtp
Connection to gmail-smtp-in.l.google.com. 25 port [tcp/smtp] succeeded!
220 mx.google.com ESMTP bz2si13656083wjc.108 - gsmtp
HELO boo.example.com
250 mx.google.com at your service
MAIL FROM: <me@example.com>
250 2.1.0 OK bz2si13656083wjc.108 - gsmtp
RCPT TO: <you@gmail.com>
250 2.1.5 OK bz2si13656083wjc.108 - gsmtp
DATA
354  Go ahead bz2si13656083wjc.108 - gsmtp
Subject: Test
From: Me <me@example.com>
To: You <you@gmail.com>

Test.
.
550-5.7.1 [2001:db8:8e3f:43c7::12      16] Our system has detected that this
550-5.7.1 message does not meet IPv6 sending guidelines regarding PTR records
550-5.7.1 and authentication. Please review
550-5.7.1 https://support.google.com/mail/?p=ipv6_authentication_error for more
550 5.7.1 information. bz2si13656083wjc.108 - gsmtp

That sucks

It is not a sane anti-spam measure, because not having a reverse name tells absolutely nothing about the email emitter or the message itself: it only indicates that the email emitter has a lame access provider. Specifically:
- spammers can have perfect reverse names;
- regular email emitter can have no reverse name because their access provider suck, and have no way of fixing it because all access providers suck the same way.
While IPv6 is the future of the Internet, this measure discourages its use. Yes, providing no reverse names suck, but it is by no mean the user's fault, and since IPv6 is more complicated to implement than IPv4, Internet actors must tolerate youth errors or people will just keep what works, i.e. IPv4.

By the way, I would love to have a perfect Internet access provider, but that does not exist where I live, and I do not think it exists at all anywhere. Google, instead of complaining about lame access providers, you are welcome to offer me a fibre service with decent upload rate, static IPv4, static IPv6 /64, customizable reverse DNS for IPv4 and IPv6, and full respect of the network neutrality. Until then, I am doing the best I can with the least bad provider available.

Workaround

As I said, their is an easy workaround. Google does not accept my mail on IPv6? Fine, I will keep using IPv4. Only for Google of course, no need to punish the whole Internet for Google's damn restrictions. With Postfix, you can do that this way (thanks to Christian Skala for his blog post about this problem). In /etc/postfix/master.fr, enable an IPv4-only SMTP client service:

smtp4     unix  -       -       -       -       -       smtp -o inet_protocols=ipv4

Then, in /etc/postfix/main.cf, define a transport map if you do not already use one:

transport_maps = hash:/etc/postfix/transport

Create that transport map /etc/postfix/transport or complete it:

gmail.com smtp4:

Finally, hash that transport map and have Postfix reload its configuration:

# postmap /etc/postfix/transport
# service postfix reload

Tags : Google, IPv6, reverse, DNS, Internet, access, provider, sucks, restriction, email, SMTP, reject, IPv4, Postfix, workaround

55 comments

tuesday 17 september 2013 à 15:05 Philippe Marzouk said : #1

Nerim has customizable reverse mapping.

I was bitten by this also as my reverse was badly configured initially. I could correct it.

tuesday 17 september 2013 à 15:19 Samaël said : #2

Thanks a lot \o/

tuesday 17 september 2013 à 15:21 Tanguy said : #3

@Philippe Marzouk : You are lucky to be eligible for a Nerim connection. I am not, probably because I only live in a small city called Paris, XIVe. That is, I am eligible to super-expensive, business-class SDSL, but not to simple, affordable offers.

Funnily enough, I tested my parents and grandparent eligibility, in several other cities: same result. In fact I was never able to find anyone eligible to affordable offers with Nerim.

tuesday 17 september 2013 à 15:25 gregoa said : #4

What I've also seen is this complaint about a missing rDNS entry for a host with a perfect reverse name ...

tuesday 17 september 2013 à 15:54 David Edmondson said : #5

What about non-gmail.com domains hosted on Google?

tuesday 17 september 2013 à 16:01 Paul said : #6

It isn't just Google, and it's going to become more widespread. As you stated, it's not a panacea, but it is a barrier. Should be noted that an email provider could potentially use this to their advantage if this is more widely adopted by not assigning PTR records to dynamic space. Give a PTR record to sending system, keep the dynamic space from having a PTR record on IPv6 and they are less likely to spam other systems (unless going through a smarthost, etc, but that's a different problem)

tuesday 17 september 2013 à 16:30 Tanguy said : #7

@David Edmondson : That would have to be handled case by case. But people that delegate their email service to Google should simply be aware that they are loosing legitimate mail.

@Paul : Is is not panacea, it is simply crazy. Having or not a PTR record has absolutely no connection to being or not a spammer. Email providers that apply such restrictions are loosing mail, and punishing people for their provider's mistake. That kind of restrictions would be as sensible as blocking Google entirely because there are spammers that use Gmail accounts.

tuesday 17 september 2013 à 16:35 David Edmondson said : #8

"Case by case" was far too much hassle for me - I added "inet_protocols = ipv4" to main.cf.

tuesday 17 september 2013 à 16:47 pywy said : #9

Yep, I observed exactly the same issue, since 16th Aug if my memory is good.
By the way, I am looking for the same workaround for exim4. Since I communicate with a lot of gmail users, I had to set my exim4 config in ipv4 only. This sucks.
I still receive gmail mail in ipv6, would be great if I could send them back in ipv6 too.

tuesday 17 september 2013 à 16:54 Tanguy said : #10

@David Edmondson : Well, in that case I would ask you a favor: every time you are asked whether or not you have a full support for IPv6, could you remember to specify “No, because my ISP does not offer me IPv6 reverse DNS and Google blocks email from my IPv6 address”? When they ask, people have to know that one actor preventing migration to IPv6 is Google.

tuesday 17 september 2013 à 17:21 Philipp Kern said : #11

Google likes data. I'm certain that the reason for this was spam. Like "there's a magnitude more spam from addresses without a PTR". People enforced PTR on v4 for ages. If you are using a business connection as you state, you should be able to request a PTR. On the other hand it's no better in Germany. The provider I'm with doesn't even offer static IPs for business customers, and I didn't have an alternative. But I think most people simply use a smarthost on a rented server somewhere instead than keeping up with more and more restrictions put into place for such address ranges.

(ObDisclaimer: I'm a Google employee but do not have any insights in Google spam filtering and am not speaking for the company.)

tuesday 17 september 2013 à 17:28 Tanguy said : #12

@Philipp Kern : Yes, and much spam is coming from the Internet, but this is no reason to refuse mail from the Internet. But I never stated I was using a business connection, whatever that may be: I am using a simple ADSL connection, waiting for some decent fiber service to come to my door.

Also, using smarthosts is one of the worst thing that happened to email, as concentrating email sending on large servers prevents recipients from doing thing IP-based blocking (smarthosts send many many many messages, and among them many spams, but blocking a smarthost would result in loosing much legitimate mail), and prevents senders to publish strict SPF policies. More generally, it goes against the possibility for anyone to publish his own services on the Internet. For instance, with that kind of restrictions, you employeer Google, would not have been able to start with two guys in a garage.

tuesday 17 september 2013 à 18:25 agi said : #13

They are somewhat paranoid regarding SMTP and IPv6.

I first hit the reverse resolution problem some days ago, which it's something that had to be fixed of course. But after fixing that they now reject mail from my nagios server (~4mails/day), only when delivered via IPv6, not IPv4, with the error:
Our system has detected an unusual rate of unsolicited mail originating from your IP address.
Unusual rate???? 3/4 mails (10 at most) with nagios messages? I don't care much, since those are notifications for a client, and it's me that takes care of them. But I don't like them ruling world's email.

tuesday 17 september 2013 à 20:35 Karellen said : #14

@pywy: How did you make that configuration? Did you use the global "disable_ipv6" option, or the more fine-grained "dns_ipv4_lookup"?

wednesday 18 september 2013 à 08:43 laurentb said : #15

Google isn't the only one, I have been forced to set a reverse DNS much earlier to be able to send to other domains.

wednesday 18 september 2013 à 08:45 PAB said : #16

Thx a lot !

wednesday 18 september 2013 à 14:25 pywy said : #17

@karellen: Yep, had to use global "disable_ipv6" option, but it may not be the best solution...
Let me know if you can disable ivp6 for gmail only !(in exim4 config)
A link to a discuss on gmail forum : https://productforums.google.com/forum/#!msg/gmail/K5klFKDnUAE/Q5JPxd3pa9MJ

wednesday 18 september 2013 à 14:38 Paul said : #18

IPv6 is a "brand new world", and gives Email providers a little room to play and try different criteria for allowing you to deliver mail to their systems. PTR records could be just the beginning, with something like DKIM or SPF being another option. All of these are easy to setup for a spammer, that's quite clear. It's just another speed bump. DMARC is a different sort of option, not something you'd use for a home domain, but something that a large company should look at implementing. I could see wanting a PTR record as part of an attempt at something like domain reputation that they may use to watch for spam attacks or to limit number of messages. Large scale providers have different problems than the smaller guy, they use different solutions.

wednesday 18 september 2013 à 15:20 pywy said : #19

@karellen : I've followed your advice on "dns_ipv4_lookup" setting.
It used to be a boolean, but has become a string in exim.
By adding "dns_ipv4_lookup = gmail-smtp-in.l.google.com" in my exim4 config file, mails are sent to gmail though ipv4 only, while it keeps ipv6 to the rest of the world.
It works!

wednesday 18 september 2013 à 15:29 Tanguy said : #20

@Karellen, @pywy: If you go that way, there is another similar option that is less stable but independant of the MTA: forcing recipient MX IPv4 addresses in /etc/hosts.

wednesday 18 september 2013 à 18:40 Philipp Kern said : #21

SPF was a wrong idea to begin with and still breaks with all forwarders that don't use SRS. So I wouldn't pull that off as an argument.

thursday 19 september 2013 à 14:57 Jean-Marc Liotier said : #22

This reminds me of the bad old times of half-assed antispam rules in the 90's - some SMTP servers used to consider IPv4 without proper reverse spammy.

Get a Hurricane Electric Ipv6 tunnel - they will let you set the reverse pointer of your choice: https://www.linode.com/wiki/index.php/IPv6_Reverse_DNS

wednesday 25 september 2013 à 04:46 Cesar Peralta said : #23

A great post. Thank you

wednesday 16 october 2013 à 22:00 Piotr said : #24

Merci beaucoup Tanguy ...

friday 08 november 2013 à 08:15 Jean-Marc Liotier said : #25

Merci beaucoup !

thursday 14 november 2013 à 00:16 Dmitriy said : #26

Many lots of tanks!

saturday 16 november 2013 à 14:57 Jocelyn Delalande said : #27

The fail in that workarround is with people hosted at google with their own domain...

Any return on that cases ?

saturday 16 november 2013 à 15:03 Jocelyn Delalande said : #28

Sorry to multi-comment but I found that, more hackish but handling cases such as the one I mentioned

: http://blog.hqcodeshop.fi/archives/122-Fixing-Googles-new-IPv6-mail-policy-with-Postfix.html

wednesday 27 november 2013 à 03:24 peterlim said : #29

the workaround using ipv4 as transport is work. PTR record is not a practical solution. it only stick to one domain. if u run public mail service that serve more than 1 domain, the rest of the domain will be rejected by gmail. even your MX record point to the mail server which has correct PTR record. so Google is very poor on IPv6 mail support, because yahoo mail have no problem with it. no choice, use IPv4 with gmail, they can't talk ipv6, thats my conclusion

wednesday 27 november 2013 à 10:00 Tanguy said : #30

@peterlim : They do not refuse messages simply because the reverse DNS does not end with the same domain as the MAIL FROM address, do they? That would be a completely inappropriate rule, that they do not validate themselves of course.

friday 29 november 2013 à 12:32 prietus said : #31

sorry. the reverse resolution restriction already exists in ipv4 mail servers relaying. due the bunch of spammers out there.

friday 29 november 2013 à 13:50 Tanguy said : #32

@prietus : I know, but using reverse DNS to determine whether or not a message is a spam is stupid, as is has absolutely no link to a message's legitimacy. Spammers can have perfect reverse DNS, while many honest sender are unable to get one.

sunday 08 december 2013 à 02:15 Neil said : #33

Thanks for the post, helped me workaround the problem quickly.

tuesday 10 december 2013 à 22:16 Hans said : #34

#29: Your claim is untrue. Google does not reject mail just because the PTR is pointing to a different domain. I operate a (very) small ISP relaying mail for different users and there was never such a problem. Also if you were true, this would be true for IPv4 deliveries as well.

#32: I disagree. A missing PTR record is more often than not a sign of a not properly maintained mail server. If you can't get proper RDNS, you have to switch your ISP. Any competent mail admin knows that this is the way to go because not only Google but many many big size and small size ISPs do not accept mail from hosts without a valid PTR. And this is going on for years already. Yes, dedicated spammers will often have working PTR, but blocking out badly maintained (and thus often abused) mail servers is also helping and widely accepted in the real world.

Face it: Sending mail from a host without proper RDNS (and no, it does not need to match the mail sending domain) is not only best practice but just a requirement these days.

Regarding Google Mail: It is not possible to fully understand (as an outsider) by which criteria Google classifies mail as spam or ham. However it seems that it helps to have matching SPF records for the domains used in the mail (at last the envelope sender).

wednesday 22 january 2014 à 02:40 Emily Smith-Lee said : #35

http://slnlaw.blogspot.com/2014/01/through-google-glass-mxgoogle-error.html

monday 10 february 2014 à 10:49 Michael said : #36

#34: 100% Agree.

tuesday 11 march 2014 à 16:11 Nisse said : #37

@Tanguy: spam-viruses does not have perfect rDNS. so they all get blocked.
All home networks on DHCP get blocked by rDNS.
They are the most likely to be sending spam without knowing it and the least likely to fix the problem.
spam-viruses using a proper smtp host is blocked very fast.

And as good as SPF is as an idea most companies don't have the qualified IT support to handle it.
just look at all the spf records ending with ~all saying "I have listed good servers but you should accept all other emails as-well."
And because it's not widely spread it's the server admins blocking bad SPF mail that get's yelled at.

And lastly when it comes to IPv6 and spam, you need to be as strict as possible, you can't allow a spammer with millions of IPs to spam from all of them without any restrictions.
There isn't a blacklist in the world that can keep a database of all bad ipv6 addresses and be functional

tuesday 11 march 2014 à 21:39 Andi said : #38

It's even worse. If the Google resolvers tempfail on IPv6 PTR they also reject with permanent error code. We have this from time to time with perfectly matching forward/reverse DNS and five minutes later the very same mail passes.

Thanks Google, your are reaching the AOL/Hotmail Bottom

friday 21 march 2014 à 08:14 Lisa Smith said : #39

Simply awesome post. thanks

sunday 08 june 2014 à 15:50 login01 said : #40

Tanguy,

There are only two types of people that do not create a reverse record and DMARC, they are SPAMMERS and people that are new to IPv6 security. In either case, if you want to do business with IPv6 enabled companies, you must follow the rules.

sunday 08 june 2014 à 19:31 Tanguy said : #41

@login01 : Thank you for insulting me. I have a DMARC record, that is easy to do, but I do not have a reverse record for IPv6, I am neither a spammer nor new to IPv6 security. Guess what the reason is: defining a reverse record is made possible by one's ISP, and mine does not offer that possibility. Neither does any ISP available where I live, in fact almost all of them offer no IPv6 access at all, so I am already glad mine does.

monday 04 august 2014 à 12:51 Alex Lennon said : #42

Fantastic. I had been wondering what was going on here. Very irritating as I was suddenly getting emails to all sorts of people bounced.

Works like a charm. Thanks

monday 29 december 2014 à 21:01 Holger said : #43

@Tanguy: If your ISP does not offer you to set a reverse DNS entry for an IPv6 address you simply cannot expect to be able to deliver e-mails reliably via IPv6, so don't do it. Perhaps keep keep asking your ISP regularly to improve their services. - We have been rejecting mail from servers without a reverse DNS entry on IPv4 since 2006 - if there is a temporary DNS error we reply with a 4xx error - if their DNS server clearly says "no PTR" we reject with a permanent 5xx error. Rejecting mail like this does NOT mean the sending server relays spam, it merely means it's managed by a clueless mail systems administrator (who doesn't deserve to be in business). Unfortunately there are plenty of clueless admins, so we maintain a whitelist, with entries maintained on request from our customers - if they are missing important e-mails they'll tell us. - The main reason why there is so much spam these days is because clueless sysadmins are allowed to get away with setting up inferior systems far too often.

wednesday 31 december 2014 à 18:37 Tanguy said : #44

@Holger : Your assumption about servers without a reverse DNS record is insulting to their administrators. To be specific, when you say “it merely means it's managed by a clueless mail systems administrator (who doesn't deserve to be in business)”, this is wrong, as the only thing it means is that the servers is using an Internet connection managed by lame people: that may be the system administrator, but usually it is not, and he has no way to fix that.

thursday 16 april 2015 à 14:15 Maribeth said : #45

Fantastic...simple awsome post you shared.. its great deal. I need to learn more

wednesday 22 april 2015 à 02:26 Jake said : #46

I'm actually very very happy Google requires some intelligence from sender domains, if you don't know what you're doing don't host email! PTR, SPF, and DKIM should all be required! Thanks Google!

wednesday 22 april 2015 à 09:54 Tanguy said : #47

@Jake: As for Holger, please read my article before making insulting assumptions on me. I perfectly know whay I am doing with email. Actually, I know it even better than some big operators that are known not to even have people reading their abuse and postmaster mailboxes for instance. And I also know not having a reverse DNS is abnormal, but the thing is I CANNOT fix that! Only m ISP could, but they would not. And no, changing ISP is not an option, as NO ISP in my living area offers reverse DNS for IPv6.

tuesday 25 august 2015 à 06:05 justin said : #48

I just had to disable IPv6 for a similar but different gmail issue.

Gmail was bouncing mail as spam. with a different error. Now the underlying cause might be the same (no reverse lookup for ipv6 or no SPF record or whatever) however the actual error was NOT that, it was

" Our system has 550-5.7.1
detected that this message is likely unsolicited mail. To reduce the
550-5.7.1 amount of spam sent to Gmail, this message has been blocked.
Please 550-5.7.1 visit 550 5.7.1"

So, what's up with that? I should have got

"Our system has detected that this
550-5.7.1 message does not meet IPv6 sending guidelines regarding PTR records
550-5.7.1 and authentication. Please review"

but did not.

monday 26 october 2015 à 11:51 Huub Knops said : #49

Seems that google has changed its IPv6 PTR policy, right?

Iam noow able to send mail from a server with IPv6 with no propper IPv6 PTR record and I don't get the previous message anymore. Can anyone confirm?

friday 30 october 2015 à 02:45 Waltman said : #50

Thanks for this writeup! For some reason I'd been able to send mail to gmail for 2+ years after you wrote this, but I just hit it for the first time today. Your instructions worked perfectly.

friday 27 november 2015 à 13:31 eric said : #51

There is a slight problem with this rule though, it's when the recipents domain names uses google for their MX, under their own domain name (I stumbled on the case earlier.
so rather than creating a rule for each domain, i went to force all postfix trafic though ipv4 by changing the line :
inet_protocols = all
to
inet_protocols = ipv4
in /etc/postfix/main.cf
(ubuntu 15.04, postfix 2.11.3-1ubuntu2)

friday 18 december 2015 à 20:28 Robb Allan said : #52

The solution works with Mac OS X servers, with one modification: chroot has to be explicitly disabled in the service definition line in master.cf, i.e.,

smtp4 unix - - n - - smtp -o inet_protocols=ipv4

saturday 23 january 2016 à 15:04 Legion said : #53

For me it works perfectly.

thursday 28 april 2016 à 23:12 mario_26 said : #54

Merci pour cette info, toujours au top Tanguy ! ;)

++

wednesday 04 january 2017 à 22:29 greg said : #55

Je viens de rencontrer le problème!
Merci beaucoup du coup de main!

Big Up pour utiliser PluXml ;-)

Rss feed of the article's comments

Tanguy Ortolo