International Characters in E-mail Addresses

2020-06-18

Um, What?

The company I work for has several websites that send out e-mails for a number of reasons, and last week we noticed some Norwegian e-mails that didn’t send, and .NET only gave the error message: The client or server is only configured for E-mail addresses with ASCII local-parts: redacted@someplace.kommune.no.

The problem turned out that the local-parts, the part of the e-mail address before the @ contained Norwegian characters, æ, ø, and å.

I’ve never come across this problem myself before, mostly because I was under the impression that all of us on this spinning blue dot had the same agreement that all things internet should be English, but no. Our municipalities have apparently opened up the possibility for national characters in e-mail username (local-parts), which kinda makes sense since a lot of our names have these characters.

So what’s the problem?

Well, our e-mail server obviously wasn’t configured to handle this, which should have been an easy fix. We use Postfix, and some quick googling reveals that it’s a simple setting to turn on, smtputf8_enable = yes, but oh-no, of course it wouldn’t be that easy!

We fired up a new instance of Postfix and enabled the UTF8 setting and ran some tests, which all failed. .NET still threw the same Exception as before.. Wtf!

Turns out you have to tell the code as well. Ok then.

Instead of doing just: (simplified)

new SmtpClient().Send(message);

You have to give the client an extra options:

var smtp = new SmtpClient {
  DeliveryFormat = SmtpDeliveryFormat.International
};

smtp.Send(message);

No biggie, problem solved. Nope! The same error was thrown..

Logged into the Postfix server and checked the logs and it says the connection was terminated after the EHLO command. Again, wtf? That doesn’t make a lick of sense. Why would having national characters in the to field make .NET abort the connection before it even tried to submit the e-mail to the SMTP server. If I removed the national chars and resubmitted, it went through fine and turned up in the logs. But with the chars, it seems that .NET aborted itself before even trying.

Ok. Time to go old-school. SSH back into the server and telnet localhost and try to manually write the e-mail for queuing. That worked. The e-mail was accepted and added to queue. Yay! Success!

The success was of course short-lived. I checked the inbox, no e-mail, then I checked the logs, which stated that Google Apps SMTP bounced the e-mail because of a Syntax Error in the local-parts. FUCK! Btw, if you go the Gmail interface and try to send an e-mail with national chars in it, it works fine, as long as you’re sending to another Google account.

Mandrill to the Rescue

Or..?

We’re slowly migrating our servers to using Mandrill for sending e-mails because maintaining your own SMTP service is pure and utter madness, but this particular solution wasn’t migrated yet. Easy solution, migrate it!

Added the Mandrill code and changes the queue handler to post it to their API. Ran the code and the mail was successfully posted and all was good in the land. Waited patiently in my glorious inbox for all of 30 seconds before logging into the Mandrill interface in frustration to check where my freaking e-mail was and why it hadn’t turned up in my inbox.

It wasn’t there…

Changed the code to send without national chars, post to API, turns up in Mandrill and my inbox. Re-added the chars, post to API, but it doesn’t turn up in Mandrill or my inbox. It just disappears without so much as a goodbye party.

Switched the code over to using Mandrills SMTP server instead of their API, and lo and behold, get the same fucking annoying error: The client or server is only configured for E-mail addresses with ASCII local-parts.

#rage

Why oh why doesn’t the Mandrills API give any error back about this. I guess we’ll never know.

SendGrid

Another big player in the “we send e-mails for you” landscape is SendGrid. Some easy google-fu reveals they’ve had the same issue for years and refuse to give a proper answer or timetable for a fix. Okey…

AWS SES

Amazon Web Services has their own e-mail sending platform, aptly called Simple Email Service, and a few articles damning Mandrill and SendGrid praise AWS for having the all-mighty solution to all of the worlds problems.

Fire up an account, validate an e-mail, and run a testcode without national chars, which of course works. Ok so far.

Add the national chars, and… guess what. Same exception!

OMFG! Rage intensifies!

Conclusion

So, none of the big guys in the SMTP business supports international characters. What’s a dev to do?

We took the easy way out. Slapped a sticker on the registration form: ASCII or GTFO

PS

We had to contact the few people who had managed to signup with their weird e-mails to get them to send us alternative e-mail addresses, but of course our Google Apps account can’t send to those e-mails, because they’re not Google Apps.

#facepalm