Testing secure email transmitting locally
I inherited some legacy Python code that sends emails the following way (code simplified):
message = MIMEMultipart()
message["From"] = email_from
message["To"] = email_to
message["Subject"] = email_subject
message.attach(MIMEText(email_msg, "plain"))
context = ssl.create_default_context()
with smtplib.SMTP(os.getenv("SMTP_HOST"), os.getenv("SMTP_PORT")) as server:
server.ehlo()
server.starttls(context=context)
server.ehlo()
server.login(os.getenv("SMTP_USER"), os.getenv("SMTP_PASSWORD"))
server.sendmail(email_from, email_to, message.as_string())
server.quit()
I wanted to run and test this code without modifying it at this point and without using the actual mail server. That's why I set off to find some locally runnable email server solutions that fit well within the docker-compose stack I was already using.
The most promising servers I found were the following:
- maildev/maildev (previously known as djfarrelly/maildev) and
- mailhog/MailHog
Both come with convenient Docker images that run out of the box and have documentation for the most important configurations. They also feature nice web front ends to view the received emails that don't differ that much on first sight and the REST-API to request this data has all the needed information.
In the end I settled on MailHog as it had the most recent stable version (summer 2020 vs. summer 2019).
Unfortunately what both servers also have in common is that they don't support SSH and STARTTLS that I need to use as you can see from the code snippet above. Both have feature requests in their repositories about this functionality but there seems to be no activity in this regard. Browsing through those tickets I stumbled about recommendations to use a reverse proxy to work around this problem and there was one specific mention of Simple Mail Forwarder and I decided to give it a try.
Though I take security very seriously it is not my strongest subject and I'm always happy if solutions exist that set things up securely by default and provide the needed documentation to fill the gaps as needed.
In this case I had to bite my way through it and probably spent way too much time on it. In the end I got it working and here I want to protocol how I got there.
Getting the MailHog container running
This step is fairly easy: Add this to your services inside docker-compose.yml and define the environment variables in your .env file:
email_test_server:
container_name: ${COMPOSE_PROJECT_NAME}_email_test_server
image: mailhog/mailhog:v1.0.1
ports:
- ${PUBLISH_PORT_EMAIL_TEST_SERVER_FRONTEND}:8025
networks:
- default
It actually exposes the ports 8025 (web interface over HTTP) and 1025 (SMTP server). As I only need to access the later from within the container network I didn't forward it in the ports section. Otherwise I didn't need to change much and am immediately able to access the front end and the API that are both unsecured.
Creating the glue
The next step is to get the Simple Mail Forwarder (SMF) to work and now it's getting interesting. I made this iteratively as I was unsure how the certificate generation would work. So I started it up with the following setup:
email_test_forwarder:
container_name: ${COMPOSE_PROJECT_NAME}_email_test_forwarder
image: zixia/simple-mail-forwarder:1.4
depends_on:
- email_test_server
environment:
SMF_DOMAIN: email_test_forwarder
SMF_CONFIG: "@email_test_forwarder:to@email_test_server:testPw"
SMF_RELAYHOST: email_test_server:1025
TZ: Europe/Berlin
networks:
- default
This creates the SMTP user @email_test_forwarder with password testPw and forwards every message received that is directed to the @email_test_forwarder domain to to@email_test_server. During the setup process SMF complains about the domain names without a top level domain. This might be solved by using something like @email_test_forwarder.local but I didn't try it out as it worked anyway.
SFM takes care of generating self-signed certificates for you if don't provide it some yourself. The important thing here is setting SMF_DOMAIN correctly.
When you run this image in your container it works well. But when I ran the Python code to connect to the server I got the following error message:
[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1091)
That's reasonable: By default Python doesn't want to connect to a server using a self-signed certificate. So I need to install it in the Python container.
For figuring out what to do next executing the following command from within the Python container was helpful:
openssl s_client -starttls smtp -connect HOST_EMAIL:SECURE_PORT
or in my case:
openssl s_client -starttls smtp -connect email_test_forwarder:25
It returns among other things the server certificate. I identified this certificate as the one generated as /etc/postfix/cert/smtp.ec.cert inside the email_test_forwarder container.
To share this file easily I pulled it out from the forwarder container and saved it (and smtp.cert) in a new data/smtp_test_cert/ directory.
Now to reuse those saved certificates I based a simple Dockerfile on the original one and copied those files over:
FROM zixia/simple-mail-forwarder:1.4
COPY data/smtp_test_cert/* /etc/postfix/cert/
Additionally I changed the section in my docker-compose.yml:
email_test_forwarder:
container_name: ${COMPOSE_PROJECT_NAME}_email_test_forwarder
build:
dockerfile: email_test_forwarder/Dockerfile
depends_on:
- email_test_server
environment:
SMF_DOMAIN: email_test_forwarder
SMF_CONFIG: "@email_test_forwarder:to@email_test_server:testPw"
SMF_RELAYHOST: email_test_server:1025
TZ: Europe/Berlin
networks:
- default
Adjusting the Python container
At last I have to install those certificates in my Python docker container. This can be achieved by a small addition to the respective Dockerfile:
FROM python:3.7
[...]
COPY data/smtp_test_cert/*.cert /etc/ssl/certs/
RUN ln -s /etc/ssl/certs/smtp.ec.cert /etc/ssl/certs/36cc48aa.0
[...]
The hash that needs to be used for the link (in this case 36cc48aa.0) can be found by executing the following command and appending ".0" to it.
openssl x509 -noout -hash -in /etc/ssl/certs/smtp.ec.cert
The only thing that is left to do is configuring the SMTP connection with the following values:
SMTP_HOST=email_test_forwarder
SMTP_PORT=25
# for user info see SMF_CONFIG in docker-compose.yml
SMTP_USER=@email_test_forwarder
SMTP_PASSWORD=testPw
SMTP_FROM=zzz@email_test_forwarder
Now I can test the application without having to worry about external mail servers.
