How we scale our servers

Using Mailcoach, you can send email campaigns of all sizes. Some of our users have large subscriber lists and send hundreds of thousands of emails in a short time. We also see that multiple users send such large campaigns simultaneously.

In this blog post, we'd like to share how we handle the server load that goes with sending a lot of emails.

Hosting Mailcoach

If you need automatic scaling, you might use a serverless solution like AWS Lambda (optionally in combination with Laravel Vapor). For most use cases, they can handle immense workloads.

However, for Mailcoach, using AWS Lambda was not an option. We focus on privacy: all tracking options are optional and turned off by default. Another aspect of the focus on privacy is our desire to adhere to GDPR rules as much as possible.

Some of the GDPR rules are open for interpretation. Ask a bunch of legal experts on how things should be implemented, and you'll get a bunch of different opinions. We wanted to be on the safe side, and chose a very strict interpretation. Our application only uses servers that are located in the EU and that are owned by EU companies. That's why using AWS is not an option for us.

Mailcoach is hosted on UpCloud servers. A server hosts the main application, and servers handling all work in the background using queued jobs. The queue itself is handled via Laravel Horizon. Preparing an email and sending it is also a queued job. So, when we have a large volume of mail that needs to be sent out, we need extra servers to handle queued jobs.

Automatically create and destroy servers

Using UpCloud's API we can create and destroy servers programmatically. All queued jobs are stored in Redis running on a central server. We have created a server image on UpCloud that contains the Mailcoach application and will connect to that Redis server to pick up jobs.

In an early version of Mailcoach, all code to start and stop servers was tailor-made for Mailcoach and UpCloud. Because we think others might also like our approach, we extracted our code to a package called spatie/laravel-dynamic-servers. In Mailcoach, we now use this package.

You can think of laravel-dynamic-servers as a sort of PHP-based version of Kubernetes that has 5% of its features but covers that 80% use case. For most PHP and Laravel developers, this package will also be easier to learn and use.

The package is driver based. It ships with support for UpCloud (because we needed that ourselves), but the community already created a DigitalOcean driver, and creating a driver of your own is easy.

Typically, on your hosting provider, you would prepare a server snapshot that will be used as a template when starting new servers.

After the package is installed and configured, you can use PHP to start and stop servers.

Here's the most straightforward way to start a server via PHP code:

use Spatie\DynamicServers\Facades\DynamicServers;

DynamicServers::increase();

To start a server, the package will start a queued job that makes an API call to your server provider to spin up a server. It will also dispatch subsequent jobs to monitor the entire starting process of a server.

Stopping one server is equally simple:

DynamicServers::decrease();

In most cases, you would use these methods directly, though. The package also offers a method called ensure. You can pass it the number of servers you want to have available in total.

DynamicServers::ensure(5);

If fewer servers are currently active than the number given, more servers will spin up. The package will destroy a few if there are more servers than that number.

Usually, you would have code that calculates the number of servers you need and pass that number to ensure.

Here's a simplified version of the calculation logic we used at Mailcoach. We use Horizon's WaitTimeCalulator class to get the waiting time of the queue.

use Laravel\Horizon\WaitTimeCalculator;
use Spatie\DynamicServers\Facades\DynamicServers;

$waitTimesOfAllQueues = (WaitTimeCalculator::class)->calculate();

$maxWaitTime = max($waitTimesOfAllQueues);

// 1 server for every 5 minutes of wait time
$amountOfServersNeeded = floor($waitTimesOfAllQueues / 60 / 5); 

DynamicServers::ensure($amountOfServersNeeded);

So, when the wait times are long, the number of needed servers will be passed to ensure. When the queues are empty, $amountOfServersNeeded will be zero. When zero is passed to ensure, all dynamic servers will be destroyed.

Of course, this logic should be executed frequently. The package has a method determineServerCount which will be executed every minute through a scheduled command. You would typically use it in a service provider:

use Laravel\Horizon\WaitTimeCalculator;
use Spatie\DynamicServers\Facades\DynamicServers;
use Spatie\DynamicServers\Support\DynamicServersManager;

// in some service provider

DynamicServers::determineServerCount(function (DynamicServersManager $servers) {
	$waitTimesOfAllQueues = (WaitTimeCalculator::class)->calculate();
	
	$maxWaitTime = max($waitTimesOfAllQueues);
	
	// 1 server for every 5 minutes of wait time
	$amountOfServersNeeded = floor($waitTimesOfAllQueues / 60 / 5); 
	
	DynamicServers::ensure($amountOfServersNeeded);
});

In closing

Using the spatie/laravel-dynamic-servers package, we can automatically scale the capacity for sending emails. Whenever large campaigns are sent out, we see in our Slack channel that servers are being created and deleted.

Mailcoach is the best service for sending out email campaigns. We also offer email automation that allows you to easily build a drip campaign. You can manage your content and templates via powerful HTML and Markdown editors. Start your free trial now.