Using jobs instead of commands in the schedule of a Laravel app

In a standard Laravel application, you can schedule Artisan commands in your console kernel. While that works for most projects, we took a different route in Mailcoach. Instead of scheduling commands, we're scheduling jobs.

In this blog post, I'd like to explain why and how we do this.

Using commands in a schedule

Using Laravel's scheduler, you can execute commands at specific internals. My blog freek.dev is powered by a typical Laravel application. Let's take a look at a part of its schedule.

namespace App\Console;

// imports...

class Kernel extends ConsoleKernel
{
    protected function schedule(Schedule $schedule)
    {
        $schedule->command(RunHealthChecksCommand::class)->everyMinute();
        $schedule->command('responsecache:clear')->daily();

        $schedule->command('site-search:crawl')->daily();
        $schedule->command('backup:run')->dailyAt('3:00');
        
        // other commands
    }

    // ...
}

You're probably already very familiar with this. You can use the command method on $schedule to schedule any command you want, and you can chain on an interval. The only thing to note here is that, instead of passing a command signature, you can also pass the fully qualified class name of a Command.

Making commands tenant-aware

In typical applications, most functionality is coded in the app itself. The hosted version of Mailcoach, let's call it Mailcoach Cloud, is a special kind of Laravel application. Most functionalities are provided by the spatie/laravel-mailcoach package, which you can buy if you want to self-host mailcoach.

The spatie/laravel-mailcoach package contains a couple of Artisan commands which should be scheduled as explained in our installation docs.

/*
 * in app/Console/Kernel.php of an app
 * where you use spatie/laravel-mailcoach
 */ 
protected function schedule(Schedule $schedule)
{
    // ...
    $schedule->command('mailcoach:send-automation-mails')->everyMinute()->withoutOverlapping()->runInBackground();
    $schedule->command('mailcoach:send-scheduled-campaigns')->everyMinute()->withoutOverlapping()->runInBackground();

    $schedule->command('mailcoach:run-automation-triggers')->everyMinute()->runInBackground();
    $schedule->command('mailcoach:run-automation-actions')->everyMinute()->runInBackground();

    $schedule->command('mailcoach:calculate-statistics')->everyMinute();
    $schedule->command('mailcoach:calculate-automation-mail-statistics')->everyMinute();
    $schedule->command('mailcoach:rescue-sending-campaigns')->hourly();
    $schedule->command('mailcoach:send-campaign-summary-mail')->hourly();
    $schedule->command('mailcoach:cleanup-processed-feedback')->hourly();
    $schedule->command('mailcoach:send-email-list-summary-mail')->mondays()->at('9:00');
    $schedule->command('mailcoach:delete-old-unconfirmed-subscribers')->daily();
}

The Mailcoach Cloud application is multi-tenant using a single DB. Each team that registers at Mailcoach Cloud is seen as a tenant. Behind the scenes, we use our own spatie/laravel-multitenancy package to make Mailcoach Cloud tenant-aware.

In Mailcoach Cloud, all Artisan commands above should be run for every tenant. We could have gone ahead and created a special version of each command that is tenant aware. But we'd rather not do that and execute the exact same commands as in spatie/laravel-mailcoach.

The solution we used here is to loop over each tenant and wrap each command in a job. Let's take a look at the simplified code:

/*
 * in app/Console/Kernel.php of mailcoach.app
 */ 
protected function schedule(Schedule $schedule)
{
   Team::with(['subscriptions'])
      ->select(['id', 'name', 'trial_ends_at'])
      ->each(function (Team $team) use ($schedule): void {
         $schedule
            ->job(new TeamArtisanJob($team, 'mailcoach:send-automation-mails'))
            ->name("{$team->name} - mailcoach:send-automation-mails")
            ->everyMinute();

         $schedule
            ->job(new TeamArtisanJob($team, 'mailcoach:run-automation-triggers'))
            ->name("{$team->name} - mailcoach:run-automation-triggers")
            ->everyMinute();

         $schedule
            ->job(new TeamArtisanJob($team, 'mailcoach:run-automation-actions'))
            ->name("{$team->name} - mailcoach:run-automation-actions")
            ->everyMinute();
            
         // all other mailcoach commands.
            
    });
}

Each command is wrapped in a TeamArtisanJob We use queued jobs instead of commands because if there are a lot of tenants, executing all commands for all tenants sequentially might take too much time. By using queued jobs, we can perform the necessary concurrently by using multiple queue workers.

Let's look at the simplified code of TeamArtisanJob and discover what happens there. I've left some details out to make a good example.

namespace App\Domain\Team\Jobs;

// imports left out for brevity...

class TeamArtisanJob implements ShouldQueue, ShouldBeUnique
{
    use Dispatchable;
    use InteractsWithQueue;
    use Queueable;
    use SerializesModels;

    public function __construct(
	    private Team $team, 
	    private string $command,
    )
    {
    }

    public function handle()
    {
        $this->team->execute(fn () => Artisan::call($this->command));

        Team::current()?->forget();
    }
    
    public function uniqueId()
    {
        return $this->team->id.$this->command;
    }
  
    public function uniqueFor()
    {
        return 60 * 15; // 15 minutes
    }
}

You can see that the artisan command itself is passed to a closure given to the execute method on $this->team. This method is available because the Team model extends from the Tenant base class provided by spatie/laravel-multitenancy.

The execute method will first perform all tasks to make a tenant the current one. The most important task for this example is setting global scopes. After that task each, query gets an extra where clause that looks like this (pseudo code) where team->id = <team-id-where-execute-is-being called-on>.

After that global scope is set, the closure passed to execute will be executed. In our case, Artisan::call($this->command) will be called (and remember, $this->command contains a command signature from the spatie/laravel-mailcoach package, eg. 'mailcoach:send-automation-mails'. Each query inside the command will be scoped to the right team. After that closure is executed, the global scope will automatically be removed by the spatie/laravel-multitenancy package.

So with this setup, we've made all of the commands of spatie/laravel-mailcoach package tenant-aware without changing the source code of those commands.

In the code of TeamArtisanJob, you also see that we're using the ShouldBeUnique trait. This will ensure that only one job for this command and team will be on the queue. Should our queue be so slow, the TeamArtisanJob is not executed on time, then running the scheduler again won't result in a duplicate job on the queue.

In closing

In most applications, you won't need the strategy as explained above: you could create commands that do the necessary work for all tenants at once (or let those commands dispatch jobs that do all the work).

For Mailcoach Cloud, which uses the self-hosted spatie/laravel-mailcoach package at its core, this is an excellent way of making commands that were not written with multitenancy in mind, multi-tenant aware.

Even if you're not creating a multi-tenant aware app, it might be helpful to execute regular scheduled commands as jobs so that they can be executed concurrently. Laravel makes it easy to schedule queued jobs.

Mailcoach is the best service for sending out email campaigns. We also offer email automation that allows you to quickly build a drip campaign. You can manage your content and templates via powerful HTML and Markdown editors. Start your free trial now.