Projectors in Depth

Up until this point we've covered the technical basics of event sourcing: how to store events and project them. It might leave you wondering if we've actually gained much by using an event-driven approach. On the surface it might seem like we've added several more steps to achieve the same goal.

That perception will change with this chapter: we're going to look at what a simple concept like projectors actually allow us to do.

---

Have you ever had a client ask about a new feature, where you had to tell them it would only work on data that's added after the feature was deployed? An example: imagine your client wanting to do some in-depth analysis on customer behaviour. Maybe they'd like to identify customers that took less than 10 minutes to check out their cart. Now what if we didn't keep track of the checkout date and time yet? You would need to add a new checkout date field on the cart, which would be null for all carts existing before this feature.

With event sourcing however, we've got more flexibility. The fact that "an event happened" is information that's usually missing in a traditional application where you don't keep track of every change. An event sourced system on the other hand keeps track of when events were originally dispatched. This essentially gives us a chronological log of everything that happened in our system.

So without having to change the database schema, we can solve our client's problem. There are in fact two solutions: we could either add a field on the Cart projection, or we could make a dedicated projection for this kind or report. Let's start with the first.

---

Our code is running in production and we need to add a new field to an existing table. We'd start with a migration:

public function up()
{
    Schema::table('carts', function (Blueprint $table) {
        $table->dateTime('checked_out_at')->nullable();
    }
}

Next, we need to make sure this field is projected as well. Let's go to our CartProjector. The onCartCheckedOut method was already implemented:

class CartProjector extends Projector
{
    // …

    public function onCartCheckedOut(CartCheckedOut $event): void
    {
        $cart = Cart::find($event->cartUuid);

        $cart->writeable()->update([
            'state' => CheckedOutCartState::class,
        ]);
    }
}

Now we need to know when the event happened. Our event itself doesn't keep that data (since we didn't add a field for it), but there still is a way to determine its creation date. Whenever an event is serialized to the database, it's wrapped in an EloquentStoredEvent model — a normal Eloquent model. It doesn't only store the serialized event though; it also keeps track of some meta data, including its creation date. If we can retrieve the stored event model for this event, we can access its data. Luckily every event class automatically gets some meta data attached to it, including the ID that's used to store it in the database.

Knowing all of that, retrieving the stored event is nothing more than a normal database query:

$storedEvent = EloquentStoredEvent::find($event->storedEventId());

Bringing it all together, we'd refactor our projector like so:

class CartProjector
{
    // …

    public function onCartCheckedOut(CartCheckedOut $event): void
    {
        $cart = Cart::find($event->cartUuid);

        $storedEvent = EloquentStoredEvent::find($event->storedEventId());

        $cart->writeable()->update([
            'state' => CheckedOutCartState::class,
            'checked_out_at' => $storedEvent->created_at,
        ]);
    }
}

However, we're not done yet! If we'd push these changes to production, they would only affect carts that were checked out after our deploy.

Remember how I explained that event sourcing enables us to rebuild our application state from scratch? We could essentially throw away all of our data, except the events themselves, and rebuild everything again. That's exactly what we're going to do: throwing away all carts, and replaying the events. We could in theory throw away and rebuild everything, but you can imagine how that would take lots of time. So instead of starting from scratch, we're only going to replay the events related to carts.

Before replaying, we'll need a way to reset our carts table. Projectors have a built-in solution for this: the resetState() method:

class CartProjector extends Projector
{
    public function resetState(): void
    {
        Cart::query()->delete();
    }

    // …
}

This method is run whenever we replay the projector. The final thing to do after deploying our changes is to actually replay this projector. This is done with the following artisan command:

php artisan event-sourcing:replay \
  "App\\Cart\\Projectors\\CartProjector"

After this command is finished, the events will have been replayed and the cart projector would have written the checked out date to all carts, including those in the past.

---

Throwing away all carts and rebuilding them can become quite expensive over time. Imagine a production system with years of data; rebuilding all carts could take hours, maybe even days. You wouldn't want your system to be down in "maintenance mode" all that time. We're going to cover deployment strategies for such cases in a later chapter, but it's also worth mentioning another solution.

Instead of changing the cart itself, we could make a new projection, dedicated to the report our client is asking. They want to know what customers checked out their cart in less than 10 minutes, so let's make a dedicated table that keeps track of the amount of time a cart was active. Here's the migration:

Schema::create('cart_durations', function (Blueprint $table) {
    $table->uuid('uuid');
    $table->uuid('cart_uuid');
    $table->dateTime('created_at');
    $table->dateTime('checked_out_at')->nullable();
    $table->integer('duration_in_minutes')->nullable();
});

And here's the projection:

class CartDuration extends Projection
{
    protected $guarded = [];

    public $timestamps = [];

    protected $casts = [
        'created_at' => 'datetime',
        'checked_out_at' => 'datetime',
        'duration_in_minutes' => 'integer',
    ];
}

Finally, we'll have to make a projector. We're going to make a new class, but we'll listen to already existing events. More specifically: the CartInitialized and CartCheckedOut events:

namespace App\Cart\Projectors;

// …

class CartDurationProjector extends Projector
{
    public function onCartInitialized(CartInitialized $event): void
    {
        $createdAt = EloquentStoredEvent::find($event->storedEventId())->created_at;

        (new CartDuration)->writeable()->create([
            'uuid' => $event->cartUuid,
            'cart_uuid' => $event->cartUuid,
            'created_at' => $createdAt,
        ]);
    }

    public function onCartCheckedOut(CartCheckedOut $event): void
    {
        $checkedOutAt = Carbon::make(
            EloquentStoredEvent::find($event->storedEventId())->created_at
        );

        $cartDuration = CartDuration::find($event->cartUuid);

        $durationInMinutes = $checkedOutAt
            ->diffInMinutes($cartDuration->created_at);

        $cartDuration->writeable()->update([
            'checked_out_at' => $checkedOutAt,
            'duration_in_minutes' => $durationInMinutes,
        ]);
    }
}

Whenever a cart is initialized, this projector will create a new CartDuration projection, and store the date the event was triggered. Note that we're repurposing the cart's UUID as the duration's UUID. Since Cart and CartDuration are a one-to-one relation, it's ok to reuse it. Technically you wouldn't even need a dedicated UUID for this projection, but Laravel expects it to be there nevertheless.

Next, when a CartCheckedOut event is triggered, we update the existing CartDuration for that cart, and calculate the duration in minutes. When this projector is run for previously dispatched events, it will create a new CartDuration for all existing carts because of the CartInitialized event, as well calculate the duration when those carts were finally checked out.

The only thing left to do is deploy these changes, and replay the events on our newly created projector:

php artisan event-sourcing:replay \
  "App\\Cart\\Projectors\\CartDurationProjector"

You can see how we've created another interpretation of the same event stream by creating a new projector that listens to the same events. It's once again an illustration of how events are the only source of truth, and projections are only a side effect.

Besides being able to act upon things that happened in the past, there's another benefit we've gained: once our CartDuration projections are generated, it's very easy to read from them. We don't have to do any in-memory calculations or database joins: the data is right there, whenever we need it, in the right format. This also means that you could make dedicated projections for specific pages, even when those projections are based on the same event stream. No need to dynamically build an admin dashboard based on the projected carts, let's just make a dedicated projection for that dashboard.

Finally, there's no more fear of missing out: you can add however many projections you want after your application has gone in production. As long as the events are stored, you're free to replay them as many times and in whatever way you'd like. We'll cover the power of projections even further when we discuss CQRS, a topic for a later chapter.

/** @test */
public function cart_initialized(): void
{
    $projector = new CartDurationProjector();

    $event = CartInitializedFactory::new()->create();

    $projector->onCartInitialized($event);

    $cartDuration = CartDuration::find($event->cartUuid);

    $this->assertNotNull($cartDuration);
}

/** @test */
public function cart_checked_out(): void
{
    $projector = new CartDurationProjector();

    $cartDuration = CartDurationFactory::new()
        ->createdAt(Carbon::make('2021-01-01 10:00'))
        ->create();

    $event = CartCheckedOutFactory::new()
        ->withCart($cartDuration->cart)
        ->createdAt(Carbon::make('2021-01-01 10:25'))
        ->create();

    $projector->onCartCheckedOut($event);

    $cartDuration->refresh();

    $this->assertEquals(
        25,
        $cartDuration->duration_in_minutes
    );
}

# Caveats

Before ending this chapter though, I need to mention a few caveats.

The first one I mentioned already: replaying events can be time consuming. There are cases where this isn't a problem, but sometimes you need to be aware that your deployment strategy might become more complex.

Next, there's also the matter of which data to store in events. While making changes to projections is easy, making changes to events (like adding a property or removing one) is very difficult. Changing an event might have big consequences when replaying them. This is also a topic we'll spend a chapter on later in this book.

---

Despite these caveats, I think there's lots of value in using an event sourced system. While some problems are harder to solve because of event sourcing, others become trivial. We'll dedicate the future chapters to further exploring everything that's to be gained from using event sourcing. There is indeed even more possible thanks to the simple concept of storing events.