Setting up Async Event Sourcing with SmoothPHP


Event sourced applications tend to be async from the write side of the application to the projection side. In SmoothPHP all events are pipped into a queue system and then the event handlers are on the queue processor. Making all event dispatchers a different async process.

There are many different ways to achieve this separation, the simplest we have opted for is queue system. But we do have plans to use PThreads in the future to achieve better performance. Another idea is to use Gearman (not been updated for a while though).

So this guide will show you how to set up the default Async setup of redis and supervisor to achieve Async event processing. This is based on the Laravel version of the package and using a debian/ubuntu box the process should be the same for other OS’s.

How it works out of the box. Aggregate are saved and the events are passed into a event bus. Listeners of the event bus get passed every event the system generates. A listener takes events and passed them into a queue system in there serialized form. Another process (laravel worker) then takes those events/jobs out of the queue system and passes them to an event dispatcher, this then goes to subscribed listeners who do the work (writing a projection for example).

Install Redis

Redis is used for the jobs/events to sit in between the write site and the read side. It enables the write side to store the event for the read side/event dispatcher to read later.

Installing

As root type in the following commands

cd /tmp
wget http://download.redis.io/releases/redis-stable.tar.gz
tar xzf redis-stable.tar.gz
cd redis-stable
make
make install
cd utils
./install_server.sh
update-rc.d redis_6379 defaults
echo "bind 127.0.0.1"  | sudo tee -a /etc/redis/6379.conf
chmod 700 /var/lib/redis
chown redis:root /etc/redis/6379.conf
chmod 600 /etc/redis/6379.conf
service redis_6379 restart

Note the bind 127.0.0.1 command which makes redis only listen to local host. You may wish to change this but you must secure your instance if you do.

Install Supervisor

Pretty easy

sudo apt-get install supervisor

Then you need to set up a job for supervisor to supervise.

sudo nano /etc/supervisor/conf.d/smoothphp-worker.conf

Add in the following config file remembering to edit the command path and the user. If you wish for more processes handling events increase the numprocs setting.

[program:smoothphp-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/artisan queue:listen --sleep=3 --tries=3
autostart=true
autorestart=true
user=www-data
numprocs=1
redirect_stderr=true
stderr_logfile=/var/log/supervisor.log
stdout_logfile=/var/log/supervisor.log

Once saved you are ready to tell supervisor to reload config files and start the process

supervisorctl reread
supervisorctl update

Supervisorctl Status

Now you are ready to process events async. If you wish to change the way events are processed you just need to write a new implementation of the Event Listener Interface and process them how you want.

Getting Setup with SmoothPHP


Getting setup with SmoothPHP CQRS and Event sourcing. In this first video in our series we cover getting a VM ready for development. Install SmoothPHP and the SmoothPHP Laravel adapter. Configure SmoothPHP and install the Eventstore table.

In the next video we can cover building out first aggregate and events.

Comments are most welcome as I don’t have much experience with screencasting.

What is Command Query Responsibility Segregation (CQRS)?


What is Command Query Responsibility Segregation (CQRS)? It is a pattern formed by Greg Young, an independent software consultant who was the pioneer of CQRS and Event Sourcing. The pattern describes using a different model for updating information (write) than the modal used for reading data. For many complex situations this separation can be highly valuable, for simpler tasks it can be total overkill and a detriment to projects. It should be made clear CQRS is not an architecture – it is just a simple pattern that examples architecture that would otherwise not be possible.

What is Command Query Responsibility Segregation, Diagram

Greg provides a good explanation of what CQRS is which we will expand on.

Think in your mind what a command is and what a query is. Commands mutate the state of the domain – this can be in terms of CRUD which we all love and know so well. Commands are nothing more than self describing DTO’s, for example ‘RegisterMember’ or ‘CancelOrder’. Typically their class name is a verb, describing the intent of the action. Commands do not return any value, as they are write only actions. Queries are similar, in that their class name will describe their intent. However, they will not affect the data store, instead simply finding a set of information based on a given set of parameters. They are read only operations, and may look like FindMembersByRegistrationDate(DateTime registrationDate) : MemberCollection. This is the foundation of CQRS, and as you can see, it’s really, really simple.

Imagine the following code:

CQRS Example

interface MemberService
{
    registerMember(id,email,password):void
    findMember(id):Member
    findMemberByEmail(email):Member
    addMembersAddress(address,id):void
}

Applying simple CQRS you would get two separated services:

interface MemberWriteService
{
    registerMember(id,email,password):void
    addMembersAddress(address,id):void
}
interface MemberReadService
{
    findMember(id):Member
    findMemberByEmail(email):Member
}

And that is simply what Command Query Responsibility Segregation is, nothing really more, you keep your read and your writes systems segregated. This enables you to change the way your programming mind works, a refreshing break from the standard ORM CRUD methodology.

The biggest benefit is realising that you can architect your commands and queries differently, and host your services separately. You can have a couple of write servers and many read servers. The rate in which you need to scale read and write services are different and in many applications the queries are many orders of magnitude more frequency than commands, so applying CQRS enables more targeted and effective scaling.

We will cover the other benefits in other articles as next we will be talking about Event Sourcing.