Setting up Async Event Sourcing with SmoothPHP


Event sourced applications tend to be async from the write side of the application to the projection side. In SmoothPHP all events are pipped into a queue system and then the event handlers are on the queue processor. Making all event dispatchers a different async process.

There are many different ways to achieve this separation, the simplest we have opted for is queue system. But we do have plans to use PThreads in the future to achieve better performance. Another idea is to use Gearman (not been updated for a while though).

So this guide will show you how to set up the default Async setup of redis and supervisor to achieve Async event processing. This is based on the Laravel version of the package and using a debian/ubuntu box the process should be the same for other OS’s.

How it works out of the box. Aggregate are saved and the events are passed into a event bus. Listeners of the event bus get passed every event the system generates. A listener takes events and passed them into a queue system in there serialized form. Another process (laravel worker) then takes those events/jobs out of the queue system and passes them to an event dispatcher, this then goes to subscribed listeners who do the work (writing a projection for example).

Install Redis

Redis is used for the jobs/events to sit in between the write site and the read side. It enables the write side to store the event for the read side/event dispatcher to read later.

Installing

As root type in the following commands

cd /tmp
wget http://download.redis.io/releases/redis-stable.tar.gz
tar xzf redis-stable.tar.gz
cd redis-stable
make
make install
cd utils
./install_server.sh
update-rc.d redis_6379 defaults
echo "bind 127.0.0.1"  | sudo tee -a /etc/redis/6379.conf
chmod 700 /var/lib/redis
chown redis:root /etc/redis/6379.conf
chmod 600 /etc/redis/6379.conf
service redis_6379 restart

Note the bind 127.0.0.1 command which makes redis only listen to local host. You may wish to change this but you must secure your instance if you do.

Install Supervisor

Pretty easy

sudo apt-get install supervisor

Then you need to set up a job for supervisor to supervise.

sudo nano /etc/supervisor/conf.d/smoothphp-worker.conf

Add in the following config file remembering to edit the command path and the user. If you wish for more processes handling events increase the numprocs setting.

[program:smoothphp-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/artisan queue:listen --sleep=3 --tries=3
autostart=true
autorestart=true
user=www-data
numprocs=1
redirect_stderr=true
stderr_logfile=/var/log/supervisor.log
stdout_logfile=/var/log/supervisor.log

Once saved you are ready to tell supervisor to reload config files and start the process

supervisorctl reread
supervisorctl update

Supervisorctl Status

Now you are ready to process events async. If you wish to change the way events are processed you just need to write a new implementation of the Event Listener Interface and process them how you want.