Symfony2 as a thread (in English)

A few weeks ago I was trying to improve performance of my Symfony 2 application. I tried to find some solution for the bottleneck I experienced (which was some heavy I/O usage during the booting of the Symfony 2 kernel). Since I couldn't find anything on heavy I/O usage, I started to think about what my perfect solution would look like.

Before going on, I would like to mention that if you're just looking for performance optimization for your Symfony 2 app, than this is not your read. There are some excellent slides from Seldaek @ http://slides.seld.be/?file=2011-10-20 High Performance Websites with Symfony2.html which are really helpful. This post is just a brainfart, which just isn't suited for live environments (yet). It's merely a proof of concept that works in an environment that is not scalable.

At this stage my app was already using a reverse proxy caching mechanism (the internal sf2 cache kernel). And for the parts which couldn't be cached I was using some ESI's. Furthermore it used APC caching for a few large resultsets (e.g. menu structure of the website).

Because of the heavy disk I/O (due to the autoloading of the many bundles it was using), I thought of a situation where a booted kernel was always there, just handling requests one after another that never terminates. Just like a threaded desktop application. I was instantly in love with this idea, building a threaded application with Symfony2 was something new, that might just work. It would also be a big mindshift, thinking about the possibilities this can open up for PHP.

Yeah I know; it's exotic, experimental, unstable and it's just waiting for those first comments like "PHP/Symfony is never intended to be used that way". Sure, most of them (maybe all) are true. But again, this is just a proof of concept. Maybe a direction in which PHP will grow in future releases? Who knows? For now it's just for fun :-)

Concept

To have a threaded Symfony2 application and still have the GUI served over the webserver, we need a parallel process besides the webserver so the webserver can hook into it. I chose zeroMQ for this purpose (because of the ease of installation and usage). A PHP script called from the webserver will connect to a zeroMQ implementation on a specific port. On the other side a PHP process is started (a booted Symfony 2 kernel), which has an eternally running while{} loop to handle every request it receives. The kernel handles the request and sends it back to the webserver worker, which returns the received response and terminate the request. Schematic view below.

Setting it up

First you need to install 0mq. Installation instructions are OS dependant. See http://zeromq.org/intro:get-the-software for more info. (Compiling on CentOS was pretty straightforward)

Then you'll need your PHP bindings, so we can use the zeroMQ library from PHP. A PHP extension is provided as a PECL, and is built by running

also add "extension=zmq.so" to your php.ini, and restart your webserver

Then create the PHP thread/process side

And of course the webserver side of it

Everywhere I wrote Symfony2, any other PHP framework or codebase can be used. But because of the great abstraction of  the Request object inside the Symfony2 framework, this was pretty easy to get most of the stuff working quickly. Inside the 0mq.php script you can't access server variables like ($_GET, $_POST, $_SESSION, etc), all of these are wrapped in the Request object before it is send to zeroMQ, So the 0mq.php script is completely agnostic of the existence of a running webserver.

Some drawbacks

Seriously, don't use this in live environments! A few drawbacks should be tackled before using this kind of pattern in a live environment.

Security

Because in a normal environment each request is terminated by the webserver. So from the start of a request we're 100% sure we have a clean environment with no residual variables that could expose delicate information of the previous request (maybe from another user). 

Memory

Ever built a long running/heavy duty PHP task for the CLI? If yes, then you'll most certainly have encountered some memory leaks. Mostly due to some library (or your own code ;-)), or maybe PHP itself. Because PHP is not so much used in these kind of situations, there was never the necessity to take these prerequisites too much into account. Most of the PHP internals is probably quite clean, but you might encounter some problems when running the entire Symfony2 stack for a longer time (e.g. Doctrine can be quite memory hungry when doing many data operations)

Worker matching

Each worker of your webserver should be able to connect to your thread. But if there's just one thread.., you might just as well configure your webserver to have just one thread. So we should have as many 0mq threads as webserver workers in order not to have the webserver waiting.  I didn't do much investigation in all kinds of 0mq design patterns, but there's probably one with a broker and load balancer available which could overcome this problem.

Performance

This solution wil result in a permanent memory load on the webserver having as many running kernels as workers (if you have 8 workers, you'll always need the memory load compared to 8 users in a normal environment). Initially this setup will cost you more. In the long run it should peform better because a kernel is booted one time only for each worker. So every request following the first should perform better. Because it isn't my primary goal anymore (because of practical implications) to use this in production environment I didnt do any peformance measurements.

Conclusion

It became quickly clear that a threaded PHP application can't be run in a live environment (yet). From the initial performance problem I drifted somewhat away to just a "threaded PHP application". But thinking about the possibilities for future development this could just be something new in the PHP world that could thrust some new and really innovative solutions.