Profiling Drupal: XHprof, Uprofiler, Tideways with PHP7 on Linux and Windows
Profiling Drupal: XHprof, Uprofiler, Tideways with PHP7 on Linux and Windows
Being able to analyze and monitor - with ease - the performance of your application is a key part to the success of any web based project. Not only because a slow site might negatively affect conversions, but as Drupal 8 is shifting away from websites and more into complex web based applications, specific business transactions are becoming more important. Considering Drupal's historical poor performance when it comes to "backend" operations, having proper profiling tools is a must.
The RPS (requests per second) metric becomes meaningless in applications where user interaction is the norm, and is only relevant now for brochureware.
In this post I will be discussing the Tideways Profiler: a FREE and OPEN SOURCE maintained drop-in replacement for XHProf and UProfiler that has support for PHP7, Windows and Linux and is actively maintained. But this is not just a drop-in replacement for XHprof and UProfiler.... to keep the business running - Tideways - the company behind the free and open source PHP extension provides a cloud service (very much like the proprietary blackfire.io) where you can collect, monitor and analyze your profiling sessions. If you want to stay on the cheap side, you can still use any of the free XHprof and UProfiler compatible tools such as XHGUI.
Why is Tideways important? Because if it was not for it there wouldn't be any available free and open source profiling tool that you can use with PHP7 and that is actively maintained. Worth mentioning here is that you can still profile with the XDebug extension (also free and open source) but it is not suitable for continuous performance monitoring neither for production usage. And to be honest, it is a pain to use even in development environments as the application becomes orders of magnitude slower when enabled.
At some point a big PHP "customer" called Facebook realized that they needed a tool to profile PHP software that could be deployed to production environments. Facebook developed such a tool in-house and open sourced it somewhere between the end of 2008 and the start of 2009. They worked on it for a couple of years until the project was abandoned in 2013. The original PECL extension can be found here.
Then a fork of XHprof called UProfiler took the lead, being taken care after by Friends Of PHP, a small group of individuals that includes Fabien Potencier and Pierre Joye (among others).
During those years, integrations with XHprof/UProfiler poped for almost any platform (such as the Drupal XHProf module) as well as external tools to collect, analize and monitor profiling data such as XHgui.
But as projects mature and the individuals and companies behind them start to seek feasible business models to keep the business running (because just giving away simply won't work), things start to change. And so hapend with UProfiler. With Fabien being the main maintainer and seeking monetization on the hard built PHP ecosystem, they released the proprietary Blackfire.io knowing that the arrival of PHP7 would mean that - unless someone big took the lead and upgraded the current profiler - people would rather pay a handful of bucks for a proprietary solution (as PHP now had found it's way into more serious business where being cheap/free is not important anymore). It was obvious that they had no interest in anyone keeping up the work on UProfiler or forking it out, otherwise this thread would have not been locked.
Then Tideways was born. A small company with talented and dedicated individuals that forked the UProfiler extension, made it PHP7 compatible, added full Linux and Windows support and built a cloud based business model while keeping the PHP Extension free and fully backwards compatible with XHprof and UProfiler tools.
If you deploy the free PHP extension with the Tideways daemon, profiling data can be collected directly on your production environments (without affecting performance) and sent to the Tideways cloud service to be analyzed.
The Tideways website already contains detailed instructions on how to setup Tideways extension and Daemon on Linux. They also give instructions for Windows, but I feel those to be a bit lacking and will expand them here.
The first thing you need is the Tideways PHP extension that you download from the CI service APPVeyor:
Look for the latest successful build (the most recent one with the "green light"):
Then choose which build you need for your specific platform (PHP Version, Thread Safety and x86/x64):
Then navigate to the artifacts tab, where you fill find the download link for the compiled PHP extension:
As usually, download the php_tideways.dll into your extension folder (/ext) and enable it in your php.ini file:
After enabling the PHP extension in PHP.ini you should see it in a phpinfo() dump:
Download the Tideways.php PHP library from here:
And store it somewhere in your system, such as d:\_Webs\xxx\Tideways.php
Now configure the extension in your PHP.ini file:
extension=php_tideways.dll tideways.auto_prepend_library=0 auto_prepend_file = D:\_Webs\xxx\Tideways.php tideways.connection=tcp://127.0.0.1:8136 tideways.api_key=MYAPIKEY tideways.sample_rate=50 tideways.collect=profiling tideways.monitor=full
The tideways.api_key value here should be the API key for an application that you have setup through the Tideways Cloud Service:
They offer 30 day trials so you can easily test this without commitment.
To get your environment to send data to the Tideways cloud service you need to deploy the Tideways daemon, that you can download from here.
If you are just trying out the profiler, you can start the daemon manually from the console using:
But if you are going to deploy this on production environments you need to setup the daemon as a windows service.
Unfortunately, as the tideways daemon is not a .Net native windows service, the recommended way to deploy it as a service is to use the Non Sucking Service Manager:
The NSSM manager will ensure that the daemon is up and running, and to restart it in case it crashes or gets stuck.
Remember to let the daemon through your firewall, otherwise it won't be able to report data to the Tideways cloud service.
The rest of the Tideways setup instructions are much the same as the ones for Linux.
You should also get ahold of the Tideways Chrome Extension. Usually on production environments you don't capture full call traces (due to performance reasons) and you only profile a percentage of the requests (see the Tideways official setup guide for more details about this). Using the Chrome extension you can tell the profiler to capture full traces during a period of time directly from your browser.
After deploying the Tideways extension and daemon, you should install the Tideways Drupal module.
Once enabled, you will see a small report in your status page:
There is nothing more to do from Drupal. Under the hood what the module does is tell the Tideways extension meaningful information about each request, so that you can find useful information in the Tideways cloud reports.
Launch your website on Chrome and using the Tideways Chrome exension, start a full trace capturing session:
Then the most recent captured traces from the manual session will start showing (right above the regular continous monitoring data):
Choose any specific business transaction and access the full data:
Out of the box you get full traces, callgraph, timelines and much more. If you need anything custom, you can always add custom instrumentation.
To get a deeper look into the possibilities of Tideways, use the official documentation.