Setting up Code Syntax Higlighting with Drupal

While setting up (still in progres..) this website we found the need to enable easy Code Highlighting to be done through our WYSIWYG editor of choice: CKeditor. What looked like a 30 minute task, ended up in a more than 3 hour adventure. Let's see what happened.

• Understanding the Basics
• Choosing the right syntax highlighting library.
• Puting the pieces together
• Final words

Understanding the basics

Code higlighting is this:

var i;
var fib = []; //Initialize array!

fib[0] = 0;
fib[1] = 1;
for(i=2; i<=10; i++)
{
    // Next fibonacci number = previous + one before previous
    // Translated to JavaScript:
    fib[i] = fib[i-2] + fib[i-1];
    alert(fib[i]);
}

Having code higlighting in your site greately improves readabilty of technical coding articles.

The theory of how code higlighting works was clear from the start. Code higlighting usually works by having the 'code' or 'pre' tags surrounding your piece of code, then a javascript plugin is in charge of parsing and styling that piece of code. The code or pre tags might have a class attribute that indicates the programming language of the snippet, and some of these plugins have automatic language detection.

The HTML for the previous piece of code looks like this:

<pre>
<code class="language-javascript">var i;
var fib = []; //Initialize array!

fib[0] = 0;
fib[1] = 1;
for(i=2; i&lt;=10; i++)
{
    // Next fibonacci number = previous + one before previous
    // Translated to JavaScript:
    fib[i] = fib[i-2] + fib[i-1];
    alert(fib[i]);
}</code></pre>

At first everything looks easy, we just need 2 things:

• [1]  plugin for the CKEditor that will allow to create this "pre" and "code" content with ease, we don't want editors to input raw HTML. If possible, this plugin should alow live preview of the formatted code.
• [2] Some sort of "frontend" plugin that will detect, process and style all "pre" and "code" blocks found in every page.

Choosing the right Syntax Hyghlighting library

The first thing anyone would do is scout for available Drupal modules (usually to get some inspiration, not from a site builder's perspective).

This is the first one that comes up: https://www.drupal.org/project/syntaxhighlighter

This module doesn't look good, and it is focused on integration with Tiny MCE and WYSIWYG module's, something that doesn't work with our CKEditor without WYSIWYG setup.

What it does is give us a good hint: the SyntaxHighlighter javascript library seems to be widely adopted and a good candidate to be included into our site through this site's customization module (for each site developed at Sabentis - the people behind DrupalOnWindows.com - we have an exclusive module for that site's tweaks and customizations, and also a theme).

This is were the nightmare starts. We were unable to determine at first which version of the plugin to use, the stable versions (3.0.83 and 3.0.9)  are quite out of date. The GIT setup for the different versions is a mess, and the author recomends not to use the master branch because it's at development. We decided to go for the 3.09, but for our surprise the distributed download is not even ready for deployment because it has uncompiled SASS stylesheets and the key JS files needed to setup the plugin are duplicated in different locations - with different source code of course -.

Forward about an hour and we had tried integrating all possible versions (master, 3.0.83 and 3.0.9)  without success. We even found out that this plugin depends on another plugin (not stated anywhere!), the XRegExp plugin.

We keep looking and find this Sandbox: https://www.drupal.org/sandbox/eduardoa/2157351. But not even worth trying.

After some more investigation we find out that the CKEditor has an official plugin, the Code Snippet Plugin that depends on the HighlighterJS library.

This looks good now, we have a CKEditor plugin to manage (and live preview!) the creation of Code Snippets, but we still need a frontend module/plugin to run the Highlighter plugin when vieweing our pages. That would be the HighlightJS module (makes sense to use the same plugin that the editor is using, to have a coherent looking of the formated code between edition and visualization).

Puting the pieces together

Looks like we will need no custom coding to achieve what we want. Take these steps to setup Syntax Highlighting:

• Download the Code Snippet Plugin into your ckeditor/plugins folder.
• Enable the plugin for the Input Format you will be using at /admin/config/content/ckeditor. You might need to drop the toolbar icon to you current toolbar setup.
• Download the HighlighterJS library and drop it in a sites/all/libraries/highlightjs folder (note the name of the folder is not the same as in the download package).
• Download and enable the HighlightJS module (It's composed of two submodules, the Highlight JS Syntax Highlighter and the Highlight JS Filter, you should only enable the former one).

Remember to add the icon to the current toolbar after enabling the plugin in the CKEditor settings.

After doing this, you should be able to start adding code snippets to your content.

What's up with the Highlight JS Filter submodule?

The HighlightJS module does not know anything about the editor you are using, so they developed an input filter to help solve some negative interactions between user input and other usual Drupal input filters by:

• Doing a check_plain() over on yout code snippet
• Protect newlines from line break converter
• Some other stuff...

We found that the CKEditor Code Snippet plugin already encodes HTML entities, so doing another check_plain() on the contents ends upd with double encoded data and undesired output.

To make the Filter work with CKEditor Code Snippet, you must comment the line that double encodes our data in highlight_js_filter.module:

function highlight_js_filter_escape($text, $type = 'code', $lang = '') {
  // Note, pay attention to odd preg_replace-with-/e behaviour on slashes
  $text = check_plain(str_replace('\"', '"', $text));
  [...]
}

 In this way you can still benefit from the other functions performed by the Highlight JS Filter.

Final words

Our internal development guidelines obligate us to assess every new module that we incorporate into our projects, and the HighlightJS was a new one. Initial inspection denoted bad coding standards in the module, including the use of hook_init, and an overkill implementation. For our use case, we could have implemented the part that we need of the module less than half of the code and hooks it uses now. This blog is a toy project, so we didn't bother and left it as is, but the lesson to remember here is that you should never completely trust contributed code, we have found that many bottlenecks are originated in bad coded contributed modules when doing project rescue.