Continuous integration and testing with Drupal on AppVeyor

You can now easily test your Drupal projects on AppVeyor. Currently, AppVeyor is the major player in CI regarding Windows Servers. On other CI systems (Travis, Bitbucket pipelines) you are limited to Docker containers for the *nix platform. (This will soon change as some CI will throw Windows containers into the mix).

Until then, the only tool to CI your Drupal (or any PHP project) on a Windows based environment using IIS is AppVeyor.

In this post I'll show you how we setup free (because AppVeyor is free for open source/public projects) for the MS SQL Server driver for Drupal.

After opening a windows account, go the the + New Project menu.

AppVeyor automatically integrates with major code repository providers such as GitHub, BitBucket, Gitlab and Visual Studio Online. In our case, we will use the "raw" Git option, to connect to any Git repository (in this case the drupal.org one):

All the configuration for the project is done through an appveyor.yml file that you commit to the root of your repository. The reference for this file can be found here:

https://www.appveyor.com/docs/appveyor-yml/

In this file you configure the build script, services and any other behaviour required for your CI process.

We prepared a completely automated build and test script to download drupal, install it on a local IIS site and run the complete core test suite:

version: 1.0.{build}
skip_tags: true
init:
- ps: 
services:
- mssql2014
- iis
install:
- cmd: >-
    net start MSSQL$SQL2014
    powershell -command "Set-Service 'SQLAgent$SQL2014' -StartupType Manual"
    net start W3SVC
    powershell -command "new-item c:\tools\php\ext -itemtype directory"
    powershell -command "new-item c:\tmp -itemtype directory"
    rmdir c:\tmp /s /q
    powershell -command "new-item c:\tmp -itemtype directory"
    powershell -command "(New-Object Net.WebClient).DownloadFile('https://github.com/Microsoft/msphpsql/releases/download/4.1.5-Windows/7.0.zip','C:\tmp\sqlsrv.zip')"
    powershell -command "(new-object -com shell.application).namespace('C:\tmp').CopyHere((new-object -com shell.application).namespace('C:\tmp\sqlsrv.zip').Items(),16)"
    copy /Y "c:\tmp\7.0\x64\php_pdo_sqlsrv_7_nts.dll" "c:\tools\php\ext\php_pdo_sqlsrv.dll"
    powershell -command "(Get-Item c:\tools\php\ext\php_pdo_sqlsrv.dll).VersionInfo"
    rmdir c:\tmp /s /q
    powershell -command "new-item c:\tmp -itemtype directory"
    powershell -command "(New-Object Net.WebClient).DownloadFile('http://windows.php.net/downloads/pecl/releases/wincache/2.0.0.8/php_wincache-2.0.0.8-7.0-nts-vc14-x64.zip','C:\tmp\wincache.zip')"
    powershell -command "(new-object -com shell.application).namespace('C:\tmp').CopyHere((new-object -com shell.application).namespace('C:\tmp\wincache.zip').Items(),16)"
    copy /-Y "c:\tmp\php_wincache.dll" "c:\tools\php\ext\php_wincache.dll"
    powershell -command "(Get-Item c:\tools\php\ext\php_wincache.dll).VersionInfo"
    cinst -y OpenSSL.Light
    SET PATH=C:\Program Files\OpenSSL;%PATH%
    sc config wuauserv start= auto
    net start wuauserv
    cinst -y --allow-empty-checksums php -version 7.0.9
    cd c:\tools\php
    copy php.ini-production php.ini
    echo date.timezone="UTC" >> php.ini
    echo extension_dir=ext >> php.ini
    echo extension=php_openssl.dll >> php.ini
    echo extension=php_wincache.dll >> php.ini
    echo extension=php_pdo_sqlsrv.dll >> php.ini
    echo extension=php_com_dotnet.dll >> php.ini
    echo extension=php_sockets.dll >> php.ini
    echo extension=php_mbstring.dll >> php.ini
    echo extension=php_soap.dll >> php.ini
    echo extension=php_curl.dll >> php.ini
    echo extension=php_gd2.dll >> php.ini
    echo extension=php_gettext.dll >> php.ini
    echo zend_extension=php_opcache.dll >> php.ini
    echo opcache.enable=1 >> php.ini
    echo opcache.enable_cli=1 >> php.ini
    echo opcache.memory_consumption=128 >> php.ini
    echo opcache.revalidate_freq=1500 >> php.ini
    echo opcache.max_accelerated_files=8000 >> php.ini
    echo wincache.ucenabled=1 >> php.ini
    echo wincache.ucachesize=128 >> php.ini
    echo wincache.fcenabled=0 >> php.ini
    echo realpath_cache_size=5M >> php.ini
    echo realpath_cache_ttl=1800 >> php.ini
    echo pdo_sqlsrv.client_buffer_max_kb_size=24480 >> php.ini
    echo Setup certificate store
    powershell -command "(New-Object Net.WebClient).DownloadFile('https://curl.haxx.se/ca/cacert.pem','C:\cacert.pem')"
    echo curl.cainfo="C:\cacert.pem" >> php.ini
    SET PATH=C:\tools\php;%PATH%
    powershell -Command ($env:Path)
    powershell -command "new-item c:\composer -itemtype directory"
    cd /d C:\composer
    php -r "readfile('http://getcomposer.org/installer');" | php
    powershell -command "(Get-Item C:\composer\composer.phar).length"
    powershell -command "'@php C:\composer\composer.phar ' + $([char]37) + '*' | Out-File C:\composer\composer.bat -Encoding ASCII"
    SET PATH=C:\composer;%PATH%
    cd /d C:\projects\
    composer create-project -n drupal-composer/drupal-project:8.x-dev
    cd /d C:\projects\drupal-project
    composer config repositories.drupal composer https://packages.drupal.org/8
    composer require drupal/sqlsrv:~2.0
    xcopy /S /I /E %cd%\web\modules\contrib\sqlsrv\drivers %cd%\web\drivers
    composer config repositories.1 git https://%GITLABUSERNAME%:%GITLABPASSWORD%@gitlab.com/david-garcia-garcia/mssql.git
    composer require david/mssql
    powershell -command "'@php %cd%\vendor\drupal\console\bin\drupal ' + $([char]37) + '*' | Out-File %cd%/web/drupal.bat -Encoding ASCII"
    cd /d C:\projects\drupal-project\web
    drupal about
    drupal site:install standard --langcode="en" --db-type="sqlsrv" --db-host="localhost\SQL2014" --db-name="mydrupalsite" --db-user="sa" --db-pass="Password12!" --db-port="1433" --site-name="SQL Server Drupal Site" --site-mail="admin@example.com" --account-name="admin" --account-mail="admin@example.com" --account-pass="admin" --no-interaction
    drupal about
    cd /d C:\projects\drupal-project
    powershell -command "(New-Object Net.WebClient).DownloadFile('https://www.drupal.org/files/issues/2294731-39-phpunit-windows.patch','%cd%\patch.patch')"
    git apply patch.patch --directory=web
    powershell -command "(New-Object Net.WebClient).DownloadFile('https://www.drupal.org/files/issues/use_the_php_binary-2748883-15.patch','%cd%\patch.patch')"
    git apply patch.patch --directory=web
    powershell -command "(New-Object Net.WebClient).DownloadFile('https://www.drupal.org/files/issues/simpletest_is_broken_on-2605284-61.patch','%cd%\patch.patch')"
    git apply patch.patch --directory=web
    powershell -command "(New-Object Net.WebClient).DownloadFile('https://patch-diff.githubusercontent.com/raw/hechoendrupal/drupal-console/pull/3134.patch','%cd%\patch.patch')"
    git apply patch.patch --directory=vendor/drupal/console
    cd /d C:\projects\drupal-project\web
    drupal module:install simpletest
    choco install -y urlrewrite
    powershell -command "New-WebSite -Name 'MyWebsite' -PhysicalPath 'c:\projects\drupal-project\web' -Force"
    echo 127.0.0.1 www.mysite.com >> %WINDIR%\System32\Drivers\Etc\Hosts
    powershell -command "Import-Module WebAdministration; Set-ItemProperty 'IIS:\Sites\MyWebsite' -name Bindings -value @{protocol='http';bindingInformation='*:80:www.mysite.com'}"
    SET PATH=%systemroot%\system32\inetsrv\;%PATH%
    echo Change default anonymous user AUTH to ApplicationPool
    appcmd set config -section:anonymousAuthentication /username:"" --password
    echo Setup FAST-CGI configuration
    appcmd set config /section:system.webServer/fastCGI /+[fullPath='C:\tools\php\php-cgi.exe']
    echo Setup FACT-CGI handler
    appcmd set config /section:system.webServer/handlers /+[name='PHP-FastCGI',path='*.php',verb='*',modules='FastCgiModule',scriptProcessor='C:\tools\php\php-cgi.exe',resourceType='Either']
    iisreset
    NET START W3SVC
    powershell -command "wget http://www.mysite.com/"
test_script:
- cmd: >-
    cd /d C:\projects\drupal-project
    mkdir c:\testresults\
    php web/core/scripts/run-tests.sh --php php --all --verbose --url "http://www.mysite.com/" --xml c:\testresults\
artifacts:
- path: c:\testresults
  name: Results

You can see the CI process here with build results:

https://ci.appveyor.com/project/david-garcia-garcia/sqlsrv

If you host your projects in Bitbucket or GitLab AppVeyor has out of the box integrations that will update build statuses for pull requests, branches, commits, etc.

You can use the above testing script as a base template to implement CI and testing in your Windows based Drupal projects (private or public) against a MS SQL Server.

In order for Javascript based tests (full browser intergation tests) to work with the drupal test suite, you need to install and setup PhantomJs on your testing machine. Use these commands:

- cmd: choco install phantomjs
- ps: Start-Job -Name "test" -ScriptBlock {cmd /c "phantomjs --ssl-protocol=any --ignore-ssl-errors=true vendor/jcalderonzumba/gastonjs/src/Client/main.js 8510 1024 768 false 2>&1 >> c:\gastonjs.log"}