In a previous post, I discussed how to use custom cookies on Pantheon and how to provide a browser based functional test using the WebDriverTestBase class from Drupal 8.
While it worked great, these tests can be a bit of a bear to get up and running locally. Installing Selenium and chromedriver can be daunting and unwieldy and difficult to replicate across members of the team so they can write and run tests locally. Leveraging the same stack in CI can be an equal challenge.
I decided to investigate different solutions for solving this. While Nightwatch is now included with Drupal core, I had some difficulty in getting it running. I could not find a way to use a different profile for testing, so I could not figure out how I could install the modules and configuration I would need under test. The .drupalInstall Javascript function also assumes the running script has access to PHP, and if you are operating in Docker containers, this may not be possible with your setup. It would be great to use this, but for now I don't think it is quite there yet. Not for my needs at least. Nightwatch relies on Selenium as well, so even if I could get it going it only solves half of the problem.
Hello Cypress
I've known about Cypress for a while but never had an opportunity to take advantage of it until now. Cypress provides Docker images to use, which is great because I have been developing projects within Docker for two years now. I never install any dependencies on my host machine anymore, which makes it able to scale across the project team and clients regardless of technical skills - a true strength of Docker. On top of that, Cypress has no reliance on Selenium at all.
Cypress is capable of doing nearly anything you can think of. It comes with an Electron app if you want to watch the browser run through the tests, and can also run entirely headless. For this post, it will be running headless only.
To add Cypress to the project, I added the image to my docker-compose.yml file:
cypress:
image: cypress/included:4.0.0
container_name: "myproject_cypress"
working_dir: /app
volumes:
- ./tests/cypress:/app/cypress
- ./tests/cypress/cypress.json:/app/cypress.json
Upon starting up, Docker will pull this image and create a container for me. We will be able to use this later using a Docker command. First, we need to add a directory in the project root called "tests" with a directory inside of it called "cypress". These get mounted into the container per the 'volumes' directive above.
Inside of the tests/cypress directory, we need to provide a config file called cypress.json that is used to provide configuration values and runtime variables to tests. It does not require a lot to start with:
{
"baseUrl": "http://testing.docker.localhost",
"cookieDomain": ".docker.localhost",
"pluginsFile": false,
"supportFile": false,
"fixturesFolder": false,
"video": false,
"pageLoadTimeout": 10000
}
To start with, I have set video to false (prevent recording of any tests) and set the page timeout to 10 seconds (the default is 60). Cypress can take video recordings of tests, but I do not need that feature. Feel free to experiment with it. The baseUrl is set to our testing domain, and I am passing a custom config value called "cookieDomain" with a static value of ".docker.localhost".
With that in place, we can write our first test. The Cypress API is very simple and expressive, and all tests are written in Javascript. Not only does it read pretty straightforward for anyone, it also makes writing tests a more accessible than writing them in PHP (again, I aim to provide tools for the team at any skill set).
Goodbye WebDriverTests
Let's go ahead and convert the test from the previous post to a test for Cypress.
Here is the original test we are replacing that was using WebDriverTestBase:
<?php
namespace Drupal\Tests\mymodule\FunctionalJavascript;
use Drupal\FunctionalJavascriptTests\WebDriverTestBase;
/**
* Class CookieScriptTest
*
* @group mymodule
* @package Drupal\Tests\mymodule\FunctionalJavascript
*/
class CookieScriptTest extends WebDriverTestBase {
/**
* {@inheritdoc}
*/
protected $strictConfigSchema = FALSE;
/**
* {@inheritdoc}
*/
protected static $modules = ['node', 'user', 'mymodule', 'system'];
/**
* {@inheritdoc}
*/
protected function setUp() {
parent::setUp();
$this->container->get('module_installer')->uninstall(['page_cache']);
$this->container->get('theme_installer')->install(['mytheme']);
$this->container->get('config.factory')->getEditable('system.theme')->set('default', 'mytheme')->save();
}
/**
* Test that mymodule/js/cookies.js copies legacy cookies.
*/
public function testOldCookiesAreCopied() {
$cookies = [
'COOKIE_ONE' => 'foo',
'COOKIE_TWO' => 'bar',
];
$this->prepareCookies($cookies);
$cookie_one = $this->getSession()->getCookie('STYXKEY_COOKIE_ONE');
$cookie_two = $this->getSession()->getCookie('STYXKEY_COOKIE_TWO');
$this->assertEqual($cookie_one, 'foo');
$this->assertEqual($cookie_two, 'bar');
}
/**
* Helper method to set a cookie value for testing.
*
* @param string $cookie_name
* @param null $cookie_value
*/
protected function prepareCookies(array $cookies) : void {
$session = $this->getSession();
foreach ($cookies as $name => $value) {
$session->setCookie($name, $value);
}
// This is required to "set" the old cookies in the request and have the test browser
// add them for the next request. Essentially, "click the site logo".
$this->click('.header__branding a');
}
}
Here is that same test rewritten for Cypress in Javascript. We create our spec test, cookies.js, inside of tests/cypress/integrations:
describe('Existing user cookies should be functional in Pantheon.', function() {
beforeEach(function() {
cy.setCookie('username_ticket_cookie', 'foo');
cy.setCookie('security_tickets', 'bar');
cy.visit('/');
});
it('Legacy cookies are preserved and copied for Pantheon', function() {
cy.getCookie('STYXKEY_username_ticket_cookie').should('exist');
cy.getCookie('STYXKEY_username_ticket_cookie').should('have.property', 'value', 'foo');
cy.getCookie('STYXKEY_security_tickets').should('exist');
cy.getCookie('STYXKEY_security_tickets').should('have.property', 'value', 'bar');
});
it('Writes the current domain into the cookie', function() {
cy.window().then((window) => {
const domain = window.getCookieDomain();
expect(window.getCookieDomain()).to.equal(Cypress.config().cookieDomain);
cy.getCookie('STYXKEY_username_ticket_cookie').should('have.property', 'domain', domain);
cy.getCookie('STYXKEY_security_tickets').should('have.property', 'domain', domain);
});
});
});
Very clean and clear! The Cypress API is very simple to use. Now we can go ahead and try to run the test, but first, we have to provide an environment.
One difference between this and the WebDriverTestBase test is that we cannot tell Cypress to install modules for the test, or even to install a new instance of Drupal from scratch to the local database. We can get around that though by just creating a fresh site with Drush:
drush @site.env si CUSTOM_PROFILE --db-url=(database info) --sites-subdir=testing.docker.localhost
CUSTOM_PROFILE is simply the name of an install profile made for this project that ensures we have the bare minimum environment setup needed to run this and other Cypress tests. It contains modules and configuration. I run this once to provide the environment. You could also run this every time you test (locally), especially if you are actively working on the install profile to make it suitable for testing. Alternatively, you can just restore a database from backup, like a copy of production, for example. It's really up to you and your needs. Note: my tests don't rely on real content, but yours may, so you may want to think about restoring from a production database for different tests dependent on content. Our tests revolve around cookies and AJAX based on cookie values, so no actual content is needed.
Lets go ahead and have Cypress run the tests. We're going to tell Docker to run the Cypress image and pass it the command 'npx cypress run':
docker-compose run --rm cypress npx cypress run
We can also alias this command with Make or Ahoy, to reduce typing and make it easier for people to use (Docker commands can be lengthy and a lot to remember). Here is an example of an alias for Ahoy:
cypress:
cmd: docker-compose -f docker-compose.cypress.yml run --rm cypress npx cypress run "$@"
usage: Run Cypress tests.
Now that command is simply "ahoy cypress", or "ahoy cypress -- ARGS" if you want to pass additional arguments. Simple to remember.
We're good to roll, and in a blink of an eye, Cypress has executed and evaluated our test.
Running: cookies.js... (1 of 1)
Existing user cookies should be functional in Pantheon
✓ Legacy cookies are preserved (2158ms)
✓ Writes the domain into the cookie (1425ms)
That sure is fast! Since we are not doing a full install of Drupal per run or using Selenium, our total test execution time has dropped from 2 minutes to 3 seconds. It's hard to argue with that. This results in faster feedback loops and faster iterating over tests and code, less downtime for devs and improved build times in CI.
Fun, fast and clean.
Cypress in Continuous Integration
Pushing code and having it build in CI, we can take full advantage of Docker and Cypress here too, as well as run our full test suite. For example, in TravisCI:
script:
- docker-compose exec php composer install -n --optimize-autoloader
- docker-compose exec php (drush path) si CUSTOM_PROFILE -y --db-url=(database info) --sites-subdir=testing.docker.localhost
- docker-compose exec php phpunit --testsuite unit,kernel,functional
- docker-compose run --rm cypress npx cypress run
Nothing different here, these are all the same commands we use to install Composer dependencies, create a testing site with Drush, and run our tests!
All in with Cypress
I am a full Cypress convert now. Being untethered from Selenium and Drupal classes, this can be used to test Drupal 6 through 8 and beyond for all of our UI needs. Cypress is usable for any platform, not just Drupal, so it can be used just about anywhere. It's dead simple to use, well supported, can be extended with custom plugins, and executes lightning fast.
It originally took me 2 full days to get Selenium installed and running in Docker, while figuring out the proper configuration so it runs the original tests, and then a lot of debugging CI to get it to do the same. By throwing all that out and using Cypress, I was able to get up and running and convert in just a few hours.
I find it to be an exceptionally powerful and valuable tool for QA and confidence. If you are not doing any automated testing like this, I implore you to check it out. If you have QA teams, this will save them loads of time over manually clicking and interacting with the site to free them up to provide more valuable QA feedback that machines can't really give you.