Skip to main content

Testing Drupal with WebDriver browser mode vs Headless browser mode

There is not a lot of documentation available about what's the difference between running a browser in WebDriver mode vs Headless so I did some digging...

by Jibran Ijaz /

Apparently, there are two ways to run Chrome for testing:

  • As WebDriver
  • As Headless

WebDriver:

There are two ways to run Chrome as WebDriver:

Using Selenium:

Run Selenium standalone server in WebDriver mode and pass the path of ChromeDriver bin along with the config e.g. Selenium Dockerfile

This works fine with Nightwatch standard setup, \Drupal\FunctionalJavascriptTests\JavascriptTestBase and also with Drupal core's new \Drupal\FunctionalJavascriptTests\WebDriverTestBase.

Using ChromeDriver:

Run ChromeDriver in WebDriver mode e.g. chromedriver Dockerfile

This works fine with Nightwatch, JTB, and WTB.

Headless:

Using Chrome

Run Chrome browser binary in headless mode. e.g. Chrome headless Dockerfile

Nightwatch is not working with this set up, at least I was unable to configure it. See https://github.com/nightwatchjs/nightwatch/issues/1390 and https://github.com/nightwatchjs/nightwatch/issues/1439 for more info. \DMore\ChromeDriver can be used to run the javascript tests.

Using ChromeDriver

Using Selenium ChromeDriver can be run in headless mode something like this:

const fs = require('fs');
const webdriver = require('selenium-webdriver');
const chromedriver = require('chromedriver');

const chromeCapabilities = webdriver.Capabilities.chrome();
chromeCapabilities.set('chromeOptions', {args: ['--headless']});

const driver = new webdriver.Builder()
  .forBrowser('chrome')
  .withCapabilities(chromeCapabilities)
  .build();

DrupalCI is running ChromeDriver without Selenium and testing Nightwatch and WTB on it.

Conclusion

The question is which is the best solution to run Nightwatch and JTB/WTB tests using the same setup?

  • We had seen some memory issues with Selenium containers in the past but we haven't run into any issue recently so I prefer this and you can swap Selenium container to use different browsers for testing.
  • We have also seen some issues while running ChromeDriver in WebDriver mode. It just stops working mid-test runs.
  • I was unable to get Headless Chrome working with Nightwatch but it needs more investigation.
  • Headless ChromeDriver setup on DrupalCI is quite stable. For JTB this would mean that we could use anyone from \Drupal\FunctionalJavascriptTests\DrupalSelenium2Driver and DMore\ChromeDriver.

Please share your ideas and thoughts, thanks!

For more info:

Posted by Jibran Ijaz
Senior Drupal Developer

Dated

Comments

Comment by Paulmicha

Dated

Might be worth adding to this list that there's an alternative setup I successfully tested in april using the browserless/chrome image with Cucumber and Puppeteer for behavioral tests.

YMMV but to give a rough idea, here's the relevant docker-compose.yml extract :

b-tester:
image: node:9-alpine
command: 'sh -c "npm i"'
volumes:
- ./tests:/tests:cached
working_dir: /tests
network_mode: "host"

# See https://github.com/GoogleChrome/puppeteer/issues/1345#issuecomment-3451…
chrome:
image: browserless/chrome
shm_size: 1gb
network_mode: "host"
ports:
- "3000:3000"

... and in tests/package.json :

"devDependencies": {
"chai": "^4.1.2",
"cucumber": "^4.0.0",
"puppeteer": "^1.1.0"
},
"scripts": {
"test": "cucumber-js"
}

... and to connect, open a page and take screenshots, e.g. in tests/features/support/world.js :

const { setWorldConstructor } = require("cucumber");
const { expect } = require("chai");
const puppeteer = require("puppeteer");
const PAGE = "http://todomvc.com/examples/react/#/";

class TodoWorld {

... (snip)

async openTodoPage() {
// See https://github.com/joelgriffith/browserless
// (via https://github.com/GoogleChrome/puppeteer/issues/1345#issuecomment-3451…)
this.browser = await puppeteer.connect({ browserWSEndpoint: 'ws://localhost:3000' });
this.page = await this.browser.newPage();
await this.page.goto(PAGE);
}

... (snip)

async takeScreenshot() {
await this.page.screenshot({ path: 'screenshot.png' });
}

... (snip)
}

setWorldConstructor(TodoWorld);

→ To run tests, e.g. :
$ docker-compose run b-tester sh -c "npm test"

(result in tests/screenshot.png)

I ran out of time to make a prototype repo, but my plan was to integrate https://github.com/garris/BackstopJS

Pagination

Add new comment

Restricted HTML

  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd> <h2> <h3> <h4> <h5> <h6>
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.
Not sure where to start? Try typing "hello" or "help" if you get stuck.