Skip to main content

Using Behat and DrupalDriver? Beware pathauto.

Here at PreviousNext we love behat. We use it as part of our CI builds to make sure that we're not introducing regressions and it's a great fit for our Agile methodologies. But we recently had some issues with weird behaviour in our tests, such as page manager pages not working. Read on to find what was the cause and how we tackled them.

by Lee Rowlands /

Background

In case you hadn't heard of it before, Behat is a PHP testing framework for Behaviour Driven Development. It lets you write human-readable stories that describe the behaviour of your application, that can be automatically executed to test if it's working as expected.

For some of our projects, we've embraced Behat so much that we've taken to writing the acceptance criteria for our stories in the Given, When, Then format. This has the advantage that our tests are already written when we start the story. So because we want to adapt our tests to the natural language of the acceptance criteria, rather than the other way around, we are regularly adding to our FeatureContext class to define the custom step definitions.

In order to make these step definitions as flexible as possible, we're using the DrupalDriver on some of our projects instead of the BlackBox or DrushDriver.

This means we have access to the full Drupal API's in our step definitions and any content changes that are made as part of the tests can be cleaned up in the after scenario method.

Using the Drupal Driver

In order to use the Drupal Driver you need to configure your behat.yml to nominate the api_driver. Our behat template looks like below. The ${app.url} and ${app.dir} placeholders are populated from our build.properties file using a phing build target so each developer can specify their local environment settings, such as the URL to their dev site, and the directory it lives in.

annotations:
  paths:
    features: features/annotations

closures:
  paths:
    features: features/closures

default:
  paths:
    features:  features
    bootstrap: features/bootstrap
  extensions:
    Behat\MinkExtension\Extension:
      base_url: ${app.uri}
      goutte: ~
      selenium2: ~
    Drupal\DrupalExtension\Extension:
      api_driver: "drupal"
      blackbox: ~
      drush:
        root: ${app.dir}
      drupal:
        drupal_root: ${app.dir}

drupal7:
  filters:
    tags: "@api"
  extensions:
    Drupal\DrupalExtension\Extension:
      api_driver: "drupal"
      drupal:
        drupal_root: ${app.dir}
      drush:
        root: ${app.dir}

So with that in place you can access Drupal API's in your custom step definitions.

The problem we hit

So what was our problem? Well testing manually, functionality worked just fine, but testing with Behat would net weird results. Our tests would pass fine for the first few steps, then after a particular point in the test suite, every path that used a page-manager page would fail.

The problem was easily diagnosable by adding:

And then show last response

to our steps and inspecting the page. Using this we could see that the standard node page view was being used instead of our page manager variant.

Our project scaffold includes a basic Behat feature to check that the home-page returns a 200 Ok response, for this project it too was a page manager page and even it was failing. We knew for certain that the front page wasn't a 404, but that's what Behat was telling us, and if we visited the home-page immediately after running the tests it in fact was a 404.

As soon as we cleared the cache - we got the home page back.

Diagnosing the issue

Now this is one of those times and another one of those reasons where contributing to core makes you more knowledgeable and your day-job just that little bit easier :-).

Because of our familiarity with core, we were able to diagnose the issue straight away. It was clear that something was triggering a menu rebuild and because we run Behat using phing from outside the Drupal root, we weren't getting all of the necessary include files loaded, particularly those include files that defined the default page manager variants.

Now the DrupalExtension knows this, and if you use the 'and the cache has been cleared' the step definition it provides in DrupalContext is aware of this. It does a chdir() to the Drupal root first, then resets the working directory afterwards. But it was clear that something else was triggering a menu rebuild during our test suite.

So knowing that menu_rebuild() is the function that does the rebuild and that it lives in includes/menu.inc, we opened up the file and hacked in some code to print the backtrace when the function was called. We re-ran our tests using phing and saw this trace right where things started going astray.

Array
(
    [0] => menu_rebuild
    [1] => menu_get_item
    [2] => _pathauto_path_is_callback
    [3] => _pathauto_set_alias
    [4] => pathauto_create_alias
    [5] => pathauto_user_update_alias
    [6] => pathauto_user_insert
    [7] => user_module_invoke
    [8] => user_save
    [9] => Drupal\Driver\Cores\Drupal7::userCreate
    [10] => Drupal\Driver\DrupalDriver::userCreate
    [11] => Drupal\DrupalExtension\Context\DrupalContext::assertAuthenticatedByRole
    [12] => call_user_func_array
    [13] => Behat\Behat\Definition\Annotation\Definition::run
    [14] => Behat\Behat\Tester\StepTester::executeStepDefinition
    [15] => Behat\Behat\Tester\StepTester::executeStep
    [16] => Behat\Behat\Tester\StepTester::visit
    [17] => Behat\Gherkin\Node\AbstractNode::accept
    [18] => Behat\Behat\Tester\ScenarioTester::visitStep
    [19] => Behat\Behat\Tester\OutlineTester::visitOutlineExample
    [20] => Behat\Behat\Tester\OutlineTester::visit
    [21] => Behat\Gherkin\Node\AbstractNode::accept
    [22] => Behat\Behat\Tester\FeatureTester::visit
    [23] => Behat\Gherkin\Node\AbstractNode::accept
    [24] => Behat\Behat\Console\Command\BehatCommand::runFeatures
    [25] => Behat\Behat\Console\Command\BehatCommand::execute
    [26] => Symfony\Component\Console\Command\Command::run
    [27] => Symfony\Component\Console\Application::doRunCommand
    [28] => Symfony\Component\Console\Application::doRun
    [29] => Behat\Behat\Console\BehatApplication::doRun
    [30] => Symfony\Component\Console\Application::run
)

So most of our steps start with Given I am logged in as a user with the role "foo" because we're testing functionality on the behalf of a particular user, and this in turn uses DrupalContext::createUser(), which uses user_save(), which in turn invokes hook_user_insert()

Now because our site uses pathauto, pathauto_user_insert() fires which in turn calls menu_get_item() which can't find an item for the given user object (as it's being created) so a menu rebuild occurs, and because this is outside the Drupal root, we get a partial rebuild, as a fair swag of include files aren't loaded.

Fixing the problem

So because Behat and DrupalExtension live in an Object oriented world, fixing it wasn't too difficult, our FeatureContext already extends DrupalContext, so we just reimplemented the assertAuthenticatedByRole() method, being sure to add:

'path' => array('pathauto' => 0)

to the user object passed to DrupalDriver::userCreate(). This prevents pathauto from calling menu_get_item() and thereby rebuilding the menu.

Then we fired up our tests again, and got a little bit further but hit the same issue, but with a different backtrace.

This time the trace went something like this -> DrupalDriver::createNode() -> node_save() -> invoke hook_node_insert() -> pathauto_node_insert(). I.e. the same thing again but for nodes, so we added the same code to anywhere our FeatureContext called DrupalDriver::createNode(), cleared the cache using drush and then ran it again.

Success!

Summing up

  • Behat and DrupalExtension are awesome, coupled with phing and a CI build, you can automate your testing process and run a test suite against your acceptance criteria to ensure you don't get regressions.
  • Working with the Drupal driver means you can get pretty creative with your step definitions and access the full Drupal API. But because you're running Drupal from outside the Drupal root, sometimes things can go astray.
  • If they do, having a solid knowledge of core built up from your experiences as a contributor can help you pinpoint the issues without much loss of time
  • Backtraces are your friend
  • Be sure to check out Boris Gordon's session on phing and CI at Drupalcon Prague.

Posted by Lee Rowlands
Senior Drupal Developer

Dated

Comments

Comment by moshe weitzman

Dated

Awesome tip.

Would be great to see your tests and step definitions.

Comment by Lee Rowlands

Dated

Sure thing, will cover those in a future post

Comment by SJ

Dated

Nice writeup! Thanks for sharing!

Comment by attiks

Dated

does phing has a task for behat or is this custom?

Comment by Lee Rowlands

Dated

We have our own phing build targets for behat because there are some target dependencies we run before, like cache clear, db sync, registry rebuild, updating database etc.
We have targets for behat for local dev (more verbose) and for Jenkins (output in junit format).

Pagination

Add new comment

Restricted HTML

  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd> <h2> <h3> <h4> <h5> <h6>
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.
Not sure where to start? Try typing "hello" or "help" if you get stuck.