Embracing Capybara

Editor's Notes

• #2: I’m going to talk about Embracing Capybara: how to understand Capybara, train it to do new tricks, and discipline it when it misbehaves. First, show of hands…
• #3: OK keep them up…
• #4: Sometimes it just doesn’t do what you want it to do.
• #5: I work at Lonely Planet, on a fairly complex single-page app, and we use Capybara for our tests.
• #7: In fact, we use a lot of Capybaras.
• #8: Like, a whole herd of Capybaras running together in parallel.
• #9: We, too, have been bitten by our Capybaras. We got through it, learned a lot, and now we can’t imagine living without our Capybara.
• #10: Capybaras are the largest rodents in the world. They are related to guinea pigs, but are the size of a big dog or small pig. They live in South America, and they’re pretty damn cute. Capybara is also the name of a Ruby gem written by this guy, Jonas Nicklas. It helps you write integration tests for web apps by pretending to be a user with a web browser. It takes a black box approach to testing, letting you go to URLs, click links and buttons, fill in forms, and check that the rendered page contains what you think it should have in it. It doesn’t replace test runners like RSpec, Cucumber and Minitest, it works with them. It has built-in integration with the most popular testing frameworks, but is test-framework agnostic.
• #11: Three parts: Front end is a Ruby-based DSL for browsing the web: actions like visiting URLs, clicking links and buttons, interacting with forms, and querying for contents of the page. Powering this is a flexible selector engine: supports CSS selectors, XPath, finding by content or form input labels. On the back end, Capybara has pluggable drivers for various browsing engines. Out of the box, it supports RackTest and Selenium WebDriver. RackTest is a headless, pure Ruby browser simulator. Very fast, no setup required, works everywhere, but doesn’t support JavaScript execution. Selenium drives a real web browser: Firefox, Chrome, IE, etc. Executes JavaScript. Great for compatibility testing and JavaScript-heavy apps, but slow to run, and asynchronous execution can be difficult to understand. Other options via gems: e.g., Poltergeist integrates with PhantomJS, a headless WebKit implementation. Most people use Selenium WebDriver. Why not code directly against the WebDriver API? Capybara is more Ruby-like, more flexible, and makes it easier to handle asynchrony.
• #13: Part of what makes Capybara so appealing is that it feels like magic. This is also what can make it frustrating. To work effectively with Capybara, you need to learn to think like a Capybara.
• #14: Capybara automatically waits for asynchronous operations to complete. When you try to find an element that isn't on the page, it waits and retries until it is there, or a timeout duration elapses. Default is 2 seconds. That's pretty short, so if the sign-in is slow, you might get timeout errors.
• #15: You can wrap a block in 'using_wait_time' to give it a longer timeout duration.
• #16: In Capybara 2.1, you can also pass a 'wait' option to individual methods.
• #17: Methods that unambiguously identify a fixed number of elements. This isn't fully comprehensive, but most methods fall into one of these categories.
• #18: Anything where Capybara can't guess how many elements you’re expecting, or what they are.
• #19: This test may fail when it should pass.
• #20: We can tell Capybara how many results we're expecting, and it will wait for it to become true, or time out.
• #21: When something is supposed to be removed from a page after an action, Capybara needs to wait for that, too.
• #22: If we tell Capybara that we're expecting the text not to be there, it will wait for that to be true.
• #23: Capybara's RSpec matchers are smart, so this works too. It knows whether you said expect/to (or should) or expect/not_to (or should_not).
• #24: We’ve seen examples of simple Capybara code written in an imperative, script-like style, but in big apps, it can get unwieldy. Luckily, Capybara speaks Ruby, so we can use all of the tools of the language—classes, inheritance, delegation, blocks—to teach Capybara abstractions and encapsulate the details of our application.
• #25: This Cucumber scenario is very tightly coupled to the page it is testing. It’s ugly enough here, but in a big app this style of Cucumber is much worse, with URL paths and CSS selectors duplicated everywhere. You see this a lot in apps that used an older version of the cucumber-rails gem. It used to generate a file of default Cucumber step definitions that looked just like this.
• #26: Jonas, the author of Capybara, wrote a blog post called “You’re Cuking it Wrong” explaining why this is bad practice. Aslak Hellesøy, the author of Cucumber, has since disavowed web_steps.rb, and it is no longer created by default.
• #27: Here, the underlying URL and page markup details have been encapsulated within step definitions. This is much more readable for non-technical stakeholders. Still, it is very coupled to specific UI implementation details.
• #28: Now the Cucumber test concisely describes the business requirements.
• #29: You may be tempted to simply move the tightly-coupled test code into the step definition, but this is hardly better. It is still boring and brittle.
• #30: We can refactor to page objects that model the UI of the application, and create an object-oriented testing API that can be reused across step definitions.
• #31: The application tester object can be added to the Cucumber world. It encapsulates access to the Capybara session, and provides methods that represent pages in the app, or services it provides.
• #32: Each page encapsulates markup details by providing logical methods for the elements on the page. These return Capybara Node objects, which support methods such as click, fill_in, finding descendent elements, etc. The entire Capybara DSL is available, scoped within that node. For more complex elements, you can encapsulate them in their own page objects that provide higher-level APIs. For example, a date picker could be wrapped in an object that lets you set a date or choose “Today”.
• #33: Let’s see how that’s used again.
• #35: Long selectors like this are extremely brittle. There's no shame in adding CSS IDs and classes to your markup for your tests. Difficulty writing tests can be a code smell: clean, semantic markup makes for easier testing. Try (A)TDD. Page Objects help here, too. Navigating through a chain of page elements gives useful errors and stack traces.
• #36: Sometimes, despite your best efforts, Capybara misbehaves. Here are some tips you can use for getting it back in line.
• #37: Add this to your Cucumber step definitions. If a step is failing, throw “And I debug” right before it and watch what happens in the browser. If it looks like it’s working, continue in the debugger. If it passes, you probably have a synchronisation issue, where the step isn’t waiting for the operation to complete.
• #38: Capybara can save screenshots. This is a simple way to save a screenshot after each scenario. You can get more clever with this, such as only saving after failures. Our code is pretty complex, but there is also a gem that makes it automatic.
• #39: One annoying thing with Firefox: sometimes tests fail when it doesn’t have focus. Firefox doesn’t fire certain events when it’s in the background. focusmanager.testmode is a hidden Firefox configuration that disables this. Google for it and you’ll find examples of how to set it. New versions of Firefox often break WebDriver, so keep that gem up to date using “bundle update selenium-webdriver” often. Also, disable Firefox’s auto-update in CI. Linux handles events differently than Mac OS and Windows. Keep that in mind when diagnosing failures in CI. Make sure your xvnc or xvfb window is large enough to display the page properly. We had to configure the default geometry in Jenkins. You’ll get a clean profile in Firefox with no add-ons installed by default. Capybara Firebug is a gem that automatically installs and activates Firebug into Firefox when Capybara runs. Really useful, but slows down your tests. Use a Cucumber profile to activate it.
• #40: The usual resources apply.
• #42: The most important thing to do: prioritise fixing flaky tests as a team. Flaky tests are ones that sometimes pass, sometimes fail, seemingly at random. It’s all too easy to get into a situation where one flaky test becomes many, your build is always red, and you start to ignore failures. Your tests lose their meaning, and then one day you ship a bug that the tests would have caught. That really stinks.