Run Rails Test Suite with Dockerized Selenium on Gitlab CI

2018-04-10 in Development

This post was originally written for Neoteric Design.

With just 3 drop-in tweaks, it’s possible to run Rails’ System Tests on Gitlab CI, or other Docker-based continuous integration services.

We’re a small team that, between active development and legacy support, has dozens of projects to pull from that might need to be worked on. While the backbone of most projects is the familiar components of our CMS, each project we work on is tailored to the needs of the client and we can’t rely on our internal knowledge alone. So, each project ships with a robust test suite, featuring a healthy mix of unit and integration tests.

We use RSpec because we find the DSL allows us to write code that is close enough to the English business logic from the client. Starting with Rails 5.1 we migrated fully to Rails’ System Tests for integration testing. It gives us a lot of goodies, like database transaction cleanup, and automatic screenshots of failures.

While the Docker executor makes a lot of sense for the CI environment, we typically don’t need it for dev or production. We still need to be able to run our tests locally, so we strive to make things work in both places, with minimal configuration changes. We use selenium-webdriver with chromedriver to execute tests that involve javascript locally, but installing chrome and chromedriver on our runner each time would be slow and painful. Thankfully, Selenium provides Docker images that are ready to go, and Gitlab Runner’s Docker support lets just just add it as another networked service alongside the database.

Let’s walk through an annotated copy of our .gitlab-ci.yml configuration to see all it’s doing for us.

# .gitlab-ci.yml

image: "ruby:2.5.1"

# Add Postgres and Selenium Docker services. A Firefox container is also available.
services:
  - postgres:latest
  - selenium/standalone-chrome:latest

# Set our environment variables. Note it's supplying where Gitlab's
# Docker service will make Selenium available at. The Postgres
# config matches a database.yml.gitlab we'll add later.
variables:
  POSTGRES_DB: "test_db"
  POSTGRES_USER: "runner"
  POSTGRES_PASSWORD: ""
  RAILS_ENV: "test"
  SELENIUM_URL: "http://selenium__standalone-chrome:4444/wd/hub"

# Cache gems in between builds. We use the project path slug
# as the key because one cache per project works well enough
# for us
cache:
  key: ${CI_PROJECT_PATH_SLUG}
  paths:
    - vendor/ruby

# Setup shell commands. We need nodejs for asset compilation,
# And libgmp for the bcrypt gem. Then we override the database
# configuration with our Gitlab configuration. Now, we can
# install our gem dependences, bundle to the vendor folder so we # can cache them, and finally prep the database.
before_script:
  - apt-get update -q && apt-get install nodejs libgmp-dev -yqq
  - cp config/database.yml.gitlab config/database.yml
  - gem install bundler rubocop --no-ri --no-rdoc
  - bundle install -j $(nproc) --path vendor
  - rails db:schema:load

# We have two jobs, first, we lint the project with Rubocop to
# keep us honest and clean
rubocop:
  script:
  - rubocop

# Then we run our RSpec suite. When we run tests against
# Selenium, Rails will save screenshots of failures. We capture
# Them as artifacts so we can grab them through Gitlab's UI
# later.
rspec:
  script:
  - rspec spec
  artifacts:
    when: on_failure
    expire_in: 1 week
    paths:
    - tmp/screenshots/
    - log/

And here’s the database config for running on CI. The CI is always a one-off, isolated environment, so we don’t need the typical project specific naming. So, we use generic names here so we can recycle this file as-is in any project. CI having it’s own clearly defined database configuration is one concession that’s proven worth it, especially when we can make it as easy as dropping in this one file.

# config/database.yml.gitlab
test:
  adapter: postgresql
  encoding: unicode
  pool: 5
  timeout: 5000
  host: postgres
  username: runner
  password: ""
  database: test_db

In our RSpec configuration, the only change we have to make is to check if we’re and use that instead of loading a local chromedriver. When using a remote Selenium we do need to tell Capybara where our app is for Selenium to connect to. We set that through the host! method provided by Rails’ SystemTestRunner. This was the key bit not covered by Gitlab’s examples or other articles we read on the topic.

# spec/support/capybara.rb

# Hide Puma start up notification
Capybara.server = :puma, { Silent: true }

RSpec.configure do |config|
  config.before(:each, type: :system) do
    driven_by :rack_test
  end

  config.before(:each, type: :system, js: true) do
    if ENV["SELENIUM_URL"].present?

        # Make the test app listen to outside requests, for the remote Selenium instance.
            Capybara.server_host = '0.0.0.0'

      # Specify the driver
      driven_by :selenium, using: :chrome, screen_size: [1400, 2000], options: { url: ENV["SELENIUM_URL"] }

      # Get the application container's IP
      ip = Socket.ip_address_list.detect { |addr| addr.ipv4_private? }.ip_address

      # Use the IP instead of localhost so Capybara knows where to direct Selenium
      host! "http://#{ip}:#{Capybara.server_port}"
    else
        # Otherwise, use the local machine's chromedriver
      driven_by :selenium_chrome_headless
    end
  end
end

When the app needs to make network requests, say to an external API, we use Webmock to stub those requests in testing. In strict mode, Webmock will raise an error if a request isn’t whitelisted or in your provided stubs. Capybara needs to be able to talk to the remote selenium, thankfully it’s easy to add it to the whitelist.

# spec/support/webmock.rb

WebMock.disable_net_connect!(allow: ['localhost', '127.0.0.1', /selenium/])

As a side note, breaking up our RSpec configuration into these initializer-style, single-purpose, drop-in configuration files makes juggling all these test suites much easier!

With this setup, we’ve been able to simply drop in these 3 files (.gitlab-ci.yml, database.yml.gitlab, and the updated capybara.rb) to several Rails 5.1+ projects and have a working CI pipeline.

rails

testing

ci/cd