Creating a Selenium grid

Its real power shows when it is used in a grid, where multiple browsers on multiple OSs can be run simultaneously — this is a must-have setup for every QA and front-end developer team. Here, I’ll try my best to show how to set up such a grid with a usage example.

Structure

A grid consists of a hub and one or more nodes. The hub, as the name suggests, is a central point, where all the work is managed. This is also a component which clients willing to do some web tests connect to. You may see the name “Selenium Server” — this is a capplication that can, depending on the parameters given, serve as a hub or a node.

A node in fact comprises a Selenium client, browser driver and the actual browser. The Selenium client is the aforementioned “Selenium Server” application (but run in client mode, blame Selenium developers for the confusion) with a configuration stored in a JSON-formatted file. The browser driver is browser-dependent: for Firefox it’s called GeckoDriver (formerly known as wires), for Chrome/ Chromium it is ChromeDriver. Note on Firefox: GeckoDriver is a newer approach to browser automation. Underneath, it uses Marionette, Firefox’s own automation protocol, which is not directly compatible with Selenium, thus Gecko driver is required. You may see some older articles on the web telling you to install an XPI extension — this is no longer the case.

An important thing to mention is the fact that the browsers on nodes are running headless. To achieve that, I will be using wrappers – simple shell scripts that add extra parameters. They may also be used to force other parameters, like setting window size.

Read also: The Future Of Frontend Framework

Installation

The installation steps shown below were done on Debian Linux.
Install Java first. I find it very hard to say which version would be the best, but Java 11 LTS should do the job. For my guide, I will be using OpenJDK 1.8. Make sure you are a superuser before performing the steps below.

Preparation

1. Create a system account under which all the applications will run:

# useradd -d /opt/selenium -m -k /dev/null -r -s /usr/sbin/nologin selenium

2. Download the latest Selenium Server JAR (v3.141.59 – yes, that’s pi. No, it’s not a Raspberry) and put it in
   /opt/selenium/selenium-server-standalone-3.141.59.jar
3. Install a few packages required by headless browsers:

# apt-get install xvfb libgtk-3-0 libdbus-glib-1-2 libxss1 libasound2

Installing the hub:

4. create a systemd service in /etc/systemd/system/selenium-hub.service:

[Unit]
Description=Selenium Hub
Wants=network.target
[Service]
User=selenium
Group=selenium
SuccessExitStatus=143
ExecStart=/usr/bin/java -jar /opt/selenium/selenium-server-standalone-3.141.59.jar -role hub
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target

5. enable and start the service:

# systemctl enable selenium-hub.service
# systemctl start selenium-hub.service

Installing a node – Firefox

6. Download Firefox in a .tar.gz format and unpack it to /opt/firefox-x.y.z/ where x.y.z is its version (v69.0.3 in my case) — you may download multiple versions and create a node for each one.
7. Download GeckoDriver and unpack it to /opt/gecko-driver-x.y.z/ — the GeckoDriver version has nothing to do with the Firefox version, currently its version is v0.26.0 and it supports the current and previous Firefox releases.
8. Create a wrapper file that will start Firefox in headless mode. Other settings can be added here, like proxy, window size, etc. Put it in /opt/firefox-x.y.z/firefox-headless:

#!/bin/bash
/opt/firefox-69.0.3/firefox --headless "$@"

9. Create a configuration for this node in /etc/selenium/node-firefox-69.0.3.json:

{
    "capabilities":[
        {
            "browserName":"firefox",
            "maxInstances":2,
            "platform":"LINUX",
            "version":"69.0.3",
            "firefox_binary":"/opt/firefox-69.0.3/firefox-headless"
        }
    ],
    "maxSession":5,
    "register":true,
    "registerCycle":5000,
    "nodeStatusCheckTimeout":5000,
    "nodePolling":5000,
    "role":"node",
    "unregisterIfStillDownAfter":60000
}

10. Create a systemd service in /etc/systemd/system/selenium-node-firefox-69.0.3.service

[Unit]
Description=Selenium Node
Wants=network.target selenium-hub.service
After=selenium-hub.service
[Service]
User=selenium
Group=selenium
SuccessExitStatus=143
ExecStart=/usr/bin/java -Dwebdriver.gecko.driver=/opt/gecko-driver-0.26.0/geckodriver -jar /opt/selenium/selenium-server-standalone-3.141.59.jar -role node -nodeConfig /etc/selenium/node-firefox.json -hub http://localhost:4444/grid/register
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target

11. Enable and start the service:

# systemctl enable selenium-node-firefox-69.0.3.service
# systemctl start selenium-node-firefox-69.0.3.service

12. If all went well you may open a web console: http://webtests:4444/grid/console (where “webtests” is the name of your host) and see that it has a browser node connected.

Installing a node – Chrome

13. Download a stable build in .deb format and unpack it. We only need a part of that package – the actual browser – unpacked to /opt:

# ar xv google-chrome-stable_current_amd64.deb
# tar -xvf data.tar.xz --strip 3 -C /opt "./opt/google"
# mv /opt/chrome /opt/chrome-77.0.3865.120

14. Download ChromeDriver specific to your version of Chrome – for Chrome 77 the ChromeDriver version is 77.0.3865.40 – and unpack it to /opt/chromedriver-77.0.3865.40/chromedriver
15. Create a wrapper file that will start Chrome in headless mode. Other settings can be added here, like proxy, window size, etc. Put it in /opt/chrome-77.0.3865.120/chrome-headless:

#!/bin/bash
/opt/chrome-77.0.3865.120/google-chrome --headless "$@"

16. Create a configuration for this node in /etc/selenium/node-chrome-77.0.3865.120.json:

{
    "capabilities":[
        {
            "browserName":"chrome",
            "maxInstances":2,
            "platform":"LINUX",
            "version":"77.0.3865.120",
            "chrome_binary":"/opt/chrome-77.0.3865.120/chrome-headless"
        }
    ],
    "maxSession":5,
    "register":true,
    "registerCycle":5000,
    "nodeStatusCheckTimeout":5000,
    "nodePolling":5000,
    "role":"node",
    "unregisterIfStillDownAfter":60000
}

17. Create a systemd service in /etc/systemd/system/selenium-node-chrome-77.0.3865.120.service:

[Unit]
Description=Selenium Node
Wants=network.target selenium-hub.service
After=selenium-hub.service
[Service]
User=selenium
Group=selenium
SuccessExitStatus=143
ExecStart=/usr/bin/java -Dwebdriver.chrome.driver=/opt/chromedriver-77.0.3865.40/chromedriver -jar /opt/selenium/selenium-server-standalone-3.141.59.jar -role node -nodeConfig /etc/selenium/node-chrome-77.0.3865.120.json -hub http://localhost:4444/grid/register
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target

18. Enable and start the service:

# systemctl enable selenium-node-chrome-77.0.3865.120.service
# systemctl start selenium-node-chrome-77.0.3865.120.service

19. If all went well you may open a web console: http://webtests:4444/grid/console (where “webtests” is the name of your host) and see that it has both browser nodes connected.

Read also: What is PHP used for? How can we use it in a project?

Usage

Below is a very simple Java snippet utilizing our newly created grid. In order to run it, include the org.seleniumhq.selenium:selenium-serverlibrary:

Capabilities capabilities = new FirefoxOptions();
// Capabilities capabilities = new ChromeOptions();
RemoteWebDriver driver = new RemoteWebDriver(new URL("http://webtests:4444/wd/hub"), capabilities);
driver.get("http://www.example.com/");
System.out.println("Page title is: " + driver.getTitle());
driver.quit();