Set up Parallel Testing

Distribute tests to multiple build agents to run them in parallel.

Build agent machine

In the context of testing web apps, a build agent is considered as a computer with browser, which is driven by test scripts to verify the app.

Build agent machines are be physical or virtual machines, as long as they can communicate to the BuildWise server and the target web sites.

  • Virtual Machines in computers on LAN

    Use virtualization software such as VMware, Oracle VM VirtualBox or Parallel Desktop for Mac. Virtual Box is free.
    You can download free 90-day VM images from Microsoft.

  • Cloud instances

    AWS, AZure and Vultr all offer hosting Windows Server 2016 instances. The prices are all quite reasonable, staring about $50 per month.

  • Physical machines on LAN

    Reuse existing physical machines.

Once the machine is set up, install Git and Chrome (and other browsers if necessary).

Verify connection

Verify that you can visit

  • BuildWise server
  • Your test server (the application under testing)

in your browser.

Install BuildWise agent

A BuildWise agent is installable software that runs test jobs, perform the following steps repeatedly:

  1. Check the server whether there is an active build? If yes,
  2. Get the test script from the server
  3. Run the test script
  4. Return test results to the server.

BuildWise Agent are available on Windows and Mac OS X. As BuildWise Server is 100% open-source (in Ruby), software companies can develop own agents that can work with BuildWise server.

Install BuildWise agent is simple, just download and run the installer, accepting all default options.

Starting with 3 agents

More agents will speed up test executions, but also require more infrastructure maintenance effort.

If you are not sure, we recommend starting with 1 BuildWise Server + 3 Virtual Box VMs in one computer. A typical laptop nowadays with 16GB memory can comfortably run them, and you probably can get it all set up within hours, no need seeking permissions.

General Settings

Here is an example:

  • BuildWise Server URL

    The URL of the buildwise server, e.g. http://buildwise.server. Click ‘Visit’ link to verify the connection.

  • Agent Name

    The unique agent name (set by license, with exception of Site License) to the Server.

  • The wait period (in seconds) to check active builds or next tests

  • Preferred browser for running tests in

    The support of this depends on the test scripts. BuildWise Agent will pass an environment variable BROWSER to test execution, it is up to test scripts to take advantage that.

  • Test results

    The directory where BuildWise Agents will save test results (and screenshots in case failures) to. The ‘Archive test results’ tell Agent to retain the history of test results (in timestamped directory name).

  • Start Agent on Launch checkbox

    The agent process (talking to the server) is not started by default, to prevent confusion in case the agent machine is not ready. The user can click the ‘Start’ button under the Dashboard tab to start it.

    With this option is checked, BuildWise Agent will automatically starts the agent process on start.

Configure BuildWise agent for project

To make an agent to run tests for one project, two settings need to be in place:

  • The test project checked out (from Git) on the Build agent machine

    This only needs to be done once. The directory needs to match the one set in BuildWise Project settings.

    In above example, for application WhenWise, its working directory on agents can be C:/work/whenwise (Windows) or /Users/zhimin/work/projects/whenwise (Mac).

    It is important that the parent repository of checked out project source on BuildWise Server are the same as the ones on build agents.

  • Environment variables such as test server URL, …, etc

    Special environment variables set set in Agent to pass to test execution. For example, we commonly use BASE_URL to set target server URL, so that Agent 1 -> Server 1, Agent 2 -> Server 2, …, to achive parallelism. Of course, you can set multiple agents with the same URL, this can serve as a form of load testing as well, but the test scripts need to handle side effects (test exeuction need to independent).

    Below is an example of use BASE_URL in Ruby scripts, quite simple, right? 

    def site_url(default = $BASE_URL)
   ENV['BASE_URL'] || default
end

#...
driver.get(site_url)

    Please note that environment variable has to be associated to an application name, so that Agent can support execution of multiple projects. Here is a sample configuration.

Configure BuildWise agent for pre-execution

Due to the nature of UI test automation, test execution can be suspended, or unepxected siturations (such as browser hangs) can happen. This can affect next build. So BuildWise run a set of pre-test-execution commands, it is configured as below:

Start up BuildWise Agent on boot

You configure your machine to automatically run BuildWise Agent at startup.

© 2006 - 2024 AgileWay Pty Ltd. Powered by SiteWise CMS