• Introduction
• Install BuildWise Server
• Set up build project
• Test Frameworks
• Sequential Test Execution
• Set up Parallel Testing
• BuildWise Agent for Windows
• BuildWise Agent for Mac
• Parallel Test Execution
• Build Control
• Reporting
• Administration
• Utility

Parallel Test Execution

• View error screenshot
• Create To-test-list
• Priority - dynamic ordering
• Auto rerun - reduce false alarms

View error screenshot

BuildWise agents take screenshot of the browser when an error occurs, which might help analyze the failure quickly. Click ‘Failure’ link.

Click ‘Screenshot’ tab to view.

Create To-test-list

When a UI test step starts, it will first lists all the to-be-executed test scripts files.

By default, all test script files under a folder (e.g. spec) will be included as the regression suite. This is set in Rakefile:

desc "Running tests distributedly"
task "ci:ui_tests:full" => ["ci:setup:rspec"] do
  build_id = buildwise_start_build(:project_name => BUILDWISE_FULL_PROJECT_ID,
                                   :working_dir => File.expand_path(File.dirname(__FILE__)),
                                   :ui_test_dir => ["spec"],
                                   :excluded => excluded_spec_files || [],
                                   :distributed => true
  )
  #...
end

Priority - dynamic ordering

With every test script file (in distributed build), you can see a priority number (shown in last column). This determines the execution order. In other words, tests with higher priority will run before the ones with lower priority.

The priority is auto set by BuildWise based on two factors:

1. Recent build results of each test script file

   The failure of most recent build will get high weighting.

2. Average execution time

   For tests about the same priority, the tests took longer time will get slight higher weighting.

You can also override the default order, by simply clicking the priority number and set a new value.

The execution order changed.

Auto rerun - reduce false alarms

With parallel test execution using BuildWise agents, BuildWise supports auto-rerun, that is, if one test failed on one build agent, server will give it another go by reassigning the test to another build agent. The auto-rerun can be turne on / off in project settings.

Here is a re-run example. The test failed on Agent Win7-BWA-06, then rerun it on Win7-BWA-04 and it still failed.

The below is rerun successful case, which means, we saved a false alarm.