• Introduction
• Install and Setup
• Get started
• Work with project
• Test Execution
• Navigation
• Productivity
• Refactoring
• Debugging Tests

Debugging Tests

Debugging test scripts is quite different from debugging code, so are the technicques.

• Keep browser open after one test execution
• Analyse test error ouput
• Run selected steps against current browser
• Write to console

Keep browser open after one test execution

When a functional test execution fails, the first instinct of the testers is to check the current state of the application (browser if the target application is a web app). However, we cannot leave the browser (using webapp as an example for illustration, the same logic appliies to desktop and mobile apps) open for every test exectuion, as there will be too many browser windows. So the dilemma is: we want to close browser windows by default, but keep the browser open when debugging test scripts.

The solution in TestWise is simple: when an individual test case is run, TestWise will provide a mechanism to leave the browser open. Running an individual test case (as shown below) means running a single test case in a test script file which may contain many test cases. Please pay attention to the differences of execution icon, context menu text and execution result panel.

 

If your test scripts follow the convention (described below), the browser will remain open after the test execution finishes. How? The magic is uneless debugging? in after(:all) execution hook.

after(:all) do
  driver.quit unless debugging?
end

debugging? is a function defined in test_helper.rb. It checks two environment variables (RUN_IN_TESTWISE and TESTWISE_RUNNING_AS) which will be set accordingly by TestWise based on user’s choice of execution mode.

def debugging?
  return ENV["RUN_IN_TESTWISE"].to_s == "true" && ENV["TESTWISE_RUNNING_AS"] == "test_case"
end

The above code is in Ruby/RSpec syntax. The technique is applicable to most frameworks though.

Analyse test error ouput

When a test execution failed in TestWise, execution results (with timings) is displayed in the execution pane at the bottom.

 

A red dot indicator will be shown in the editor to highlight the line where the failure is. If there are multiple erors (a test script may contain many test cases), click icons to navigate to the error lines quickly.

Most of the time, the reason of failure is not as obvious as the above one. To analyse, click Test Output tab to view the test output, espeically the error stack trace, which is very useful for identify the cause of the failure.

 

You can click the highlighted file_name:line to navigate to the test statement quickly.

Run selected steps against current browser

This is the most liked feature by TestWise users. During the development (and debugging) of an automated funtional test script, we often need to try many times before get it correct and stable. For example, when we use Selenium to test a web app, a tester might try different locators to identify one element on a web page. If the locator was not correct, a tester needs to re-run the whole test script to verify. This is time consuming, frustrating and inefficient. How likely will a tester keep refining test script until getting a reliable locator, if every single try needs waiting for a 5-minute execution? Quite often, test automation attempts are failed for this reason.

TestWise, from v1, has a feature: run selected test steps against current browser. This idea was from Watir, the framework supports attach browser. However, Selenium WebDriver does not have this feature. Due to its great benefits (easily saving hours of work for each programmer/tester at AgileWay projects), we implemented the feature to Selenium WebDriver.

1. Run one individual test case

    

2. The browser remains open

   TestWise will keep browser open in this execution mode. Don’t close that browser window.

    
    

3. Select test steps and ‘Run Selected Scripts Against Current Browser’

    

   These selected test steps will be included in a newly created test script file debugging_spec.rb and get executed.

    

   As expected, it failed (because we haven’t changed anything). However, this is different from the last execution.
    

   • Only the selected test steps are executed, not all in the original test script.
   • In a speical test script file
   • The execution is against the last opened browser window.

    

4. Modify the special test script and run it again

    

   In above case, the correction of name attributed fixed the test. You can now copy this correct test step back to the original test script file and continue the work.

Please note, in this special debugging test script file (debugging_spec.rb), once it ‘connects’ the last running browser window, any arbitrary test steps can be tried (adding new, for example) in the test script, and run as many times as you want.

Write to console

This is the most basic but useful debugging technique: printing the text out. Here is an example: the test scripts prints out the ‘booking number’ on a receipt page, using standard puts in Ruby. The output shows in TestWise’s console.

 

The printed out text will be shown in stdout as well