False positive virus alert on TestWise.exe

By mark.

We use free "Bat to Exe Converter"" to create an exe file (with icon) to launch TestWise. Unfortunately, a few Anti-Virus software reported false positive on it, please be assured that TestWise is virus free. Virus Total, an online virus scanning service, reports :

"5 out of 41" Ant-Virus software detect TestWise positive, they are Comodo, Jiangmin, McAfee+Artemis, McAfee-GW-Edition and TrendMicro.

If you happen to have the annoyance, you can try to start TestWise with batch file included.

If you are interested in why exe file generated by "Bat to Exe Converter" may cause false virus alert, please read this.

Load testing with RWebSpec

By mark.
This is one of most asked questions on iTest: "Can it do load testing?". As more projects getting to acknowledge "Automated UI (functional) testing is not possible":http://blog.objectmentor.com/articles/2009/09/29/ruining-your-test-automation-strategy, Load testing gradually is becoming the replacement of 'Automated Testing', which is not right of course. Users of iTest will disagree this permissive view of functional testing, but that's not the topic today. Back to the question, our answer is "iTest is a functional test tool, but the functional test scripts created may be used for load testing, and work has started on creating a load testing IDE". What's special about it?
  1. Reuse functional test scripts

    Different from most current load testing scripts (which work on HTTP request levels), our loading test scripts use same syntax of functional test scripts: RWebSpec, which is known to be very readable. Furthermore, you can reuse the functional test scripts for load tests with only slight modifications.

  2. See load test scripts in action in Firefox browser

    Although execution of load tests is in headless mode, ie, seeing test running in a browser during development greatly improve the experience and confidence. Load tests with RWebSpec just allow you that.

  3. Can verify page contents just as functional testing.
The IDE is still under development, but you can try load tests using RWebSpec now (from command line). You need Java 1.6 installed first, see ReadMe.txt included 1. Download iLoad2 v0.1 here 2. Unzip to C:\ to make application folder C:\iLoad2-01 3. Start a command window, enter the following command to add application to PATH.
C:\>cd iLoad2-0.1
C:\iLoad2-0.1>SET PATH=C:\iLoad2-0.1\bin;%PATH%
4. Preview load testing run in Firefox browser ("install JSSH plugin":http://TestWise.com/documentation/firefox-support first)
spec -fs samples\agile_travel\deny_access_preview_rwebspec.rb
5. Run load tests with a number of virtual users.
spec -fs samples\agile_travel\deny_access_rwebspec.rb
The output shows how long does it take for each virtual user:
  Thread[1] 6.861s
  Thread[3] 7.455s
  Thread[2] 7.674s
  Thread[4] 9.456s
Now let's example the actual load test scripts:
run_with_virtual_users(4) {
  browser = open_browser("http://travel.agileway.net", {:resynchronize : false})
  home_page = HomePage.new(browser) # We cam reuse page objects!
  assert browser.text.include?("You did not provide any details for authentication.")

Verify database records in functional testing

By mark.
Function testing, in a certain way, is verifying the data (retrieved or stored at the database) through the user interface layer. Sometimes, it might be easier to verify database records directly, some SQL skills are required though. RWebSpec provides an easy way to do so, in a easy 3 steps. # Connect to the database # Choose the table # Use easy syntax or SQL to do verification
 story "Check Mysql database" do
    connect_to_database mysql_db(:host : "localhost", :database : "lavabuild_local", :user : "root", :password : ""), true
    load_table("projects") # now can use Project.xxx for easy-to-user datbase query on this table

    Project.count.should == 1  # record count
    Project.first.name.should == "BuildMonitor"
    Project.last.name.should == "BuildMonitor"
    Project.exists?(:name : "BuildMonitor")
Check out the _verify_database_spec.rb_ in demo project.

Testing complex sites

By mark.
We are not living in a perfect world, some web sites are created without automated testing in mind. So we have some sites with complex javascripts, elements without IDs or duplicated IDs. . Here is one, the test scripts generated from TestWise/Watir recorder won't work.
  scenario "Site with complex JavaScripts: Recorded" do
    select_option("intFrom", "Brisbane")
    select_option("intDepMonthYear", "Jan 2010")
    select_option("intDepDay", "Sun 03")
    select_option("intRetMonthYear", "Feb 2010")
    select_option("intRetDay", "Wed 03")
    select_option("intAdults", "2")
1. Line 3: _click_link_with_id("toSYD")_ failed with error 'Unable to locate element, using :id, "toSYD"' *Reason:* a javascript window pops up as user enters text in destination text field, however that event is not fired when running the test scripts. *Solution:* Click the image to bring up the javascript popup window manually.
   image(:id, "toBoxPlusSign").click 
# Line 10: _click_button("Go")_ had no effect *Reason:* There are more than one button with caption 'Go' on the page
*Solution:* Click the specific button by using :index
    button(:text : "Go", :index : 2).click # click 2nd 'Go' button
Here is the complete test script:
  scenario "Site with complex JavaScripts:  Working version" do
    select_option("intFrom", "Brisbane")
    image(:id, "toBoxPlusSign").click # bring up the popup    
    select_option("intDepMonthYear", "Jan 2010")
    select_option("intDepDay", "Sun 03")
    select_option("intRetMonthYear", "Feb 2010")
    select_option("intRetDay", "Wed 03")
    select_option("intAdults", "2")
    button(:text : "Go", :index : 2).click # click 2nd 'Go' button

Work with frames

By mark.
TestWise Recorder does not support operations in frames yet. The work around is simple: Ues Watir syntax for operations in frames. Let's walk through with an example. Here is a sample page with an iframe embedded, its html source fragment of frames as below:

Data-Driven Testing in TestWise

By mark.

Data-Driven Testing means tests' input and output are driven from external sources, quite commonly in Excel or CSV files. For example, if you have a list of user logins with different roles, and login process is the same but with different assertions. In this case, you can extract the pure test data out in an Excel spreadsheet, use the same test script to load test data and execute one by one. Because RWebSpec/Watir tests are in fact Ruby scripts, it is quite easy to do so.

For impatient readers, you can see and run the sample by opening the demo project C:\Program Files\TestWise\samples\demo\demo.tpr in iTest, run test script file: database_spec.rb.

The Excel file: C:\Program Files\TestWise\samples\demo\testdata\users.xls contains 3 username-password combination.

Valid Login agileway agileway Login successful!
User name not exists nonexists smartass Login is not valid
Password not match agileway badpass Password is not valid

Test Script

require 'rwebspec'
require 'spreadsheet'

spec "Use Excel for data driven web tests" do
  include RWebSpec::RSpecHelper

  before(:all) do

    # Load Excel file
    excel_file = File.join(File.dirname(__FILE__), "testdata", "users.xls")
    excel_book = Spreadsheet.open excel_file
    @excel_sheet1 = excel_book.worksheet "Users" # or use 0 for first sheet

  before(:each) do

  after(:all) do
    close_browser unless debugging?

  scenario "Load user list from an Excel spreadsheet to test multiple user logins" do
    # Iterate each row in the spreadsheet, use data for test scripts
    @excel_sheet1.each_with_index do |row, idx|
      next if idx == 0 # ignore first row
      login, password, expected_text = row[1], row[2], row[3]
      enter_text("userName", login)
      enter_text("password", password)
      click_button("Sign In")
      page_text.should include(expected_text)
      failsafe{ click_link("SIGN-OFF") } # if logged in OK, try log out


New feature: Script Library

By mark.

‘Script Library’ (a collection of searchable test scripts) provides a convenient way to learn writing RWebSpec or Watir test scripts. To use it, click ‘View’ -> ‘Script Library’ (or clicking the ‘Library’ tool on toolbar in v1.6.5)

The script library will be shown on the right (as below).

You can view all or browse by categories (RWebSpec, Watir and assertions), also can search by text description. Once an entry is selected, the test script is below, clicking ‘Insert’ button (or double clicking it in upcoming v1.6.5) will insert it into current editor (at the current caret).

New feature: Run to this line

By mark.

‘Running test up to specific line’ (or ‘breakpoint’ alike) is one of the most requested feature, it is now available in v1.6.4. Right mouse click on a test statement in editor, then select ‘Run to this line’. TestWise will execute this test case just as ‘running an individual test case’ except the execution will be paused on the line. (If selected line is an assertion statement, TestWise will choose next available operation line, such as click_link, enter_text)

You can resume the test execution by clicking ‘Resume’ button.

TestWise Tips - Test Report

By mark.

After (or during) test execution, click the Excel icon

to get test report in Excel

For formal reporting, it is best to be generated as a part of continuous integration.