The election campaign is reaching the climax in Australia (election day on this Saturday). Last night, one voter asked Prime Minster a question of impact of troubled QLD health payroll system on upcoming E-Health system. In Queensland, this yet another IT project failure attracts a lot political attention, just have a look at news headlines below:

• Qld Govt blames IBM for health payroll bungle
• Qld may kill IBM payroll contract, seek damages Health payroll debacle: IBM to fight contract scrapping Queensland health payroll not fully tested

QLD health payroll failure is after similar one in the same city: Brisbane City Council to dump troubled payroll (this may not be done IBM though). And surprisingly, in this IT world, reputation or recent track records seems doesn't matter much, Gold Coast City Council (80KM from Brisbane) pays $55m to health payroll firm.

Globally, the biggest IT disaster believed to be UK NHS (based on this ZdNet article): : "experts have warned that the project could actually cost more than xC2xA330 billion, making it the greatest IT disaster in history."

The especially sad part of all this is that while these big companies walked away with millions, we as tax payers are ultimate victims.

Auditor-General's report on QLD Health Payroll: Pay system not properly tested: report.

Once one asked me, "What didn't they just fix the system? Given all the attentions (including demonstrations), 3 months is a long time, isn't it? I saw you fixing defects in our system very quickly."

My answer: "If they don't have automated regression testing in place (most likely not), fixing a defect may break other functions. Manual testing is not good finding out that quickly. Before I check-in fixes, I run some related regression test suites in TestWise, and our continuous build server runs full regression tests every day. That's why Steve McConnell says 'The only practical way to manage regression testing is to automate it'"

So will our elected politicians learn the lesson? Let's hope.

Selenium and Watir, as we know, are the two most popular open-source testing frameworks. The next big thing is WebDriver, which is now merged with Selenium 2. Thanks Jari Bakken (who is also the creator of Celerity), we have Ruby bindings for Selenium 2 and even Watir wrapper of it: watir-webdriver, which some might refer it as Watir 2.0. With watir-webdriver, users can pretty much reuse existing Watir scripts, with minor changes.

What are the benefits of using watir-webdriver comparing existing Watir 1.6.5?

• Run same test scripts in IE, Firefox, Google Chrome and HTMLUnit (headless)
• No JSSH plugin required for running tests in Firefox
• Synergy of combination of Selenium and Watir

In last appearance at Watir Podcast, I said future TestWise versions will support WebDriver. Now you can try web-driver in IDE!

1. Download TestWise-2.0b1-setup.exe and install
2. In TestWise, open sample project c:\program files\testwise\samples\demo\demo.tpr (if you are using TestWise first, it shall be pre-loaded)
3. Select the first test script file (left mouse click the file name on the left): ajax_rwebspec_spec.rb,
   choose your browser (IE, Firefox or Chrome), then run it
    

Please note that both Selenium 2 and TestWise2 are at early development stage, so please use Watir 1.6.5 and TestWise 1.9 for production use.

For some sites (especially in .NET world), users are presented a native windows' authentication window (like below) before can access to it.

The standard open_browser in Watir won't work (get stuck), as it will be waiting response from the browser.

  open_browser("http://secure.site.com") # will stuck 

To add a bit more complexity, we expect the same tests to pass whether user has already logged in or not.

The solution is:

• Detecting Login Window using Ruby's timeout
• Use AutoIt to fill user name and password in native Windows Login Window.

    begin
      Timeout::timeout(5) {
        open_browser("http://secure.site.com")
      }
    rescue Timeout::Error : e
      # debug "Timeout error on get to site, maybe asking for login"
      autoit = WIN32OLE.new('AutoItX3.Control')
      win_title = "Connect to edam.bluefirems.com.au"
      ret = autoit.WinWait(win_title, '', 10)
      if ret
        autoit.ControlSetText(win_title, "", "Edit2", username)
        autoit.ControlSetText(win_title, "", "Edit3", password)
        autoit.ControlClick(win_title, "", "Button3")
      end
   end

The above test scripts will work, but not readable. By applying 'Extract Function' and "Move to Helper" Refactorings in TestWise, we get:

    # ...
    rescue Timeout::Error : e
      # debug "Timeout error on get to site, maybe asking for login"
      # calling helper function in test_helper.rb
       
      login_as("zhimin@secure.site.com", "Secret")    
    end

At TIST conference, after my demonstration, on tester asked me "We saw you running the same test (changing password) twice, before and after refactoring, how could it work second time? The data has been changed from first run.". The answer was database reset, ie. the database is reset to the known state before running each test. As I have been doing this for a number of years now, it almost become second nature to me in terms of automated testing. To make the concept of database reset easy to remember, I call it "The Simpsons Pattern".

Benefits of database reset is so important that you probably will appreciate only after experience it. Once I asked a business analyst to rate his view of importance on database reset in scale of 1 - 10. He answered "9.... No, 10".

• Develop tests without worrying of data changes caused by tests, what a feeling!
• Faster test execution. Use seeded data records instead of creating data from scratch every time.
• Always know data record states, great even for manual inspections of applications.

My typical RWebSpec test scripts like this:

suite "Group of related test cases" do
  include TestHelper

  before(:all) do
    open_browser
  end

  before(:each) do
    reset_database
    login_as("zhimin")
  end

  after(:each) do
    failsafe{ logout }
  end

  test "One test" do
    # might change application records
  end

  test "Another test case" do
    # doesn't matter the above test is run or not, as we start fresh
  end
end

Now comes the hard question, how could you implement database reset as testers? No, it has to been done by developers. If your team is doing Agile (or claiming so), go ask them, and it will be of great help for them as well. From experience, the better architecture, easier to implement database reset and the earlier the better.

StoryWise, one of Agileway's Agile Testing Suite product, is a flexible web-based project management tool.

StoryWise is free and open source, released under the terms of the GNU General Public License v2 (GPL).

• Requirement Management
• Defect Tracking
• Project Wiki
• Time Tracking
• Gantt chart and calendar
• Requirement Traceability

Apart from being free, simple and easy to use, StoryWise distinguishs itself from others with one unique feature: Requirement Traceability, a term nowdays gradually fades out due to high hype with low benefits but lots of maintainence work.

StoryWise is different, it allows Requirement Traceability to be achieved simply and effortlessly. StoryWise was first demonstrated at 3rd Tianjin International Conference on Software Testing with very positive feedbacks.

Check it out!

Here are the slides and videos I presented at 3rd Tianjin Software Testing conference

StoryWise + TestWise = Effortless Requirements Traceability in Automated Testing (pdf)

Video Demonstrations xEFxBCx8D Quicktime

1. Create a requirement in StoryWise
2. Create an automated functional test case for it
3. Refactor it!
4. Implement Traceability
5. Generate Requirements Traceability Matrix
6. Quick TestCase Navigation
7. Question Time: Testing non-web apps, using TestWise to test TestWise

Here are the slides and videos I presented at ANZTB 2010 conference.

Presentation Slides (pdf)

Video Demonstrations (Quicktime)

1. Manual testing (webapp)
2. Automated testing (webapp)
3. Create a new project in TestWise
4. Update test scripts using Page Class
5. Collaboration: Avoid Confrontation- 'It worked on my macine;
6. Collaboration: Acceptance Driven Test Development with TestWise
7. Question time: Testing non-web apps, using TestWise to test TestWise

I will be delivering a presentation "Testers and Programmers Collaboration in Automated UI Testing" at upcoming ANZTB 2010 conference, see you there! !http://www.iceaustralia.com/anztb2010/pix/gui/pix_splash.jpg!

Web 2.0 sites such as GMail grow rapidly, which brings big challenges to web test automation. Different from traditional web sites with "synchronous request/response model", new web sites handle requests asynchronously using AJAX. For example, in GMail, clicking 'Send' button does not land you on 'Email Sent' page, instead you see a '_Sending..._' banner appears and replaced with '_Email sent successfully_' later. The difficulty of testing AJAX web sites are not knowing when the request is processed and verify dynamic updated page contents. The approach used iTest2 is simple, after AJAX call, keep trying next operation with specific time. During that time frame, operation error will be ignored (and try again after interval),

    button(:value,"Transfer").click
    # try up to 10 seconds, waiting receipt date 'showing up'
    try(10) { label_with_id("date").should == today }

    # try up to 60 seconds every 10 seconds.
    try (60, 10) { click_button('Next') }

To assert specific HTML element is shown or hidden

    assert_hidden(:div, "info")  # 
...
 is hidden
    assert_visible(:div, "info")

To assert specific HTMl element is added or removed in a web page

    assert_exists("label", "email_sent") #  is removed
    assert_not_exists("label", "sending")