Introduction

TestWise is a functional testing IDE (Integrated Development Environment) for testers/programmers to develop/debug automated functional test scripts efficiently. TestWise supports the test scripts in the following free and open-source automation drivers:

  • Selenium-WebDriver, for testing web apps
  • Appium, for testing desktop and mobile apps
  • Watir, which is now an extension of Selenium WebDriver

and syntax frameworks:

  • RSpec - (Ruby)
  • unittest - (Python)
  • Mocha* - (Node.js)
  • Cucumber - (Ruby)

If you don’t have pre-determined framework, we strongly recommend Selenium WebDriver + RSpec in the beautiful Ruby language.

This user guide mainly uses Selenium WebDriver (automation) + RSpec (syntax) as the frameworks for illustration. Most of features apply to others as well. Please bear in mind the support of others might not be as good in beta versions.

For users who are new to test automation, read on for a brief introduction. You may skip this and go straight to install TestWise and run a few tests.

TestWise Philosophy

TestWise development started in 2007 and the first public release was made available in 2009 (with former name iTest2). Here are the philosophy of TestWise:

  • Open

    TestWise only supports free, open-source frameworks such as Selenium WebDriver and Appium, always. There is no proprietary ‘TestWise syntax’. All test scripts created with TestWise can be edited by any text editors and run from the command line.

  • Power of Text

    Inspired by the classic ‘The Pragmatic Programmer’ book, we believed ‘the power of text’ offers best flexibility. As a result, we don’t have complex User Interface to force users to perform tasks in a certain way.

  • Convention of Configuration

    Inspired by Ruby on Rails framework, TestWise supports common conventions (and best practices) such as Helper and Page Object Model (POM) to help users to learn test automation with ease, as well as develop and maintain test scripts efficiently.

  • Simple and dedicated for Testing only

    We want TestWise to be easy, simple and fun to use, for one purpose only: Functional Testing. However, it is easier said than done. When TestWise development was in the prototyping stage, we found the effort would have been a lot less if we created with established UI framework such as Eclipse or NetBeans. However, the compromise would be an end product just like many another programming IDEs, with many non-applicable-to-testing (and potentially confusing) features. In the end, we decided on the far more challenging way: developing TestWise from the scratch.

  • Affordable

    Testing tools has been known super expensive for a long time, often over US$10,000+ per license. Thanks to Selenium WebDriver and efforts from open-source communities, the average price of testing tools has been gradually declining, probably around $3000, still above what most testers are willing to pay.

    We priced TestWise at US$400 for a personal license about 10 years ago, and have been providing free community edition. This was because we believed that automated testing works better when whole team use it.

    From TestWise 6, we are going a step further, unregistered users can use a full-featured TestWise with minor limitations: 10-seconds viewing Test Automation and Continuous Testing tips on start and a generous limit of the number of test executions (called execution tokens). If the tokens ran out, unregistered users may request more from TestWise support site, or simply restart TestWise (and launching TestWise 6 is super quick).

Automated Functional Testing in Nutshell

Functional Testing here means the testing can be performed by testers, business analysts and customers (please note, programmers alos spend significant time doing functional testing as well for their own verification during the development of a software feature) to verify that the web application does as required by customers. Automated functional testing is conducting these tests using test scripts.

There are many benefits of automated testing, especially for regression testing. As Steve McConnell says it in a nutshell (in his classic book: Code Complete) “The only practical way to manage regression testing is to automate it” .

Here are the common steps testers follow to develop a test case:

  1. Understand the requirement (know what to test)
  2. Think and design the steps (think how to test)
  3. Get application ready (preparation)
  4. Drive the application (do what need to get there)
  5. Verify the result against expectation (check what you’ve got)

The first two steps sometimes are called ‘test design phase’; Step 3 onwards are called ‘test execution phase’. Automated functional testing differs (manual testing) from test execution phase. It is important to emphasise that test design phase is vital.

The same steps apply to automated testing as well, with some differences from manual testings. We will visit them in more detail in the context of testing web applications using RWebSpec (the test framework supported by TestWise IDE).

Step 3. Preparation: Set Initial State (‘Simpsons Pattern’)

Besides conventional preparation tasks such as test data collection, here we want to emphasise the ‘setting initial state’, ie. before test execution, the web application is in a known state.

Why? For automated test scripts, there are only two results “Correct” or “Wrong”. For a typical manual test, a tester may not write down ‘make sure no users logged in’ as a test step but may do it naturally as a part of test execution anyway. Like robots, test scripts strictly follow the direction given. If previous test leaves the user logged in, then the next test case starts from a different state, it will fail.

People, who watched ‘The Simpsons’ cartoons, will know what we mean.

How? In RSpec, you can adding appropriate test steps in the following blocks to achieve. 

before(:all)
before(:each)
after(:each)
after(:all)

Here is an example. 

before(:all) do
  @driver = Selenium::WebDriver.for(:chrome)
  @driver.get("https://whenwise.com")
end

before(:each) do
  login  # a function defined in the test helper
end

after(:each) do 
  fail_safe { logout }
end

after(:all) do
  @driver.quit r unless debugging?
end

Step 4. Drive the application

The test scripts ‘drive’ (interact with web page controls such as textbox, checkbox …etc) the web browser to perform required operations. The execution process shall be same as done by a human except without human, testers can view the test execution in a real browser (e.g. Chrome).

Here are some sample driving statements in Selenium WebDriver: 

click_link('transfer')
select_option('fromAccount', 'Savings')
enter_text('amount', "499.00")
click_button('Submit')

Step 5. Verification

In some frameworks or tools, this step is also called ‘assertion’ or ‘checkpoint’.

A typical web application verification is to check certain text present on the web page. It is important to note that as web page is written in HTML, so the checking is really against the HTML source returned (to view a web page’s HTML source, right mouse click and select ‘View Page Source’). In RSpec test framework (TestWise uses), it is written as 

expect(page_text).to include("Payment Successful")

Here are some more complex verifications 

assert_link_present_with_text("TestWise with Id")
assert_checkbox_not_selected("checkbox1")
assert !div(:id, "info").visible?

If you want to learn more about test

Testing Process in Nutshell

How does test automation fit your current testing process? As a matter of fact, it is quite easy. There are two stages when a tester performs testing against one requirement:

  • Test Design
  • Test Execution

The test design phase is the same regardless you automating it or not: test data, steps and verification. Even a tester is told to automate one test case, he will perform manual testing at least once.

The automation work starts when you feel confident this test may be automated, and then start the converting designed test steps in executable test scripts.

 

Five-Minute Ruby Programming Guide

The open-source test framework, such as Selenium-WebDriver or Appium, comes with a choice of programming languages such as Ruby, Python, Java, JavaScript and C#. We recommend Ruby, a scripting language with powerful scripting in expressive, concise and elegant syntax. If you know Ruby, Congratulations, you can skip this. If you don’t know Ruby, Congratulations! you are going to learn a fast growing popular programming language, and it is not difficult to learn. Instead, you will find it is fun to learn and use Ruby.

We here list some concepts and syntax (with examples)that you will need to get started on writing your own test cases, and you will learn more as you go. Good Ruby programming skills will certainly help you to write more creative test scripts. But we can assure you that you don’t need much of that to get started. TestWise will help you on the way too.

Warning: This is not a formal Ruby tutorial (and here is one by Chris Pine). Its purpose is for non-programmers to get some programming concepts quickly.

Comments

Comments are used to document the scripts, and should be ignored during test execution. In ruby script, Comments start with “#”.

In the following example, the first line is a comment, while the second is not. 

 # Mi, the following line return error, but I can't see why. Please help.
expert(driver.page_source).to include("Functional testing is fun")

Here is an example of appending a comment at the end of test step, and this step will be executed. 

 # two buttons with same caption 'Login', use ID instead.
driver.find_element(:id, 'loginBtn1').click

String

When we enter some texts in text box on a web page, these texts are treated as Strings, and presented in the script as quotes, such as “TestWise IDE”, “399.00”. 

"James" + " Bond" # => "James Bond"
1 + 2 # => 3
"1" + "2" # => "12" as it is string adding

Scope

All programming languages have certain structures (hence, the intended instructions can be delivered to computers). A scope can be seen as a container of scripts, and the variables defined in the scope are only available to the scope. A scope starts with ‘do’ and ends with ‘end’.

Our test case definition uses scopes, by that one test case does not mix with another one. Here is an example. 

it "my test case" do
  # steps in the scope
  if true do 
    #in the inner scope
  end
end

Variable and Assignment

In programming, the process of storing a value in a computer’s memory for later use is called Assignment. The name we give to that value is a variable. The syntax is as below: 

variable_name = some_value

Here are some examples. 

app_id = driver.find_element(:id, 'appId').text # get application id from 'div'
driver.find_element(:id, 'application' + app_id) # click a specific application

The following scripts assign a TransferPage to the variable ‘transfer_page’, and call the page functions (see below) on it. 

transfer_page = expect_page TransferPage
transfer_page.select_from_account 'Savings'
trasnfer_page.enter_amount "123"

Compare Equal

As you can see, ‘=’ is used for assigning value to variables. How can I check that the values are the same? By using ‘==’ for equal, and ‘!=’ for not equal. 

expect(driver.find_element(:id, 'receipt').text).to eq('123')
 # today is special helper function
expect(driver.find_element(:id, 'receiptDate').text).to eq(today)

Logical operations

Now we quickly move to more advanced topics. You won’t be likely to use it in your first batch of test cases. But it is good concept to know. We will show some (quite readable) examples here. 

raise('page has error') if page_text.include?('Oops')
if receipt_number > 100 then
  driver.find_element(:link_text, 'Bonus').click
end
if page_text.include?('Success') then 
  driver.find_element(:link_text, 'Ok').click
else
  driver.find_element(:link_text, 'Cancel').click
end

Function

A function (or called ‘Method’) is a collection of pieces of scripts that may be reused.

Here is a definition of function ‘login_as’ that takes two parameters (with one optional). 

def login_as(login, password='password')
  driver.find_element(:name, "username").send_keys(login)
  driver.find_element(:id, "pwd").send_keys(password)
  driver.find_elmment(:id, "login-btn").click
end

Here are two usages of the function. 

login_as('bob', 'thebuilder') # login using 'bob'/'thebuilder'
login_as('john') # same as login_as('john', 'password')

Learning Test Frameworks

Selenium WebDriver is a framework for testing web apps. Appium is quite similar to Selenium WebDriver as they both are based on the same WebDriver protocol. If being taught properly, people will find learning Selenium WebDriver is actually a lot easier than the other. The reason: Selenium WebDriver follows a common pattern:

Find a element, then Perform an action on it

For example, the test statement below clicks a link with text ‘Vote’ on a web page. 

driver.find_element(:link_text, "Vote").click

Can you guess what these 2 steps do? 

driver.find_element(:id, "username").clear()
driver.find_element(:id, "username").send_keys("testwise")

That’s pretty easy, isn’t it?

Please note, Selenium WebDriver is a standard framework, and the syntax is very similar across all 5 standard programming languages. This means if you master one, you will be comfortable with other 4 languages provided you have the basic knowledge of the programming language.

  • Java 

    driver.findElement(By.id("username")).clear();
driver.findElement(By.id("username")).sendKeys("testwise");

  • Python 

    driver.find_element_by_id("username").clear();
driver.find_element_by_id("username").send_keys("testwise");

Of course, there are more advanced usages in Selenium and challenges on locating certain elements (In theory, programmers shall use unique and meaningful IDs for controls on a page. However, in reality, as we all know, it is NOT the case). You may refer to this book, which contains over 200 practical recipes.

Finally, be aware of some online-courses and trainings, which are often not practical. Their teaching style is to dump various Selenium (and programming) syntax from a text book. That is why there are comments such as ‘Selenium WebDriver is hard to learn’ and ‘Learning Selenium WebDriver takes at least 6 months”. This is actually very wrong!

As the creator of TestWise, I have done the trainings and mentoring on site as well for over 10 years. Under my guidance, most new-to-automation testers developed real work test cases on the same day by using TestWise. Of course, they will not be a Selenium expect in a day, nobody can. However, they will find their knowledge learned in one day can directly apply to their day-to-day work. This gives them confidence and willingness to learn more.

© 2006 - 2024 AgileWay Pty Ltd. Powered by SiteWise CMS