FaceBook UI Test Automation (Selenium WebDriver) + CI

By Zhimin Zhan.

Today I found an excellent presentation  (YouTube linkon How Facebook do CI from F8 (Facebook Developer Conference), starting from 31". Katie Coons, software engineer at Facebook, says:

"No.1 goal of CI at Facebook is developer efficiency"

"The system must provide frequent feedback to developers. Because the sooner we let them know about a problem, the easier and faster it is for them to fix it"



Over a quarter of a million automated test execution for ClinicWise

By Zhimin Zhan.

Several years ago at a conference, when I showed a stats of 200K+ test executions for a government project I led, I could sense doubts from audience.  This week, the number of test executions conducted against my own application  ClinicWise exceeds a quarter of a million. 

   

This is achieved with the tools (TestWise, BuildWise Server + BuildWise Agent) and practices I covered in my book: "Practical Web Test Automation".  And development and testing are all done in my spare time. 

As the graph below shows, the number of automated tests grows with the development. People who had attempts with test automation will know, rare projects can achieve this.


 

The following graph shows automate testing keeps defect defects very early (typically within 10 minutes of checkin). To ClinicWise customers, the software is solid and nearly defect-free, at frequent (daily) deployments. 


Every full build will run all UI test cases. The overall test execution time is dramatically reduced by distributing to multiple BuildWise agents to run in parallel. 


Without these automated tests, ClinicWise simply won't be possible.



Why non-selenium test automation frameworks/tools will die?

By Zhimin Zhan.

There are still commercial testing tool vendors trying to get you to purchase their expensive record-n-playback tools, but in my opinion, their days are numbered, Selenium WebDriver (or equivalent) is the way to go in the domain of testing web applications. Here are some hard facts:

  • WebDriver is a W3C standard WebDriver API is a standard at W3C, the organization behind WWW.
  • Support for Cross Browser Testing

    Nowadays testing web applications on IE is not enough, as Chrome and Firefox are also the main stream browsers.

    Google and Mozilla have been supporting Selenium WebDriver for a long time, and looks like Microsoft finally on board by releasing IE WebDriver Tool for Internet Explorer 11. The chances for a proprietary testing from one commerical testing tool to get support from Mozilla, Google and Microsoftwre is very slim.

  • Support for multi-platforms

    There are increasing number of IT professionals using Mac, and some want to run tests in free OSs such as Linux.

  • Job Ads

    The demand for Selenium has already been reflect in the number of job ads. Based on Indeed's job trend, Selenium has overtaken QTP (regarded as the long-time leading automated testing tool) in 2012. Since then, the number of QTP jobs nosedived.

    The Facts: http://www.seek.com.au/

    Selenium: 169

    QTP: 43

    UFT: 8 

Run TestWise Mac on Yosemite

By Zhimin Zhan.

Apple's new OS Yosemite removed Ruby 1.8, which TestWise requires. Here is a way to get TestWise running on Yosemite: copying Ruby 1.8 from a 10.9 Mavericks to Yosemite

On Mavericks, package Ruby 1.8 to a tar file (or zip file if you like)

$> cd /System/Library/Frameworks/Ruby.framework/Versions

$> tar -cvf ~/1.8.tar 1.8

The copy '1.8.tar' to Yosemite .

$> cp /Volumes/USB/1.8.tar /System/Library/Frameworks/Ruby.framework/Versions

$> cd /System/Library/Frameworks/Ruby.framework/Versions

$> tar -xvf 1.8.tar

Done, and this only needs to be applied once. Then you can run (or upgrade) TestWise Mac as in Mavericks.  


Upgrade gems in TestWise

By Zhimin Zhan.
Sometimes, testers want to use utlitiy gems in the test scripts. For example, aws-s3 gem can be used for checking the files stored in AWS S3. Thanks to the openness of TestWise, it is pretty easy to do.
  1. Let TestWise know the new gems
    Open c:\agileway\TestWise\Gemfile in a text editor, and add your gems there.

  2. Install the new gems
    In a command window, run these 3 commands.

  3. Restart TestWise

Please note that incorrect customization of an TestWise installation may result in TestWise failing to start, and AgileWay unable to support it.

TestWise 4 is released

By Zhimin Zhan.

We are happy to announce that TestWise 4 is released, here are some highligh features:

  • RSpec 3
  • RWebSpec 6
  • New code base for faster UI operations and better version control integration
  • Experimental support for Capybara 1

Do I need to upgrade to TestWise 4?

Upgrade is recommended, but don't have to. For users who wish to continue using RSpec 2 / RWebSpec 5, please see our next blog article on how to upgrade gems (such as selenium-webdriver) in TestWise.

How do I upgrade to TestWise 4?

Depending on your license:

  • TeseWise Subscription License.
    Upgrades are always free within the subscription period. Just contact our support to get new license key.
  • TeseWise Perpetual License purchased on and after 2013-08-18.
    The upgrade is free (within one year of purchase), same as the above.
  • TeseWise Perpetual License before 2013-08-08.
    Entitled 50% discount on upgrade, contact our support and we will send a link to purchase the upgrade with discount.

Please include your original license email when contacting the support for ugprade.

Upgrade to RSpec 3

RSpec 2 RSpec 3
xxx.should be_true
xxx.should be_truthy
xxx.should be_false
xxx.should be_falsey

Upgrade to RWebSpec 6

RWebSpec 5 RWebSpec 6

specification and test_suite, which are used to group test cases (in one test script file), are now deprecated.

specification "XXX" do
end
test_suite "XXX" do
end

Standard RSpec's describe

describe "XXX" do
end
repeat_try, try, try_until
try_for
Depend on minitest/assertions Self-impelmented assert

Also I updated the test scripts in the books: 'Practical Web Test Automation', "Selenium Recipes in Ruby" and "Watir Recipes in Ruby" to be RSpec 3 compliant.

Microsoft CEO wants to merge 'testers and developers', cut traditional testers.

By Zhimin Zhan.

Today, Microsoft announced that it will cutting up to 18,000 jobs in the next year". Two days ago, this report revealed the new Microsoft CEO's vision for team structures, in particular, the change of role on 'software testing'.

"Engineering teams have traditionally been split between program managers, developers and testers. Yet with new cloud methods of building software, it often makes sense to have the developers test and fix bugs instead of a separate team of testers, Nadella said in an interview last week after unveiling his memo."

"Some of the cuts will be among software testers,"

The end of job 'Software Tester'

By mark.

In Australia, we still call people perform software testing "Testers". As we know from the book "How Google Tests Software", it is called "Software Engineer in Test" (SET) at Google. When I was in America last year, I saw many "SET" job ads on STARWEST conference.

Today, following a link from LinkedIn Update email, I was quite shocked that how quickly and big the change is: The number of SETs jobs are 10 times more than software testers

"Software Engineer in Test": 27,384 results

"Software Tester": 2,348 results

The Happiest Jobs In America: Software Tester

By mark.

I expressed "Software testers are the best jobs" a few times in my presentations, especially doing test automation.  It is not just me think so, according to Forbes:   Software quality assurance engineers (a fancy name for software tester) is the happiest job in America (2012).

Based the (happy) index ranking, Software tester even leads No.2 by a big margin.

The future for software tester who can write automate test scripts will be even better, according Simon Stewart (Selenium Lead from Facebook), "Facebook has no testing department, he believes this (testers write scripts) will become more widespread".

Secrets of developing quality web applications super fast : automated testing + CI

By mark.

I talked and wrote a lot about test automation and continuous integration, which are the key reasons behind the success of some big names such as Facebook and LinkedIn. ("FaceBook pushes two releases per day"; LinkedIn joined to this elite club by "luring" Kevin Scott, the senior vice president of engineering and longtime Google veteran, "completely overhauled how LinkedIn develops and ships new updates" (wired magazine called this 'software revolution'). While few of my audience or readers deny the importance and benefits of automated testing and CI, some of them (I feel) thought these can only be done by software giants. Many failed (including some wise testers who purchased TestWise) due to the lack of motivation.

As a matter of fact, test automation and CI are vital to small projects just as to the big ones, if not more. Automated tests and CI compensate the usually limited testing and deployment resource often found in small teams. The very few companies who have done well wouldn't share how they apply test automation and CI (like Google and FaceBook), as they usually regard it as a secret. To be fair, while the technologies are usually open (such as Selenium, Jenkins), the application of test automation and CI varies (not much) due to the nature and culture of the business.

I share our secret here, on how we applied automated testing and CI to one of our new product: ClinicWise, an online clinic management software. The project was started as a side project about a year ago. A relative of mine, a dentist was about to open a dentist clinic, looking for a dental clinic software. He has used many of them before, but was not happy with any of them (too many bugs, complex and very expensive). For some reason, I offered a try to develop a customized application for him. Looking back now, it sounds a bit crazy, as I had no background in this health care area and the client is 8000km away.

Here are 'the secrets':

Initial Stage

  1. get skeleton application up running with database migration, unit test, code coverage, automated UI tests and deployment. It is not that hard, even if you haven't done it before. The key is to keep the skeleton application simple.
  2. set up BuildWise server on the integration server, running all CI steps within BuildWise. This requires understanding of CI and build language. If you read and follow my book Practical Web Test Automation, which contains instructions and the sample build scripts.
  3. implement common components such as user sign in, password reset, user management, access control, CRUD, pagination, file uploading, audit logging, etc. No matter which language you use, there are mature libraries for them. Make sure to add some automated UI tests as you go, and pass them in your build server. Discipline!
  4. add exception reporting. When an error occurs, the system will send you an email with the full stack trace.
  5. enhance the UI. Bootstrap and Font Awesome make it very easy to add progressional looks to your application.
  6. talk to the client to identify two (just two) key components must have right now. For clinical software: client management and appointments.

After 40 hours (I counted) in spare time, the first production release was out. The client could put it in use: registering new clients and booking appointments.

Ongoing

  • consult with the client to understand the business request. Use the demo server for communication proved time-saving.
  • implement the feature/enhancement or fix the bug
  • check in and kick a build in CI server (in our case, BuildWise)
  • if all tests pass, release to the demo site for the client to check
  • release to the production on a time convenient to the client (such as mid-night)

The automated tests and CI prevent numerous errors from going to the production, also enable upgrades to latest version of frameworks and libraries (some scary stories there, thanks god for automated tests!). Software releases is NOT a stressful activity. The client is getting used to new releases, "it is getting better and better". On his recommendation, several dental and physio clinics signed up with ClinicWise.

Summary

Without test automation and CI, ClinicWise is not possible. The process prevented many bugs (most from ourselves, some from dependent library and infrastructure changes), the customers only see stable and 'keep getting better' software. More importantly, the frequent release enabled us getting quick feedback to keep development and our customers happy: the software what they really want!

Here are some build server screenshots: