Test Management with TestLink

TestLink is a great open-source tool to manage test cases. It helps to organize all the testcases, version control and the testing process. It can be integrated with any bugtracking tool (mantis, bugzilla… etc). Also, TestLink can be use to see if all requirements are covered with testcases. and it provides varies kind of test reports too. Before working with TestLink, let’s have some knowledge about the test case.

What is in a test case?

Testcase describes action and reaction of a feature in a software product. In one hand, test case document can be use as a breakdown of requirement document. On the other hand, it can be use to estimate testing effort. Testcase document contains these fields.
Test Case ID, Test Case Title, Test Case Summary, Precondition, Step actions, Expected Result, Actual Result, Inputs, Status(Pass/Fail)

This is a template of a test case document.

Install TestLink

Just like Mantis, You can install TestLink with five easy steps (in a windows server). For more details, Check the installation manual locate at <testlinkfolder>\docs\testlink_installation_manual.pdf.

1) Install xampp or wamp (apache, mysql, PHP package) -http://www.wampserver.com/en/

2) Download the latest stable version of TestLink - http://sourceforge.net/projects/testlink/files/

      3) Extract mantis to wamp project location (example - C:\wamp\www\)

4) In config.inc.php  file, change

                        log directory ($tlCfg->log_path) edit the path to C:\wamp\www\testlink\logs

                        upload directory ($g_repositoryPath) to C:\wamp\www\testlink\upload_area

5) Browse the project from your web browser (example -http://<ip>/Testlink) and follow the installation instructions.

Note : Default credentials to login testlink are, user: admin; password: admin

Excel to TestLink

Now, the most important part! Before you start working with TestLink, I recommend you to create a test case document in a excel sheet. Have a look at my sample test case document – (click here to download). Your testcase document need to be exactly same format.

Important! – Look at the steps and expected result numbering system. Don’t use 1) 2) ... instead of 1. 2. ...  that will cause errors.

TestLink only understand XML format of the testcase document. So, now you need a tool to convert your excel testcase document to XML format. Click here to download the XMLConverter. Double click the “Exceml.exe” file inside the “XMLConvert\Debug” folder. Browse and locate the excel testcase document and “Convert to Xml”. You will find the converted file inside “Xml” folder. Now you are ready to upload testcases to TestLink.

Work with TestLink

1. Create a Project – First you need to create a project. Go to the “Test Project Management” section to create a project. 
   


2. Select the Project – Make sure you are currently selecting the correct project. 
   


3. Create Test Plan – Use “Test Plan Management” to create a test plan.
   


4. Create a Test Build/Release - Specify the Build of the Project you are going to test.
   


5. Import Test Cases for the project. - Now you need to import your testcase XML file from test specification. 
   


6. Add test cases to the Test Plan – In "Test Plan Content" section, click “Add / Remove Test Cases” to add selected testcases to the test plan. 
   


7. Assign Test Cases to Test Engineers. 
   


8. Test case execution. 
   


9. See Reports and Charts. 
   

Tip: You can integrate TestLink with your bug tracking tool. Click “Issue Tracker Management“ under “system” and set configurations. Make sure to assign and activate the bug tracker when you creating projects.

{ Read More }

Behavior Driven Development with Cucumber

While searching a better automation solution for web applications, I could find a great automation framework which is called “Cucumber”. Simply, Cucumber is a BDD supported testing tool that executes automated tests. It is easy & flexible solution that supports java-selenium development. Unlike selenium robot framework, this BBD solution has a huge technical support on internet.

What is BDD?

BDD (Behavior Driven Development) is a good extension to the conventional TDD (Test Driven Development).

TDD is widely using by developers to write their unit tests. Add a test, run entire test suit and write some code on newly added test, rerun all tests and refactor is the main concept of TDD.

BDD has a different approach. Its mainly focus on the behavior (input and outcome) of the functionality. In other words, BDD is a combination of user story (a short introduction of tests) and acceptance criteria.

Example :

          As a [role] I want [feature] so that [benefit]

          Given [initial context]

          When [input event]

          Then [outcome]

Cucumber is used to execute tests written in this format. The language that Cucumber understands is called 'Gherkin'. Gherkin is just a combination of four keywords (given, when, then, and). It's very easy to write tests with Gherkin. also, even non-technical people can understand the tests and scenarios. Check below example which is a test written in Gherkin.

Feature: Search on Google

  As an user

  I want to search on Google

  So that I can see results

  Scenario: results are shown

    Given the page is open "http://www.google.com"

    When I search for "Cucumber"

    Then a browser title should contains "Cucumber"

Now, let’s have some fun with cucumber!

Setup your first cucumber project

1. Download sample cucumber-java-selenium project from here - https://github.com/michalvich/cucumber-jvm-selenium-example
   
   Note 1 : Check the directory structure. You can see ‘pom.xml’ in the root directory of the project. ‘pom.xml’ is the fundamental unit of work in Maven. It is contains information about the project and configuration details used by Maven to build the project. 
Note 2 : You can see the .feature file at '..\src\test\resources\com\michalvich\cucumber\selenium\example'. That is the cucumber test case document which is written in Gherkin.
Note 3 : You can see java code at '..\src\test\java\com\michalvich\cucumber\selenium\example', which is the code we should write to execute our cucumber test cases.


2. Install Java, create a user variable “JAVA_HOME” and set it to jdk installation directory (example - C:\Program Files\Java\jdk1.7.0_21)


3. update/create the Path environment variable in the user variables and append the %JAVA_HOME%\bin


4. Check java is installed successfully. ( type “java –version” in command prompt)


5. Download eclipse IDE (https://www.eclipse.org/downloads/)


6. Download maven (http://maven.apache.org/download.cgi)


7. Read the installation instruction at http://maven.apache.org/download.cgi#Installation or follow these steps.


1. Add the M2_HOME environment variable in the user variables with the value C:\Program Files\Apache Software Foundation\apache-maven-3.2.1
2. Add the M2 environment variable in the user variables with the value %M2_HOME%\bin
3. update/create the Path environment variable in the user variables and append the value %M2% 


8. Check maven is installed successfully. (type “mvn –version” in command prompt)

Note : Make this small change before proceed. Open the 'pom.xml' file. find this set of codes
<dependency>
       <groupId>org.seleniumhq.selenium</groupId>
       <artifactId>selenium-java</artifactId>
       <version>2.28.0</version>
</dependency>
Change the version no "2.28.0" to "2.41.0"


9. Go to the place which pom.xml located and type,
1.  “> mvn package” (to build the project)
2. “> mvn install” (optional)
3. “> mvn eclipse:eclipse” (to generate the eclipse project files from your POM)


10. Open eclipse and set the maven repo directory (in eclipse - windows - preferences - java - built path -> add M2_REPO --- maven repo directory (C:/Users/Test/.m2/repository))


11. Open eclipse and import the project ( file – import – general - existing project in to workspace)


12. Run the project as a junit (run – run configurations – browse the project and the class)

How to write your own code?

• First, write a cucumber test case document, save as a .feature and put it in the same directory which locate ‘GoogleSearch.feature’.
• Then, Create a empty java class with the same name and just run it. You can see the code snippets print in your eclipse console window.
• Finally, Copy and paste the snippets to implement missing steps. Write few lines of codes for test. you can have any help from internet.

Look! How easy it is! Software automation is not a nightmare anymore!

{ Read More }

Defect management – Track and Trace Defects with MANTIS

Defect management is all about controlling bug life cycle using a defect tracker. You can find many open source and commercial defect tracking tools such as Mantis, Bugzilla, Trac, Flyspray, JIRA... etc. By this post I wish to discuss about famous open source bug tracking system – MANTIS.






What is Bug life cycle?

Defect in a software product is passing several stages in it’s lifetime. New Assign Open Reopen Verified Rejected Deferred        Defect tracking system helps to track and control changes in these stages.

Why MANTIS?

Mantis is one of the best open source bug tracking tool which has heavy usage around the world. It is light, fast, providing descriptive reports and easy to use. It has email notification support too. Also you can manage users, projects, tags, custom fields, global profiles, plugins and configurations.
Mantis Bug Tracking project was started way back in 2000. After 6 years Version 1.0.0 was released. In November 2008, project was switched from SVN to Git revision control tool. In July 2012, the MantisBT organization on GitHub became the official repository for the Project's source code. By today, the stable mantis version is 1.2.17.

There are four easy steps to install MANTIS in a windows server

           

1) Install xampp or wamp (apache, mysql, PHP package) - http://www.wampserver.com/en/

      2) Download the latest stable version of MANTIS - https://www.mantisbt.org/download.php

      3) Extract mantis to wamp project location (example - C:\wamp\www\)

4) Browse the project from your web browser (example - http://<ip>/mantis) and follow the installation instructions.

Defect reporting Tips and Tricks

Check the project and version before you report a defect in MANTIS. There are few common fields in any defect tracking system. Study the characteristics of these fields. Bug Category or Issue Type (Defect/Feature Change Request/ Improvement) Severity (Critical/ Major/ Medium/ Low) Priority (Urgent/ High/ Medium/ Low) Assignee Summary Description Steps To Reproduce File uploader

• Severity and Priority - Identifying Severity and Priority is the fundamental objective when you reporting a defect. Severity describes the impact of system functionality while Priority describes the urgency of getting fixed. Let me show you an example:
• If user cannot login to the system, it is a High Priority & High Severity defect.
• If a project title or heading text is misspelled, it will be a High Priority & Low Severity defect.
• If some link of function is not working which is rarely use by users in the system, that should be logged as a High Severity & Low Priority defect.
•  If a misspelled text appears in a page which is rarely use by users in the system, that is a Low Priority and Low Severity defect.

I can see few Common mistakes which QA engineers normally make. Make sure you are not among those people. Not providing enough details about actual result (not mentioning error log, not attaching screen shots) Not providing reproducing steps, locations and URLs. Not providing expected results Not providing the environment (OS & browser version) which the issue occurred.

{ Read More }

How to Test a Web Form

Testing a web form is a fundamental challenge for any QA engineer.  But, to do it in professional way, you need to have a clear vision. How and what to test in a web form? 

What is a web form?

Web form is a bundle of components (text labels, text fields, text area, drop down fields, radio buttons, check boxes, submit button… etc. ) in a web page.  I'm assuming that you know the behavior and characteristics of these web components.

Today, I’m going to give you very important set of Tricks & Tips in web form testing.

	Test mandatory fields - First of all, just submit the form without fill or select any field. You should see required field validations in front of all mandatory fields. (You can Identify mandatory fields by the * mark display beside the field).
	Test trim functionality – Now, you know the required fields in your web form. Then you add spaces to all required fields and submit. Space is a blank character. You shouldn’t able to submit the form with space in the required fields. If you are able to do, you should report a bug & ask to trim all the required fields.
	ADD/EDIT/DELETE/VIEW – Most of the web forms are based on data management system. So, we have to test workability of basic 4 sqls (insert into, update, delete and select). Try to add a record, edit a record, delete a record and view a record. You can check data from either database or front-end.
	Special character testing – Send special characters (example - ~!@#$%^&*()_+`-=<>?,./:";'{}[]\|.... etc) through the form fields and try whether you can break the app.
	Test emails - If the form sending emails in graphical format then make sure it displays correctly in all the main email clients.
	Field validation -   Your form fields may have validations for field length, email format, phone no format… etc. Then, you need to validate data and check validation messages against the fields.
	Client side validation Vs Server side validation –we use Client side validations (JS validation) to validate user inputs before submit the form. That is light & fast. If JS validations are faster, why server side validations?  Basically because, web browsers have an option to disable java scripts. So, someone can bypass JS validations & send harmful data to the server. As a QA engineer you should not forget to check availability of both validation types.

Important: If site functionalities are not working while JS disable, that is acceptable. Real problem is, if someone can send direct requests while JS disabled.

	Post and Get methods – There are two commonly use http request methods. Get and Post. Both can be used depend on the situation. What we need to know is, GET method adds data to the URL and data is visible to everyone in the URL. Now, change URL parameters manually and try to do some unauthorized activity. If you could, you found a BUG!
	Security testing - I always believe a good QA engineer should be a good hacker! No joke! If you are a good hacker, you should be good security tester too. SQL injection The medium which data carrying from web page to DB, is called sql query. If you know the behavior of the query, you may able to access db in irregular way. Let me show you an example.

Consider a sql query like this:
     SELECT * FROM Users WHERE Username='$username' AND Password='$password' 
     If we insert below values,
     $username = 1' or '1' = '1
     $password = 1' or '1' = '1
     Then the query will be 
     SELECT * FROM Users WHERE Username='1' OR '1' = '1' AND Password='1' OR '1' = '1'   
And it’s always true and returns all values in users table.

What you can suggest to improve security of a web application?

     (1) Include a Captha – captha is use to prevent robotic access. It will avoid spammers who using auto-filling software programs.
     (2) Use a secure protocol – have you seen some web sites starts with https:// while others are http://? ‘Https’ is combination of ‘http’ and SSL/TLS, Which is more secure protocol. Communication is encrypted and no one can access data.

User Interface testing – Formatting, alignment, images, colors will make a better user experience.  UI testing is a separate topic to be discussed. Just for now, I’ll give you a small hint: By today, more than 80% of internet users are using Chrome and Firefox browsers (source : http://www.w3schools.com/browsers/browsers_stats.asp). So, make more weight on those browsers while you testing UI.

 

    
Before leave I’ll give you two additional tips.

Tip 1: Give suggestions to improve quality of the application. But you are never allow to exceed the requirement specification. Stay beyond the limit & release your suggestions.

Tip 2: During your test, keep an eye on the URL to make sure you are testing in same test environment. With my experience, I see some QA engineers are start testing in QA environment and end in production. Don’t get entangled. Keep the eye on URL.

{ Read More }

Selenium in 3 Days – Day 3 – Be an expert in Selenium Robot Framework.

Today, the last day of your selenium training, I will start with four FAQs. 
  

      Q.   Can I change the location where the log and result files create in?

A.    Yes, you can. you can give a directory where the logs and result files stored in. It is not mandatory to exist the directory. Here, ‘<foldername>’.

>pybot --outputdir <foldername> <test_suit_name>

      Q.     Can I execute selected test cases from my test suite?

      A.      Of course you can! Suppose you have a test case named ‘testcase001’ in your test suit. You can execute only that particular test case using below command.

>pybot --test testcase001 <test_suit_name>

Further, you can check the usage of pybot with below command.

>pybot --help

      Q.     How can I find the locator of a web element?

      A.      You can locate elements by several ways. by id, by name, by xpath, by link text, by partial link text, by tag name, by class name, by css selector.. etc. but with my experience, best strategy to locate an element is 'xpath'. If you are using firefox, you are having the firebug which is the best tool to find locators.  Just press 'F12' in your keyboard. Basic HTML knowledge would be sufficient to find the html code line of your web component. Just right click on it and choose 'Copy Xpath'. Or else, you can use the firefox addon – “element locator for web driver”.  Also you can use the addon – “firepath” to customize xpaths.

You can find basic xpath syntaxes from here-http://www.w3schools.com/xpath/xpath_syntax.asp

      Q.     If my selenium script running in a windows PC, can I schedule it to automate?

      A.      Yes, you can save your pybot command as a executable .bat file and schedule it with the windows scheduler.

From here, I’m going to talk about two major topics.

Working with windows dialog box

When you working with web applications, you may have to download several objects in a web page. (example – text/csv files, excel/word documents, zip files, jar files… etc ), in that case firefox will ask you to confirm before download. 

Look. This dialog box is NOT a component of the browser. It’s a windows dialog box. You neither can handle it through selenium2 library nor builtin library. So, how can we overcome this?

You have two major options to do this.

• With the help of AutoIt library.
• Download AutoIt library from – https://code.google.com/p/robotframework-autoitlibrary/downloads/list
• Refer ‘README.txt’ for installation instructions.
• After successful installation, you can see a folder named ‘AutoItLibrary’ inside <python directory>\Lib\site-packages
• Import the library inside the resource file. (Library     AutoItLibrary)
• Send simulated keystrokes to the active window using ‘Send’ keyword.
• Example 1–  Send   !s   for send Alt+s and choose ‘Save File’ option.
• Example 2 – Send   {ENTER} for send Enter key.

But, wait! There is a limitation on this. You CANNOT use AutoIt library in linux systems. So we have to move to another option.

• By changing the firefox profile attributes.

This is the best option to work with windows dialog box. The default firefox profile which is using by selenium2library is locating at ‘<python_installation_dir>\Lib\site-packages\Selenium2Library\resources\firefoxprofile’. Just open the ‘prefs.js’ file and change below firefox profile configurations.

firefoxProfile.setPreference("browser.download.folderList",2);

firefoxProfile.setPreference("browser.download.manager.showWhenStarting",false);

firefoxProfile.setPreference("browser.download.dir","c:\\downloads");

firefoxProfile.setPreference("browser.helperApps.neverAsk.saveToDisk","text/csv");

Note 1- Value of ‘browser.helperApps.neverAsk.saveToDisk’ should be change according to the document type which you suppose to download.

Example – for zip files ‘application/zip’, for jar files ‘application/java-archive’

Note 2 – Instead of changing default FF profile, you can create a separate FF profile. Refer ‘https://support.mozilla.org/en-US/kb/profile-manager-create-and-remove-firefox-profiles’ for more information. In this case, you need to mention ‘ff_profile_dir’ with the ‘open browser’ keyword.

Working with databases.

If you are a selenium developer, you may need to connect to a database and verify data which you submit through the web page. How can you do that? I’ll explain it using an example.

How to connect an oracle DB using selenium web driver? 

There are 4 easy steps to do that.

Step 1 : Download and install robotframework-databaselibrary (http://franz-see.github.io/Robotframework-Database-Library/)

Step 2 : Install oracle client, make TNS at tnsnames.ora (Location- <downloaded directory>/network/admin/tnsnames.ora)

Example :   BOBA64 =

                        (DESCRIPTION =

                             (ADDRESS_LIST =

                                 (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.101.230)(PORT = 1521)

                                 )

                               )

                                       (CONNECT_DATA = (SID = BOBA))

        )

Step 3 : Download and install DbApiModule – “cx_Oracle” (http://cx-oracle.sourceforge.net/)

Step 4 : Connect to the db using ‘Connect To Database Using Custom Params’ keyword. (refer http://franz-see.github.io/Robotframework-Database-Library/api/0.5/DatabaseLibrary.html)

 Example : Connect To Database Using Custom Params   cx_Oracle     '<username>', <password>', '<tns>'

Today you have exposed to more complex scenarios in selenium robot framework.   If you have further questions, contact me anytime.  I’m always ready to help you. 
For now, keep practice!  That is the only option to be an automation expert! Good luck! 

{ Read More }