Advanced Test Management - TestRail™

If you are looking for an advanced test management tool over testlink, I have found an outstanding solution from Gurock Software. Unlike testlink, this is not a freeware. But, it has all features that you expect from a test management tool. TestRail is a web-based test case management tool which enables to monitor, evaluate & organize software testing effort. In this article, I would like to discuss few main features, benefits and basic components of TestRail.

Main benefits of TestRail

√  Organize test cases and test suits

√  Mange test runs and report issues on the go

√  Tracking test results

√  Distribute test reports

√  Boost testing productivity

√  Tracking test progress

Key features of TestRail

√  Add test cases or import test cases via xml, csv file.

√  Create milestone and add test plans to the milestone.

√  Create Test runs by including testcases.

√  Issue Tracker integration -  TestRail can be integrated with bugzilla, GitHub, JIRA, Mantis, Pivotal, LightHouse… etc. This helps to report and link issues to a testcase.

√  Track progress of Test execution in every aspect.

√  TestRail plugin to the JIRA – Using this plugin user can link the requirement (JIRA ticket) to the test case. On the other hand, requirement (JIRA ticket) shows all the relevant testcases within the story.

√  Customizable & graphical interpretation of test reports.

Dashboard - Dashboard Overview
TestRail Dashboard view enables users to have an overview of their projects. ToDos, Project milestones, Recent Activities and graphical interpretation of test runs are the key features.

Note: for more information about ToDo and Milestone, see below sections "Milestone" and "ToDo"

Test Cases
Users are allowed to create testcases and testsuits here. Key features of TestRail – Test Cases section are,

√  It enables to rearrange test cases between test suits (sections).

√  Move and copy test cases between test suits or test projects.

√  Customize test case templates.

√  Estimate the time for each individual test case.

√  Attaching files to the test case.

√  Import testcases in csv or xml formats.

√  ‘Tests and results’, ‘defects’ and ‘history’ of a particular test case can be monitored.

√  Markdown options: (e.g **Bold** - two astricts before and after a text will bold your text.)

Hint: In a section (testsuit), group testcases by application modules or functional areas. (Not by product version or testing types)

Test Runs and Results
Test Runs and Results section helps users can perform a test run prior to software release & obtain the test result. Key features of TestRail – Test Runs and Results section are,
Add Test Run

√  Assign test run to particular users

√  Attach a milestone to the test run

√  Select all or specific test cases to the test run, based on selected criteria. (e.g: high priority test cases)

Execute Test Run and Result

√  Execute the test & Add test result (e.g: Passed, Failed, Blocked, Retest)

√  Report test execution time

√  Report bugs relate to Test case into an integrated bug tracking tool (e.g: JIRA)

√  Email, Print or Export (in xml or csv format) test result

Add Test Plan

√  Add a test plan (collection of test runs) to group test tuns

√  Attach a milestone to the test plan

√  Add configurations (e.g: browsers or OS) to a test plan

Defect Integration

Testrail allows administrators to integrate to an issue tracking tool. (e.g: JIRA) After integration has done “Test runs and Results” section allows users to,

√  Push (link) details from Test result to the JIRA story.

√  Add a new bug in JIRA instantly.

Also, the ‘References’ section under “Test Cases” section allows users to,

√  Link JIRA stories (requirements) to the particular test case.

Milestone

√  User can use milestone to separate releases

√  User can set start and end date to a milestone

√  User can assign the milestone to a test run or test plan

 

ToDo

ToDo represents an individual task for each member of a test team. Administrator (Test Lead) can assign tests to members of test team.

Reports
There are a lot of overview charts are available within the each section (Milestone, Test run and results, test cases, defects, activities). For additional reporting options, Testrail facilitates various types of customizable reports. Few examples are shown below.

√  Milestone Report

√  Workload Summary

√  User can assign the milestone to a test run or test plan

√  Defect Report

√  Coverage Report

√  Activity Report

√  … etc

Few other features of Testrail reports are, 

√  User can set access permission of their customized reports

√  User can schedule a test report to generate regular basis.

√  User can print, download or email reports.

Tool Comparison – TestLink Vs TestRail

Here is the list of advantages of TestRail over TestLink.

√  Testrail interprets real time testing progress. Also, it enables editing tests during a test run. But, Testlink cannot do the same.

√  Testrail has a JIRA plugin that can use to monitor test progress within the JIRA.

√  Testrails allows to Clone Tests between projects and TestLink doesn’t.

√  Testrail allows batch operations (cloning, editing, deletion)

√  Testrail has a advanced reporting module over testlink. Customizable reports, dashboard graphs and schedule reports you can only fine in Testrail.

√  Testrail provides both cloud and on-premise while Testlink doesn’t support cloud.

√  Testrail help Taskboards, which is a great help to an agile team. But, Testlink doesn’t have Taskboard facility.

√  Since Testrail is a commercial Tool, it has wide support and service. But a freeware like Testlink doesn’t.

References

https://www.gurock.com/testrail/

http://testlink.org/

http://qasrilanka.blogspot.com/2014/10/test-management-with-testlink.html

https://www.youtube.com/channel/UCaCU2W8q6eYOPpdxcFKfCVA