Welcome to part 2 of our series on the Agile QA process. The first part of the series gave an overview of and introduction to the stages of the Agile QA process. Part 2 of the series will take you through the steps to follow to set up your own process from scratch.
Setting up an Agile QA process from scratch can be quite daunting. One must keep in mind that having a working QA process from a previous organisation does not mean it would work or fit all scenarios. So, on that note, instead of providing a hard set of guidelines to follow, I am going to share the thought process to help you set up your own framework to fit an organisation so you can hit the ground running:
- Know your product
First and foremost, the QA must understand what type of product the application under development is and its purpose. This is a good way to start gathering high level information about the system’s architecture and the different systems that could be involved. This allows the QA to anticipate what types of testing should be carried out based on experience. For example, if the system under development is a retail web application, the first things that come to mind are UI testing, backend testing, cross-browser testing, performance testing etc.
- Know the product’s pain points
Now that we know what the product is and its purpose, it is time to get an understanding of what the product pain points are. This is usually done by conducting interviews with the product owner and business analyst, and sometimes with product end users. The main goal of this step is to understand the functionality that is considered high risk and make sure these risks are catered for in the overall test strategy. Here are a few sample questions that can help flush out product pain points:
- What are the most common bugs leaking into production?
- Which functionalities/ modules are considered unstable?
- Were there any security breaches before?
- Is the system performant?
- Know your product’s audience
To create a cost-effective and value-driven QA framework, the QA should know who will be using the actual software product. Understanding the target audience will allow the QA to tailor the process around the users’ needs. This will then allow the QA to shift the testing focus to features that are considered high-value, and as a result, save significant time and money.
For example, the actual end-users only use Google Chrome’s latest version in their organisation to navigate the product website. This means:
- Cross-browser testing can be ignored in this scenario.
- Testing the product on the latest version is top priority.
- Testing the product on the latest version -1 could be considered.
- Create the QA process document
While working in an Agile environment promotes building working software over creating comprehensive documentation, it doesn’t necessarily mean that we should skip the whole documentation process altogether. Essentially, the goal of the QA process document is to maintain QA activities in a structured manner and to serve as a guide for newly hired QAs.
Generally, the QA process document should contain:
- Description of the QA steps detailing what, when, and how QA fits in every stage of the Agile development process e.g.
| Agile Development Stage/ Scenario | High-level QA Steps |
|---|---|
| Once work item User Story/ Acceptance are written | QA needs to review the user story/ acceptance criteria for: – Completeness – Clarity – Redundancy – Consistency – Testability |
| When acceptance criteria has changed | Revisit test cases and estimate testing effort and additional regression tests if required. |
| When a bug is discovered | Record and triage bug with the project team members. Track the bug progress – the end state should either be closed or deferred depending on the bug severity and priority. |
| When a bug is fixed, or code update is deployed | Understand the change and its impact with project team members. Then plan manual and/ or automated regression tests, and execute those tests. |
| Testing is completed on a work item | Communicate test results and observations with the project team members to get feedback. |
- The test strategy
| Content | High-level Description |
|---|---|
| Testing objectives | Testing objectives are usually aligned with the project or business goals e.g., some projects focus on delivering high application stability while others focus on constant delivery by means of quick feedback and defect prevention rather than defect detection. |
| Life cycle of work items | Define the “Definition of Ready” and “Definition of Done” criteria for various work items (Task/ Bug). |
| Types of Testing | Details the What, Who, Where, When, and How for each testing type: – Unit Testing – Acceptance Testing – Regression Testing – Exploratory Testing – Performance Testing – Security Testing, etc. |
| Roles and Responsibilities | Defines the team member’s role and responsibilities on each test level. |
| Test Automation Strategy | Defines how automation frameworks will be utilised throughout testing. It includes planning which tests should be automated, target test environments, setting up test data, execution logs and reporting test results. |
| Test Tools | Details the tools to be used for each test level e.g. Postman or SOAP UI for API testing, SQL Server Management Studio for database testing, JMeter for performance testing, etc. |
| Test Environments | Details the environment for development, testing, and UAT. |
| Test and Defect Management Tools | Details the management tools to capture various testing metrics. These metrics will then be used to evaluate the effectiveness of the test strategy e.g. Azure DevOps, TestRail, JIRA, etc. |
- Implement the QA process and monitor the results
Once the QA process is documented and communicated across the project team, it’s now time to implement the process and evaluate its effectiveness. The aim is to ensure that critical bugs are not released to production. Here are a few metrics that can be used to measure software quality in Agile:
- Number of bugs found in testing
- Number of bugs found in production
- Test Effectiveness = IB/ (IB + EB)
- Where:
- IB = Internal bugs found in testing
- EB = External bugs found in production
- Where:
- Test Coverage (% of code covered in automated testing)
By following these steps, you can set up an Agile QA process that works for your organisation and the specific product involved. Look out for the final part in the series covering best practices for the Agile QA process.
By Alvin Maranan
Alvin has 10 years experience in manual and automated testing. Having ISTQB Advanced Level Test Analyst and Technical Test Analyst certifications under his belt, he seeks to continuously improve both manual and automated testing practices and establish robust QA processes to ensure high quality outcomes.
