Extensive testing before any release is not only necessary but also become essential for any company for their client and their clients's client,vendors,partners. Due to frequent change in IT industry,testcase base grows heavy in size. To run those huge cases,we need a strong manual team . Involvement of huge manual test engineers is nothing but loss human power.As they have to test the same test cases over and over for each release. Industry found out a common solution in the form of Automation testing. It is a sequence or series of actions performed by automation tool. The actions can or can not be dependent on human. During 2001 to 2005, the initial days of automation testing, automation team used to be a part of quality assurance or manual testing team.
A automation team was defined as a set of technology enthusiastic functional test engineers who writes VBA,or some small code base to make their job easy. Gone those days when team mainly used to depend on manual testing and automation was given a lower priority.
With the involvement of software design model like waterfall to v model to now days agile or rapid development ,automation becomes obvious. Agile and rapid development are meant to develop and change in a speedy way.So Automation needs to provide support in the same way.The nature of automation starts from unit testing to full execution, On demand execution to scheduled execution. Testing started rely on robot. Magnitude of testing is full day and night.When adopting such a huge speed, as a company, we need to adopt some strategies.
Here are the top ten strategies to make test automation as a service:
Must Read::Framework and Tool Evaluation || Action Concept in QTP
1. When to use Automation:
Go for automation testing if the project is long term and features will be tested again and again.Automation testing is only preferable if and only if the nature of testing is repeatable. Automation takes time to be mature but effective to catch bugs. It will give QA more confidence to look into the non automated area.So use automation in such cases where-
- It saves time
- Increase accuracy
- Reduce cost
- Better ROI
- Better coverage
2. Test Tool:
|Image Credit https://www.qasource.com|
Many a times, the tool sales person /team showcase their tool,in such way, that the tool they are selling can automate everything a customer need. But reality is far from the show. So a expensive tool does not really make a functional team member an automation expert. Even they can not replace an automation expert.
Now the last aspect of tool selection could be o select some tool where your most of the knowledge base are. Simply if your resource is familiar to some technology or test tool or language, prefer the related tool.This will save time of initial learning and give automation a good start.
Test tool selection depends on the following factors:
- Affordability [Cost involves for this tool]
- Flexibility [How flexible the tool company about selling the tool or license]
- Compatibility [Can the tool support the UAT]
- Functionality[How easy the GUI or what feature the tool is offering]
- Usability[how easy to operate the tool]
- Maintainability [How easy to maintain the scripts]
If you are justifying a tool and not a member of management, please involve them from day one. And if you are a manager, please join the discussion to understand about the tool.If any team wants to get benefited with automation testing,management support is must. They must fund for R&D activities,pilot projects,fund for architecture development etc. Mostly it is top -down approach rather than a bottom up activity.
Must Read:: HTML Report in OATS::: HTML Report in QTP
Reporting should be the next major aspect of strategy. From the beginning, we need to standardize the reporting. Also we need to upgrade the reporting whenever necessary. Reporting should be customized or can be drilled to the lowest level. Creating various levels of abstraction is a very good approach.I like the jUnit/TestNG report for selenium. The consumers may be different for different level of reporting. Functional engineers may be drilling down till last level,Managers may be interested about how many scripts,how many pass etc. Top management may be looking at overall percentages.
Always tailor automation and information to meet automation objectives.
Must Read:: Top 12 framework Features:::Top 10 Framework Features
We need to design a structured approach by which automation test engineers can produce maximum testcases by providing minimum efforts. We need to ensure maximum performance.Prepare best practices . Best would be to adopt a modular approach. Better to use a library driven approach rather putting everything in a script. Decomposition of the flow should be based on independent tasks. Not on the basis of a big flow. The independent task lets call Units should have only one trigger point and only one outcome. With these units, one can make flows.
Always create layer of framework. The top layer should be a simple script with simple English. GEB and SPOC are very popular in this aspect. The test inventory should be very user friendly to a non technical user.
There are many IDEs(Integrated development environment) available in the market.One should design the framework such a way that it can overcome the limitation of the tool's GUI Part.
So framework development is not only a conceptual thing. There are finance area also attached to it. Be very careful to understand the requirements,tool selections and hiring.
5. Test Script:
Test script should be independent but structured. Each test case should be easy maintainable.Follow Hermetic Test Pattern for test development. Even now single ton approach is also nice.Test cases should be accumulated in easy and random way. Functional team should set testcase as per their requirement.Decomposition of the flow should be based on independent tasks. Not on the basis of a big flow. The independent task lets call Units should have only one trigger point and only one outcome. With these units, one can make flows.
Another way of automation is to check the heath of the test cases. And keep updating your inventory. The more we will test our testcases more we will get the quality. It will surely reduce pesticide paradox in automation.
6. Clear Expectation:
Prepare a capability analysis of the engineers who are working in the team. second,prepare an estimation of the scripts according to the complexity[may be simple ,medium,complex,very complex]. Now set an target like in a day ,how many simple,medium,complex and very complex you can deliver.Estimate accordingly. I would suggest take 1-2 days buffer while planing and provide estimate accordingly.
Second important aspect is to get transparency. Yes. depending upon the maturity of the employees , we must be transparent to them. Information is power in our IT industry. In few cases, it is seen that the information is not shared to the team. Manager or lead poses to be over-smart thinks that this act gave them lot of power. Actually this is a poor managed team where actually nobody knows where the project stands.
The best approach is to divide and rule approach. Look at your requirements then identify the smaller requirements. Now identify the smallest unit testcase that can be automated. Once this identification is done,look for reusable functions,common approaches. Build the helper functions. Then start automating with maximum re usability. In this way, many automation testcases can be automated within very short time.
Next to make test inventory scale-able and maintainable always depend on test data. Use testdata for triggering and running large scale testcase. There are many approaches to get or generate test data. But framework should support multiple format like-CSV,Spreadsheet,text file, random generation,xml. etc.Plug and play with these data source can give your test framework wings.
Use different tools to understand the progress in milestone by milestone manner.Budget control,statistical tools like six sigma,lean,Failure analysis,Fish-bone, process control ,key performance indicators,regulars,Standardized reporting should be used to analysis different scenarios.
Retrospective meetings should be well prepared rather just jotting few points and go unprepared.
Every claim good or bad should be supported by supporting data.
In automation industry , we have two categories of test engineers.
1. Designer- Mainly Test Architect
2. Automator- Who will automate the test.
Automators are multi talented like a developer. They need to understand the requirements, create test case,create testscript,create supporting testdata, execute the test,analysis of the failures,re execution, reporting. So it is always better to have programming knowledge in automators.
How many engineers need to be hired depends on the load of work you will have. If the project follows waterfall model,you will get sometime to take a look of the work and estimate accordingly. But if the project follow a agile , then it is tough to get the projected load. So it is always better to have one or two shadow worker in buffer.Most of the companies now are creating talent pool.A pool of engineers with required skillset.
As I said earlier , It is no more a manual test engineer's job to automate application,It requires dedicated automation developer now a days. Apart from a dedicated team,the automation team requires a manager or a technical architect to drive the team. They should be from automation background. If the hiring is not correct way, like a manual test engineers given automation responsibilities or a manual manager is guiding automation team, then in most of the cases team lacks of motivation and innovation.
9. Operational Model:
As in agile methodology the load of the work is little unpredictable and requirements may get changed. As a result the traditional operative model of billing[Hours calculation,timesheet entry] may not work for both the vendor and client. In this changing moment of software development it may be good to go cloud's way of billing. Deliver the agreed quantity and get the billing. It is always a win win situation for both the parties as it follows pay as use mode. This solves the not used,un billable resource problem. I have seen some model like the total no of scripts delivered, number of test execution happened,number of regression run,number of bugs logged.So according to your model,use quality center or jenkins to create more demand ,execution,daily execution,report to earn more.
Inside organization, follow different quality standard,documentation or CMMi level requirements.IT security,Data security,audit security,compulsory regulation,certification, quiz should be part of operation model. Employees should be encouraged to take the certifications,documentations,audit very seriously.
10.Do More with Automation:
Automation not only test your application but can do many more things. All tedious manual job like taking backup of the scripts,updating internal portal, posting report can be done with automation. I have heard one of the team automated the billing portal which was getting input data from review portal. I have experienced the automation of QC to OATS[Oracle Test Management] migration with the power of Selenium.
GUI testing is the mostly tested area in automation. But the success of GUI testing depends on many other factors. In my previous organization, in certain runs we have experienced 90% failures. While analyzing the failures we got to know that these were due to synchronization issue. Application was slow in nature. We use to give second round of execution of the failure testcases, and the 90% of them used to pass!!!.
After researching, we used direct database update,Seeded data and APIs to reduce the usage of GUI. That gave us speed of execution , less failure due to sync issues and more accurate bugs.
Well, more good was,we started supporting API testing as well.
Continuous Integration with automation testing is one of the prime factor for automation success.
These are the basic ten strategies that will surely support the TAAS. Let me know your views as well.