My reply would be 80%-20%....80% not required 20% required. Automation Testing is activated if there are two conditions
1. if you are doing regression testing for single or multiple dataset
2. if your application is stable enough to perform the test repeatedly
Perspective of automation testing can be divided into three parts
1. Development phase
2. Execution phase
3. Maintainance phase
When we get a big flow to automate , generally seen very less time is given to make it done.Management often misunderstood the term automatic and automated. Allocated time is not sufficient as the development team pushes the patching window or crosses the deadlines.This time an automation test engineers thinks to find out all the reusable components so that the development time becomes less.We use a lot of loop to do same activity again and again. We try to automate what ever is possible. Our objective is to check an outline to check the overall completeness.Smart enough to complete the task???
standard become important as more people will be developing in a very short span of time. Getting feedback from the framework users will help for latter acceptance. Automation Test development is a dedicated effort, there is nothing called free lunch.(often management asks for free or extra time of a tester to automate the flow.) Yes,we can automate in a shorter span of time. but shall we??
No!!. we must not automate poor tests.Efforts put into automating high value,high quality tests will improve the test execution and provides greater pay back. Similarly, simply automating manual testcases may not give us better ROI or a best automation suite.Select the tests judiciously.
Don't try to automate very complex flows at first.Automation should go in a slow but steady pace.Identify the proper testcase while automating the complex flows. If you automate chaos,all you will get a faster chaos.
Smoke testcases are designed to check health of the builds,releases. This type of tests saves hundreds of hours for testers and developers. Smoke testcases are good place to start the automation development, as they run almost everyday,on every checkin,or on every build.This is a less value add for testers error prone and boring.
Come to execution phase,most of the time after a successful completion of development , our scripts got stuck.This is the time when we have very less no of time to complete the task. 80% people say that development of the scripts are not proper. Even I found a instance where "for" loop sucks if it got stuck in between of the loop.Understanding the loop also a big factor for the execution master who is in charge of execution. sometimes we think if we can write statements instead of loop.Basically we look for consistency of testscripts at this point.
We need to structure our automation regression test for most successful execution.The main objective of regression suite to give confidence and assurance,not detecting bugs.
The second most important issue is data issue.Most of the time either data is not present or wrong data causes the execution to stop.Even using manual testdata sometimes gives wrong output. The third point may be data-hell condition. When we create a lot of modular scripts and try to create data for modular approach we create same data again and again.Say i need to change a set of data there is no mechanism to make the this latent change reflect everywhere. We have to open the child script datasheet one by one and change. there is no consistency at all between the data. These causes problem when running the scripts
The last part is Maintainance ...when the script is old and running for say 3/4 years ..it is extremely difficult without a proper comment or explanations.At this time we try to understand from code itself what it is doing. Remember the more number of lines of code is costly during maintenance .
In agile, we often automate some flows which are unstable. Actually, there is no value add while automating unstable functionalities as maintenance overhead is huge.