Results for Idea of Framework

How To Create Test Data for Advanced Test Structure or Test Framework

August 07, 2015
In my last couple of posts, Structure of Automation Test Cases and
Best Test Structure::Hermetic Test Pattern , I have shown each testcase
should be independent to each other. But to make them real
independent it requires testdata manipulation. Testdata also need to
be independent ad unique in nature.

Why it is required?
It is required to stop data pollution.Stop using polluted data usage
inflow and outflow. 
I am writing few common methods to make testdata independent:

Ways:
  1. Use testcase id or name at the end of the testdata like Cutomer_Name=”Rabin”+”Test123”;
  2. Use time stamp to make testdata unique like-Customer_Name=”Rabin”+”3103201515082015”;
  3. Use random number generator to create unique testdata.like-“Robin”+rand.nextInt(900) + 100;
I like the Random number generation method and if it is not possible to use, I generally use the testcase id.
More note on Random number generation:
RandomStringUtils.random(10, false,true);

you need to import - org.apache.commons.lang3.RandomStringUtils package
this is very useful when generating 10 digit random number.
AtomicInteger counter = new AtomicInteger(1);
counter.getAndIncrement();

This is also very useful to get unique number.


image credit:http://www.sqledt.com/
How To Create Test Data for Advanced Test Structure or Test Framework How To Create Test Data for Advanced Test Structure or Test Framework Reviewed by Animesh Chatterjee on August 07, 2015 Rating: 5

Best Test Structure::Hermetic Test Pattern

August 07, 2015
In my last post-Structure of Automation Test Cases, I have shown how we can improve the test structure and make a better one.Too many tests will make feedback loop delayed and too less tests may not have enough coverage.Sometimes too many tests are added to a single E2E(End to End) script.This kind of test design will make it spaghetti. The goal is to design a significant approach that will lower down the feedback loop.
Now lets looks at the question below:

  • What is the biggest benefit or goal ? 
  • What is the cost ? 
  • How is the team? 
  • What training team needs? 
  • What is holding automation back? 
We have to prioritise the issues and resolve one by one. Now in this post I am trying to venture the pattern on which the structure is made. The test structure is made on Hermetic Test Pattern. As per the Hermetic test pattern each test should be completely Independent and should not depend on other test. So,It should be atomic in nature and self sufficient.As per the maintenance concern,each test case should be maintained concern,each testcase should be maintained separately and if any dependency outside of the test found(may be a 3rd party library,3rd party service) test engineer should cut the linkage with the service.
The best way to start is to look into smaller objectives but add test slowly to make it mature.
Advantages: Each testcases are atomic in nature so no dependency on anything. Test takes clean start as precondition creates all test data No polluted data inflow our outflow. We can execute the test in different order on need based. We can create more molecularity which increase code re usability. As each testcase is atomic,parallel execution is still possible.The deep advantages are:

  1. Increase confidence in software quality.
  2. Earlier time to produce the software to the market means more revenue.
  3. Reduce the cost in testing.
  4. Consistent repeatable testing
  5. Run tests unattended.
  6. Find more regression bugs
  7. Run tests more often
  8. Better quality software
  9. Coverage is high
  10. Find more bugs
  11. Test on different systems.


Disadvantages: Duplication of code increases if not created modular type. Design testdata for each testcase Execution time increases. High resource utilisation.
Best Test Structure::Hermetic Test Pattern Best Test Structure::Hermetic Test Pattern Reviewed by Animesh Chatterjee on August 07, 2015 Rating: 5

How to give test URL to my framework

January 26, 2012


This section is dedicated to my viewers.Here they can ask question and I will be happy to help or redirect to some page what is best suitable for them. 







Hi Animesh
How to change/give environment name dynamically -to my framework? .Right now we are hardcoding in our scripts.

Regards
Veenetha
Hi Veenetha,
Thank you very much for writting me!! I am trying my level best to solve your query.

Well veenetha,there are several ways how you can insert environment name into your framework:----
1)Ofcourse,I agree,the hardcoding into a script is one of the options.
 String  URL="www.abc.com"
2)The second improvement could be,you can create a dialog box(input to take the environment.This you will place master/global driver and the value you will pass to the child scripts.

Or,you get the URL from user and save it to the notepad and read it dynamically afterwords.
3)Create an environment file for providing URL.I found advantage on creating a separate file for environment details.Updation is very easy and effective.
Say active_URL="google.com"
Tomorrow you can change it to
  active_URL="yahoo.com"
  active_URL="google.com"


My personal suggestion is to go with either 2 or 3.
How to give test URL to my framework How to give test URL to my framework Reviewed by Animesh Chatterjee on January 26, 2012 Rating: 5

Features of today's Automation Testing Framework Part-3

January 08, 2012
Hope you have enjoyed Part one and part two of Framework Building. Now I can say It is bit updated framework. This post will talk more about the fine tuning your framework development.
Portability:When I said Distributed nature in part two What I mean that any test can be operated from any machine.Let us improve the framework little bit more. I can copy the master driver which will automatically create the folder structure add up the necessary dependent files and will be ready to run any testcases.
Ease of Functionality:  This can be achieved when we can give user a platform where user can plug in and plug out any extra feature he/she required/not required respectively.Like Manager , business analysts and functional people can control global testing without knowing the tool.This will truly become Skill Utilization too.
Capture all : Yes!!! that is the key for a successful  framework.Always capture Step information,log,Success criteria,Exit and entry criteria and Error Information.
Scoping and Descoping:   The more you make your framework user friendly , by providing Easy controlled and controllable actions ,More it will be a go to tool for all.This feature will allow business user to scope a set of testcases or scenario for a specific business flow.
Advanced Coverage: The more you fine tune your application the more the test engineers are going to use it.If your Framework supports Negative testing ,their is nothing like it.Manual persons also can jump into your framework to speed up their application testing.It is nothing but to increase your coverage i.e Automation Coverage This simply saving manual effort.
NO Object Repository:  Now a days testing demands very fast execution and report generation.Please refer one of my post on Having Repository.So now a days people like to have a pattern based testing where there will be no Object Repository.Pattern will detect the type of field and will do actions.
Hassle Free Integration: Framework should support a Hassle free way to integrate a newly build module. I suggest a GUI component to do this integration. By this an Automation test engineer or a functional test engineer can make a code and can submit for further use.Framework should not be so complicated that integration of modules will take lot of time and it requires a lot of effective testing time.
Implementation of Source control System : This is one of the basic criteria to make an advanced framework.Yes!!! Track your source code to make everybody understand what kind of change is applied upon the code.History will make sure if the piece of code is untouched or error prone and who has done what kind of change.There is always a chance that the new code is error prone and can create regression. For that case we can always go back to previous version.   
Recordings: New framework should support video recording of each and every testcase execution report. Based on the requirements, framework should support the pass video or fail video along with steps and logs.
 
Features of today's Automation Testing Framework Part-3 Features of today's Automation Testing Framework Part-3 Reviewed by Animesh Chatterjee on January 08, 2012 Rating: 5

How Object repository can affect execution performance?

September 22, 2011
 image
Introduction:
There are several approaches in the market to generate your script and  the framework implementation. But we need to think a manner where script development time ,script maintenance time becomes less.Most importantly we need to find an approach where the execution time is also become less.
What is the importance of downing the execution time?
Well, i am inspired by a philosophy that automation test engineers should not loose time by executing test script. All the test script should be configured once and should be able to run unmanned manner. More over these scripts will run in a computer assisted environment. Computer is not that intelligent as we are.If you insert enough intelligence into script it might slow down the process and on the other side intelligence is less..there is a chance that script will error out. So what is the best approaches to automate your script that will serve long without issue is a question.


Role of Object Repository
Object repository is a place where your testing tool use to store information about objects under test for a flow. This may be .tsr file or a straight xml file.During run time script takes reference of  this file and identify the objects.
image
It is seen if the OR size is less that means test engineers are doing more descriptive programming.Descriptive programming is very powerful but it is a time consuming process.Hence it will slow down the process.
On the other hand if the OR file is big.The seek time will be high. There might be a scenario where object conflict will occur.This also slow down the process.
image
Here this picture shows if OR files grows high in Size how the response time of the scripts increase.
image
image
As per experience, An O.R file should be between 5 to 12 MB..Above that or below that performance is not as good as we expected to be.

Check out here how to speed up an automation framework
Technorati Tags: ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
Windows Live Tags: Object,repository,execution,performance,Introduction,script,framework,implementation,manner,development,maintenance,Most,importance,philosophy,automation,computer,environment,intelligence,error,Approach,Role,tool,objects,reference,size,Descriptive,scenario,Here,files,response,Above,Check,scripts,engineers
WordPress Tags: Object,repository,execution,performance,Introduction,script,framework,implementation,manner,development,maintenance,Most,importance,philosophy,automation,computer,environment,intelligence,error,Approach,Role,tool,objects,reference,size,Descriptive,scenario,Here,files,response,Above,Check,scripts,engineers
Blogger Labels: Object,repository,execution,performance,Introduction,script,framework,implementation,manner,development,maintenance,Most,importance,philosophy,automation,computer,environment,intelligence,error,Approach,Role,tool,objects,reference,size,Descriptive,scenario,Here,files,response,Above,Check,scripts,engineers
How Object repository can affect execution performance? How Object repository can affect execution performance? Reviewed by Animesh Chatterjee on September 22, 2011 Rating: 5

How file size can affect execution time?

September 18, 2011

ktip
Introduction:
There are several approaches in the market to generate your script and  the framework implementation. But we need to think a manner where script development time ,script maintenance time becomes less.Most importantly we need to find an approach where the execution time is also become less.
What is the importance of downing the execution time?
Well, i am inspired by a philosophy that automation test engineers should not loose time by executing test script. All the test script should be configured once and should be able to run unmanned manner. More over these scripts will run in a computer assisted environment. Computer is not that intelligent as we are.If you insert enough intelligence into script it might slow down the process and on the other side intelligence is less..there is a chance that script will error out. So what is the best approaches to automate your script that will serve long without issue is a question.
Please read here for the best Approach for automation.
File Structure and size:
Once you finalized your approaches the next important aspect is the component development. Yes the most important component of any framework is File.
Now what ever framework we develop ,the structure always resides inside the Hard disk.Even if we came up with  high speed data bus,high speed data transfer mechanism,during execution it is seen the number of files ,structure and file size slow down the process.
  • if the file size is very less  that means we need to incorporate a lot of files….. the seek and latency time goes high
  • if the file size is high that means we are including every thing inside one library and seek time becomes high.

image
The guideline for file design is written below:
  • try to write same functions into a file but do not club all the function into a single file.
  • Number of files which will be loaded first (like-common functions –common to the entire flow..needs to be called almost every steps, path library) should be less 2/3.
  • library specific to a module will be loaded dynamically and will be released after use.
  • Garbage collector should be called after dropping one file . it depends on the scenario to scenario.
  • More over any point of time the files are handled can not exceed 3/4.
  • Try using lightweight HTML reporting instead of Excel Reporting.(excel-2000 and above takes heavy space in RAM….it is always greater than 2 MB.
  • Using of CSV as a input file is the best approach. One more best approach is to use XML.
How file size can affect execution time? How file size can affect execution time? Reviewed by Animesh Chatterjee on September 18, 2011 Rating: 5

How to Speed up Automation Framework?

September 17, 2011

ktip
Introduction:
What is the catch for defining and developing new framework? somebody might tell the cost for development should low. High productivity.Easy maintenance might be another aspect.But we never consider the time factor for running the scripts. After all this is computer assisted testing.Who cares about the timing? As long as it is a unmanned execution ,nobody cares for execution. This article talks about the a brief idea about how to speed up an automation execution time.
Every scripts are based on some framework.May be a record playback or a serious framework. So if you can configure your framework effectively, run time or execution time will go low.
I have jotted down few improvement areas 
We need to design our automation such a way that we can swap to different tools when needed. We need to preserve the tool independence. Apart from these test engineers should take advantages of some of the downtime or judiciously takes time away from regular tasks that have a due date pretty format, a lot can be accomplished.Object oriented design allowed us to play into test harness in a standard way and reuse existing components to test new features,minimizing the code(power of re usability).
Speeding up automation does not mean a good architecture or a well defined framework. Even though with lot of investment if we create a good architecture for extendable automation framework and script design.it may still fail even though if found high quality bugs.Initial success is no guarantee of long term success.Technical aspects are not the only factors that influence a project.

Before going into details of speeding up automation,we must consider 2 aspects of architectures.

  1. Technical architecture(Software engineering/experience)
  2. Business architecture (Goal of business,bug test case automation)
The business architecture should always exists. It is not only automator's property or a part of his workflow. Manager should push others to learn about automation framework.Manager also should know and understand the power of automation.
The framework creator should be given enough time or encouragement to share his knowledge and expertise.The engineers also need to understand the programming knowledge and enhance the skill.
The automation effort should be tracked in to a good way in a holistic manner.
Speeding up automation also can be found by finding the real objective. The objectives needs to be identified,reduce manual labor costs,improved product quality ,increased testing flexibility,free manual resource for more area to test,reducing strain on network resources as fewer testers are required.Strong support from management continued with the assurance that issues such as funding,test design quality,originally from outside the team and affecting the automation's progress needs to be addressed.Sometime it is better to have a view from outsiders about the process,framework and code.The feedback may a give a lot of internal changes.

Independent testing team concept is a booming concept.Now a days it has become a service line for many companies.Independent testing service optimizes resources availability provides standardize methodology,quality. This process also ensures minimizing defect risks for predefined requirements. For really going good, I recommend, to divide the group into two different ways:
  • Testers
  • Core automation engineers
Testers: Mainly the test engineers who drives testing(Functional testers,QA or acceptance testers).They are more close to the business. They understand the business goal.
 The responsibilities are:   
  • Defines the testing scope,define what is to be tested.
  • Defines the test plan that contains testing quote.
  • Create manual test scripts,entry and exit criteria(outline of the test). 
  • Create report as a proof after executing the testcase manually
  • Supply test data as when it is required
Core Automation Engineers: They are the one who translates the manual scripts into an automation testcases. They may be far from business goals more close to technical aspects nearer to developers. They should have a great programming knowledge.
The responsibilities are:
  • Prepare a road map/test plan for each system to be tested.
  • Prepare automation test script to cover or apply each test quote
  • Write reliable scripts to collect final outcome(It should contain report,bugs,logs etc) suitable for manual test engineers.
  • Become an expert in execution who can understand the script,API.
The separation between testers and automation engineers is necessary as  testing and emulation of user input should not clouded and cross contaminated. Automation engineers needs to have testcases to automate. The testers group are the one who provides them. To bring up speed, testers need to have a constant flow of manual scripts towards automation. Many automation projects either fail or become slow if they don't perform this activity seriously. Converting manual test engineers to automaton test engineer gives a poor result or the strategy may fail.But this way of creating a testing team could be an issue in hiring as testing team grows high. Recent trend is to hire developer vs tester ratio becomes 1:2 from 1:3 to 1:4.So funding could be an issue.

The test automation architect is a key player who is responsible for developing and maintaining the test framework.Architect defines and deploys best practice and coding standards to be used in implementation of the tests and train  test engineers to do minor changes in the abstraction layer. So when your test set is little mature, hiring a test architecture will definitely ed up the scripting and further process.

Abstraction Layer:your abstraction layer should be a balanced one. If the abstraction layer is too thin(too many fields are set in a specific test script), it could lead to a cumbersome maintenance activity. (if any of these filed gets updated,hundreds of field needs to be maintained). If your abstraction layer is too thick(too large or too advanced), the complexity will be too huge for this layer

If we are advancing towards an upgraded framework,test engineers need to learn new technologies,new techniques. It means we need more upgraded class room training or offline training. So hiring experts or creating an expert group who can train and step in when needed,is a good idea.But that may significantly hit the ROI.But we can not overlook the cost of learning and upgraded yourself.

When your testing process are really matured and want to do more speedy testing. Daily Build Engineers come into play. The person in charge of daily builds ensures that the test batch is executed ,makes the first analysis of the result and pass the work needed to analyze the test result to the automation test engineers or QA.A good CI(Continuous Integration) process is must.Hudson or jenkins or cruse control etc will make the automation process more speedy.
  



How to Speed up Automation Framework? How to Speed up Automation Framework? Reviewed by Animesh Chatterjee on September 17, 2011 Rating: 5
Powered by Blogger.