Monday, September 19, 2011
ISTQB Exam Weighting
ISTQB Exam Weighting
1) Test Concepts (++++)
2) Test Cases (++)
3) Test Plan (++)
4) Test Builds (+++)
5) Reports (+)
6) Testing Planning (++)
7) Validation (++++)
8) Requirements (+++)
9) Test Forms (+)
10) Control Procedures (++++)
11) Testing Deliverables (+++)
12) Test Execution (+++)
13) Test Automation Concepts (++)
14) Problem Resolution (+)
Weighting Key in the Test
+ = 1 - 5%
++ = 6 - 15%
+++ = 16% - 30%
++++ = over 31%
1) Test Concepts (++++)
2) Test Cases (++)
3) Test Plan (++)
4) Test Builds (+++)
5) Reports (+)
6) Testing Planning (++)
7) Validation (++++)
8) Requirements (+++)
9) Test Forms (+)
10) Control Procedures (++++)
11) Testing Deliverables (+++)
12) Test Execution (+++)
13) Test Automation Concepts (++)
14) Problem Resolution (+)
Weighting Key in the Test
+ = 1 - 5%
++ = 6 - 15%
+++ = 16% - 30%
++++ = over 31%
Friday, September 16, 2011
Money making, software testing career and secrets of a richest tester
This is a guest article from Pradeep Soundararajan. He is a Consulting Tester, Satisfice Inc & Software Testing Magician. Reach him at his blog Tester tested
These days a lot of people who pass out of engineering and science colleges are interested about software testing as a career. When I passed out at a time when the IT had started to boom back in India, most of the fresh graduates with whom I interacted didn’t even know there existed jobs or careers like software testing.
I was offered a job as a tester in a start up for 7440 rupees a month compared to fresh developers (who were picked from better institutes from where I graduated) being paid 34,500 rupees a month.
Today there isn’t such a huge difference between what testers and developers get paid and I consider this generation to be luckier than my generation without ignoring the idea that my generation might have been luckier than its previous generation.
When I started my career as a software tester, I didn’t find any training centre, which could coach me on software testing, and I lacked guidance. I didn’t know about Google and its power of search.
In the organization I worked for, there existed a senior software tester, not by designation or for the technical competence but just that he joined that organization 6 months before I did. He happened to coach me. I blindly believed all that he said about testing. I believed him and never questioned him.
By believing whatever he said I think I was becoming dumb. I looked for someone who could coach me and found two great people, one a developer and other a software architect in the organization whose ideas were much impressive than the senior software tester.
The duos were more open to questions from me as compared to the so-called senior software tester. When I questioned all things that I heard from the so-called senior software tester, I found that most of what the senior tester said was highly idiotic.
I realized that my quest in life was to see myself doing good or great testing in future. To do that, I must learn, I must learn, I must learn, I must practice, I must practice, I must practice…
What do I learn? What do I practice?
When I asked for information about software testing, some of my friends sent me material that was nothing more than, – types of testing, techniques of testing, different types of documentation, process of testing and development.
A question that I asked changed my life and you might want to know what that question is: Is there something beyond what all these people think software testing is which I can learn?
Now that leads to more questions. If it exists, where does is exist? Who has the information? How can I find it?
That lead me to discovering James Bach one of the world’s leading expert tester. His career graph is one of the most impressive career graph I have seen till date. He is a school drop out at 8th standard and yet became the youngest Test Manager of the world at the age of 20 in Apple Computers. He even helped Microsoft in Test Specification and was expert witness in court cases that involved investigations of the computer world. He has traveled to most countries where software testing is being done and has carried over consulting assignments there. He is a kind of tester that can make most testers in the world feel ashamed of their lack of skills, knowledge and maybe money. That reminds me to say, he has made lots of money.
I thought this man must have a secret with him that other software testers don’t know and I wanted to learn that. I found that James Bach is very similar to Jackie Chan as he considers skilled testing to be a mental martial art. Sorry, James doesn’t have any testing certification that you know of and he thinks certification doesn’t help, so don’t try to think of certification when you are thinking of James Bach, the great tester and guru of software testing.
I had to pass through several mental martial arts tests before I became his full time student. Let me not take you through the entire story but let you know that I reached a stage where he hired me to represent his company in India.
I don’t like comparing myself with others and run a rat race but some of my friends who were comparing with me were very disappointed as I progressed. I travel around the world speaking and coaching at international conferences. I am featured as an expert tester sometimes (which I acknowledge, I am not) in other countries. I have a fan following for my blog. I am an independent consult, working on different projects in a day and for different clients from different parts of the world. I coach, consult, speak, write, think, test, manage and learn software testing and problem solving. I was interviewed by CNBC as they considered me a problem solving expert and wrote a column for them as Expert problem solver. I was invited to manage testing for an organization products and services division with about three years of working as a software tester. I have tested over a hundred and twenty three products, so far.
Reputation means more money but if you do things just for gaining reputation you won’t get it. Reputation is a little tricky. People think it is about doing things what other people like but I think it is about other people liking what you are doing.
Don’t worry about too many “I”, I have written in this article and for the moment, think if you have so many “I” to say or probably even more, in testing that makes people to approach you for consultation, you would be making more money than you ever imagined you would make as a tester.
I want to see Indian testers make more money than what they have been making. That’s precisely why I am writing this article for you all.
To start in the journey, apply this heuristic: Question everything that – you hear, you see, you feel, you want to see, you want to hear, you want to feel, you don’t want to hear, you don’t want to feel and other things you think you missed.
How to apply this heuristic?
Let me give you an example to get you started: There is a common myth (which means something is fundamentally wrong but people blindly believe it) by which most testers to my knowledge in India live: Testing is done to improve Quality
Who said the above statement?
Why should I believe it?
By having the above idea that testing improves quality, can any tester on Earth say how much quality he has improved?
If a tester can’t say that then there is something wrong with the fundamental behind it.
Improve what quality?
What is quality?
Who defines what quality is?
Does a tester define what quality means?
If I go to a hotel and the hotel owner says he serves quality food and I as a customer think the quality is not good, whose view is important?
How can merely finding bugs improve quality?
So, if a tester reports 5000 bugs and the developer quits the organization the same day, has the quality improved?
So, if a tester finds 10000 bugs and doesn’t report them, has the quality improved?
In the above case, testing did happen, and hence did the quality improve?
If I as a tester report 50 bugs, and the developer in a context of fixing bugs introduces 100 more bugs, has the quality improved?
Why do all other testers don’t understand the fundamental that it is a developer who can improve the quality?
As a tester, isn’t my job to find information about quality than trying to think of improving the quality?
Oh my God! I have been misguided all this while. So what’s testing then?
Isn’t the above question, a good question?
Didn’t I learn from this that many people around us are fooling and that is what is stopping me from becoming someone like James Bach?
Do I want to be fooled?
Should I allow people, bugs, documents to fool me?
These days a lot of people who pass out of engineering and science colleges are interested about software testing as a career. When I passed out at a time when the IT had started to boom back in India, most of the fresh graduates with whom I interacted didn’t even know there existed jobs or careers like software testing.
I was offered a job as a tester in a start up for 7440 rupees a month compared to fresh developers (who were picked from better institutes from where I graduated) being paid 34,500 rupees a month.
Today there isn’t such a huge difference between what testers and developers get paid and I consider this generation to be luckier than my generation without ignoring the idea that my generation might have been luckier than its previous generation.
When I started my career as a software tester, I didn’t find any training centre, which could coach me on software testing, and I lacked guidance. I didn’t know about Google and its power of search.
In the organization I worked for, there existed a senior software tester, not by designation or for the technical competence but just that he joined that organization 6 months before I did. He happened to coach me. I blindly believed all that he said about testing. I believed him and never questioned him.
By believing whatever he said I think I was becoming dumb. I looked for someone who could coach me and found two great people, one a developer and other a software architect in the organization whose ideas were much impressive than the senior software tester.
The duos were more open to questions from me as compared to the so-called senior software tester. When I questioned all things that I heard from the so-called senior software tester, I found that most of what the senior tester said was highly idiotic.
I realized that my quest in life was to see myself doing good or great testing in future. To do that, I must learn, I must learn, I must learn, I must practice, I must practice, I must practice…
What do I learn? What do I practice?
When I asked for information about software testing, some of my friends sent me material that was nothing more than, – types of testing, techniques of testing, different types of documentation, process of testing and development.
A question that I asked changed my life and you might want to know what that question is: Is there something beyond what all these people think software testing is which I can learn?
Now that leads to more questions. If it exists, where does is exist? Who has the information? How can I find it?
That lead me to discovering James Bach one of the world’s leading expert tester. His career graph is one of the most impressive career graph I have seen till date. He is a school drop out at 8th standard and yet became the youngest Test Manager of the world at the age of 20 in Apple Computers. He even helped Microsoft in Test Specification and was expert witness in court cases that involved investigations of the computer world. He has traveled to most countries where software testing is being done and has carried over consulting assignments there. He is a kind of tester that can make most testers in the world feel ashamed of their lack of skills, knowledge and maybe money. That reminds me to say, he has made lots of money.
I thought this man must have a secret with him that other software testers don’t know and I wanted to learn that. I found that James Bach is very similar to Jackie Chan as he considers skilled testing to be a mental martial art. Sorry, James doesn’t have any testing certification that you know of and he thinks certification doesn’t help, so don’t try to think of certification when you are thinking of James Bach, the great tester and guru of software testing.
I had to pass through several mental martial arts tests before I became his full time student. Let me not take you through the entire story but let you know that I reached a stage where he hired me to represent his company in India.
I don’t like comparing myself with others and run a rat race but some of my friends who were comparing with me were very disappointed as I progressed. I travel around the world speaking and coaching at international conferences. I am featured as an expert tester sometimes (which I acknowledge, I am not) in other countries. I have a fan following for my blog. I am an independent consult, working on different projects in a day and for different clients from different parts of the world. I coach, consult, speak, write, think, test, manage and learn software testing and problem solving. I was interviewed by CNBC as they considered me a problem solving expert and wrote a column for them as Expert problem solver. I was invited to manage testing for an organization products and services division with about three years of working as a software tester. I have tested over a hundred and twenty three products, so far.
Reputation means more money but if you do things just for gaining reputation you won’t get it. Reputation is a little tricky. People think it is about doing things what other people like but I think it is about other people liking what you are doing.
Don’t worry about too many “I”, I have written in this article and for the moment, think if you have so many “I” to say or probably even more, in testing that makes people to approach you for consultation, you would be making more money than you ever imagined you would make as a tester.
I want to see Indian testers make more money than what they have been making. That’s precisely why I am writing this article for you all.
To start in the journey, apply this heuristic: Question everything that – you hear, you see, you feel, you want to see, you want to hear, you want to feel, you don’t want to hear, you don’t want to feel and other things you think you missed.
How to apply this heuristic?
Let me give you an example to get you started: There is a common myth (which means something is fundamentally wrong but people blindly believe it) by which most testers to my knowledge in India live: Testing is done to improve Quality
Who said the above statement?
Why should I believe it?
By having the above idea that testing improves quality, can any tester on Earth say how much quality he has improved?
If a tester can’t say that then there is something wrong with the fundamental behind it.
Improve what quality?
What is quality?
Who defines what quality is?
Does a tester define what quality means?
If I go to a hotel and the hotel owner says he serves quality food and I as a customer think the quality is not good, whose view is important?
How can merely finding bugs improve quality?
So, if a tester reports 5000 bugs and the developer quits the organization the same day, has the quality improved?
So, if a tester finds 10000 bugs and doesn’t report them, has the quality improved?
In the above case, testing did happen, and hence did the quality improve?
If I as a tester report 50 bugs, and the developer in a context of fixing bugs introduces 100 more bugs, has the quality improved?
Why do all other testers don’t understand the fundamental that it is a developer who can improve the quality?
As a tester, isn’t my job to find information about quality than trying to think of improving the quality?
Oh my God! I have been misguided all this while. So what’s testing then?
Isn’t the above question, a good question?
Didn’t I learn from this that many people around us are fooling and that is what is stopping me from becoming someone like James Bach?
Do I want to be fooled?
Should I allow people, bugs, documents to fool me?
Want to Get Paid to Play Games? Become a Game Tester!
We are covering almost all topics from software testing industry. Very less testing professionals are aware of Game Testing industry and Jobs. Game Testing industry is expanding really fast and now there are many “game design” and “video game testing” openings available in market. Here is a very interesting article on game testing which will give you a brief idea about game testing industry!
This is a guest article written by a Game Testing Expert “Malini”.
Game Testing Industry Introduction:
The video game testing industry is set to become the largest industry. In spite of the recession, there was no dearth in the sales of the game titles, although the game console sales were hit and the game testing companies had to revamp their strategies.
Gaming had its ups and downs over the years but it continues to grow leaps and bounds. Facebook application games are really path breaking with budding developers experimenting their knowledge. Episodic games are the new thing. Games for the iPhone are the new frontier.
So, no one in the game industry knows where games will be even two or three years from now. The only thing they know is that everything is changing and that the games that are released in a few years will be different from what we have now.
Following Jobs are made available by Gaming Industry:
Video game programming Jobs (designing video games)
Video game testing jobs
Designing Video games requires skilled and experienced video game designers. Testing video games is equally challenging as game tester needs to have a solid writing skill, very good communication skill and habit to keep attention for details.
Video game testers play critical role in game development industry. As video game programmers spend years deigning video games and video game tester needs to make sure it’s ready for release in very short time span.
What is a typical Game Testing Process?
Computer games take from one to three years to develop (depending on scale). Testing begins late in the development process, sometimes from halfway to 75% into development (it starts so late because, until then, there is little to play or test).
Once the testers get a version, they begin playing the game. Testers must carefully note any errors they uncover. These may range from bugs to art issues to logic errors. Some bugs are easy to document but many are hard to describe and may take several steps to describe so a developer can replicate or find the bug. On a large-scale game with numerous testers, a tester must first determine whether the bug has already been reported before they can log the bugs themselves. Once a bug has been reported as fixed, the tester has to go back and verify that the fix works – and occasionally return to verify that is has not reappeared.
Game Testing Strategy:
Evaluation of game rules:
Game rules adequately explain operation of all components of the game including features, free games etc. Game functions as defined by rules.
UI, Functional, Performance and Compatibility test:
Verify Games outcome and data are correctly shown when games are played. Verify Game Functionality such as Game Progress, game outcomes, handling of incomplete and re-started games, multi player games.
Verification the Integration points:
Check if game win determination aligns with game rules.
Reviewing gaming procedures:
Procedures will be reviewed by System management, player account management, tournaments and promotions.
Infrastructure and security review:
Require to verify all equipment and network implementation. Secure and reliable operation for example time synchronization, OS reliability and security.
How to Test Games?
This process is almost similar to product or web application testing. Here is the typical game testing process:
Identification: First analyze and identify the game rules and behavior
Functional Testing: Ensure game works as intended. This also includes integration testing with third party tools used if any.
OS and Browser compatibility: Most critical game testing part is to ensure game works on required Operating systems. For online games check functionality on all intended browsers.
Performance testing: This becomes critical for online games if gaming site handles betting on game. Game testers must verify if Game Testing site smoothly handles customer load.
Multi player Testing: For multi player games you need to verify the game functionality to handle all players and functionality with fair distribution of game resources to all players.
Reporting: Bug reporting to developers. Bug evidence need to produced and submitted through bug reporting system.
Analysis: Developers hold the responsibility to fix the bugs.
Verification: After the fix, bug need to be verified by the testers to confirm that it shouldn’t reappear.
Game Testing Tips:
1) Understand Random Number Generator evaluation (RNG): This is very important to add unpredictability in game. In most games this RNG system is used to map game outcomes.
2) First identify the “game algorithm” from Source code to identify issues in game application.
3) Verify the source code for appropriate use of random numbers and error handling. (Only if you know the source code)
4) Validate and evaluate the game predefined rules.
5) Verify consistency of game rules.
6) Make sure offensive content or material is never displayed.
7) Regularly Check Game history and system event logs.
8 ) Make sure Games outcome are displayed for a reasonable time.
9) Irrespective of Single/Multi player games we need to validate bandwidth and client software.
10) Verify Minimum/maximum limits of bets, deposits and other critical game symbols.
11) Verify correct game and system operation after game fail over and recovery.
12) Always verify all reports for data accuracy. Verify reports for date, time, number of wins, money etc.
13) Test System requirements. This is very important in game testing. Verify all the infrastructure and security requirements, Game equipments, network and game synchronization with OS.
14) Make sure sufficient information is always available to users to protect game players.
Game Testing Jobs:
Gaming field is getting much better day by day and Game Career as a Game designer or tester is very bright. There are many game testing professionals making decent amount of money as a video game testers, working from home. Present Internet generation bringing massive innovations and scope to grow. IT and Non-IT people are willing to spend their free time to play online and video games. “Game testing from home” is now a new trend to earn money. We can clearly see that it’s getting into our daily activities.
If anyone of you is trying hard getting into gaming industry then you need to have interest and passion that drives you to success. Due to addition of vast and complex new games, Game QA is no longer less technical than general software QA. Game testing was widely considered as a “stepping stone” position but now it’s becoming full-time job opportunities for experienced testers.
If you have passion for games and good understanding of testing methodologies, becoming a successful game tester is not difficult for you!
This is a guest article written by a Game Testing Expert “Malini”.
Game Testing Industry Introduction:
The video game testing industry is set to become the largest industry. In spite of the recession, there was no dearth in the sales of the game titles, although the game console sales were hit and the game testing companies had to revamp their strategies.
Gaming had its ups and downs over the years but it continues to grow leaps and bounds. Facebook application games are really path breaking with budding developers experimenting their knowledge. Episodic games are the new thing. Games for the iPhone are the new frontier.
So, no one in the game industry knows where games will be even two or three years from now. The only thing they know is that everything is changing and that the games that are released in a few years will be different from what we have now.
Following Jobs are made available by Gaming Industry:
Video game programming Jobs (designing video games)
Video game testing jobs
Designing Video games requires skilled and experienced video game designers. Testing video games is equally challenging as game tester needs to have a solid writing skill, very good communication skill and habit to keep attention for details.
Video game testers play critical role in game development industry. As video game programmers spend years deigning video games and video game tester needs to make sure it’s ready for release in very short time span.
What is a typical Game Testing Process?
Computer games take from one to three years to develop (depending on scale). Testing begins late in the development process, sometimes from halfway to 75% into development (it starts so late because, until then, there is little to play or test).
Once the testers get a version, they begin playing the game. Testers must carefully note any errors they uncover. These may range from bugs to art issues to logic errors. Some bugs are easy to document but many are hard to describe and may take several steps to describe so a developer can replicate or find the bug. On a large-scale game with numerous testers, a tester must first determine whether the bug has already been reported before they can log the bugs themselves. Once a bug has been reported as fixed, the tester has to go back and verify that the fix works – and occasionally return to verify that is has not reappeared.
Game Testing Strategy:
Evaluation of game rules:
Game rules adequately explain operation of all components of the game including features, free games etc. Game functions as defined by rules.
UI, Functional, Performance and Compatibility test:
Verify Games outcome and data are correctly shown when games are played. Verify Game Functionality such as Game Progress, game outcomes, handling of incomplete and re-started games, multi player games.
Verification the Integration points:
Check if game win determination aligns with game rules.
Reviewing gaming procedures:
Procedures will be reviewed by System management, player account management, tournaments and promotions.
Infrastructure and security review:
Require to verify all equipment and network implementation. Secure and reliable operation for example time synchronization, OS reliability and security.
How to Test Games?
This process is almost similar to product or web application testing. Here is the typical game testing process:
Identification: First analyze and identify the game rules and behavior
Functional Testing: Ensure game works as intended. This also includes integration testing with third party tools used if any.
OS and Browser compatibility: Most critical game testing part is to ensure game works on required Operating systems. For online games check functionality on all intended browsers.
Performance testing: This becomes critical for online games if gaming site handles betting on game. Game testers must verify if Game Testing site smoothly handles customer load.
Multi player Testing: For multi player games you need to verify the game functionality to handle all players and functionality with fair distribution of game resources to all players.
Reporting: Bug reporting to developers. Bug evidence need to produced and submitted through bug reporting system.
Analysis: Developers hold the responsibility to fix the bugs.
Verification: After the fix, bug need to be verified by the testers to confirm that it shouldn’t reappear.
Game Testing Tips:
1) Understand Random Number Generator evaluation (RNG): This is very important to add unpredictability in game. In most games this RNG system is used to map game outcomes.
2) First identify the “game algorithm” from Source code to identify issues in game application.
3) Verify the source code for appropriate use of random numbers and error handling. (Only if you know the source code)
4) Validate and evaluate the game predefined rules.
5) Verify consistency of game rules.
6) Make sure offensive content or material is never displayed.
7) Regularly Check Game history and system event logs.
8 ) Make sure Games outcome are displayed for a reasonable time.
9) Irrespective of Single/Multi player games we need to validate bandwidth and client software.
10) Verify Minimum/maximum limits of bets, deposits and other critical game symbols.
11) Verify correct game and system operation after game fail over and recovery.
12) Always verify all reports for data accuracy. Verify reports for date, time, number of wins, money etc.
13) Test System requirements. This is very important in game testing. Verify all the infrastructure and security requirements, Game equipments, network and game synchronization with OS.
14) Make sure sufficient information is always available to users to protect game players.
Game Testing Jobs:
Gaming field is getting much better day by day and Game Career as a Game designer or tester is very bright. There are many game testing professionals making decent amount of money as a video game testers, working from home. Present Internet generation bringing massive innovations and scope to grow. IT and Non-IT people are willing to spend their free time to play online and video games. “Game testing from home” is now a new trend to earn money. We can clearly see that it’s getting into our daily activities.
If anyone of you is trying hard getting into gaming industry then you need to have interest and passion that drives you to success. Due to addition of vast and complex new games, Game QA is no longer less technical than general software QA. Game testing was widely considered as a “stepping stone” position but now it’s becoming full-time job opportunities for experienced testers.
If you have passion for games and good understanding of testing methodologies, becoming a successful game tester is not difficult for you!
Test cases for Change Password
Test cases for Forgot Password are mentioned below:
* Field names should be as per the requirements.
* Each field should be having character limit. Character limit for each field will be mentioned in requirement document or in technical specification document if not in either of the documents then it will be definitely available in database diagram.
* Copy and Paste functionality can be allowed in username field but only paste functionality should be allowed in Password field. Copy and Paste functionality can be performed either from hot key like Ctr+C and Ctr+V or from context menu by right click from mouse. Copy functionality is not allowed in Password field because of security reasons.
* Valid and invalid characters in each field. If the requirement tells that Username field can accept alphabets, special characters and numeral values test cases for each set of values containing only alphabets or numeral values or special characters and with also with the different combination should be checked.
* Test case having invalid current password and valid New Password and Confirm Password. Proper error message should be shown.
* Mismatch in New Password and Confirm Password but valid current password. Proper error message should be shown.
* Test case for password having maximum and minimum number of characters for each field new password and confirm password.
* Error message stating each field is mandatory or Required if any of the field is blank and user click on Update button.
* Successful message on change of password. If requirement mentions about email being send on changing password then separate test cases for that. * Separate test cases for email contents should also be written.
* Test cases for Reset or Cancel button should be written if present in screen.
* Field names should be as per the requirements.
* Each field should be having character limit. Character limit for each field will be mentioned in requirement document or in technical specification document if not in either of the documents then it will be definitely available in database diagram.
* Copy and Paste functionality can be allowed in username field but only paste functionality should be allowed in Password field. Copy and Paste functionality can be performed either from hot key like Ctr+C and Ctr+V or from context menu by right click from mouse. Copy functionality is not allowed in Password field because of security reasons.
* Valid and invalid characters in each field. If the requirement tells that Username field can accept alphabets, special characters and numeral values test cases for each set of values containing only alphabets or numeral values or special characters and with also with the different combination should be checked.
* Test case having invalid current password and valid New Password and Confirm Password. Proper error message should be shown.
* Mismatch in New Password and Confirm Password but valid current password. Proper error message should be shown.
* Test case for password having maximum and minimum number of characters for each field new password and confirm password.
* Error message stating each field is mandatory or Required if any of the field is blank and user click on Update button.
* Successful message on change of password. If requirement mentions about email being send on changing password then separate test cases for that. * Separate test cases for email contents should also be written.
* Test cases for Reset or Cancel button should be written if present in screen.
Difference Between Performance Testing, Load Testing and Stress Testing – With Examples
← Tips for Writing Test Cases
Q. What is difference between Performance Testing, Load Testing and Stress Testing?
1) Performance Testing:
Performance testing is the testing, which is performed, to ascertain how the components of a system are performing, given a particular situation. Resource usage, scalability and reliability of the product are also validated under this testing. This testing is the subset of performance engineering, which is focused on addressing performance issues in the design and architecture of software product.
Performance Testing Goal:
The primary goal of performance testing includes establishing the benchmark behaviour of the system. There are a number of industry-defined benchmarks, which should be met during performance testing.
Performance testing does not aim to find defects in the application, it address a little more critical task of testing the benchmark and standard set for the application. Accuracy and close monitoring of the performance and results of the test is the primary characteristic of performance testing.
Example:
For instance, you can test the application network performance on Connection Speed vs. Latency chart. Latency is the time difference between the data to reach from source to destination. Thus, a 70kb page would take not more than 15 seconds to load for a worst connection of 28.8kbps modem (latency=1000 milliseconds), while the page of same size would appear within 5 seconds, for the average connection of 256kbps DSL (latency=100 milliseconds). 1.5mbps T1 connection (latency=50 milliseconds) would have the performance benchmark set within 1 second to achieve this target.
For example, the time difference between the generation of request and acknowledgement of response should be in the range of x ms (milliseconds) and y ms, where x and y are standard digits. A successful performance testing should project most of the performance issues, which could be related to database, network, software, hardware etc…
2) Load Testing:
Load testing is meant to test the system by constantly and steadily increasing the load on the system till the time it reaches the threshold limit. It is the simplest form of testing which employs the use of automation tools such as LoadRunner or any other good tools, which are available. Load testing is also famous by the names like volume testing and endurance testing.
The sole purpose of load testing is to assign the system the largest job it could possible handle to test the endurance and monitoring the results. An interesting fact is that sometimes the system is fed with empty task to determine the behaviour of system in zero-load situation.
Load Testing Goal:
The goals of load testing are to expose the defects in application related to buffer overflow, memory leaks and mismanagement of memory. Another target of load testing is to determine the upper limit of all the components of application like database, hardware and network etc… so that it could manage the anticipated load in future. The issues that would eventually come out as the result of load testing may include load balancing problems, bandwidth issues, capacity of the existing system etc…
Example:
For example, to check the email functionality of an application, it could be flooded with 1000 users at a time. Now, 1000 users can fire the email transactions (read, send, delete, forward, reply) in many different ways. If we take one transaction per user per hour, then it would be 1000 transactions per hour. By simulating 10 transactions/user, we could load test the email server by occupying it with 10000 transactions/hour.
3) Stress testing
Under stress testing, various activities to overload the existing resources with excess jobs are carried out in an attempt to break the system down. Negative testing, which includes removal of the components from the system is also done as a part of stress testing. Also known as fatigue testing, this testing should capture the stability of the application by testing it beyond its bandwidth capacity.
The purpose behind stress testing is to ascertain the failure of system and to monitor how the system recovers back gracefully. The challenge here is to set up a controlled environment before launching the test so that you could precisely capture the behaviour of system repeatedly, under the most unpredictable scenarios.
Stress Testing Goal:
The goal of the stress testing is to analyse post-crash reports to define the behaviour of application after failure. The biggest issue is to ensure that the system does not compromise with the security of sensitive data after the failure. In a successful stress testing, the system will come back to normality along with all its components, after even the most terrible break down.
Example:
As an example, a word processor like Writer1.1.0 by OpenOffice.org is utilized in development of letters, presentations, spread sheets etc… Purpose of our stress testing is to load it with the excess of characters.
To do this, we will repeatedly paste a line of data, till it reaches its threshold limit of handling large volume of text. As soon as the character size reaches 65,535 characters, it would simply refuse to accept more data. The result of stress testing on Writer 1.1.0 produces the result that, it does not crash under the stress and that it handle the situation gracefully, which make sure that application is working correctly even under rigorous stress conditions.
Q. What is difference between Performance Testing, Load Testing and Stress Testing?
1) Performance Testing:
Performance testing is the testing, which is performed, to ascertain how the components of a system are performing, given a particular situation. Resource usage, scalability and reliability of the product are also validated under this testing. This testing is the subset of performance engineering, which is focused on addressing performance issues in the design and architecture of software product.
Performance Testing Goal:
The primary goal of performance testing includes establishing the benchmark behaviour of the system. There are a number of industry-defined benchmarks, which should be met during performance testing.
Performance testing does not aim to find defects in the application, it address a little more critical task of testing the benchmark and standard set for the application. Accuracy and close monitoring of the performance and results of the test is the primary characteristic of performance testing.
Example:
For instance, you can test the application network performance on Connection Speed vs. Latency chart. Latency is the time difference between the data to reach from source to destination. Thus, a 70kb page would take not more than 15 seconds to load for a worst connection of 28.8kbps modem (latency=1000 milliseconds), while the page of same size would appear within 5 seconds, for the average connection of 256kbps DSL (latency=100 milliseconds). 1.5mbps T1 connection (latency=50 milliseconds) would have the performance benchmark set within 1 second to achieve this target.
For example, the time difference between the generation of request and acknowledgement of response should be in the range of x ms (milliseconds) and y ms, where x and y are standard digits. A successful performance testing should project most of the performance issues, which could be related to database, network, software, hardware etc…
2) Load Testing:
Load testing is meant to test the system by constantly and steadily increasing the load on the system till the time it reaches the threshold limit. It is the simplest form of testing which employs the use of automation tools such as LoadRunner or any other good tools, which are available. Load testing is also famous by the names like volume testing and endurance testing.
The sole purpose of load testing is to assign the system the largest job it could possible handle to test the endurance and monitoring the results. An interesting fact is that sometimes the system is fed with empty task to determine the behaviour of system in zero-load situation.
Load Testing Goal:
The goals of load testing are to expose the defects in application related to buffer overflow, memory leaks and mismanagement of memory. Another target of load testing is to determine the upper limit of all the components of application like database, hardware and network etc… so that it could manage the anticipated load in future. The issues that would eventually come out as the result of load testing may include load balancing problems, bandwidth issues, capacity of the existing system etc…
Example:
For example, to check the email functionality of an application, it could be flooded with 1000 users at a time. Now, 1000 users can fire the email transactions (read, send, delete, forward, reply) in many different ways. If we take one transaction per user per hour, then it would be 1000 transactions per hour. By simulating 10 transactions/user, we could load test the email server by occupying it with 10000 transactions/hour.
3) Stress testing
Under stress testing, various activities to overload the existing resources with excess jobs are carried out in an attempt to break the system down. Negative testing, which includes removal of the components from the system is also done as a part of stress testing. Also known as fatigue testing, this testing should capture the stability of the application by testing it beyond its bandwidth capacity.
The purpose behind stress testing is to ascertain the failure of system and to monitor how the system recovers back gracefully. The challenge here is to set up a controlled environment before launching the test so that you could precisely capture the behaviour of system repeatedly, under the most unpredictable scenarios.
Stress Testing Goal:
The goal of the stress testing is to analyse post-crash reports to define the behaviour of application after failure. The biggest issue is to ensure that the system does not compromise with the security of sensitive data after the failure. In a successful stress testing, the system will come back to normality along with all its components, after even the most terrible break down.
Example:
As an example, a word processor like Writer1.1.0 by OpenOffice.org is utilized in development of letters, presentations, spread sheets etc… Purpose of our stress testing is to load it with the excess of characters.
To do this, we will repeatedly paste a line of data, till it reaches its threshold limit of handling large volume of text. As soon as the character size reaches 65,535 characters, it would simply refuse to accept more data. The result of stress testing on Writer 1.1.0 produces the result that, it does not crash under the stress and that it handle the situation gracefully, which make sure that application is working correctly even under rigorous stress conditions.
Subscribe to:
Posts (Atom)