Power of QEngine lies at utilizing the script once recorded against all possible environment. One such feature is using the script recorded against application in one language against the same application in different language.
Scenario,
You have an application which is available in more than one language, say English & French. And you have automated the application in English language using QEngine. You can use the same automation script recorded against English language with the application in French language. This can be achieved by configuring the Locale Settings and configuring substitution in the localized language for English language keywords in the I18NEditor.
To configure the Locale settings, goto Settings -> Locale Settings and enable "Locale Needed" option, and select the Language and Country and click on "I18NEditor" link in the page. The I18NEditor will show all the keywords from the scripts in the suite. You should configure proper substitution for the keywords in the localized language. And then save all the configuration and settings. This will play the script against configured language application.
To run the test against English language, simply uncheck the option "Locale Needed" in the Locale Settings and then save the settings.
This is very powerful settings where you can enable the scripts recorded in one language to be played back against any other language without re-recording.
Thanks
Raghavan
Read good blog on Performance Testing in Google Testing Blog. The blog is posted by Goranka Bjedov. It is good read though. I had pasted below the content which speaks about performance testing.
| Quote: |
| This post is my best shot at explaining what I do, why I do it, and why I think it is the right thing to do. Performance testing is a category of testing that seems to evoke strong feelings in people: feelings of fear (Oh, my God, I have no idea what to do because performance testing is so hard!), feelings of inadequacy (We bought this tool that does every aspect of performance testing, we paid so much for it, and we are not getting anything done!), feelings of confusion (So, what the heck am I supposed to be doing again?), and I don't think this is necessary.
Think of performance testing as another tool in your testing arsenal - something you will do when you need to. It explores several system qualities, that can be simplified to:
Capacity - is the infrastructure sized adequately Scalability - can the system grow to handle future volumes Stability - does the system behave correctly under load So, I do performance testing of a service when risk analysis indicates that failing in any of the above categories would be more costly to the company than performing the tests. (Which, if your name is Google and you care about your brand, happens with any service you launch.) Note that I am talking about services - I work almost exclusively with servers and spend no time worrying about client-side rendering/processing issues. While those are becoming increasingly more important, and have always been more complex than my work, I consider those to be a part of functionality tests, and they are designed, created and executed by functional testing teams. Another interesting thing about performance testing is that you will never be able to be 100% "right" or 100% "done. Accept it, deal with it, and move on. Any system in existence today will depend on thousands of different parameters, and if I spent the time analyzing each one of them, understanding the relationships between each two or each three, graphing their impact curves, trying to non-dimensionalize them, I would still be testing my first service two years later. The thought of doing anything less filled me with horror (They cannot seriously expect me to provide meaningful performance results in less than a year, can they?) but I have since learned that I can provide at least 90% of meaningful information to my customers by applying only 10% of my total effort and time. And, 90% is more than enough for vast majority of problems. So, here is what I really do - I create benchmarks. If I am lucky and have fantastic information about current usage patterns of a particular product (which I usually do), I will make sure this benchmark covers most operations that are top resource hogs (either per single use or cumulative). I'll run this benchmark with different loads (number of virtual users) against a loosely controlled system (it would be nice to have 100 machines all to myself for every service we have, which I can use once a day or once a week, but that would be expensive and unrealistic) and investigate its behavior. Which transactions are taking the most time? Which transactions seem to get progressively worse with increasing load? Which transactions seem unstable (I cannot explain their behavior)? I call this exploratory performance testing, and I'll repeat my tests until I am convinced I am observing real system behavior. While I am doing this, I make sure I am not getting biased by investigating the code. If I have questions, I ask programmers, but I know they are biased, and I will avoid getting biased myself! Once I have my graphs (think, interesting transaction latencies and throughput vs. load here) I meet with the development team and discuss the findings. Usually, there is one or two things they know and have been working on, and a few more they were unaware of. Sometimes, they look over my benchmark and suggest changes (could you make the ratio 80:20, and not 50:50?) After this meeting, we create our final benchmark, I modify the performance testing scripts, and now this benchmark will run as often as possible, but hopefully at least once a night. And, here is the biggest value of this effort: if there is a code change that has impacted performance in an unacceptable way, you will find out about it the next day. Not a week or a month later (How many of us remember what we did in the last month? So, why expect our developers to do so?) Here is why I think this is the right thing to do: I have seen more bad code developed as a result of premature performance optimizations - before the team even thought they had a problem! Please don't do that. Develop your service in a clean, maintainable and extensible manner. Let me test it, and keep regression testing it. If we find we have a problem in a particular area, we can then address that problem easily - because our code is not obfuscated with performance optimization that have improved code paths that execute once a month by 5%. I can usually do this in two - four weeks depending on the complexity of the project. Occasionally, we will find an issue that cannot be explained or understood with performance tests. At that point in time, we look under the hood. This is where performance profiling and performance modeling come in. And, both of those are considerably more complex than performance testing. Final word: monitor your services during performance tests. If you do not have service related monitoring developed and set up to be used during live operations, you do not need performance testing. If the risks of your service failing are not important enough that you would want to know about it *before* it happens, then you should not be wasting time or money on performance testing. I am incredibly lucky in this area - Google infrastructure is developed by a bunch of people who, if they had a meeting where the topic would be "How to make Goranka's life easy?", could not have done better. I love them - they make my job trivial. At a minimum, I monitor CPU, memory and I/O usage. I cannot see a case when you would want to do less, but you may want to do a lot more on occasion. |
[ Currently: Working ]
Read good blog on Performance Testing in Google Testing Blog. The blog is posted by Goranka Bjedov. It is good read though. You can access google testing blog at http://googletesting.blogspot.com/
| Quote: |
| This post is my best shot at explaining what I do, why I do it, and why I think it is the right thing to do. Performance testing is a category of testing that seems to evoke strong feelings in people: feelings of fear (Oh, my God, I have no idea what to do because performance testing is so hard!), feelings of inadequacy (We bought this tool that does every aspect of performance testing, we paid so much for it, and we are not getting anything done!), feelings of confusion (So, what the heck am I supposed to be doing again?), and I don't think this is necessary.
Think of performance testing as another tool in your testing arsenal - something you will do when you need to. It explores several system qualities, that can be simplified to: Speed - does the system respond quickly enough Capacity - is the infrastructure sized adequately Scalability - can the system grow to handle future volumes Stability - does the system behave correctly under load So, I do performance testing of a service when risk analysis indicates that failing in any of the above categories would be more costly to the company than performing the tests. (Which, if your name is Google and you care about your brand, happens with any service you launch.) Note that I am talking about services - I work almost exclusively with servers and spend no time worrying about client-side rendering/processing issues. While those are becoming increasingly more important, and have always been more complex than my work, I consider those to be a part of functionality tests, and they are designed, created and executed by functional testing teams. Another interesting thing about performance testing is that you will never be able to be 100% "right" or 100% "done. Accept it, deal with it, and move on. Any system in existence today will depend on thousands of different parameters, and if I spent the time analyzing each one of them, understanding the relationships between each two or each three, graphing their impact curves, trying to non-dimensionalize them, I would still be testing my first service two years later. The thought of doing anything less filled me with horror (They cannot seriously expect me to provide meaningful performance results in less than a year, can they?) but I have since learned that I can provide at least 90% of meaningful information to my customers by applying only 10% of my total effort and time. And, 90% is more than enough for vast majority of problems. So, here is what I really do - I create benchmarks. If I am lucky and have fantastic information about current usage patterns of a particular product (which I usually do), I will make sure this benchmark covers most operations that are top resource hogs (either per single use or cumulative). I'll run this benchmark with different loads (number of virtual users) against a loosely controlled system (it would be nice to have 100 machines all to myself for every service we have, which I can use once a day or once a week, but that would be expensive and unrealistic) and investigate its behavior. Which transactions are taking the most time? Which transactions seem to get progressively worse with increasing load? Which transactions seem unstable (I cannot explain their behavior)? I call this exploratory performance testing, and I'll repeat my tests until I am convinced I am observing real system behavior. While I am doing this, I make sure I am not getting biased by investigating the code. If I have questions, I ask programmers, but I know they are biased, and I will avoid getting biased myself! Once I have my graphs (think, interesting transaction latencies and throughput vs. load here) I meet with the development team and discuss the findings. Usually, there is one or two things they know and have been working on, and a few more they were unaware of. Sometimes, they look over my benchmark and suggest changes (could you make the ratio 80:20, and not 50:50?) After this meeting, we create our final benchmark, I modify the performance testing scripts, and now this benchmark will run as often as possible, but hopefully at least once a night. And, here is the biggest value of this effort: if there is a code change that has impacted performance in an unacceptable way, you will find out about it the next day. Not a week or a month later (How many of us remember what we did in the last month? So, why expect our developers to do so?) Here is why I think this is the right thing to do: I have seen more bad code developed as a result of premature performance optimizations - before the team even thought they had a problem! Please don't do that. Develop your service in a clean, maintainable and extensible manner. Let me test it, and keep regression testing it. If we find we have a problem in a particular area, we can then address that problem easily - because our code is not obfuscated with performance optimization that have improved code paths that execute once a month by 5%. I can usually do this in two - four weeks depending on the complexity of the project. Occasionally, we will find an issue that cannot be explained or understood with performance tests. At that point in time, we look under the hood. This is where performance profiling and performance modeling come in. And, both of those are considerably more complex than performance testing. Both great tools, but should be used only when the easy tool fails. Tools, tools, tools... So, what do we use? I gave a presentation at Google Test Automation Conference in London on exactly this topic. I use open source tools. I discuss the reasons why in the presentation. In general, even if you have decided to go one of the other two routes (vendor tools or develop your own) check out what is available. You may find out that you will get a lot of information about your service using JMeter and spending some time playing around with it. Sure, you can also spend $500K and get similar information or you can spend two years developing "the next best performance testing tool ever," but before you are certain free is not good enough, why would you want to? Final word: monitor your services during performance tests. If you do not have service related monitoring developed and set up to be used during live operations, you do not need performance testing. If the risks of your service failing are not important enough that you would want to know about it *before* it happens, then you should not be wasting time or money on performance testing. I am incredibly lucky in this area - Google infrastructure is developed by a bunch of people who, if they had a meeting where the topic would be "How to make Goranka's life easy?", could not have done better. I love them - they make my job trivial. At a minimum, I monitor CPU, memory and I/O usage. I cannot see a case when you would want to do less, but you may want to do a lot more on occasion. |
Another QEngine play settings which will be useful to improve script efficiency is Exception Handling. Below i have briefed on QEngine Exception handling feature.
Exception Handling
There were four different exception that can occur during playback.
* Popup Exception - During playback, due to some error scenario some alert popup may be shown. The popup may not be shown during recording the script. Due to this playback may fail. In QEngine to handle such scenario Popup Exception handling can be used. In popup exception you have following options to handle error scenario,
Report & Coninut Play - This will report about the popup appeared in the reports and then proceed to play further action
Capture Screen & Continue Play - This will capture the screen shot when the popup appeared and show the same in the reports.
Stop Play - Stop the playback and generates report.
Close & Continue Play - This will close the popup and continue to play the script.
Accept popup & Continue Play - The "OK" button in the popup will be pressed and the playback will continue.
Cancel & Continue Play - The "Cancel" button in the popup will be pressed and the playback will continue.
Call Script - Another script in the suite can be called to handle the error scenario.
Close Error popup at the end of script execution - This will close any popup remain open at the end of script execution.
* Web Exception - During playback, due to some server error the required page may not be displayed. In such scenario, the playback on the particular page will fail. In QEngine to handle such scenario Web Exception handling can be used. Web Exception will be determined by matching the Response Code with the configured Error code OR matching the page url with the configured Error page URL OR matching the page title with the configured error page title. In web exception you have following options to handle the error scenario,
Stop Play - Stop the playback and generate reports.
Report & Continue Play - Report the Web Exception details and will proceed to play back the script.
Call Script - Another script in the suite can be called to handle the error scenario.
* Error Handling - During playback, if there is any failure in the script, then the Error Handling will be invoked. In QEngine you have following option to handle the Script Failures
Call Script - Another script in the suite can be called to handle the error scenario.
* JavaScript Errors - During playback, if there is any Java script error occurs, then the same can be made to notified in the report.
To configure Exception Handling goto Settings -> Exception Handling and choose any of the exception handling to handle the error scenario.
Exception handling will be useful in handling any unexpected application scenario.
Thanks
Raghavan
[ Mood: Neutral ]
[ Currently: Working ]
A new Technical paper on QEngine Data Driven Testing is published in QEngine website.
To download the technical paper, visit,
Thanks,
Raghavan
In continuation to my earlier blog today i am covering the Ajax Element Handling settings.
Ajax Element Handling
The application under test may have used "AJAX" technology to build the application. In "AJAX", the element can inserted into the web page dynamically without loading the web page at all. If you automate such an application, playback will be successful if the element is inserted in to the web page without any delay. If there is some network delay or the application response is poor, then there is a possibility that the playback will fail, unable to find the particular element in the web page. In such scenario, you can enable AJAX element handling setting in QEngine. This will pause the playback until the element inserted into the web page. You can also configure timeout, which will pause the playback until timeout. If the element is inserted before the timeout configured the play will continue with further script actions immediately.
To Configure "AJAX Element Handling" Goto "Settings -> Replay Settings" and enable the option "Enable AJAX element handling" and configure the "Maximum time to wait".
You can enable AJAX element handling from the script using the Built-in function "enableWaitForElement("
AJAX Element Handling will help in resolving some unexpected failures in playback due to dynamic nature of the application. This will reduce time in debugging some unexpected failures.
Thanks
Raghavan
QEngine has various settings that will help running scripts efficiently.
I will explain various QEngine settings in series of blog post starting with this.
Fast Mode Playback
In QEngine, the scripts got recorded will also include the time taken while recording. During Playback, the play will pause for the time taken during recording and then proceed to further action. In such case if you take more time during recording then the play will also pause for such time, even though the application loaded properly in the browser. Using "Fast Mode" playback, the play need not wait for the time taken during recording and will do all the recorded action immediately after the application loaded properly in the browser. You can also insert minimum pause time between the actions.
To configure Fast Play Mode Goto "Settings -> Replay Settings" and choose the option "Fast - Run as fast as possible" and save the settings.
You can configure Fast Mode Play using a Built-in Script function "setFastMode()". You can also disable Fast Mode Playback using "setFastMode("false")".
Fast Mode Play will enable you to run the scripts in shorter time framework which in turn will improve productivity.
In my next blog i will explain "AJAX Element Handling" setting in QEngine.
Thanks
Raghavan
In this blog i am going to speak about functional testing Ajax based application using QEngine.
AJAX means Asynchronous Javascript And XML
First i will brief how normal application will be recorded and played back in QEngine,
On start recording and clicking on the elements in the web page will generate script for QEngine. The script will look like below
| Code: |
| setWindow("title", clickButton.... |
| Code: |
| waitForElement(" |
