Mobile Labs architected Trust™, an add-in to QTP/UFT, to maximize script reuse through a robust object model that helps insulate testers from the differences between platforms like iOS and Android™. But do you know that Trust scripts can also be reused for load testing and service monitoring?
Mobile Load Testing using QTP/UFT, Mobile Labs’ Trust, and HP LoadRunner
The adjacent video shows a LoadRunner load test scenario that utilizes a mobile transaction timer on a real mobile device. In the video we submit a Trust script to LoadRunner and ask LoadRunner to begin executing a combined load and response time test. The Trust script logs in to a mobile web site and LoadRunner correlates a simulated backend load with response time on the device.
Because the test uses a real mobile device and because the script is able to measure the elapsed time needed for the website to respond, the test is able to produce accurate end-user experience data. Running such a script on a real mobile device (in this case, an Apple® iPad® Mini) is the only way to develop accurate user experience data; VuGen scripts can only approximate end user experience times. With the real device and the Trust script, we are measuring actual user experience because we are monitoring a live device.
The test is able to show minimum, average, and maximum response times and we can use the standard deviation as an indicator of how consistent the response time was. Re-using the script required only adding “StartTransaction” and “EndTranscation” transaction timers to the script. LoadRunner is able to use these measurements in its analyses.
As the video shows, using QTP/UFT and Mobile Labs Trust with LoadRunner can significantly enhance your load tests.
It is also possible to use a third-party suite like Shunra to generate complex network conditions while using the Trust script and LoadRunner to gather information about what a user experiences under given load conditions within varied network states. Since it is impractical to run thousands (or hundreds of thousands) of mobile devices, a Trust script can serve as a probe to give an accurate reading of how server loading affects end user experience. With deviceConnect™, which provides device access management via an internal device cloud, you can locate devices in various locations to understand the additional effects of network topology on response time. The service received by a device on the data center LAN can be compared to the service received by a device on a remote network or on wireless.
Mobile Service Monitoring using QTP/UFT and HP BSM/BPM
HP’s Business Service Management (BSM) and Business Process Monitoring (BPM) software products can also-use Trust scripts with transaction timers to measure process and service performance as well as app health. A LoadRunner agent can pass Trust script metrics to the BPM product; BSM can then be used for analysis.
The suite can report on application health (is it up, running, and responsive?), service levels (is the app meeting service goals?), and can point out areas or transactions of greatest concern (and greatest payback).
As with all Trust scripts, the robust ‘mobi’ object model assists in re-using scripts across platforms (e. g. iOS and Android); enrolling Trust transactions in a BSM scenario is as simple as identifying the script, identifying the app as a business application and the Trust script as a business transaction for the data collector. Once this is done, you set a schedule and monitoring thresholds and the transactions are run according to your schedule and the data is gathered. In the BSM analysis below, we ran three Trust transactions a total of 259 times to develop data on response time, service goals, and availability.
BSM can also provide graphical analyses of availability and performance, showing where goals were met and where goals were missed:
To sum up, Mobile Labs’ Trust scripts are highly reusable, and useful for more than just functional or regression tests. Scripts with transaction timing can be built or re-used to serve as probes for load testing with LoadRunner or for service monitoring with HP BSM/BPM.