For enterprise mobility professionals, what is the real value of automated testing v. manual testing and how can you measure its impact on mobile initiatives?
When it comes to mobile app and mobile web testing, mobile developers, testers and quality assurance professionals have their work cut out for them. Besides keeping up with mobile demand and constantly creating better, faster and more dynamic mobile apps and mobile websites, enterprise mobility professionals have important decisions to make about testing.
The decision to leverage manual testing v. automated testing is not one to take lightly. Whether to pick one testing methodology over the other, or to incorporate both, depends on a couple of important factors.
First, enterprise mobility teams need to identify the best strategy for each app or mobile website that they are testing. For example, if an app is only updated a couple of times a year, manual testing would likely make more sense based on time and available resources.
But, if a mobile app or website is updated constantly, with numerous new releases every few weeks, then automated testing would help streamline the testing process making it easier for the entire team to stay on schedule and deliver on time.
Second, it is important to consider the resources at your team’s disposal. How many people are available to perform manual tests? If performing manual tests is a strain on time, human talent and/or budget, then automated testing may be a better way for your team to excel and to meet demand. In other words, you should test smarter, not harder.
Also, when considering incorporating automated testing, it is important to think about what tools to use. Do you have developers, testers and quality assurance professionals in-house who thrive using open-source testing platforms, such as Appium? Or would your team be better suited to commercial solutions such as Jenkins or HP-UFT? Do not forget to consider the individual skill sets of team members. It is important to analyze and to really understand the skills and limitations of your team and to choose the testing methodology and tools to ensure the best chance of success. With the right tools, mobile developers, testers and quality assurance professionals will be more productive when developing and testing.
Finally, enterprise testing professionals are working in a high-stakes environment where the need to be faster, better and more efficient is extremely important. Not only must releases go out on time, but teams must adhere to strict budgets, often while working with limited resources. In this type of scenario, implementing an automated testing strategy and continuous delivery/continuous integration initiatives can make a significant impact on helping enterprise mobility teams to utilize resources efficiently to meet mobile demand.
By choosing the right solutions and testing tools and by working with knowledgeable partners, enterprise mobility teams can create an efficient mobile testing lab where managing devices, leveraging automated testing, and using sophisticated open-source and/or commercial tools is a streamlined, easy process.
Not convinced? Here’s a look at two real-life scenarios where enterprise mobility professionals implemented automated testing leveraging a device cloud solution for significant positive impact on budget, resource allocation, and mobile demand.
Scenario #1: U.S. Mass Media Conglomerate
In our first scenario, a major U.S. mass media conglomerate needed to improve operations in its retail stores by boosting customer experience using a mobile app. The goal was to develop an app to help store employees streamline the process of signing up customers and managing Internet, wireless and cable services for both new customers and its millions of existing customers.
If you were to visit one of its retail stores, you would see store associates using iPads to help customers to update account information and to add new services to their plans. Based on this reliance on iPads, the organization’s mobile developers, testers, and quality assurance professionals needed a solution to not only help them develop this new app, but to continuously update, debug and push out new releases.
The enterprise mobility team simply needed to build a high-performing app to increase the productivity of store associates to close sales and improve customer satisfaction. In addition, the enterprise team needed to test the performance of this app on the iPads that the associates use daily.
This same team was also tasked with developing and maintaining a customer facing app to serve customers who do not want to visit a retail store to pay their bills, request service, or to make changes to their accounts. It was paramount that the team provide a robust level of self-service to satisfy customer demand.
Based on the level of work involved and the need to maintain these apps for internal and external users, the enterprise mobility team decided to leverage automated testing in conjunction with a mobile device cloud solution.
Because of its ability to help mobile professionals to control the chaos of mobile development, testing and device sharing, the team chose deviceConnect™. This private mobile device cloud benefits the entire team by providing one shared platform for team members to develop, test and check the performance of apps across several other mobile devices. deviceConnect is tool-agnostic, and as a result supports a wide variety of automated testing tools, making it an ideal solution for teams wanting to leverage automated testing.
deviceConnect helped the enterprise mobility team to quickly develop, test and constantly improve the apps to help employees and customers to better manage accounts and services. By moving from manual testing to automated testing, the team had some pretty impressive results.
But, before we get into the ROI and results from leveraging automated testing with deviceConnect, it is important to examine the impact that manual testing had on this team.
Before implementing automated testing, a team of three mobile testers were responsible daily for running over 50 manual tests, 3 times per day, on six different devices. From a cost standpoint, the team has determined that it costs around $0.85 to run each manual test. When they added up the numbers, they determined that it costs around $27,000 per year to manually test on six devices.
Here’s what happened when the team began automated testing:
By implementing test automation, leveraging deviceConnect, the cost of each test is reduced by $0.80, to $0.05 per test, resulting in a ROI of $300,000 for the team in six months just from leveraging automated testing instead of relying on manual testing.
In addition, by implementing automated testing, the organization’s team of three also added two hours back to their workday. With this extra time, they can do more testing across additional devices.
Besides becoming more efficient, increasing productivity and streamlining operations, automated testing enabled this team to do more with the same headcount, making existing resources more productive than ever.
Scenario #2: U.S. Vehicle Manufacturer
The enterprise mobility team for a prominent U.S. manufacturer of vehicles also needed a solution to help manage and perform automated testing. But, this team also struggled to manage and share devices for testing.
Because deviceConnect is tool-agnostic, the testing team could leverage its preferred automated testing tools with a platform that could help them share the devices across a geographically dispersed team without having to physically ship or hand-off devices.
Here’s what happened when the team began automated testing:
By bringing together automated testing and the ease of managing devices with deviceConnect, this team could test faster than ever, bringing the time to run test cases from one week to one day.
The true value of automated testing
As both scenarios outlined above reveal, automated testing helps enterprise mobility teams in several ways. Whether through better resource allocation, saving costs and time, or by enabling teams to deliver higher quality apps, pairing automated testing with a solution like deviceConnect streamlines the process for enterprise mobility enabling teams to get the most value from automated testing.
Want to learn more about deviceConnect and automated testing? Check out our video demo.