+1 404-214-5804
Enterprise Mobility Blog, Mobile Testing Strategy and Techniques

When to Use Automated Mobile App Testing vs. Manual App Testing

Manual vs. Automated mobile app testingLast updated July 29, 2016.

Let’s say your organization has an iOS app and Apple releases a new version of iOS (sound familiar?). The more automated test cases you already have on hand, the faster you’ll be able to pinpoint what updates need to be made.

With that in mind, does it make sense to automate all testing? Well, possibly.

There are times when it may be best to conduct testing manually and other times when automated testing is the better choice.

Let’s break it down.

When To Use Manual App Testing

The most common use of manual app testing is during the development phases of a project, while developers are coding new functionality.  Developers can quickly access devices using a device access manager like Mobile Labs deviceConnecttm to perform initial app validation.

In the Agile world, the completion of incremental function needs quick and early validation that the code behaves the way the developer – and designer – intends. A developer can use a real mobile device to make a quick check of the code and can then hand the managed device off for remote access by the designer or product owner.

Quickly accessing the right hardware for such confirmations makes a faster path to what Agile calls “done done” – complete code that satisfies the user story and the product owner. T

he goal of manual testing in this case is to get a quick course correction to ensure that app functions are correct.  There may be no return for automating test cases for code that won’t ship until it has been modified or completed.

Some mobile apps or hardware features can require manual testing. For example you may want to integrate with a camera function such as barcode scanning. Since there really isn’t a way to simulate this function, manual testing is the best way to go.

Manual testing may also be handy for validating the externals and the appearance of the app. Manual tests can sort out whether the app has the right icon, if it handles being interrupted and then re-dispatched, and whether the overall look and feel of the app is correct.

This last dimension is truly aesthetic, and takes a designer’s eye. Automation hasn’t yet progressed into the judgment realms of “cool” and “beautiful.”

For the overall effect, the mind’s eye can hardly be bettered.

Finally, manual testing may be useful to verify spelling, placement of objects, proportions, consistency of UI standards, and general design standards.

Are the buttons all supposed to say “Logon” or “Login?” Do we “delete” or “remove?”  Are the names of the states spelled correctly?  Is the right version of the company logo being used and are the colors consistent with marketing’s defined palette?

These details are mostly fit-and-finish items and don’t tend to vary once they have been done correctly. Of course if they do vary, then automated tests can be used to ensure that the spellings and terms remain correct.

When To Use Automated Mobile App Testing

Once a user story is accepted, you may be ready to code test cases for regression and cross-platform testing.

The ability of the app to carry out its function on a variety of platforms and constantly changing platforms raises the typical case for automation.

The question isn’t so much “does it work as we expect?” but “does it still work as we expect?” on a newer OS, on new or different devices, or even on different operating systems.

A classic case for regression testing hits the iOS community about once a year.

Last fall, it was iOS 9; this year, it’s iOS 10. A number of object classes have changed either appearance or function, and regression tests that ran on iOS 9 can provide a roadmap and inventory of where the problems (if any) are in iOS 10.

When regression tests fail in these kinds of scenarios, you get a quick look at what needs to be done for the app to support a new operating system.

The existence of a suite of regression test cases can save critical time and can be the difference between releasing an app in a timely fashion and lagging behind Apple.

Apple is getting better at moving its user base to new releases so delays in getting updated apps out to users is almost guaranteed to cause frustration.

The need for regression testing occurs when development delivers new function, when a bug is fixed, when tools or libraries used in development are updated, and when the operating system (and/or browser) is updated.

A suite of regression test cases pays the greatest dividends in the most volatile environments – and mobility certainly fits the bill.

We at Mobile Labs recently used regression testing to update our own iOS apps. We ran automated regression tests on our apps using Mobile Labs Trust™. We put a project plan together to address the problems revealed by the tests and met the release date of the newest iOS.

Developing A Sound Mobile App Testing Strategy

When you’re testing new functionality or a single function, manual mobile app testing is often the way to go.

On the other hand, whenever you’re performing cross-platform tests or simple regression testing, automated mobile app tests can deliver a sound return on your investment.

Want to learn more? Check out our latest posts!

Don’t forget to download our eBook on Amazon, to stay ahead of the curve in 2017!

Michael Ryan

Michael Ryan serves as Mobile Labs’ chief technology officer. In this role, Ryan provides the technological vision and drives Mobile Labs Trust’s product road map. Ryan has more than 35 years of experience in leading software development teams that design and build robust and market-leading solutions for large-scale enterprise customers among Fortune 1000 companies. Most recently, Ryan was with Fundamental Software where he worked on large-scale systems CPU emulation architecture, design, and implementation. Prior to Fundamental Software, Ryan was director of development, Sr. VP of R&D, and finally, Chief Technical Officer for CASE tool vendor KnowledgeWare, Inc. Ryan served as senior staff systems engineer, field manager, and regional technical support manager for mainframe manufacturer Amdahl Corporation.

More Posts - Website

Leave a Reply

You must be logged in to post a comment.

Why Mobile Labs?

Mobile Labs provides enterprise-grade, next generation mobile application testing tools. With a focus on security, agility and affordability, Mobile Labs delivers solutions to help you deliver quality mobile apps for Android, iOS and Windows platforms while also helping manage mobile devices in a private, secure cloud.

Contact Mobile Labs

3423 Piedmont Road NE
Suite 465
Atlanta, GA 30305
+1 404-214-5804
twitter  facebook linkedin google-plus SlideShare