In the final post in this 5-part blog series, we explore how creating an efficient feedback loop enables enterprise mobility teams to monitor performance and to fix any issues that may arise during testing.
How can you improve continuous delivery without any feedback on performance issues?
In our last four blogs on continuous delivery, we discussed the importance of setting up a device cloud, the benefits of testing and debugging on real devices, and the reasons to build a suite of automated tests for a continuous delivery process flow.
But what happens next? After successfully setting up a continuous delivery flow with all necessary tests in place, you need a way to collect results and to receive real time reports. After all, how can you catch and fix any issues if you are unaware of any trouble in the first place?
The answer lies in setting up a continuous feedback loop. By setting up a closed loop process, you team will receive feedback and notifications about any issues as soon as they occur. When you have quick feedback, your mobile developers, testers and QA can isolate issues and debug quickly to keep releases on track.
But, what should you include in your feedback loop?
4 Important sources for feedback
For your enterprise mobility team to really be agile, you need to receive feedback in a way that is easy to digest. With so many tests running, your team could potentially receive “feedback overload” making it difficult, if not virtually impossible, for your team to sort through. To make feedback meaningful and easy for your team to act on, you want to put solutions in place to give you targeted, focused feedback. Here are four of the most important sources for your team to monitor.
#1: Automated test results
The automated tests that you set up, whether unit, functional or performance will feed their results back into your continuous delivery pipeline or to a test case management system. Regardless of where you are getting your results, they must be managed and analyzed by your team. Any execution issues or defects must be addressed by your team quickly to stay on track with releases. Make sure to make your automated tests rock solid so that in the event of a failure, there is no guessing if the test failed due to issues with the test or with the product.
#2: APM monitoring
When making sure that your app is running at its peak performance, it is important to measure how the system is running. APM monitoring tools can help provide real time performance metrics about the health of the system. APM covers a lot of different areas such as performance and availability of the systems, transaction timing on the device and the network, as well as bottleneck detection. All of the data provided gives you insights into how your application is working from the app side all the way back to the systems that support the app. If an issue arrises you can get an immediate alert if something is going haywire. Once an alert is triggered, everyone can jump in and diagnose the issue quickly. Making sure that your critical transactions are successful is key to having a quality app.
One way to get earlier feedback from APM is to leverage the APM tools during pre-production testing. Having this data before the app hits the users can help narrow down issues that can be fixed quickly, reducing costly issues one the app has been deployed.
#3: User monitoring and feedback
If you want a better understanding of how your users are interacting with your mobile apps or websites, then user monitoring reports can help you better understand their behavior. By reviewing these reports, you can see which features of an app are being used, or how users are engaging with different areas on a mobile website.
User monitoring delivers actual, real time information about key trends, user-timing and other metrics. You can use these findings to innovate and improve the mobile experience.
#4: Direct End User Feedback
But, test results are not the only feedback your team receives about apps or mobile websites. Don’t forget about your users, or mobile consumers. This audience has high expectations for mobile experiences, and checking out any feedback on your app or mobile website is beneficial for you team to consider when making improvements.
One way to get feedback is to monitor the app store reviews. Figuring out why your “star” rating is moving up or down gives valuable feedback about the quality of your application. Reviews can give insights into problems with the application, but it can also give you insights to what customers want more of in the app or on the mobile website.
Early feedback can also come from early release programs for your application. Apple has a program called TestFlight to get early releases of your application out to end users so you can get feedback earlier. You can also use crowd sourced testing programs, like Rainforest and Applause to get users to run through your application and provide valuable feedback.
Want to learn more about continuous delivery and setting up a device cloud? Download our recent webcast to learn more about the 5 steps to launch a continuous delivery strategy and the benefits of testing and debugging on real devices.