Testing apps and getting feedback from users has never been easy in mobile development. So what service will make testing Android apps easier? I’ve compiled a set of available frameworks and features for Android application beta testing and will give highlights on my pick: TestFairy.
TestFairy follows simple steps while distributing the build. Just upload your android build (.apk) file to your TestFairy account and invite the beta testers. Once you upload the build, TestFairy server processes your apk file and injects the required code to your apk. Once your build is processed and ready for download, you can invite beta testers via email or from TestFairy official community page. Now you’re done. Once users start playing with the app, you’ll see the reports real time.
- Test users and group management.
- Allows inviting users and providing role based user access. You can be either an administrator, developer or a tester.
- You can define multiple administrators to control different projects (Enterprise license only).
- You are allowed maximum of 500 testers for any free type account. However the enterprise license goes endless.
- The free account can host maximum of 10 applications.
- Once a new version of the application is available, it notifies test user with email and a link to download the application. Alternatively users can download the TestFairy testers’ app and install the available application versions.
- Know how much time spent on testing each build.
- Geo chart gives information on different locations the application is tested from.
- You can also have the application deep analysis like activities per device name, activities per session, checkpoints, etc.
- Coverage information, including OS coverage, resolution coverage device demographics.
A geographical chart gives an idea of where your app is being tested from, while coverage information includes details on operating systems and device resolutions. What’s more, you can see a chart of activities and fragments covered per device, as well as sessions and users that participated in testing.
TestFairy Geo Chat Report
Checkpoints help to monitor when a tester has reached key points in your app. To monitor checkpoints, you don’t need to integrate with their API; just use a simple code snippet to register a checkpoint.
Here’s the kicker: TestFairy monitors all aspects of an application and generates reports. It monitors sessions, battery life, CPU usage, allocated runtime memory, threads and more.
TestFairy takes screenshots of each activity and fragments on the different devices applications are tested on. You can filter the screen grabs by activity or fragment, and spot the differences across devices.
Provides information on how the user is using the application. In the image below, heatmap report will give brief idea of the buttons, or widgets clicked by user and what’s trendy on a particular screen. This report is a visual screen shot of the application screen (a sample below). This sometimes helps in taking serious business decisions.
Heatmap Analysis Report
Helps in identifying the defect and the place of its occurrence. TestFairy records the video and takes screenshot sequences for each crash log. Again, TestFairy stands out as you don’t have to do any coding. Everything happens in the background while you upload your build. This feature may not be as advanced as “Crashlytics”, but it fulfills the basic needs for identifying the crash root cause.
Beta Tester Feedback
Along with video feedback and crash reports, an enterprise account allows for collecting feedback from individual testers, who simply shake the device to send a message.
TestFairy provides many options to free users. Free account is limited to number of applications, testers and group. There is no official record for the application limit on their website, but it is restricted to 10 apps. Below are some enterprise license feature highlights:
- Enterprise account supports unlimited apps, users and distribution list.
- White labeling for your enterprise account. You can have you own label and customization.
- Advance reporting like heatmap analysis, CPU and memory usages.
- Taking beta testers’ feedback.
- Crash reporting with series of screenshots and video recording.
TestFairy is simple when it comes to integrate with CI (Continuous integration) tools like Bamboo or Jenkins. CI tools can use the below script to upload the scripts.
Number of test cycles even for a simple software component is practically infinite. Inclusion of TestFairy helps to distribute and get early feedback from internal/external testers and other stakeholders. TestFairy has treated me well, but what works for you?