Home > Resources > Development Tools > App Testing Service

Understanding the App Testing Service

Introduction

Introduction

The Amazon App Testing Service (ATS) is a tool that you can use to test your Android app prior to submitting it to the Amazon Appstore. This free service tests your app for common issues that can prevent it from being compatible with Amazon and other mobile devices. Additionally, this service provides access to device tests that enable you to see how an app looks and performs on a virtual device hosted in the Amazon device cloud. Most Android tablet apps pass the App Testing Service tests with no additional development required.

How the App Testing Service Works

The app testing service performs two types of tests on Android apps:

  • Compatibility test: Analyzes your Android app’s APK file and lets you know if your app has any of the following problems:
    • Uses software libraries that might prevent your app from being compatible with the Amazon Appstore. In this case, the test report will suggest what libraries to use instead.
    • Uses Amazon APIs incorrectly and documentation to fix the issues.
    • Contains features that are not supported by certain devices and instructions on how to degrade gracefully.
  • Device test: Amazon tests your app on real devices and performs the following tasks:
    • Installs and launches the app.
    • Explores your app using appXplorer. appXplorer explores your apps, identifying different activities.
    • Emails you test results that include test events, screen shots of the app, CPU usage, memory heap utilization and detailed logs. Test results may vary based on the technology used for developing the user interface of the app.

About appXplorer

The appXplorer tool is optimized for apps that use native Android UI widgets to build the user interface. Apps that build user interfaces using technologies such as OpenGL and HTML5 are not effectively explored in the current version. appXplorer does not explore apps that require any form of user authentication. Device testing will not be performed if the app is an invalid APK. Apps that use specific APIs of level 18 and above will not work on the test device.

If appXplorer is unable to install, launch, or explore the app on an actual device, you will be sent an email notification telling you that that ATS was unable to test the app. This issue may happen because of an invalid APK, unsupported OS and API version, or an ATS internal error. If you receive this message, check your app for compatibility, and re-submit the app for testing.

Interpreting Test Results

Log in to your developer account to see the test results for your app. Amazon never shares test results publicly.

Compatibility test results will be available within 90 seconds. Device test results may take up 6 hours depending on the traffic that Amazon experiences on the service. If the appXplorer testing load is very high, test results can be delayed up to 24 hours from the time of app submission.

You might find that if you test the same app, unchanged, multiple times using the App Testing Service that your device test results from appXplorer might change slightly. Your results might vary because appXplorer can explore the app using different app flows, which may result in navigating to different screens and detecting app crashes only during a specific app flow. This scenario is not common but may happen with some apps. Your compatibility test results will not vary.

Submitting Your App from the App Testing Service

After you have fixed any issues identified by the App Testing Service, and your app passes testing, you can submit your app to the Amazon Appstore for publication. If you do not want to submit your app immediately and are signed to your developer account in when you run the test, Amazon will save your test results and you can submit your app later.

Note that all apps submitted to the Amazon Appstore are subject to further review by Amazon’s content operations team. Amazon tests submitted apps to verify that each app works as outlined in its product description, does not impair the functionality of the mobile device or put customer data at risk once installed, and complies with the terms of the App Distribution and Services Agreement and the Amazon Appstore Content Policy Requirements.