My personal take on this popular topic, specific concerning Android testing.
People have different views on different types of automated tests with regard to Android. I’ll categorise them into 3 groups as mentioned in this official Android testing pyramid coupled with the techniques as of November 2017:
Test functions/ methods that each handles a small piece of your business logics. There are usually thousands of these tests in a typical app. These thousands of tests take a few minutes to execute (each takes a few to a few hundreds millisecond for plain Java tests or up to a few seconds for Robolectric-based tests).
Unit tests should be a part of of your CI process. They should run for every
git push or at least
In my opinion, you should write these tests as soon as possible, while writing every logic class or method or immediately after finishing them.
Recommended technologies: Junit + Mokito
Component/ Functional/ Integration tests
Usually focus on the UI/ UX of one screen at a time. Square’s Spoon can be used here to capture the screenshots. Espresso can be used to test Android’s unique behaviours like deep-linking. I usually use Espresso to test a small flows involving 2–3 screens/ steps e.g. e-commerce add-to-bag mini flow consisting of product screen and shopping bag screen.
Of course, there are usually fewer tests here compare to unit tests and Espresso tests take longer to execute (plus they require an Android device). There could be dozens or hundreds of Espresso tests.
I’m not sure about the perfect time and procedure to run Espresso tests.
Recommended technologies: Espresso
UAT/ end-to-end tests
As the name suggests, an add-to-bag tests here would include all the steps from the first screen user sees after a fresh installation to the end of the flow (shopping bag screen here). Calaba.sh tests are English-like and can be written by non-programmers e.g. QA people.
Similar to Espresso tests, Calaba.sh tests require an Android device and take longer to run compared to unit tests. There could be dozens or hundreds of Calaba.sh tests.
I’d recommend running Calaba.sh tests before sending a pre-release APK to QA. Fastlane + Fabric Beta (Crashlytics) are ideal for this purpose.
Recommended technologies: Calaba.sh
There are many articles on the Internet about the types of automated tests and the testing pyramids, but Martin Fowler’s is my number one favourite.