iOS Testing Strategy

Recently our team at Kiprosh started using FoneMonkey for Automation testing of iOS apps. FoneMonkey is free and has strong support for both iPad and iPhone devices.

Here are the various tools that we evaluated and shortlisted for iOS testing and then finalized our testing strategy for iOS apps (tools marked in color green are recommended)

Unit Testing

• Built in XCode based unit testing using OCUnit - (little complicated and requires too many steps to create the unit test, process isn't automated)
• GTM - Google Toolbox for Mac - (suitable) http://code.google.com/p/google-toolbox-for-mac/wiki/iPhoneUnitTesting
• GHUnit - (most suitable, easy to setup, has GUI, but documentation and other build issues)
• Mocking - OCMock and OCHamrest

Automation Testing tools and framework

• FoneMonkey by Gorilla Logic (http://www.gorillalogic.com/fonemonkey) - Free and easy to use
• Sikuli (http://www.sikuli.org/index.shtml)
• Bromine (http://code.google.com/p/bromine/)
• Frank (programmatic functional tests) Blend of Cucumber
• UIAutomation Instrument (Record and play) [JBehave, Cucumber] - (programmatic functional tests)
• Cucumber (http://cukes.info/) - OpenSource
• DeviceAnywhere (Commercial)
• KIF (Keep it functional) - OpenSource

Integration / Automated Builds

• Hudson with xcodebuild
• Code coverage (gcovr) with Cobertura XML

Testing Strategy

Finally we formulated following testing strategy for iOS apps

1) GHUnit for unit testing

2) With memory management, we must verify that when allocation fails we get expected return value as nil rather than garbage

3) Automation test suite (using FoneMonkey mostly or Sikuli or DeviceAnywhere)

4) Finally plug all the unit tests and automation test suite with CI using Hudson

5) Integrate often