Background
IPhone is being deployed or tested by 80% of Fortune 500 Companies
Mobile initiatives are becoming more popular, terrible small scale data given by Soasta.
Dont always use device statistics
Dig a little deeper
Think about regions
Gartner
Challenges
Traditional dev practices incompatible with mobile
Apps require constant refinement to tune the user experience
Rapid change requires rapid development and deployment
Mostly sound like good practice, not challenges!
Recommendations
Collaborate. Speak to users. Really. How revolutionary!
Add UX testing to your skillset
Could use persona's and tours
Analyse how apps are used in production by real users
Adopt agile. Lets say, be agile or work with agility.
Mobile is different?
Multiple OS
iOS
Less version fragmentation due to upgrade strategy
Android
Fragmented versions
Device Diversity
Tablets
Mobiles
In Between
Network Diversity
Networks make amendments to Android for their OS
Example, no source given?
Mobile Testing Iceberg
Really useful model and shows the depth of your testing
Checklist
Mobile Specific Checklist Test Conditions
Network
Network Type (Mobile Variations and WiFi)
Network Quality
Network Loss
Network Transitions
GPRS
E
3G
4G
THE WALKAROUND TEST
On the Leeds/Harrogate Train
Installation
Install
Uninstall
Reinstall
Update
Clean Devices/ability to reset
Performance
Speed / Responsiveness
CPU and Memory Usage
Monitoring tools for devices?
Battery Drain
On Device / Memory Card Storage
Interrupt
Incoming / Outgoing Calls
Within and without the app
Popup Alerts (Calendar, Alarm, SMS)
While in different states
Typing messages
Installiing
Power Cycle
Take battery out
Shutdown
Hibernation after battery run down
Extended Background
Running in the background
Device Integration
Location / GPS
Video / Still Camera
Screen Size / Resolution / Orientattion
Gestures
Accelerometer
What is this?
Device Selection
App Functionality and Technology
Deployment platforms & versions
App interaction with device
Integrated Systems
User Profile
Prevalent Devices
Preferred Browser
Concurrent Usage
Load Profile
Business Objectives
Application value to business
Risk of application failure
Questions
Who is my user base?
How old is the average user?
How many men or women are in my target group?
Which platform is used most within that user base?
Which device is used most?
Which software version is installed on most of those phones?
What kind of sensors does my app use?
How does the app communicate with the outside world?
What is my apps main use case?
iPhone and iOS configurations results in 20 configurations
App Functionality
Structure. Test what it’s made of.
Function. Test what it does.
Data. Test what it does it to.
Platform. Test what it depends upon.
Operations. Test how it’s used.
Time. Test how it’s affected by time.
Matrix for Function Across Devices
Sessions
Will often last only a few seconds
Need to deliver info in 1-2 seconds
Links
http://4ourth.com/wiki/Index
http://karennicolejohnson.com/2012/05/applying-the-sfdpot-heuristic-to-mobile-testing/
http://vimeo.com/107158748
Hands On Mobile App Testing by Daniel Knott
http://apps.testinsane.com/mindmaps/LONG-FUN-CUP-Mobile-App-Test-Coverage
http://apps.testinsane.com/mindmaps/Mobile-Testing-In-a-Nutshell
Renderers
http://www.airsquirrels.com/reflector/
App Deletion Reasons
Bad Design and Usability
Loading Time
60% users will delete an app that requires registration
KIFSU
Keep
It
Fast
Simple
Usable
Keeping Up To Date
Monitor the mobile device and software market
Know when new phones will be rolled out
Find out about the new features of the OS
Keep an eye on your target group to see if new devices are showing up in your statistics
Think twice before updating a phone to latest OS
Buy a new phone with the latest OS
If buying is not an option rent the devices
Tool Selection
Test Framework
Calabash
Device Access
Speed of access
Security
CI Integration
Device Cleanliness
Language Support
preferably C#
Gesture Support
Performance Data/Monitoring
CPU Usage
Memory
Support and Consultancy
Trends
Wearables
Contextual/Personal Tracking
Mobile Payment Processing
2014 - 19k unique Android Devices up from 12k in 2013
Weird Words
phablet!
Like the big iPhone 6!