Ian Smith - Mobile Software Testing - Facing Future Challenges

Unissons nos Talents
T O G E T H E RT A L E N T E D
1Mobile Software Testing – Facing The Future Challenges - EuroSTAR 2008 (R1)
Mobile Software Testing
Facing The Future Challenges
Ian F. Smith
Principal Consultant ismith@uk.sopragroup.com
Sopra Group Ltd. Tel: +44(0)131 332 3311
Robert Campbell
Senior Developer rcampbell@uk.sopragroup.com
Sopra Group Ltd. Tel: +44(0)1698 574 200

2Mobile Software Testing – Facing The Future Challenges - EuroSTAR 2008, 12/11/2008
Overview
 Introductions
 Mobile Systems
 Trends
 Challenges/Problems
 Approaches
 Testing Mobile Systems
 Example/Case Study
 Summary

Introductions
 Ian F. Smith
 Principal Consultant – Testing Division
 Over 28 years experience in software and system development and testing
 Strong background in telecoms and real-time/embedded systems
 Robert Campbell
 Senior Developer – Testing Division
 28 years experience in embedded development, manufacturing and test
 Focussed on testing telecoms monitoring equipment for the last 6 years
 Sopra Group
 Sopra Group, a major European IT Services Company
 Total of 11,000 staff and group 2007 Revenue of 1 Billion €
 Offices throughout UK and Europe + and a fully integrated offshore
Development and Testing Centre in Noida, Delhi
 Services:
 Application Outsourcing, Systems Integration, Testing + Specialist services
(Consulting, Information Security, Business Intelligence)
 Key markets:
 Finance, government, retail / manufacturing, energy & utilities, health,
telecoms + media

Mobile Systems
 What’s a typical mobile system?
 Various types of mobile system
 User operates mobile devices
 Supports access/operation in various locations and can often connect by various methods
 Key Characteristics of Mobile Systems
 Elements:
 Mobile Devices
 Wireless/Wired connection
 Server + Network Infrastructure
 Mobile applications
 Can run on either device or server, depending on architecture and device capability
 More internet based apps/services are being now provided for mobile use
 Increasingly, core business systems are providing mobile application interfaces

Mobile Systems - Examples
 Navigation/Route Planning
 Mobile Application Server: Navigation/Route Planner
 Handset software: Navigation Handset Client
 Content: Route information
 Access Methods: GSM/3G
 Email
 Mobile Application Server: Email Server
 Handset software: Email Client Software
 Content: Email information
 Access Methods: Desktop, Laptop, GSM, WiFi, …
 Voip Service
 Mobile Application Server: Voip Network System
 Handset software: Handset Voip Features + Agent software
 Content: Voip calls data + SMS
 Access Methods: WiFi, 3G

Trends
 Some key stats
 1.28 billion mobile handsets to be sold in 2008
[Source: Gartner]
 32.2 million smartphones sold in Q1 2008
[Source: Gartner, May 2008]
 Better support for data
(networks and devices)
 3G data services
 WiFi
 Increasing power and sophistication of
mobile devices
 More powerful processor and storage
 Software capabilities
 Connectivity
 Audio/Camera/Video
 Sensors/Accelerometers
 GPS
 Application trends
 Web 2.0 applications
 Increasing expectations
 Desktop applications on mobile devices
 Device capabilities
 e.g. LBS

Trends (Cont.)
 Handsets gaining laptop/PDA capabilities
 Processing, storage and software
 Mobile OSs
 Established players …
 Symbian, Windows Mobile, RIM
 BREW, Linux Mobile, …
 New Players
 Apple, OHA/Android, LiMo, …
 Landscape is changing
 Symbian Foundation  Open source
 Possible consolidation??
 Development Environments
 Varied environments
 Open source
 Proprietary
 IDEs/SDKs
 Improving (but still with limitations)
 Server application development

Challenges/Problems
 Cellular network services can vary significantly
 Data offerings
 Performance
 Mobile device platforms
 Many different software models + SDKs/IDEs/Toolsets
 Changing platform landscape (Symbian Foundation, Android, iPhone, …)
 Device specific firmware issues
 Application complexity
 Meeting expectations for current desktop based applications on mobile
 Application/content access
 Access Diversity
 Ubiquitous access …
 GSM/UMTS
 WiFi/WiMax
 Wired
 Security
 Trust models and signing
 Increasing amount of sensitive data on devices

Approaches
 Manage Complexity
 Application and system architecture
 Component partitioning
 Divide and conquer
 Multi-level testing (unit, component, integration, system levels)
 Get value from unit testing
 Layer UI to simplify testing and support automation
 Maximise code reuse
 Important for multi-platform developments
 Different platform models can limit this
 Consider signing testing or manufacturer specific testing
 Signing can be expensive and time consuming

Approaches (Cont.)
 Using emulation
 Doesn’t address all testing needs (telephony, performance)
 Combine with GUI based test automation tools
 Device Testing
 Confirm compatibility (device firmware issues)
 Cover device specific features
 Security Testing
 Threat models change constantly, so review security testing regularly
 Consider full operational lifecycle
 Deployment, operation, upgrade, removal

Building Mobile Systems
 Choosing platforms
 Implications
 System Architecture
 DfT – Design for Test
 Create test support features
 Handset software
 Engine (portable) + (thin) UI layer
 Different software models
 Application signing
 Consider application efficiency
 Power efficiency/management is a major factor for mobile systems
 Development environments
 SDKs are getting better, but building and testing will still be challenging
 Design for offline use
 Connectivity is improving – but devices won’t always be connected
 Also, some OSs don’t support application concurrency, so consider this

Testing Mobile Systems
 Planning testing
 Server and core system tests
 Mobile device software testing
 Using “devices” in system testing
 Design testing to cover different handset platform needs
 Aim for reuse, where practical
 Server testing
 Existing web or other test tools may be used as mobile device client
capabilities approach those of desktop client systems.
 Network & Infrastructure testing
 Understand provider capabilities and limitations
 Tools
 Consider whether additional support tools can be procured/created
to assist in testing (set-up, reporting, diagnosis, etc).
 Emulators
 Can support application level testing
 Not good at emulating telephony or real-time operations
 Combine with GUI test tools

Automating Testing
 Quick wins
 Unit test can be well automated
 Continuous Integration tools
 Even some handset code!!
 Automating device testing
 This can be tough – different environments, user interfaces, etc.
 Design UI to support automation (allow virtual user actions driven by tools)
 Test Automation Tools
 Performance Testing
 Automation is essential to effective performance testing
 Consider this in design-for-test activities.
 Skills

Example
 Webraska Navigation System
 Handset based navigation system
 Route information from server
 GPRS connection
 Bluetooth GPS
 Handset software testing project
 Evaluate handset software
 Quality
 Performance
 Usability

Case Study
 Mobile Call Generator Project
 Develop proof of concept test call generation system
 Key project objectives:
 Automated control of voice calls, messaging and data functions
 Scalable solution
 Remote control
 RemCall System
 Architecture
 Software
 Handset
 Server(s)
 Development Challenges
 Achievements
 Limitations

Case Study (Cont.)
 Multi-level Testing
 Unit, Component and System level test
 Handset Agent Software
 Piloted automation for unit testing
 Introduced CI with CruiseControl
 Handset Controller PC Software
 Quality Center Server Software

Summary
 Mobile systems offer many major opportunities but developing and testing
them pose significant challenges.
 The pressure to supply mobile access to existing business applications and
systems is growing as device capabilities improve to support these.
 New and existing businesses are bring innovative mobile systems to market.
 The handset landscape is changing rapidly, with established players evolving
and new entrants joining. This change will continue.
 Consider platforms carefully – development implications and future market.
 Architecture and component partitioning can have a marked effect on system
development and testability.

Questions
?
Thank You
www.sopragroup.co.uk

Ian Smith - Mobile Software Testing - Facing Future Challenges

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Ian Smith - Mobile Software Testing - Facing Future Challenges (20)

More from TEST Huddle (20)

Recently uploaded (20)

Ian Smith - Mobile Software Testing - Facing Future Challenges