Usability and Experience Testing for a Leading Wireless Communications Device Company All Case Studies
Collect the truths that inform true usability
The Challenge
A leading designer, manufacturer and marketer of innovative wireless solutions for today’s worldwide mobile communications market sought third–party testing of its product, separate and apart from that conducted by its internal usability team so as to eliminate any bias.
Electronic Ink was hired to conduct two unique rounds of usability testing and analysis.
The challenge: design and execute a pair of customized testing methodologies, for two separate product teams; then, analyze and report on test findings, and create a clear set of recommendations for needed improvements.
The Electronic Ink Solution
The Electronic Ink team responded by designing two testing methodologies specially tailored to the unique and complex range of variables. Both tests would target novice users, across two product usability scenarios:
- Keypad Usability Test (done for the Product Design Team) – recorded a user’s speed while entering text strings on smart phone keypads.
- “Out of box”/ Setup Usability Test (done Marketing Team) – recorded qualitative data based on a user’s experience while unpacking and setting up the most popular products in the client’s competitive set, including Blackberry Curve, Apple’s iPhone, and T–Mobile’s Dash and Sidekick models
Multiple internal pilot tests were performed before actual testing was done to ensure that the test designs and methodologies would be highly relevant, and bulletproof.
The Keypad Usability Test relied on the quantitative measurement of a standardized time–to–task scenario. However, simply timing the text string entry durations was not enough. A complex error tracking and scoring system was devised to account for human–based errors such as transposed letters, deletions and corrections, and added or omitted words.
In contrast, the “Out of Box” / Setup Usability Test was based on highly qualitative measurements. Electronic Ink developed two distinctive systems to meet this challenge:
- An emotional satisfaction data scale that tracked a user’s response, based on each critical step of the setup experience.
- A video recording system that captured actual screen sequencing during device setup. For the latter, Electronic Ink fabricated a custom Hi–Def camcorder feed from each device to capture the video. This complex system provided critical qualitative data of the user experience – including screen navigation and backtracking – for subsequent analysis.
The Electronic Ink team then performed data analysis of both testing scenarios and concluded the project with a detailed set of Findings and Recommendations for each.
Net Result and Impact
To stay competitive, the client’s Product and Marketing team needed solid, unbiased and detailed testing data – along with a set of clear findings and insightful recommendations for relevant product improvements.
Electronic Ink’s robust testing methodologies exceeded their expectations, and a set of new, ongoing tests are already in the works.