Beta Program Case Study
0 likes301 views
MobilEvolution executed the largest beta program ever for a major European wireless carrier involving 1,400 internal and external trialists across nine countries. During the six-week trial period, MobilEvolution staff identified hundreds of bugs and issues through interactions with trialists. Feedback was analyzed and prioritized recommendations were provided to the client weekly to improve the product prior to its imminent launch. The successful beta program helped address issues before launch and prepared the organization for a smooth product rollout.
Beta Program Case Study
- 1. Case Study Beta program for integrated wireless device and web service Overview arrangements with network operators in countries where the Company does not hold an equity MobilEvolution provided an end to end beta program, stake. Under the terms of these Partner Network including strategy, planning, and execution to a major Agreements, the Company and its partner networks European wireless carrier. The beta program was the co-operate in the development and marketing of largest ever undertaken by our client. global services under dual brand logos. During the course of the trial, MobilEvolution At 31 December 2008, based on the registered staff had thousands of interactions with trialists, customers of mobile telecommunications ventures uncovering hundreds of bugs, user experience issues, in which it had ownership interests at that date, the and feature enhancement requests. Feedback from Company had 289 million customers, excluding paging those interactions was aggregated, analyzed and customers, calculated on a proportionate basis in presented to the client in the form of prioritized accordance with the Company's percentage interest in recommendations. these ventures. Due to an imminent launch date, planning and executing all phases of this program quickly was Business Challenge critical. Trial participant makeup was diverse, The challenge was to including 1,400 internal and external trialists in nine plan and execute the • Company’s largest beta European countries, and across a number of operating largest beta program • Trial began within 1 week companies. A new device, web client, and web service the company had of start date were all evaluated during the trial. run. We also needed • Trialists in nine countries to hit a time window Client Profile to plan and complete • Trial ran during active development the six week-long Our client is the world's leading mobile testing period within telecommunications company, with a significant 90 days, when the presence in Europe, the Middle East, Africa, Asia product was scheduled to launch. The product was Pacific and the United States through the Company's under development during the whole span of the subsidiary undertakings, joint ventures, associated trial, leading to unanticipated device and website undertakings and investments. behaviors. In addition, internal and external trialists The Company’s mobile were spread across nine European countries. Finally, subsidiaries operate prioritized recommendations based on feedback and • World’s leading wireless technical support cases needed to be provided to the company under a single brand name. During the marketing and development organizations each week • Global operations so that they had time to adjust prior to launch. last two financial • 289 million customers years, the Company has also entered into 24916 SE 133rd Way, Issaquah, Wa 98027 • www.mobilevolution.com • +1 (206) 274-7472 © 2010 MobilEvolution, Inc. All rights reserved.
- 2. Solution feedback website. Once an agreement was reached, planning and Data collected from these listening systems was content development began immediately. Within ten aggregated with data collected by the technical days of the notice to proceed, MobilEvolution had a support team, analyzed, and distilled to the ten most Program Manager on the ground in Europe full time critical issues that needed to be addressed. This for a month. information was provided the client’s development team twice a week in the form of prioritized MobilEvolution recommendations. In all, the trial panel identified notified prospective • Program management nearly 250 bugs, hundreds of user experience issues, trialists of their • Content development and logged over a hundred feature requests. participation in the • Trialist surveys trial and scheduled Technical support was also available throughout the • Technical support kickoff events in each trial for 12 hours a day via local telephone numbers, • Incentive program country. Then we email, instant messaging, and via a web-based trouble logistics prepared the agenda ticketing system. Each support interaction was • Forum moderation and presentation captured in a tracking system. Then as we discovered • Data analysis materials for each support trends, MobilEvolution developed training of the events. In materials to help trialists avoid problems before addition, we they occurred; OTA configuration tools; FAQs to developed a welcome kit which was provided with help trialists self-resolve issues; and knowledgebase each phone, including a setup guide, a letter from articles to ensure that all support staff handled company executives, and information on how to problems consistently and efficiently. contact technical support. During kickoff events, trialists received their phones, Business Benefit learned about the service, and participated during The obvious benefit a step by step training on how to sign up for and of this program - • Identified bugs configure the service. Technical representatives were identifying and fixing present at each kickoff event to help trialists work issues prior to product • Focused marketing messages through any problems they encountered. Kickoff launch - is only one events were completed in all nine countries within of the many benefits • Created vocal avocates three weeks of the first event. our client realized as a • Contributed to product result of this beta trial. roadmap Once trialists had a functional service on the web The development team • Prepared technical and their mobile phone, MobilEvolution began a was able to address support communication regimen, contacting them every other many of the bugs and • Prepared training teams day via email or SMS. Each communication directed feature enhancements them to perform a service-related task. Then once identified prior to a week, we surveyed trialists via a web-based survey launch. High priority issues that were not addressed platform to ask them about their experience. We also have been incorporated into the product roadmap regularly included questions regarding their overall and will be available in future releases. Focusing satisfaction with the service. Trialists were able to development effort based on user feedback helps the share their experiences via a moderated newsgroup, development team have the greatest impact. monitored email box, the support team, and a
- 3. Communication with development and company needs. Management during the trial regarding status and results happened early and often. The communication Training style was to present a brief executive summary, Training targeted for the audience. Whether the including recommendations for the top ten actions audience is internal business users, a partner channel, that should be initiated each week, backed by full sales or technical support staff, MobilEvolution can details. This format allowed executives to get a sense design, develop, and deliver web- and instructor-led of progress at a glance and deep dive into any specific training to improve performance and drive down issue they wanted to explore more fully. costs. Excitement grew towards the product launch date Content Development as issues and opportunities to improve the product Relevant, understandable content for your audience. were identified, fixed, and communicated back to the MobilEvolution content developers understand sales, organization. By the time the launch announcement technical support, business, and consumer issues. was made, our client’s customer-facing teams were Our team of mobile experts helps plan, develop, and advocates of the product and well versed in its release training, marketing, and technical materials features and benefits. that have impact. Internal and external trialists were regularly surveyed Consulting on net promoter-type questions. This feedback helped the marketing organization focus and test their Consulting with a bias towards action. We examine messaging prior to presenting it to the public. and help improve business processes to leverage the information end customers provide every day. We Many of the trialists selected for this project were can also help ensure sales, product, and support influential in the European wireless community. teams are in sync. These trialists provided input and saw the product improve as a result of their feedback. In many cases, Technical Support they became personally invested in and outspoken Technical support as a means to an end. Your advocates of the product. customers are telling you every day what they love and what you need to improve. MobilEvolution’s Tier About MobilEvolution III support engineers solve your customer’s problems, Founded in 2008, MobilEvolution helps clients in the then help your business avoid the problem in the wireless industry identify and address gaps between future. their customer’s expectations and their experience. Please contact us if you would like to learn more MobilEvolution staff have deep wireless experience about MobilEvolution or our services. and expertise in developing web- and device-based tools, sales enablement programs, content, technical support, and consulting services. “Listen. Learn. Evolve.” is the MobilEvolution mantra, and we keep that concept at the heart of our projects. We help our clients listen to their customers, learn from the information hidden in the data, and evolve their product or service to meet their customer’s