Build Five-Star Mobile Apps

Transcription

1 FOR: Application Development & Delivery professionals Build Five-Star Mobile Apps by Jeffrey s. Hammond, november 7, 2012 key TakeaWays Mobile development practices Focus on Quick Feedback Shops building great mobile apps simplify their development processes to minimize the time it takes to get actionable customer feedback. This means a focus on Agile principles, simpler development tools, and smaller teams. The need for a minimum viable product drives short development cycles and immediate exposure to customers for their feedback. attention To Quality is Critical When developing Mobile apps Curated app stores make it easier for developers to reach customers, but they also introduce an approval step that varies in length. As a result, building mobile apps is more like creating shrink-wrapped software than building web applications. Product defects can kill your app ratings, so make sure you release a quality app from the start. you ll Need To Learn New Tools, Techniques, and Languages Don t be afraid to simplify existing development practices and eliminate highceremony stage-gate processes and ALM tools in favor of practical approaches that mix well with the new languages and IDEs your developers will use to build apps. Short-term flexibility and a learning culture will help you transition to mobile-first development. Forrester Research, Inc., 60 Acorn park Drive, cambridge, ma UsA Tel: Fax:

2 November 7, 2012 Build Five-Star Mobile Apps Processes: The Mobile App Development Playbook by Jeffrey S. Hammond with Phil Murphy and Shannon Coyne Why Read This Report There are hundreds of thousands of mobile apps out there, and they re all competing for the same mobile users. If you want to reach your customers with a custom mobile app, then you want to make sure you create one that will get high ratings. In short stars matter. So what s the best way to structure your development shop if you want to build a highly rated app? For this report, which is the processes report of the mobile app development playbook for application development and delivery (AD&D) professionals, we spoke with more than a dozen leading mobile app developers about how they build, test, and release their top-rated mobile apps. We ve abstracted out common development best practices that these shops use that should prove helpful in building your own five-star mobile app. Table Of Contents 2 Make Your Mobile App Stand Out From The Crowd 2 Heed These Best Practices And Lessons To Build Five-Star Mobile Apps Assemble Small, Focused Development Teams Favor Simple Development Tools Over Complex ALM Adapt Agile Principles To Meet The Demands Of Mobile App Development Balance Release Speed With An Increased Focus On Quality Pitfalls To Avoid When Focusing On Quality Releases Prioritize Gathering User Feedback And Forming A Rapid Response To It Notes & Resources Forrester interviewed 13 mobile app development shops with highly rated apps: Banjo, Brainshark, Ci&T, Cocoa Box Design, Cynergy Systems, Digby, EffectiveUI, Hipmunk, Kayak.com, NatureShare, Path, Roundarch Isobar, and Untappd. Related Research Documents Mobile Technologies That Drive Sales August 10, 2012 Build Great Mobile Apps That Drive Engagement June 28, 2012 Building Mobile Apps? Start With Web; Move To Hybrid January 5, Lessons Learned When Gathering User Feedback Forrester s Mobile Development Next Practices Identifying Your Challenges Supplemental Material 2012, Forrester Research, Inc. All rights reserved. Unauthorized reproduction is strictly prohibited. Information is based on best available resources. Opinions reflect judgment at the time and are subject to change. Forrester, Technographics, Forrester Wave, RoleView, TechRadar, and Total Economic Impact are trademarks of Forrester Research, Inc. All other trademarks are the property of their respective companies. To purchase reprints of this document, please [email protected]. For additional information, go to

3 Build Five-Star Mobile Apps 2 Make YouR Mobile App Stand Out from the crowd You re finally ready to build and deploy your first mobile app. But how do you make sure it will attract happy, repeat customers? Deploying an app to public app stores is straightforward enough, and once you do potential customers can search for and find your app if they know what they re looking for. But in reality, it s not as simple as it sounds: At last count, there were more than 500,000 mobile apps in the Apple App Store and more than 450,000 in the Google Play store. At the same time, the average US smartphone user has an average of 41 apps installed on her smartphone. 1 That math is not encouraging, so you ll need every possible advantage when competing for a coveted space on consumers mobile devices, and high ratings will significantly help you. User reviews and ratings are important drivers of download and purchase volumes, and the converse is true as well poor ratings depress download statistics. It s simple math: Five-star apps fly off the shelves; single-star apps sit on the shelf. Which would you rather build? 2 So what sort of development processes will help you build a five-star mobile app? We asked 13 different mobile development shops to share their processes, tools, and best practices. These shops all share a common trait: They all have at least one highly rated, successful mobile app in production. But that s not all they have in common: There s a recurring set of best practices these shops use to deliver a useful, usable, desirable mobile experience. Some of these practices may seem obvious, but adopting them may be harder than you think. Heed these best practices and lessons To build Five-star mobile apps Building a five-star app requires more than buying the right development tools and adopting a welldefined development methodology such as Scrum. Real success comes when you create a rapid, feedback-oriented development loop that does not scrimp on quality. The top mobile development shops we interviewed have adopted the following best practices to minimize time-to-actionable-feedback. Assemble Small, Focused Development Teams Mobile applications are called apps for a reason; they are generally smaller and less complex than traditional Java,.NET, or web applications. It makes sense, then, that when we asked five-star app development shops about the composition of their teams that we would find a pattern favoring smaller, more focused development teams. To follow the lead of these five-star development shops: Plan for more and smaller teams with two to six developers. None of the development shops we spoke with have more than eight total members working on their mobile apps. It s possible to release a five-star app with only one or two developers, but adding a full-time designer greatly helps with the feedback-gathering process. We also found that individual subteams are

4 Build Five-Star Mobile Apps 3 common: When developers use native platform software development kits (SDKs), it s common to find an ios subteam and an Android subteam that loosely coordinate their release efforts but work largely as independent units. It s also common to find front-end teams that concentrate on user experience (UX) and on-client controller code and testing and separate back-end teams that focus on integration with systems of record and cross-channel integration. Employ specialized developers but not specialized quality assurance (QA) and design. Many of the shops we spoke with build a native app for at least one platform usually ios. They believe that a pixel-perfect experience and full integration with the device platform are part of providing the best user experience. These shops employ specialized developers to focus on ios, Android, or mobile Web. The reason for developer specialization is simple: Each platform may use different programming languages, different programming frameworks, and different platform release schedules. With the frequency of new platform releases, it s all developers can do right now to stay on top of one native platform, let alone three or four. The same doesn t necessarily hold true for design and quality assurance. Although design for each mobile platform does have subtle differences (e.g., the placement of navigation bars), the teams we spoke with have found that nondevelopers can work across multiple platforms more easily. We have a dedicated team that does client development engineers dedicated to each platform exclusively for specialization purposes. But the teams work closely/in tandem. When we re in the design and development process, we take into account both platforms unique challenges. (Dylan Casey, head of product management, Path) Hire design talent and invest in information architecture. One of the hallmarks of great apps is design the screens flow naturally, the most important information is at the users fingertips, and the app seems to anticipate what users want to do next. Great UX design and task flow don t just happen, and most developers don t know the heuristics of a good industrial design. While it s possible to organically evolve a good design through constant user feedback and effective A/B testing, many of the shops we interviewed accelerate that process by investing in at least one experienced professional designer or information architect. 3 Cross-train developers for emergency maintenance. With small development teams and tight release timelines, juggling vacations, platform updates, and emergency releases can be problematic. Succession planning can also be a challenge, especially when there s so much demand for talented mobile developers. While it s good to let developers specialize on a primary mobile platform, it s also useful for them to back up other developers on a secondary platform to increase bench depth (see Figure 1). This also allows flexibility when it comes to bug fixing, testing, and feature development.

5 Build Five-Star Mobile Apps 4 Retire the QA center of excellence. When it comes to testing mobile apps, traditional testing approaches are too slow. The need for on-device testing, quick turnaround times, and specialized knowledge makes a traditional QA center of excellence less useful for mobile development teams. In every shop we talked to, the development teams take on responsibility for app testing. Manual, on-device testing was the most common approach, but several companies also deploy early versions of apps internally to other company employees or externally to a standing set of beta testers. Deploying beta versions is easy for Android apps they can be side-loaded onto users devices. It s a bit more challenging for ios apps, as Apple limits developers to 100 ad hoc licenses but that s still a good population for gathering feedback. Refocus your sourcing strategies. Some of the companies we spoke with are too small to engage large system integrators yet their apps are doing exceedingly well in the stores. They are doing just fine with smaller digital agencies and hiring freelance talent. The bottom line is that you re better off looking for mobile specialists with mobile-specific design and development talent. Specialist firms have already adopted many of the process techniques we ve described in this document, so they re culturally capable of moving fast with a minimum of overhead. If you ve spent the past few years winnowing the number of firms you work with in order to gain hourly rate efficiency, you will probably need to rethink how you are sourcing development talent to focus on skills rather than labor costs. Figure 1 Cross-Train Developers For Increased Bench Depth Product owner/ product manager (PM) Shared, cross-team roles Design team Scrum master/ project manager ios development team Android development team Web development team Lead ios dev Lead Android dev Lead web dev ios dev Android dev Web dev Cross-train Cross-train Cross-train Source: Forrester Research, Inc.

6 Build Five-Star Mobile Apps 5 Favor Simple Development Tools Over Complex ALM When we asked our five-star shops what development tools they use to build, test, and deploy mobile apps, one thing became crystal clear: They all employ a minimalist approach to application life-cycle management (ALM). With a focus on time-to-customer-feedback, there s simply not a lot of tolerance for any processes or tools that get in the way. As a result, these shops: Simplify and distribute software configuration management. All the companies we interviewed use Git or Apache Subversion to manage their source code. 4 Why? We heard a common refrain: They are simple to use, and the developers come in already knowing how to use them. With a tendency toward faster releases, the need for long-running parallel branching, streams, or baselines supported by more-advanced software configuration management (SCM) tools becomes less of a need. And with fewer developers, it s easier for teams to work on a master branch or engage in rapid branching and merging through distributed version control with local repositories. Use visual designs, prototypes, and collaboration instead of textual requirements. We didn t run into a single development shop in our interviews that uses textual tools for requirements capture or management. Some shops use a very simple approach, creating bitmap screen mockups with tools such as Adobe Photoshop or OmniGraffle. Others use more-specialized tools for wireframing or mockups such as Balsamiq. They then capture user feedback generated from these visual designs in lightweight change management tools such as Trello and Atlassian JIRA. A number of the teams we spoke with mentioned that they use file-sharing services such as Dropbox or Google Drive to share screen shots and collaborate on updates. Test apps with emulators and personal devices. One of the biggest differences that development teams face when building mobile apps is testing. You can t simply throw mobile apps over the wall to a separate test group or a QA center of excellence. Not only are mobile testing tools new and different, but they are also generally more complex. Setting up an Android emulator or ios simulator requires installing an integrated development environment (IDE). And simulator/emulator testing is not sufficient; on-device testing remains the best way to assess on-network performance. While cloud-based testing solutions are available to mobile developers, current adoption among the organizations we spoke with is minimal. The current reality is that teams use personal devices to test beta builds of many five-star apps. These devices may be purchased and pooled by development teams or owned by developers, company employees, or trusted beta testers. The development teams we spoke with are interested in a better approach but also maintain that their current approach to testing gets the job done. We simulate the app on the computer to get through an initial test scenario. But even so, there are some things we can t test on the simulator. Every day we run or walk with runens to get real feedback that we send to the developers. It s the only way to ensure real usability. (Marcio Cyrillo, VP marketing, Ci&T)

7 Build Five-Star Mobile Apps 6 Use mock objects and back-end-as-a-service frameworks to manage multilayer complexity. Several of the development teams we spoke with use test mocks to model interfaces to their existing systems of record. The mock interfaces allow front-end user interface (UI) teams to move forward on design in parallel with infrastructure teams as they flesh out linkages from external web service application programming interfaces (APIs) to internal system interfaces. In a similar way, we saw some teams employ a mobile back-end-as-a-service (BaaS) to quickly give front-end UI developers objects in Objective-C and Java. 5 Struggle with continuous integration as development grows decentralized. One recurring problem we found mobile development shops struggling with is a fractured, immature tool chain. That s partially due to poor integration APIs in Apple Xcode but also because targeting multiple platforms requires multiple IDEs and platform-specific languages and deployment steps. As a result, most developers simply build their apps in their local Xcode or Eclipse instance. Some development shops are trying to use Eclipse Hudson or Jenkins to recreate the continuous integration loops they ve been used to, but none of the shops we spoke with really feel like they have nailed it yet. Adapt Agile Principles To Meet The Demands Of Mobile App Development Let s be clear: Our research shows that if you cannot implement Agile practices as part of your mobile app development efforts, you will have difficulty maintaining a five-star mobile app. Sure, you might have a successful initial launch, but if you can t quickly follow it up, then your app will languish, starved for features, or worse it may stop working when the latest version of ios or Android is released. In the past four years, Android has had 11 releases, and ios is updated just as frequently. You cannot keep your customers from upgrading, so you ll need to plan on a minimum of two to three releases a year per platform just to tread water. Most of the mobile app developers we spoke with release far more often than two to three times a year: On average, we release four to five times a year. (Ben Zotto, founder, Cocoa Box Design) We average about 10 releases a year. (Bill O Donnell, chief architect, Kayak.com) We average a release every two weeks. (Damien Patton, CEO, Banjo) At these release rates, it s impractical to talk about using iterative methods, let alone a water-scrumfall approach. You must be capable of releasing new updates on demand. The mobile app developers we talked to don t take a dogmatic approach to Agile practices. Several used Scrum or extreme programming (XP) as an initial reference point, but they ve modified traditional Agile practices and added new ones that are particularly well suited to building mobile apps:

8 Build Five-Star Mobile Apps 7 Introducing the concept of a minimum viable product helps teams get feedback faster. When you re in a competitive market, it s tempting to add as many bells and whistles to your mobile app as possible after all, it s got to be better than your competitors apps, right? Wrong the more features you add before release, the more time it takes to develop those features. And unless you have a deeper understanding of your customers than most development teams, there s a high probability that you ll miss the mark on your first shot. Instead, ask this question: What is the minimum feature set we need to collect the maximum amount of information about customers with the least effort? 6 Creating a minimum viable product (MVP) allows you to stop guessing and start learning about how your customers are using their mobile devices. Kanban boards help manage atomic demand. At systems integrator Ci&T, a small subteam has built a highly rated mobile app for runners called runens. The runens team mixes a Kanban board with a more-traditional Scrum process because it helps it manage unpredictable demand at an atomic scale. We use our board as an organizational tool that complements our daily stand-ups. It s where we see things moving. When we first started, our backlog was immense, but as things start to move toward our release, the board s a visible confirmation we re making progress. (Marcio Cyrillo, VP marketing, Ci&T) Mobile app development efforts are often smaller than traditional application development efforts, so user stories and epics may prove too coarse-grained for teams to get a feeling of flow. Managing demand at the feature (or even defect) level lets teams further decompose larger work items while mixing in user requests or defect fixing. Using wireframes helps drive feedback and build backlog. App developer Banjo describes its development process as Agile on steroids. But there s one important design step that must take place before a Banjo development team starts a series of sprints: wireframing. Banjo uses wireframe mockups of app screens with potential users to flesh out basic design concepts and what the service should do. After wireframing is complete, the team refines its backlog to answer a simple question: How can we get the minimum viable product out as fast as possible? Several firms we spoke with mentioned wireframes as their primary requirements-elicitation technique. Prototyping sets the stage for broad-brush feedback. Wireframes are a good starting point, but they don t necessarily capture the realities of touch interaction or the finer points of UI design. That s where higher-fidelity prototypes come into play. Several of the shops we spoke with use mockups of nonfunctional screen images deployed to real mobile devices to assess how users respond to different designs. The prototypes are nonfunctional in that they have no code and no business rules behind them. What they do help with is screen flow and navigational design. Prototypes answer questions such as, Should the user tap a button or swipe to the next screen? and Is an accordion a better interaction design than tabbed folders?

9 Build Five-Star Mobile Apps 8 I m huge on prototyping. We spent about a month creating low-fidelity wireframes for our initial iphone release, and when our founders were comfortable, we moved onto prototyping and then to code. (Danilo Campos, UI/UX designer and ios developer, Hipmunk) Prototypes can be used with target customers, but more often than not the shops we talked to expose prototypes to development teams, business sponsors, or company employees to gather early feedback on interaction design. Think of a prototype as a low-cost way to institute a scientific approach to design (i.e., early stage A/B testing). Developers that use prototypes may start with two different graphical layouts or navigation flows so they can see which one users like better. The results refine the design before the team engages in the real expense of cutting code. Personas provide insight into user behavior. Several of the shops we talked to use explicit personas to guide their app design. A persona helps answer questions about a user s mobile app objectives and can provide clues to appropriate technology decisions. When NatureShare designed the Audubon Birds app, it started the process based on the team s understanding of how existing birders use the paper field guide. The team also knew that the app was likely to be used in the field and that there was a high likelihood that connectivity would be spotty. This knowledge of user behavior pushed NatureShare toward a device-centric, native app with a large amount of content. It also allowed it to put a higher price tag on the app (currently $9.99) because the team felt that the app s target users would understand that the app provided more value than the standard field guide at a fraction of the price. Journey maps add a multichannel dimension to the user experience. Whether they are taking meeting notes, traveling, banking, or even drinking good beer, customers expect a seamless, multichannel experience. As a result, an increasing number of mobile apps are not standalone but connected to a larger set of interactions. For several years, we ve seen leading interactive agencies map a customer s journey as he tries to accomplish an evolving set of tasks as part of brand engagement. 7 Now we see these design firms extending journey maps to include an elaboration of prioritized digital touchpoints (see Figure 2). A journey map that details when a mobile app is a priority engagement channel is helpful in defining a minimum viable product for an initial release and also in sharpening a development team s focus on the mobile tasks that really matter. Organic release schedules help teams meet user and market demands. Many application development teams try to normalize release schedules say, one major release a year or spring/ summer/fall/winter releases. But in the mobile world, a regular release cadence is almost an anachronism. Think about it: You re not shipping DVDs or CDs. The user gets proactively notified that a new release is available, and there s really not a strong drive for annual updates to drive support contracts. The mobile shops we spoke with adopt a release-when-ready approach.

10 Build Five-Star Mobile Apps 9 Our release time is as short as a month and as long as three to four months. It varies based on numerous factors, such as the complexity of the functions being built or an operating system update. We like to build releases around themes. You also need to consider market timing. For instance, you want to be wary of shipping a release right before Christmas, because it wipes out user ratings and resets the clock on an app. (Ben Zotto, founder, Cocoa Box Design) The bottom line is that you should expect and become comfortable with a more organic release schedule. One common pattern several companies described is a major release followed by several quick follow-up releases, with a longer period of stabilization (see Figure 3). Figure 2 Multichannel Journey Maps Plot Customer Engagement Across Channels Identify customer and stages of journey. Persona: James Wow Enjoyable Functional 1 Awareness Consideration Research Purchase Engagement Describe each step in the journey; the customer s needs and perceptions. Neutral Indicate significant steps. Missed it Frustrating Source: Strativity Group website ( Indicate primary (and secondary) devices for each step. Source: Forrester Research, Inc.

11 Build Five-Star Mobile Apps 10 Figure 3 Release Schedules Vary Based On User Need And Market Factors Features plus defect fixes V 2.0 V 2.1 V 1.2 V V V 1.0 (minimum viable product) Regression plus emergency patch New OS version released Regular internal sprint cycle plus beta testing (2 weeks) Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Source: Forrester Research, Inc. Balance Release Speed With An Increased Focus On Quality With an increased focus on achieving a speedy release, it would seem like a continuous deployment regimen and a focus on rapid releases would be a natural fit for mobile developers, right? Well, not so fast: We got what seemed like a counterintuitive set of warnings about releasing too fast from the development shops we talked to. These shops recommend a cautious deployment approach that balances release speed with an increased focus on testing and quality assurance. Here s why: Initial quality is important because of app store intermediation. Most public app stores manage the relationship between developers and customers. That s great when it comes to marketing and getting paid for apps that you develop, but it s not so great when you make a mistake and a glaring defect slips through your QA process into production. The hard truth is that you have no control over how long it takes a reviewer at Apple, Microsoft, or Amazon.com to approve your emergency patch, and you have no insight into how long the queue of apps waiting to be approved is.

12 Build Five-Star Mobile Apps 11 The consequence of putting out a build that fails or doesn t start for 20% of users is dire. To be blunt, you ll get slaughtered. I don t put a release out unless I know it s good. (Ben Zotto, founder, Cocoa Box Design) It may take a day, but it might take a week or more, and there s nothing more frustrating than watching poor reviews come rolling in when you ve already fixed the problem in a queued-up release. Before a release goes out, we go through an extensive QA process. We start with a day runthrough of the app manually according to our checklist. Then we install the build on about 30 devices. We have people use it for several days and record any bugs/crashes, and there s also a feedback button to submit any problems they have. (Damien Patton, CEO, Banjo) Expedited releases are possible but no substitute for real testing. In spite of thorough testing, problems still crop up. At least half the development teams we talked to reported defects that affected their overall ratings when they shipped a subpar update. When it was an Android app, they quickly spun a new release, and in many cases they were able to get a same-day update for ios apps by asking for an expedited review. While an expedited review may sound like a great safety valve, be careful. The teams we talked to advised against crying wolf! more than once a year. Don t use an expedited release to cover for poor QA processes but as a rare safety valve to escape a truly unforeseen problem. The opinions from the shops we spoke with were unanimous it s worth it to spend money and time to make sure there are no huge bugs before release. To balance release speed with app quality: Use mobile deployment and feedback management tools. Several of the shops we talked with swear by a new set of mobile feedback management tools. You may have never heard of them, but tools such as TestFlight, HockeyApp, and App47 make it easier to manage distribution of beta license keys, automate collection of app feedback from beta users, and instrument apps with crash detection and logging. These tools allow fast-moving teams to quickly update beta testers through over-the-air updates and aggregate diagnostic information into customizable dashboards or reports. Don t wait for new mobile operating system (OS) versions use platform betas. If you re not used to working with beta code from platform vendors, you re going to need to change your policy. The reason is simple: Most customers will upgrade to a new version of their mobile operating system the month that it comes out, and there s nothing you can do to stop them. Wait until a release goes gold, and you won t have enough time to develop and test new features. Whenever a new beta build of a target OS is released, you should upgrade development devices for a two-step evaluation process. Step 1: Test your current app to make sure the new release doesn t break your code. If it does, it s critical to push a patch to existing devices before a general

13 Build Five-Star Mobile Apps 12 release. Step 2: Your team should evaluate the new platform and how you can take advantage of it to extend your app s capabilities and get a jump on the competition. Use different device platforms for blue/green releases. In a traditional blue/green setup, DevOps practitioners keep two parallel production environments that are constantly evolving between production versions n, n + 1, and n + 2. Blue/green deployment is an operational complement to A/B testing, as it allows teams to observe actual performance and accelerate or slow deployment plans for new features and patches. When it comes to multiplatform mobile apps, we see some teams release a feature in one code base first, evaluate how it works and gather user feedback, and then decide whether to expand it across all platforms. This practice is easier if you happen to be building apps with a hybrid, web, or middleware approach, as you can update your apps without redeploying through an app store. Get to know your developer relations contacts. Several of the companies we spoke with mentioned that it s worth cultivating a relationship with platform vendors developer relations employees. They can provide valuable advice about design best practices, tip you off to potential changes in app review policies, and help facilitate the process of asking for an expedited release, should you need one. Pitfalls To Avoid When Focusing On Quality Releases The five-star development shops we spoke with warned against two release gotchas: Don t create platform envy target simultaneous ship. When you have a small team of developers supporting multiple platforms, it s tempting to target one device at a time (i.e., release an ios app first, then start an Android app). The development shops we spoke with cautioned against this, especially after the initial release of the app is out on both platforms. Release a cool new feature on ios, and the Android users will complain (or vice versa). Note that platform envy creates a potential conflict with blue/green feature testing across different platforms, so you ll need to carefully time-box your evaluation period and be ready to accelerate the release of a feature that proves popular. We ve recently been working to get our Android app up to feature parity with ios. Android users are very vocal when they feel the Android version isn t up to par with the ios version. Once we got up to par with ios on Android, our rating improved significantly. (Dylan Casey, head of product management, Path) Avoid patches bundle bug fixes with new features. One of the really nice things about mobile app stores is that it s now easy for developers to alert customers that a new version of an app is ready to download. This can make it tempting to slip changes into apps on an ongoing basis, especially if you ve fixed some niggling defects in the code base. But be careful; it s better to batch those bug fixes with at least one new feature.

14 Build Five-Star Mobile Apps 13 One mantra I ve learned: No bug fixes, only releases. Putting out a bug fix angers customers, because their expectation is that a new build should have new features. These customers are more likely to leave bad reviews. (Ben Zotto, founder, Cocoa Box Design) Prioritize Gathering User Feedback And Forming A Rapid Response To It In your current development shop, do you make your list of defects open and available for all to see? Open source projects may, but most companies don t that level of transparency is disconcerting at best. When it comes to apps deployed in public app stores, you re going to have to get used to operating in the bright white lights of a public stage. Amazon, Apple, Google, and Microsoft make it incredibly easy for users to rate apps on their platforms and offer feedback about their experiences. And if an app s rating is the first thing a potential customer looks at, the detailed comments about what customers like and don t like about an app is the next. This focus on widely available public feedback is disconcerting to many traditional development shops. You don t control it, and you can t sweep it under the rug the best things you can do is manage it. To deal with user feedback, the mobile development shops we talked with: Collect feedback early and often. Let s be honest, even the best mobile developers and designers are still working out the finer points of mobile app design. We re still in the early phases of a major transition, so the best way to judge an app s usefulness is to get user feedback and lots of it. To get our first MVP, we got about 2,000 user opinions over a period of less than two months. We solicited user input on Craigslist and used social media to find as many people as we could. We then held local focus group sessions and broke into groups with five people maximum to avoid groupthink. (Damien Patton, CEO, Banjo) Gathering early feedback on a large scale allows the Banjo team to flag patterns of feature requests or identify problematic UI designs. With the design teams standing by, the Banjo team can act quickly, often having updated screen mockups ready by the end of the day and a new prototype ready the day after in time for the next user feedback session. Assign someone to listen to app store feedback. It almost goes without saying, but is anyone in your shop actually reading the feedback users post? Are they reading it daily? Do they have the ability to walk into a Scrum and reset the development priority? If not, you re in trouble. You may not need a full-time resource, but it should be part of someone s job description, and you might even want to encourage the whole development team to monitor incoming feedback. Feedback triage is an ideal role for a product owner, but if you don t have one, then consider making it a rotating position that team developers share.

15 Build Five-Star Mobile Apps 14 There s no standalone customer support team at Kayak, so customer feedback gets spread around for everybody to see, from the CEO to QA to engineers. Everyone on the mobile team gets insight into how customers are feeling. This really helps when customers have complaints about the same problem; it gets the developers looking at that issue immediately. (Bill O Donnell, chief architect, Kayak.com) Analyze posts for recurring patterns of failure and opportunity. If you treat all feedback the same, you re likely to find yourself diminishing your efforts the developer equivalent of playing whack-a-mole. 8 It s better to take a bit of time (but not much) to analyze larger batches of customer feedback to see how you can maximize your development cycles. When it comes to managing user feedback, we think of it like a circle on the wall, like a dart board. Users throw darts at the wall, and Penultimate s mission is to get the most darts in the circle. When you take the time to analyze feedback, you start to see where stuff clumps and what s most urgent. You also need to parse the difference between I can t use Penultimate unless it does X versus It would be nice if Penultimate did Y. (Ben Zotto, founder, Cocoa Box Design) Proactively reach out to unhappy users. Passion is the key to understanding user feedback. A user who has a bad experience and doesn t care about it will simply delete the app and disappear into the ether. The ones who write about their bad experiences care. This creates an opportunity to engage with them, understand the problem, fix it, and turn that passion into positive goodwill. These users want to like your app, so try to work with them and debug the problem. Even if you can t turn the application experience around, you can still influence the entire brand experience through active engagement. And since you re communicating in a public forum, you can always gain the respect of other potential users in the process. We make an effort to reach out to users who have a problem with the app; we take that feedback seriously. We invite them to call us and talk through the problem so we understand their problems from a UI perspective where something is confusing or not a great user experience. Then we try to fix it. (Charles Rattigan, EVP, NatureShare) Build feedback systems into the app to redirect frustrated customers. When a user enters feedback on an app store, it s there for everyone to see. But several of the development shops we spoke with build their own feedback systems into their apps. These systems give users an alternative route to express their opinions, one that the developers can manage. There s not much you can do about bad feedback. What we try to do is channel bad feedback to us directly. Our apps have built into them their own feedback mechanism, so people can submit their feedback directly to us. (Bill O Donnell, chief architect, Kayak.com)

16 Build Five-Star Mobile Apps 15 Ask for positive reinforcement. One way to increase you app s overall rating is to ask repeat users to give you reviews. Some of the development shops we spoke with advised asking repeat users for a review after they ve had time to use an app. But there are some caveats: Don t annoy your users ask for a review no more than once per release. You should also add an opt-out checkbox so you can avoid irritating users who don t want to be bothered. Finally, if you ve been instrumenting your app with analytics and spent time understanding your customers, you probably know when they are happiest with your app what better time to ask for a quick review? Lessons Learned When Gathering User Feedback While there are plenty of dos when it comes to gathering user feedback, there are also a few don ts: Don t stoop to generating fake user reviews. It almost goes without saying, but it happens. You ll see an app in a public app store with dozens or even hundreds of five-star ratings and a high average, but when you actually read reviews, you ll see a lot of disappointment. Apps with artificial reviews are out there the problem is particularly insidious with free apps, because there s no cost to providing the rating. You re less likely to see the problem with paid apps, as it s more expensive to generate fake reviews for these. Beware of groupthink. Whether it s a large focus group with a take-charge ringleader or an online user review that other customers pile onto, it s easy for loud, passionate, or vindictive users to create a disproportionate impact on feedback. Try to use impartial analytics, A/B tests, and small groups to balance aspiring influencers and trolls alike. Forrester s Mobile Development Next Practices While our research uncovered five mobile app development best practices, here are some next practices that mobile development teams should focus on once they ve mastered the basics (see Figure 4): Drive downloads by mastering mobile marketing. Nothing drives downloads on a curated app store like getting featured as an app of the week or a staff pick. 9 But how do you make it to that level? The developers we spoke with couldn t point to a sure-fire strategy but did mention the importance of taking advantage of platform-specific capabilities and early adoption of new capabilities when new versions of platform SDKs are released. Think about it mobile device manufacturers are in fierce competition with each other and want to show off what makes their platform unique. Give them something to show off, and you just might get some highly effective co-marketing opportunities. Evolve a connected product strategy. A standalone mobile app is great, but a connected 360-degree experience is even better. How can you connect your mobile app with your core

17 Build Five-Star Mobile Apps 16 products and services and amplify its usefulness? For example, TripIt has a nice mobile app, but it s made better by a feature that automatically scans a customer s Gmail account and parses travel itineraries, making them automatically available in the app. Connected product experiences make mobile apps stickier by increasing the level of value they deliver. Connect mobile apps and big data to deliver contextual experiences. Think of mobile devices as the first step toward a science fiction dream Mr. Spock s tricorder. 10 As mobile devices get more sophisticated and their sensors get more varied and precise, the opportunities to collect complex events grow exponentially. Add in static sensors, geofencing, and machine-to-machine communications, and it s certain we ll see a flood of events and data generated by all these devices. This data will contain a wealth of information and value, but unlocking it will require the ability to understand and process complex combinations of events and signals. In short, harnessing mobile context will require developers to integrate mobile clients with scale-out processing to maximize user convenience. Figure 4 Best And Next Practices For Mobile App Development Best practices Assemble small, focused development teams. Favor simple development tools over complex application life-cycle management (ALM). Adapt Agile principles to meet the demands of mobile app development. Balance release speed with an increased focus on quality. Prioritize gathering user feedback and forming a rapid response to it. Next practices Drive downloads by mastering mobile marketing. Evolve a connected product strategy. Connect mobile apps and big data to deliver contextual experiences Source: Forrester Research, Inc.

18 Build Five-Star Mobile Apps 17 Identifying your challenges Where should you start? Use this self-assessment to assess your current capabilities and opportunities for improvement (see Figure 5). Figure 5 Mobile App Development Best Practices Self-Assessment No. 1: Assemble small, focused development teams. Best practices Are your dev teams small and focused? Examples include organization by operating system (OS), front end, and design, testing, and integration with systems-of-record and cross-channel integration. Are your developers specialized and trained in the platforms you support? Are your developers cross-trained for emergency maintenance? Do you have people dedicated to mobile design and user experience? Yes No No. 2: Favor simple development tools over complex application life-cycle management (ALM). Best practices Have you simplified your software configuration management (SCM) tools to account for the smaller scale of mobile app dev? Examples include Git and Subversion. Are your visual designs and prototypes capturing and managing your requirements? Are you using relevant tools to manage design, prototypes, and user feedback? Are you testing your app with emulators and personal devices? Testing on personal devices is key to measuring user experience quality and on-network performance. Do you use test mocks to model interfaces to your existing system-of-record, allowing front-end user interface (UI) teams to move forward on design in parallel with infrastructure teams? Are your integration methods and application programming interface (API) tools making your life easier or harder? Are they providing quality user experiences to your users? If not, consider re-evaluating, or if possible, simply building your app in your local Xcode or Eclipse instance. Yes No No. 3: Adapt Agile principles to meet the demands of mobile app development. Best practices Are you focusing efforts on releasing a minimum viable product (MVP) in order to gain valuable user feedback faster? Are you using a Kanban board to manage demand and measure progress moving features from the backlog? Are you using wireframes to mock up potential app screens to finalize requirements for release? Do you prototype these wireframe designs before coding? Do you have a grasp on the type of user your app will attract? Are you envisioning a relevant persona when designing your app and requirements? Have you used journey maps to gain insight into tasks that matter to your users? Are you and your shop prepared for a nonregular release schedule to coincide with feature releases, OS updates, and bug fixes? Yes No Source: Forrester Research, Inc.

19 Build Five-Star Mobile Apps 18 Figure 5 Mobile App Development Best Practices Self-Assessment (Cont.) No. 4: Balance release speed with an increased focus on quality. Best practices Have you thoroughly tested your app prior to release? Are you releasing a quality product to the app store? When releasing a bug fix, do you try to incorporate new features so users feel they re getting value from the update? Are you using mobile deployment and feedback management tools? Are you testing your app on beta versions of the OS to ensure that everything still functions properly? You cannot control when your users update, so you need to be prepared. Are you utilizing new beta functionality to get ahead of the competition? Are you anticipating and planning for future releases in advance? Are you utilizing user feedback to prioritize deployment plans for new features and patches? Do you have a good relationship with your developer relations contact at the relevant app stores? Yes No No. 5: Prioritize gathering user feedback and forming a rapid response to it. Best practices Have you gathered feedback from potential users to gain a baseline for your minimum viable product (MVP) and guide your requirements? After releasing your app, are you monitoring app store feedback and incorporating what you see into your backlog and prioritizing dev team efforts accordingly? Are you categorizing feedback as nice to have to improve user experience versus need to have to work and function? Are you proactively reaching out to unhappy users? In addition to the app store reviews, are you also channeling feedback through relevant (and not annoying) alerts in your app? Yes No Source: Forrester Research, Inc. Supplemental MATERIAL Companies Interviewed For This Report Banjo Brainshark Ci&T Cocoa Box Design Cynergy Systems Digby Hipmunk Kayak.com NatureShare Path Roundarch Isobar Untappd EffectiveUI

20 Build Five-Star Mobile Apps 19 Endnotes 1 In May 2012, Nielsen research surveyed US smartphone users on the number of apps installed on their smartphones. Source: Paul Sawers, Nielsen: US smartphones have an average of 41 apps installed, up from 32 last year, TNW Insider, May 16, 2012 ( 2 May 2011 research from MTV Networks found that recommendations from people a user knows and user reviews or recommendations were nearly tied as the top factor in discovering a new entertainment app for purchase or free download. Source: How to Encourage Mobile App Discovery and Usage, emarketer, June 15, 2011 ( 3 When a development shop performs A/B testing, it develops a hypothesis about how a change might positively affect user behavior (e.g., if we use a swipe to navigate from one screen to another, users will spend more time interacting with the app than they do with our current use of buttons ). Developers then implement the alternative action path and test it against the original (control) with groups of users. Results are collected and objective data is used to determine whether the hypothesis is proved. If the hypothesis is proved, the change is implemented. 4 Git is a popular open source distributed version control system that is lightweight and relatively simple for developers to learn. Source: Git ( Subversion is a centralized source code management repository that is widely used by application development teams. Source: Apache Subversion ( 5 Mobile BaaS offerings address the gap between front-end development proficiency and back-end infrastructure requirements. Enterprise-class back-end-as-a-service addresses scalability while providing an integration platform into existing enterprise services. See the August 30, 2012, Cloud Mobile Development: Enabled By Back-End-As-A-Service, Mobile s New Middleware report. 6 Eric Reis describes the concept of a minimum viable product as part of his Lean Startup methodology. Source: The Lean Startup ( 7 Left to their own devices, companies often neglect customers. But they don t need to. We recommend that organizations use customer journey maps to examine interactions from their customers points of view. Mapping the customer journey requires five steps: 1) collect internal insights; 2) develop initial hypotheses; 3) research customer processes, needs, and perceptions; 4) analyze customer research; and 5) map the customer journey. To get the most value from these journey maps, companies need to widely share findings, take action on insights, and sustain the learnings over time. See the February 5, 2010, Mapping The Customer Journey report. 8 Whack-a-mole is an American carnival game where the player tries to force moles back into holes with a soft mallet only to have other moles pop up. It also refers to a repetitious and futile task: Each time an adversary is whacked, or kicked off a service, he only pops up again somewhere else. Source: Wikipedia (

21 Build Five-Star Mobile Apps 20 9 Curated app stores are an outgrowth of new walled gardens controlled by platform vendors. They stand in direct comparison to web applications or traditional shrink-wrapped software, which is not curated or limited by platform vendors. 10 The tricorder was a multifunction sensing and recording device made popular in the original science fiction series Star Trek. It was originally carried by Leonard Nimoy in his role as Mr. Spock, chief science officer on the enterprise. The tricorder is now the inspiration for the Qualcomm Tricorder X Prize. Source: Mark P. Mills, With the Tricorder X PRIZE Qualcomm Launches the New Era of Metadata Medicine, Forbes, January 18, 2012 (

22 About Forrester A global research and advisory firm, Forrester inspires leaders, informs better decisions, and helps the world s top companies turn the complexity of change into business advantage. Our researchbased insight and objective advice enable IT professionals to lead more successfully within IT and extend their impact beyond the traditional IT organization. Tailored to your individual role, our resources allow you to focus on important business issues margin, speed, growth first, technology second. for more information To find out how Forrester Research can help you be successful every day, please contact the office nearest you, or visit us at For a complete list of worldwide locations, visit Client support For information on hard-copy or electronic reprints, please contact Client Support at , , or [email protected]. We offer quantity discounts and special pricing for academic and nonprofit institutions. Forrester Focuses On Application Development & Delivery Professionals «Responsible for leading development and delivery of applications that support your company s business strategies, you also choose technology and architecture while managing people, skills, practices, and organization to maximize value. Forrester s subject-matter expertise and deep understanding of your role will help you create forward-thinking strategies; weigh opportunity against risk; justify decisions; and optimize your individual, team, and corporate performance. Andrea Davies, client persona representing Market Insights Professionals Forrester Research, Inc. (Nasdaq: FORR) is an independent research company that provides pragmatic and forward-thinking advice to global leaders in business and technology. Forrester works with professionals in 17 key roles at major companies providing proprietary research, customer insight, consulting, events, and peer-to-peer executive programs. For more than 29 years, Forrester has been making IT, marketing, and technology industry leaders successful every day. For more information, visit