Great Expectations: What Users Want From the Mobile Web Experience

Transcription

1 W H I T E PA P E R Great Expectations: What Users Want From the Mobile Web Experience

2 COMPANIES FEEL THE MOBILE IMPAC T Mobile Websites and applications are changing the way we do business. Consider these examples: ebay: Its iphone app has generated $400 million since launching in 2008 Pizza Hut: In three months, Pizza Hut s mobile app surpassed $1 million in sales Marriott: More than $1.25 million in bookings in the first 100 days of its mobile Website launch With mobile, your Website is literally in your customers hands. But are you ready to meet their expectations? After years of anticipation, the mobile Web is finally on the move. By 2013, the US mobile Internet audience is expected to reach million users. i One network alone, AT&T, has seen its mobile data traffic increase by fifty times in just the last three years. ii And perhaps the biggest name in mobile consumer products, Apple s iphone, has stimulated 3 billion mobile application downloads since the opening of the App Store less than two years ago. iii With recent advances in mobile commerce, advertising cannot be far behind. In fact, the industry anticipates a $1.56 billion mobile advertising market by iv Google and Apple have both made high-profile mobile ad network acquisitions that are helping to fuel expectations. Behind the scenes, exploding smartphone sales are further driving this boom. Morgan Stanley predicts smartphones will out-ship the global PC market (that s desktops, laptops and netbooks combined) by v Smartphones already beat basic feature phones by a factor of five times vi in complex services usage, such as and Web browsing. vii With great power comes great responsibility The combination of more sophisticated devices, more powerful browsers and superior connectivity today s WiFi and 3G networks speeds can be as much as six times faster than their 2G and 2.5G predecessors have elevated end-user expectations. Consider the results of a recent Gomez study of more than 1,000 US mobile Internet users: 58% of users expect Website load speeds on their mobile devices to be comparable to, or better than, what they experience on their desktops. Web page load expectation mobile vs. home/laptop computer A recent Forrester Research study found similar results: today s patience threshold for the wired Web is just two seconds for page load time, down from four seconds only three years ago. Looking ahead, expectations will only continue to rise. Within twelve to sixteen months, the next generation in mobile connectivity, 4G networks, will make smartphones smarter, networks faster and end-users even more eager for immediate interactive gratification. The message is clear: mobile users may be willing to trade some functionality for the convenience of mobile connectivity, but they will not sacrifice speed or availability. W H I T E PA P E R G R E AT E X P E C TAT I O N S : W H AT U S E R S WA N T F R O M T H E M O B I L E W E B E X P E R I E N C E 2

3 A look at four bellwether verticals In the following pages, we ll examine performance statistics and research gleaned from four key verticals: search, retail, banking and airline. These high-impact, high-traffic, areas expose crucial insights on the relationship between mobile Web performance and market leadership. The data presented is a summary based on the following two key performance metrics that Gomez mobile Web benchmarks measure on a continuous basis: Response Time How fast a given mobile Website homepage or multistep transaction loads. Availability Percentage of successful mobile Website homepage or multistep transaction downloads out of total download attempts. For more information concerning Gomez s mobile Web benchmarks please refer to the benchmark methodology section at the end of this whitepaper. With Gomez s research, you can compare your own site s mobile Web performance to industry leaders across two vital metrics: Website availability and response time. Website availability is a measure of your ability to be reached after all, your store will only make money when it s open to customers. And response time is directly tied to consumer behavior. Bing has found that a slowdown as small as 2 seconds leads to a 4.3% reduction in revenue per user. By measuring your performance against the best, you gain a more precise understanding of where your company stands or where it needs to be. Gomez s Mobile Web Benchmark Search, April 2010 Gomez s mobile Website performance benchmark for the search sector evaluates a number of performance metrics including the response time and availability of a 3-step transaction. W H I T E PA P E R G R E AT E X P E C TAT I O N S : W H AT U S E R S WA N T F R O M T H E M O B I L E W E B E X P E R I E N C E 3

4 This link between increased usage and a faster user experience be it search or mobile Gmail reinforces something we at Google have known for a long time: fast is better than slow. Robert Hamilton, Product Manager, Google mobile team xii SNAPSHOT: GOOGLE GOES MOBILE As of January 2010, Google had 68.2% of the US mobile search market. Yahoo followed at 31.5% and Bing trailed at a distant 0.2%. xiii From January September 2009, Google s search page ranked #1 in Nielsen s top US Websites accessed by mobile phones. xiv Mobile Search If search is the 500-pound gorilla of the Web, it s an even bigger beast in mobile. Given the smaller screen size, hand-held devices encourage searching over browsing. More significantly, the anytime, anywhere appeal of smartphones stimulate impromptu searching when shopping, travelling or pursuing leisure activities. In fact, the most recent search upgrades are designed to make mobile search even faster. Google is leading a trend that would allow searches by location (leveraging GPS technology), by pictures (using built-in phone cameras) and even by voice input. The graph below reveals average mobile Website availability (up-time accessible to the user) for a three-step search process across search industry leaders over the given time period. Google had a unique audience of 27,913,000 in December 2009 (The Nielsen Company). Average Mobile Website Search Transaction Availability, March 2009 January 2010 At its low-point, availability averaged 97.44%, which represents roughly 18 hours of downtime per month. At its high-point, availability averaged 99.05%, which still represents 7 hours lost each month. Leadership: The category leader was also the lead performer, with an average of 99.62%. At the bottom end of the scale, category laggards dropped as low as 95.93%. Gomez measured mobile Website response times using a consistent, three-step search process with each search industry leader. W H I T E PA P E R G R E AT E X P E C TAT I O N S : W H AT U S E R S WA N T F R O M T H E M O B I L E W E B E X P E R I E N C E 4

5 Average Mobile Website Search Transaction Response Times, March 2009 January 2010 Transaction response times hovered within a 6-9 second window throughout the testing period. Leadership: The category leader had an overall average of 3.39 seconds. Category laggards dipped as low as seconds. Benchmarks your site should meet or beat: Availability: The average availability for the search category was 98.47%, equal to 11 hours of downtime/month. Response time: Average response time (for a three-step transaction) for the top ten search providers in an eight-month period was 7.40 seconds. W H I T E PA P E R G R E AT E X P E C TAT I O N S : W H AT U S E R S WA N T F R O M T H E M O B I L E W E B E X P E R I E N C E 5

6 Mobile is very important to our strategy and will only become more critical as consumer adoption and usage becomes more widespread. Steve Yankovich, vice president of platform business solutions and mobile, ebay xviii Focus on Mobile Commerce Shopping is proving to be as popular on mobile devices as it has been on desktops. Even in a recession year, ABI Research reports that mobile shopping increased more than 300% in one year, from $396 million in 2008 to $1.2 billion in xv Best Buy calculates the number of its mobile visitors in FY2009 to be 6.5 million; in FY2010, they expect 17 million. In a survey of 2,000 smartphone users, Compete found that at least 68% of users conduct some form of mobile shopping activity such as looking for reviews, shipping information or store coupons. xvi Average mobile Website home page availability between Dec and Jan in retail hovered around 99.16%, the equivalent of about 6 hours of downtime per month. Leadership: The category leader led performance with an availability average of 99.88%. Category laggards went as low as 97.62%. Home page response times averaged 3.92 seconds between Dec and Jan Leadership: The category leader had an overall average of 2.05 seconds. Category laggards went as low as 5.90 seconds. Benchmarks your site should meet or beat: Availability: The average home page mobile retail home page availability was 99.16%. THE NIELSEN GROUP RANKS THE TOP M-COMMERCE MERCHANTS BY VISITOR VOLUME x vii The top four as of October 2009: 1. ebay: 5.03 million visitors, up 26.7% from 3.97 million in October Amazon.com: 3.51 million visitors, up 42.6% from 2.46 million in October GameSpot: 2.58 million, up 33.6% from 1.93 million in October Netflix: 1.85 million, up 240.6% from 543,000 in October 2008 W H I T E PA P E R G R E AT E X P E C TAT I O N S : W H AT U S E R S WA N T F R O M T H E M O B I L E W E B E X P E R I E N C E 6

7 Response time: Average response time in the category was 3.92 seconds. Mobile Banking Banking-related interactions are among the most data-intensive, security-sensitive and popular of all activities on the Internet. The mobile Web is no different. From 2007 to 2008, Bank of America experienced a 300% growth in mobile customers. In April 2009, the bank had 2.2 million customers; by December 2009, that number had risen to 3.5 million. Bank of America attributes the opening of 150,000 new deposit accounts to its support for mobile banking. xix Independent research supports the bank s enthusiasm for mobile. According to ABI Research, by 2015 as many as 407 million people worldwide will bank by mobile phone, with 66 million from North America. To date, the global number of users banking by mobile phone more than doubled between 2008 and 2009, and is expected to double again in Mobile banking for retail customers will become a staple no different than online banking. Marc DeCastro, research manager for consumer banking, IDC xx Average Mobile Banking Website Homepage Availability, March 2009 January 2010 Overall, the banking industry made dramatic improvements in availability between March 2009 and January At its low-point, home page availability averaged 98.63%, representing about 10 hours of downtime per month. At its high, availability averaged 99.83%, equaling little more than 1 hour lost each month. Leadership: The category leader led performance with an availability average of 99.70%. Category laggards averaged 99.01%. W H I T E PA P E R G R E AT E X P E C TAT I O N S : W H AT U S E R S WA N T F R O M T H E M O B I L E W E B E X P E R I E N C E 7

8 Average Mobile Banking Website Homepage Response Times, March 2009 January 2010 Average response times remained consistent, reaching a slight improvement towards the end of the year. Home page response times maintained a steady average of around 3.5 to 4.5 seconds. Leadership: The category leader had an overall average of 3.02 seconds. Category laggards dipped as low as 5.08 seconds. Benchmarks your site should meet or beat: Availability: The average bank home page availability was 99.38%. Response time: Average response time in the category was 4.00 seconds. W H I T E PA P E R G R E AT E X P E C TAT I O N S : W H AT U S E R S WA N T F R O M T H E M O B I L E W E B E X P E R I E N C E 8

9 Mobile Airlines The mobile Web is a perfect fit for customers on the go. In the airline industry, it means giving flyers the ability to buy tickets, check flights and get boarding passes from the devices they have on hand. Case in point: Nielsen reports that Delta Airlines mobile site and applications receive 995,000 unique visitors per month. And according to Jupiter Research s 2010 Mobile Ticketing report, more than 2 billion boarding passes will be sold in 2010, with that number expected to reach 15 billion people by Ever since launching our platform, m-commerce has been a vital part of our mobile strategy. Many of our business customers use our mobile site to book their Lufthansa tickets. Marcus Casey, director of global online sales and mobile services, Lufthansa xxiii Average Mobile Airlines Website Homepage Availability, March 2009 January 2010 The airlines category took a significant availability hit in the late-spring, early-summer of At its low-point, home page availability averaged 94.62%, which represents more than a day s downtime per month. At its high, availability averaged 99.46%, equivalent to little more than 4 hours lost each month. Leadership: The category leader held an average of 99.72% availability. Laggards went as low as 91.18%. W H I T E PA P E R G R E AT E X P E C TAT I O N S : W H AT U S E R S WA N T F R O M T H E M O B I L E W E B E X P E R I E N C E 9

10 Average Mobile Airlines Website Homepage Response Times, March 2009 January 2010 In response times, airlines demonstrated (on the whole) steady degradation. Leadership: The best airline had an average home page response time of 3.00 seconds. Category laggards went as low as 4.87 seconds. Benchmarks your site should meet or beat: Availability: For airlines, home page availability averaged 98.26%. Response time: Average response time in the category was 3.92 seconds. W H I T E PA P E R G R E AT E X P E C TAT I O N S : W H AT U S E R S WA N T F R O M T H E M O B I L E W E B E X P E R I E N C E 1 0

11 High expectations demand top performance: How does your mobile Website compare? As adoption increases, networks accelerate, and mobile devices become even more sophisticated, mobile Web activity will prove increasingly important to just about any enterprise. But with these greater opportunities comes higher expectations for performance. Speed and availability are inherent requirements of the mobile channel. According to AT&T s Chief Technology Officer John Donovan, it s not uncommon for iphone owners to check a stock 40 times a day. Anytime a Do Not Walk sign flashes in Manhattan, people pop on their iphones for that 30 seconds. he says. xxiv With mobile Websites people expect quick, anytime transactions that work flawlessly. The mobile Website that fails to perform a transaction at speeds that mimic mobile behavior standing in line, stopped on a sidewalk, or sitting in a bus is unlikely to get consumers to use the mobile service again. How well does your site perform on the mobile Web? Use the benchmarks established in four key verticals to see how you compare to the trendsetters and performance leaders in the mobile world. Availability: Search 3 Step Transaction: 99.62% Retail Home Page: 99.88% Banking Home Page: 99.70% Airlines Home Page: 99.72% Your mobile site s average availability: Response Time: Search 3 Step Transaction: 3.39 seconds Retail Home Page: 2.05 seconds Banking Home Page: 3.02 seconds Airlines Home Page: 3.00 seconds Your mobile site s average response time: If you have limited insight into the experiences you deliver to your mobile customers, or need to improve your mobile Website performance to meet or beat mobile leaders, it is time to talk to an expert from Gomez. For deeper insights and assistance, call or visit us online at Gomez.com. W H I T E PA P E R G R E AT E X P E C TAT I O N S : W H AT U S E R S WA N T F R O M T H E M O B I L E W E B E X P E R I E N C E 1 1

12 Benchmark Methodology Gomez created the Mobile Web Benchmark series to raise awareness of the importance of mobile technology as a viable and important channel for Web application delivery. The purpose of the benchmark is to provide an understanding of how industry leaders perform on the mobile Web. The benchmarks in the Mobile Web Benchmark series include Gomez s three fundamental metrics: Availability, Response Time, and Consistency Gomez s mobile Web benchmarks can be accessed on Gomez.com Search Transaction Benchmark Methodology The Gomez Search Transaction Benchmark represents the Availability, Response Time, and Consistency of a typical Search application on a mobile device across the nation s largest online search firms. In order to select the firms for this benchmark, Gomez used comscore Inc. s Expanded Search Query Report published in July Steps Performed for Each Firm Gomez developed a specific task that a hypothetical mobile user wishes to accomplish. The task was predetermined to ensure that there are no extraneous steps in the sequence and that an equal comparison across all firms is enacted. The user s intention for our testing scenario is: Home Page. Navigate to the home page of the mobile Web site. Type in a key word and select Search. Search Results Page. This page will contain specific search results for the search term used in the script. Retail Home Page Benchmark Methodology The Gomez Mobile Retail Home Page Benchmark represents the Availability, Response Time, and Consistency of a typical online retail home page on a mobile device across the nation s largest online retailers. In order to select the firms for this benchmark, Gomez used the top 50 retailers listed in the 2009 Internet Retailer Top 500 Guide with a dedicated mobile presence. Mobile Banking Home Page Benchmark Methodology The Gomez Mobile Banking Home Page Benchmark represents the Availability, Response Time, and Consistency of a typical OLB home page on a mobile device across the nation s largest online banking firms. In order to select the firms for this benchmark, Gomez used the combined top 10 banks that appear on the FDIC s top Holding 50 Bank Holding Companies by Deposits and Top 50 Commercial Banks and Savings Institutions by Deposits. Airline Mobile Home Page Benchmark Methodology The Gomez Airline Mobile Home Page Benchmark represents the Availability, Response Time, and Consistency of a typical airline home page on a mobile device across the nation s largest online airline firms. Fortune 1000 companies rely on Gomez metrics to help maximize their Internet business effectiveness and to get the most from their technology investments. In order to select the firms for this benchmark, Gomez used the list of Top 10 US Passenger Air Carriers by Market Share as determined by the Bureau of Transportation Statistics. Availability Availability measures the percentage of successful transactions out of the total number of transaction attempts. Availability rates are dependent on a number of variables including availability of the mobile web site, ability to process a transaction without error, and ability to complete each step of the transaction within a 60-second time frame. Availability rates do not include server-side errors or technical errors incurred due to scripting problems. These errors do occur from time to time and are reflected with an N/A (not available or not applicable) message on the benchmark. Data points from all tests run on multiple testing locations and are aggregated to calculate the average availability for the test period. Response Time Response Time metrics reflect the time required to download each page in an entire multi-step transaction (including all embedded objects, JavaScript, and cascading style sheets). Data points from each successful test run are collected from the multiple testing locations and are aggregated to formulate the average response time for the test period. The resulting metric represents the average time required to execute the multi-step transaction without factoring in user latency. Frequency of Tests and Testing Locations Gomez monitors home pages and transactions from four testing agents in each of three testing locations, one in Lexington, MA, one in Chicago, IL and one in Seattle, WA. To access the mobile sites, Gomez s testing infrastructure uses nodes that are connected to the mobile internet. The tests simulate the Motorola RAZR V3 on AT&T, T-Mobile, Sprint, and Verizon. Scripting Technology All transactions collecting the three Gomez metrics use scripts generated with the Gomez Script Recorder and are deployed across the specified testing locations. W H I T E PA P E R G R E AT E X P E C TAT I O N S : W H AT U S E R S WA N T F R O M T H E M O B I L E W E B E X P E R I E N C E 1 2

13 About Compuware Gomez Gomez, the Web performance division of Compuware, provides the industry s leading solutions for optimizing the performance, availability, and quality of Web and mobile applications. The on-demand Gomez platform integrates solutions for Web load testing, Web performance management, Web cross-browser testing, and Web performance business analysis that test and measure Web and mobile applications from the outside-in across all users, browsers, devices, and geographies using a global network of over 100,000 locations. When combined with Compuware Vantage, Gomez offers the industry s only solution for optimizing application performance across the Enterprise and the Internet. Over 3,000 customers worldwide, ranging from small companies to large enterprises including 12 of the top 20 most visited US Web sites use Gomez solutions to increase revenue, build brand loyalty, and decrease costs. i ii Mary Meeker, Economy and Internet Trends Morgan Stanley, October 2009 iii iv v Mary Meeker, Economy and Internet Trends Morgan Stanley, October 2009 vi North American Technographics benchmark survey, Forrester 2009 vii North American Technographics benchmark survey, Forrester 2009 viii ix x xi xii xiii xiv xv xvi xvii xviii xix xx xxi xxii xxiii xxiv Gomez 10 Maguire Rd, Bldg3, Suite 330, Lexington, MA General Inquiries: Gomez, Inc. All rights reserved. Gomez is a registered service mark. All other trademarks and service marks are the property of their respective owners. Sales: USA UK +44 (0) Germany +49 (0)