Optimizely Personalization: Atlanta User Group Workshop Q2 2016 Hudson Arnold Optimizely Strategy Consultant hudson@optimizely.com twitter: @hudson_arnold
Today, we ll discuss... 1. What does Personalization mean? a. Testing & Personalization b. Rules Based vs. Automated c. Sample Use Cases 2. How would my team start personalizing? a. Personalization Process + Team b. Pitfalls to Avoid c. Crawl, Walk, Run Timeline 3. The fun stuff: Campaign Design a. Audience Strategy b. Brainstorm!
Testing & Personalization
Testing + Personalization Testing Testing is the process of refinement via exploration to the underlying core site structure, which 100% of visitors to each page will see. This lifts the baseline performance of the site over time. For every experiment on a given page, the winning variation becomes the new baseline. Aggregate site performance will improve as more and more winners gradually combine to form a superior default experience.
Testing + Personalization Personalization Personalization is the practice of further refining the site experience for important, distinct segments as well as to individuals. Non-personalized site shows the same experience to everyone With personalization, the site shows distinctly different experiences to different groups
Testing + Personalization How do Testing + Personalization work together?
Testing + Personalization How do Testing + Personalization work together?
Testing + Personalization How do Testing + Personalization work together? The main principle behind integrating testing and personalization is: test multiple variations and launch the winners as personalization Campaigns. Every experience in your personalization plan is a hypothesis: a testable statement proposing a cause-effect relationship between your site experience and visitors behaviors. Testing these hypotheses can help you optimize the experience. Programs also benefit by testing in parallel with personalization. Some changes to your site experience - such as fonts, color palette, or a single versus multi-page checkout process - are served to all visitors. For most companies, redesigns of the underlying site structure aren t likely to have specific applications in personalization. Optimize these site-wide changes in Testing, then build targeted experiences on top of them in Personalization.
Ongoing Optimization Testing & Personalization Example Workflow Let s assume that we run an A/B test to everyone on the homepage...
Ongoing Optimization Testing & Personalization Example Workflow New 1-Size Fits All Baseline The winner is going to become the new baseline for non-personalized visits (everyone).
Ongoing Optimization Testing & Personalization Example Workflow New 1-Size Fits All Baseline But, when we apply segmentation to the results, we see that the student segment had an interesting result.
Ongoing Optimization Testing & Personalization Example Workflow Variation 1 Variation 2 Variation 3 We then run a personalized experiment to that segment...
Ongoing Optimization Testing & Personalization Example Workflow Variation 1 Variation 2 Variation 3 The winner of which becomes a new, ongoing personalization campaign.
Rules Based vs. Automated
Rules-based vs. Automated Personalization Rules-based Good For: Capitalizing on marketing strategy (segments, personas, promotions, discounts, value propositions etc.) Reaching important, big segments Contextual Segments Audiences created manually using business logic and prior research. Each experience tested through. Behavioral Targeting Integrated 1st & 3rd Party Audiences Good For: Capitalizing on data modeling Set it and forget it implementation Less control / maintenance Reaching every visitor (or all known ) Automation Recommended Content The audience is derived and updated algorithmically to dynamically adjust templated content (ie recommendations). Difficult to iterate. 1-1 Content Attributes Frequency & Recency
Rules-based vs. Automation Continuum of Organizational Maturity Generally, more mature digital orgs layer on automation onto personalization efforts after rules-based segmentation realizes value. Regardless, investment in personalization should probably happen only after testing has itself matured. Source: Forrester Optimize Customer Experiences with Online Testing and Continuous Optimization, January 26 2016
Sample Use Cases
Roundtable 1. Do you already use targeting and segmentation in your testing practices? How so? What s preventing you from more full-bore personalization? 2. Do you personalize in other aspects of your marketing? Does that suggest site personalization? Why or why not? 3. Is rules-based or automated personalization more interesting to you? Why?
How would my team start personalizing? Process + Team
Personalization Playbook End-End Process + Milestones Discovery Implementation - Audience: Analytics - Data Connections Segmentation - Audience: Qualitative Analysis - Custom Tracking - Platform; Pages, Tags, Events, - Systems: Scope & Platform Audiences Requirements Planning - Campaign Validation (Targeted Testing) - Campaign planning & reach projections Production Reporting & Iteration - Granular Audience Reporting - Program reach and lift opportunities - Campaign testing & iteration - Creative Production (Images, Copy) - Experience management - QA
What Will Optimizely Do? High Level Strategy + Technical Implementation (Client &/Or Agency Owns Ongoing Work)
Core Personalization Team Skillsets & Team Role Depending on team structure, can also include: Analyst: Reporting, audience research via segmentation and data modeling, tracking reqs. Strategist: Campaign concepting, experiment + campaign roadmaps, business requirements, KPIs U/X: Wireframing and experiment design, usability research and persona development
Personalization Resourcing Structure: Generalized Sample RASCI Discovery - Audience: Analytics Segmentation - Audience: Qualitative Analysis - Systems: Scope & Platform Requirements Implementation - DCP Connections - Custom Tracking - Platform; Pages, Tags, Events, Audiences Planning - Campaign Validation (Targeted Testing) - Multi-threaded campaign planning & reach projections Production - Creative Production (Images, Copy) - Experience management - APIs with client, CMS, etc Reporting & Iteration - Granular Audience Reporting - Program reach and lift opportunities - Campaign testing & iteration Responsible Accountable Supporting Consulted Informed / Approve
How would my team start personalizing? Pitfalls to Avoid
Personalization without Testing Pitfalls to Avoid Experience shows that it is tempting - and risky - to go to market with personalization campaigns without a built-in plan for testing, validation, and iteration.
Unconcerted Focus on Initial Campaigns Pitfalls to Avoid Focus on Top ROI Pages to Start, Progress Incrementally Homepage Launch 1st iteration 2nd iteration 3rd iteration Landing Page Product Page Checkout
Unconcerted Focus on Initial Campaigns Pitfalls to Avoid...Not Everything at Once Focus on Top ROI Pages to Start, Progress Incrementally Homepage Landing Page Product Page Checkout Homepage Landing Page Product Page Checkout Launch Launch 2nd iteration Successful practitioners start with a tight focus on addressing top audiences and pages. (Concerted preparation & iteration in phase 1 drive successful implementation and long term momentum.) 3rd iteration Pages should be high traffic + high value, Audiences relatively high reach. 1st iteration
How would my team start personalizing? Crawl, Walk, Run
Phased Integration of Personalization Crawl, Walk, Run Phase 1 Phase 2 Phase 3 0-12 Weeks Months 3-12 Months 12-24 Platform Implementation Simple Audiences Starter Campaigns, Limited Integration of Testing + Personalization Phase 2 Planning Integration with 1st & 3rd Party Data More Campaigns drive higher Reach Integration of testing & Personalization workflows More advanced use cases Phase 3 Planning Full system integration Iteration drives ongoing improvement Data modeling & predictive analytics drives new audience strategy Use cases continually iterated Web personalization data feeds email and ad deployment Reach: 0-15% Reach: 30-60% Reach: 75-100% Pages: 1-3; only most critical ROI points Pages: Multiple campaign/audiences on top ROI pages, a validated campaign running on each next tier page Pages: Almost every page / interaction, multiple elements w/in each page # Campaigns: 2-5 # Campaigns: 10-20 ongoing campaigns # Campaigns: 25+ ongoing personalization campaigns, continually iterated on Audiences: Natively available, simple, large, simple conditions; Metro, Single Behaviors Audiences: Target intersecting audiences (ie. Past Purchaser & Geo, not just Past Purhaser), 3rd & 1st party data used, more and complex behaviors Audiences: Granular audiences discovered with analytics and modeling, old audiences iterated and improved Tactics Modules (lightboxes), image swaps, little testing Tactics Series of tests driving campaign execution and iteration. Interactive elements, more dramatic changes. 1-1 recommendations / content attributes Tactics Fully expressive strategy that integrates testing and personalization
Roundtable 1. Do you think it would be possible to be successful at personalization without following the process or without all of the skill sets that were mentioned? Why or why not? 2. Are there any other pitfalls to personalizing that weren t mentioned that you could foresee? How would they affect the proposed implementation timeline?
Audience Strategy Rationale, Reach, Technical Feasibility Audience Strategy
Audience Development Essentials Plan for high reach as well as for lift Set goals for % of Reach as Well as Lift Plan to address some chunkier audiences... Some that highly granular and actionable Be thoughtful about rules-based vs. automated Use both left-brain and right-brain strategies for quality Creative: what are the personas you envision targeting? Quantitative: what are the most actionable behavioral segments? Procedural: what is the reach, the technical feasibility, and creative requirements for each audience?
Brainstorm!
BRAINSTORM - 15 min. 1. Split into groups of 3 (1 expert, 1 intermediate, one beginner). We ll spend 5 minutes on each individual s site. 2. Define 3-5 potentially valuable audiences. 3. What are 2 important pages for influencing conversion? What are some variables on those pages you could personalize? 4. Document a concrete idea, how for one page, 1 important elements could be personalized for 3 of the audiences defined.
Q&A