Copyright 2014 Splunk Inc. Crowdsourcing the Matrix: Improving the Service Desk Experience and ITIL/ SDLC Processes Ian Thomas Problem Management Analyst, Paychex Enterprise Support
Disclaimer During the course of this presentalon, we may make forward- looking statements regarding future events or the expected performance of the company. We caulon you that such statements reflect our current expectalons and eslmates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward- looking statements, please review our filings with the SEC. The forward- looking statements made in the this presentalon are being made as of the Lme and date of its live presentalon. If reviewed aser its live presentalon, this presentalon may not contain current or accurate informalon. We do not assume any obligalon to update any forward- looking statements we may make. In addilon, any informalon about our roadmap outlines our general product direclon and is subject to change at any Lme without nolce. It is for informalonal purposes only, and shall not be incorporated into any contract or other commitment. Splunk undertakes no obligalon either to develop the features or funclonality described or to include any such feature or funclonality in a future release. 2
Agenda! About Me! About Paychex! Typical SDLC/ITIL Workflows Before and ASer Splunk! SupporLng the Real World! Crowdsourcing the Matrix! Improving Everything! What s next?! Q & A 3
About Me! Senior Problem Management Analyst at Paychex! ITIL FoundaLons CerLfied! Lean Six Sigma Green Belt Pending! 8 years with Paychex! Over 13 years of industry experience! Fan of sharing informalon efficiently to reduce spin! Splunk Fan 4
About Paychex 5
Typical SDLC/ ITIL Workflows Before and ASer Splunk
Before Splunk SDLC/ITIL InformaLon Flow Pre- Prod Test Test Engineering Bug Developer Logs = Defect Defect Tracking/ Code Repository Prod Support Bug Ticket IT Support Developer Logs = ArLcle KCS Knowledgebase RFC ITSM Monitoring IT OC Runbook Script Logs = Alert AlerLng System 7
Problems/Risks! Each group could be starlng from scratch! Same issue gets recorded separately in each repository! Knowledgebase searches are not 100% accurate! PotenLal for mullple Defects, RFCs, and Knowledgebase arlcles generated for same issue within each silo 8
ASer Splunk SDLC/ITIL InformaLon Flow Pre- Prod Test Test Engineering Bug Developer = Defect Defect Tracking/ New Splunk Code Repository Charts/Graphs Prod Support Bug Ticket IT Support Developer = ArLcle RFC KCS Knowledgebase ITSM New Splunk Charts/Graphs Monitoring IT OC Runbook Script = Alert AlerLng System New Splunk Based Alerts 9
ASer Splunk SDLC/ITIL InformaLon Flow Pre- Prod Test Test Engineering Bug Developer Logs = Defect Defect Tracking/ Code Repository Prod Support Bug Ticket IT Support Developer Logs = ArLcle KCS Knowledgebase RFC ITSM Monitoring IT OC Runbook Script Logs = Alert AlerLng System 10
SupporLng the Real World
Lost Guest DirecLons to MGM Grand Service Desk Lost Guest How can I help you? Hi, I am lost. Can you help me get back to the MGM Grand? Where are you? I don t know, I see a Starbucks Can you see any landmarks? I see a McDonalds Wait, I see another Starbucks Are you driving or walking? Driving What road are you on? Fremont Street What, you can t drive there???!!! I can and I am! What are you driving? A Rascal Scooter 21 Starbucks 10 McDonalds 12
How is This Like Online ApplicaLon Support? QuesLons commonly asked when supporlng online clients What plajorm/browser are you using? What applicalon are you in? What are you trying to do? Where are you, what are you seeing, and what is the issue? 13
Client Support Process Clients What app What Issue? What do I do? Help! KCS Knowledgebase No SoluLon Found Help! What app What Issue? Execu@ves Client Facing Service Desk Client Facing Business Support What do I do? 14 IT Support Incident/ Bug Ticket = ArLcle Developers RFC Produc@on Support Process
When Experts View Logs 15
GPS Tracker Logs How Could They Help? Lost Guest Service Desk Hi, I got lost. Can you help me get back to the MGM Grand? Sure, One second while I check your locahon in our new tool (What does this say? This doesn t help me!!!) Ok Where are you now?... Lost Guest IT Expert Hi, I got lost. Can you help me get back to the MGM Grand? Sure, I see where you are... (and where you have been) Follow Koval down to Harmon take a right etc. 16
Crowdsourcing the Matrix
360 View Client Who we Serve I want answers and help now. I don t want to be asked lots of questions. William Service Desk Rep I want to help the client now. I want to ask as few questions as possible to provide the client with the right answer the first time. Tom IT Support Desk I want to know what to do when an issue is escalated to me. I want the Service Desk to resolve issues without escalating to me. Dave Developer If we have information about a logged event I want to know it. I want to stop known issues from being escalated to me, from IT Support, Test Engineering, and the Operations Center. Miriam Test Engineer If I encounter a bug, I want to know if we have seen it before. Paul OperaHons Center Analyst When I get an alert, I want to know to respond: Is issue directly impacting our clients and how? 18
Which Logged Events Do We Care About? IT Client All Events All Events INFO ERROR More Answers Fewer Ques@ons WARN Fewer Annoyances 19
Known Event Records EventKNow Approach! What is a Known Event Record (KER)? Event class of interest idenlfied Logical buckelng schema created ê Find fields you can combine into event signature ê Mask event specific uniqueness Buckets are automahcally assigned a Known Event Record Number! When someone encounters an event with a Known Event Record perlnent to their issue, they update it with what they know in a simple interface! Everyone using Splunk/EventKNow sees the informalon in the Known Event Record and adds to it as more is learned 20
What does adding Known Event Records do? KERs Divide Event Space into Buckets All Events 21
EventKNow Known Event Record Viewer/Editor 22
EventKNow Access Points Make it Easy Open Searches - IT/Dev and Splunk Ninjas Trending Dashboards: IT OC, Dev, Problem, KERs graphed by - KER- App - RFC(APR at Paychex) and Status Username Search Client Facing Service Desks/Test/Support User Plajorm/ Browser Info 23
Improving Everything
Before EventKNow - SDLC/ITIL InformaLon Flow Pre- Prod Test Test Engineering Bug Developer = Defect Defect Tracking/ New Splunk Code Repository Charts/Graphs Prod Support Bug Ticket IT Support Developer = ArLcle RFC KCS Knowledgebase ITSM New Splunk Charts/Graphs Monitoring IT OC Runbook Script = Alert AlerLng System New Splunk Based Alerts 25
Make Everyone Behave Like an IT Expert 26
ASer EventKNow SDLC/ITIL InformaLon Flow Pre- Prod Test Prod Support Monitoring Test Engineering Bug Ticket IT OC Bug IT Support Developer Runbook Developer Script EventKNow = = = Defect ArLcle RFC Alert Defect Tracking/ EventKNow Code Repository Charts/Graphs KCS Knowledgebase ITSM AlerLng System EventKNow Charts/Graphs EventKNow Based Alerts 27
Client Support Process ASer EventKNow Clients Help! I know what you need I know what to do EventKNow Username/Event Search KCS Knowledge base I know what you need Client Facing Service Desk Help! Execu@ves Client Facing Business Support I know what to do 28 IT Support NEW Incident/ Bug Ticket = ArLcle Developers RFC Produc@on Support Process
Demo Of EventKNow Interfaces! Support Interface! Known Event Record Viewer/Editor! Dashboard! Open Searches 29
EventKNow as Connector Future OpportuniLes ProacLve CommunicaLon KCS Knowledge Base Self Service In App - Real Time Event Specific Feedback EventKNow People IntegraLon with Internal Support Tools In App Support Requests Improve Business Processes Execu@ves 30
What s Next? 30 17 32 37 41 40 26 53 31 26 32 26 31
Special Offer: Try Splunk MINT Express for Free! Splunk MINT offers a fast path to mobile intelligence. How fast? Find out with a 6- month trial* Register for your free trial: hsp://mint.splunk.com/conf2014offer Download the Splunk MINT SDKs Add the Splunk MINT line of SDK code and publish** Start getng digital intelligence at your fingerlps! *Offer valid for.conf2014 atendees and coworkers of atendees only. **Trial allows monitoring of up to 750,000 monthly achve users (MAUs). 32
THANK YOU Ian Thomas Problem Management Analyst Paychex, Inc. ithomas@paychex.com