Accelera'ng Your Solu'on Development with Splunk Reference Apps

Similar documents
End- to- End Monitoring Unified Performance Dashboard (UPD)

Hunk & Elas=c MapReduce: Big Data Analy=cs on AWS

In Depth with Deployment Server Sanford Owings

Telemetry: The Customer Experience

Expanding Uniformance. Driving Digital Intelligence through Unified Data, Analytics, and Visualization

Workflow ProducCvity in Splunk Enterprise

Real World Big Data Architecture - Splunk, Hadoop, RDBMS

Splunk Dashboard Framework What s New Nicholas Filippi Product Management, Splunk

Copyright 2013 Splunk Inc. Introducing Splunk 6

MIGRATING SHAREPOINT TO THE CLOUD

Amplify Service Integration Developer Productivity with Oracle SOA Suite 12c

Architec;ng Splunk for High Availability and Disaster Recovery

How To Use Splunk For Android (Windows) With A Mobile App On A Microsoft Tablet (Windows 8) For Free (Windows 7) For A Limited Time (Windows 10) For $99.99) For Two Years (Windows 9

The Purview Solution Integration With Splunk

Corporate Bill Analyzer

MicroStrategy Course Catalog

& USER T ECH.C W WW. SERVICE

WHITE PAPER SPLUNK SOFTWARE AS A SIEM

XpoLog Competitive Comparison Sheet

Architec;ng Splunk for High Availability and Disaster Recovery

Splunk for Networking and SDN

An enterprise- grade cloud management platform that enables on- demand, self- service IT operating models for Global 2000 enterprises

Customizing and Integrating

Buying, Using and Integrating NetSuite

Deploying the Splunk App for Microso> Exchange

Using WebLOAD to Monitor Your Production Environment

Stream Deployments in the Real World: Enhance Opera?onal Intelligence Across Applica?on Delivery, IT Ops, Security, and More

Aligning Your Strategic Initiatives with a Realistic Big Data Analytics Roadmap

ebay : How is it a hit

What s New in Security Analytics Be the Hunter.. Not the Hunted

Software Development Interactief Centrum voor gerichte Training en Studie Edisonweg 14c, 1821 BN Alkmaar T:

WhiteWave's Integrated Managed File Transfer (MFT)

Day 1 - Technology Introduction & Digital Asset Management

The Jiffy Lube Quick Tune- up for your Splunk Environment

Automatizace Private Cloud. Petr Košec, Microsoft MVP, MCT, MCSE

PRIVACY AWARE ACCESS CONTROL FOR CLOUD-BASED DATA PLATFORMS

Splunk Enterprise in the Cloud Vision and Roadmap

Grid CompuAng AnalyAcs with Splunk Finnbar Cunningham

Mobile application testing for the enterprise

Building a Splunk-based Lumber Mill. Turning a bunch of logs into useful products

Fortinet FortiGate App for Splunk

API Architecture. for the Data Interoperability at OSU initiative

EMC Data Protection Advisor 6.0

Bridge Development and Operations for faster delivery of applications

How To Avoid Snowflakes

MRV EMPOWERS THE OPTICAL EDGE.

Getting Started with Google Cloud Platform

Testing Tools using Visual Studio. Randy Pagels Sr. Developer Technology Specialist Microsoft Corporation

Building your Server for High Availability and Disaster Recovery. Witt Mathot Danny Krouk

Legal. Copyright 2016 Magento, Inc.; All Rights Reserved.

RapidStart Implementation

The Trusted Technology Partner in Business Innovation PASSION DISCIPLINE INNOVATION TEAMING INTEGRITY

Splunk Company Overview

Appendix A: Case Studies

Cross-Platform Phone Apps & Sites with jquery Mobile

BENCHMARKING V ISUALIZATION TOOL

Analytics Software for Energy Management and Building Systems Optimization and Equipment Fault Detection

JOURNAL OF OBJECT TECHNOLOGY

Splunk Enterprise Log Management Role Supporting the ISO Framework EXECUTIVE BRIEF

Delivering Cost Effective IT Services

Vulnerability Management with the Splunk App for Enterprise Security

Final Report - HydrometDB Belize s Climatic Database Management System. Executive Summary

Splunk for.net Developers

BPMS BUYER S TOOL KIT. Sample Request for Proposal for a Business Process Management Suite. Part 1 of the complete BPMS Buyer s Tool Kit

PEGA MOBILITY A PEGA PLATFORM WHITEPAPER

MulGsite Clustering and Search Affinity

Simplifying Big Data Analytics: Unifying Batch and Stream Processing. John Fanelli,! VP Product! In-Memory Compute Summit! June 30, 2015!!

A Vision for Operational Analytics as the Enabler for Business Focused Hybrid Cloud Operations

More Comprehensive Digital Intelligence - CorrelaFng Client and Server- side Data

Keeping Splunk in Check: Tools to BeGer Manage Your Investment

50 shades of Siebel mobile

Data Integration Checklist

Moving your development to the Cloud using Visual Studio Online

Java Monitoring. Stuff You Can Get For Free (And Stuff You Can t) Paul Jasek Sales Engineer

HADOOP BIG DATA DEVELOPER TRAINING AGENDA

Introducing Storm 1 Core Storm concepts Topology design

DBX. SQL database extension for Splunk. Siegfried Puchbauer

Deployment Topologies - DPAdmin An isoagroup Product

NextRow - AEM Training Program Course Catalog

Developing Secure Mobile Applications from SharePoint Presented by Seyfarth Shaw LLP and Something Digital

Implement a unified approach to service quality management.

Collaborative DevOps Learn the magic of Continuous Delivery. Saurabh Agarwal Product Engineering, DevOps Solutions

#mstrworld. No Data Left behind: 20+ new data sources with new data preparation in MicroStrategy 10

Unified Batch & Stream Processing Platform

Catálogo de cursos plataforma elearning Microsoft Imagine Academy: Microsoft SQL Server y Visual Studio

Copyright 2014 Splunk Inc. Splunking the JVM. Damien Dallimore. Dev Evangelist, CSO Splunk

Transcription:

Copyright 2015 Splunk Inc. Accelera'ng Your Solu'on Development with Splunk Reference Apps Grigori Melnik Principal Product Manager Developer PlaAorm, Splunk @gmelnik

Disclaimer During the course of this presentagon, we may make forward looking statements regarding future events or the expected performance of the company. We caugon you that such statements reflect our current expectagons and esgmates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward- looking statements, please review our filings with the SEC. The forward- looking statements made in the this presentagon are being made as of the Gme and date of its live presentagon. If reviewed aser its live presentagon, this presentagon may not contain current or accurate informagon. We do not assume any obligagon to update any forward looking statements we may make. In addigon, any informagon about our roadmap outlines our general product direcgon and is subject to change at any Gme without nogce. It is for informagonal purposes only and shall not, be incorporated into any contract or other commitment. Splunk undertakes no obligagon either to develop the features or funcgonality described or to include any such feature or funcgonality in a future release. 2

How should I build my app?

Splunk Developer Guidance EVERYTHING YOU NEED TO BUILD

Building SoluGons on the Splunk PlaAorm Splunk Reference Apps Complete, working real- world Splunk solugons built together with partners (Conducive, Auth0) Splunk Developer Guide This is unbelievable, it covers most everything I learned the hard way Bernie Macias, Technical Architect, Zillow dev.splunk.com/goto/devguide

Planning a journey PlaDorm and tools: a kitbag for our journey UI and visualiza'ons: what the apps look like Working with data: where it comes from & how we manage it Adding code: using JavaScript and Search Processing Language Packaging and deployment: reaching our first desgnagon Dealing with OAuth Aler'ng Building in telemetry with high- performance data collec'on splk.it/devguide

Splunk Reference App Demo

How we build guidance

1. Started with a QuesGons Backlog! Architecture Why should I not use transacgons? What does a typical Splunk applicagon reference architecture look like? When should I use pivot vs tstats? What common paradigms are applicable to Splunk app development? Why should I use data models? What are the typical deployment topologies? Why should I choose a specific one? What are the confounding factors on the choice of my topology? When my data source touches on many data models, should I assume complete separagon or heavy inheritance? How do I extend an exisgng data model? How do I parggon my Splunk solugons? What does CIM offer and why should I build CIM- compliant apps? What are the tradeoffs of various types of inputs? In the context of CIM, what are the tradeoffs of using my props.conf and transforms.conf and rewrigng them on How do I architect my Splunk solugon and deployment for a very large scale? indexing, completely discarding the vendor supplied field names? How do I reconcile the advantages of a clean How do I architect my Splunk solugon for the cloud? What are specific consideragons for deploying to AWS or Azure? interface & normalisagon, but at the cost of losing alignment with published vendor documentagon, and a learning What s the landscape of Splunk extension points? curve for exisgng users? How do I integrate data from Splunk into exisgng applicagons and systems? How do I manage my solugon declaragve configuragon? How do I detect/troubleshoot bad config? How do I plan and design a robust alergng and monitoring subsystem on top of Splunk? How do I log and analyze data that is not event driven (certain web feeds, html parsing, image meta data)? Compare and contrast ad- hoc searching vs background searching What should I consider for my sizing requirements? What are recommended configuragons of Splunk deployment to meet my sizing requirements? How do I handle transient faults? Should I architect my solugon to index my data in local data center (zone) or centrally? How do I effecgvely manage credengals? What are things we can automagcally degrade so we can make sure our core experience is working? What s the effect of search head locagon on my app and the overall user experience? How When something happens, how effecgvely propagate the info and react to it? How do I develop an integrated mechanism to let me connect Splunk to my MOM (messaging middleware) and index my messages? How are other What solugons on Splunk does do I prepare built? What were a the challenges? typical How have they Splunk been addressed? applica'on architecture be produc've look like? do I integrate my event Splunk genera'on into exis'ng when systems? developing & How do I package an app? deal with versioning and updates? How do I handle the requirement that app configs must be different across different server types in a distributed! Packaging and Deployment How do I piece together various parts of a Splunk app (custom search commands, mod inputs etc.)? How do I package a Splunk solugon with a single install that automagcally rolls out all the necessary dependencies? How do I manage my Splunk solugon versioning, backward and future compat? What's the best way to split up custom apps for deployment?! Development How should I set up my development environment to be producgve with Splunk? What are different ways of how I develop my Splunk app? Pros and cons of using specific SDK vs REST APIs? Pros and cons of using SimpleXML vs Advanced XML vs Web Framework How do I analyze a data source for a TA? What are the different ways of enriching the data in Splunk? What are their tradeoffs? When should I use event types and transacgons for data classificagon? How do I extend Splunk to define a custom input capability? When should I use modular inputs vs scripted inputs vs..? What are streaming vs non- streaming outputs consideragons? How do I deal with long- running scripts? Handling shutdown/restart of Splunk? Concurrency? State persistence etc. tes'ng Splunk? an app? environment (e.g. apps on search heads shouldn't have inputs enabled)?! Quality/Compliance What quality gates should I consider? What kind of para- funcgonal characterisgcs are important to consider? What heurisgcs do I use to bless/block a release? How do I test a data model? How do I prepare event generagon when building/tesgng an app? What kind of perf tesgng should I do and how? How do I test UI? How do I security cergfy my solugon? How do I design to sagsfy my retengon and compliance policies? How do I architect to design my availability requirements? How do I handle geographic disaster recovery / fault tolerance? How do I properly instrument my solugon so that I know what s happening?! Sustained Engineering How do I maintain/service/support Splunk apps? How do my customers handle updagng their customized configs once new versions of my app come out?! Business Why should I build on Splunk? 9 What kind of skill do I need my devs to have to build a Splunk solugon? What is the community building? How are current devs creagng unique experiences using Splunk I typically want to

2. IdenGfied Extensibility Surface Area Data inges'on & indexing Input ê Scripted inputs ê Modular inputs ê Custom (trained) source types ê Custom sources Data inges'on pipeline ê Field extracgons ê Field transformagons Indexing ê Custom indexes Searching Search authoring ê Custom search commands ê Macros (basic, parametrized) ê Saved searches Data classifica'on ê Event types ê TransacGons Data enrichment ê Lookups ê KV store collecgons ê Workflow acgons Data normaliza'on ê Tags ê Aliases Data mining ê cluster & dedup ê anomalousvalue ê kmeans ê predict commands Processing & repor'ng Search- 'me mapping ê Data models CIM extensions Custom UI/visualiza'ons ê Pages, views & dashboards ê JS Extensions ê CSS Extensions ê Custom setup screens Scheduled processing ê Scheduled reports Aler'ng ê Scripted alerts ê Custom alert acgons Branding & naviga'on ê Custom app navigagon & branding Manageability ê Custom splunkweb controllers ê Custom splunkd endpoints 10

3. Mined business requirements with partner 4. Formulated learning objecgves 5. Reconciled 3 & 4 with our designs 11

Data Search language AggregaGng siloed metrics into meaningful KPIs Data manipulagon Data normalizagon Sub- searches Config- driven Persistence with KV store Macros Viz: Dynamic scaling Customizing in- the box viz controls Custom nav Ux acgviges permeagng all dev Using sub- searches to correlate data TroubleshooGng searches Data mining: ExploraGon PreparaGon: filtering/deduping/ buckegng Using advanced stagsgcs funcgons Threshold- based anomaly detecgon EvaluaGng goodness /accuracy General search paqerns Search opgmizagons Ux Prototyping AdapGng 3 rd party viz library Composite charts with interacgons Dealing with high- volume data sets TroubleshooGng perf issues Post- process or not- post- process deployment implicagons Automated UI tesgng (w.selenium) Post- processing IntegraGng with 3 rd party component Unit tesgng (w.mocha) PersisGng state (per user) Plus non- funcgonal topics: App versioning Packaging InstallaGon Security review Deployment Publishing to splunkbase App cergficagon Data modeling Using lookups Building a baseline lookup table Windows of Gme/Custom Gme ranges Overlaying Gme data Serng the stage Overall Splunk app structure UI technology selecgon: Simple XML vs SplunkJS Modularity Dev & test env Dev workflow Modularity Data onboarding CIM compliance Tools

Takeaways! PlaDorm, not just an engine! On- prem and cloud! App development!= rocket science! Splunk Developer Guidance : learn and reuse for the win!! Reach out to my team (devinfo@splunk.com) and tell us about your experience

Resources dev.splunk.com/goto/devguide github.com/splunk/splunk- ref- pas- code github.com/splunk/splunk- ref- pas- test blogs.splunk.com/dev @devinfo @gmelnik 14

Related breakout sessions & acgviges A Lap Around Developer Awesomeness in the Next Version of Splunk (Itay Neeman/Glenn Block) Visualizing Data From the Ground Up: Raw Data to Interac've Graphics With Splunk (Marshall Agnew) Liberate Your Applica'on Logging! (Glenn Block/Jian Lee) Modular Inputs - If You Build It, They Will Come (ScoA Haskell) The 'State' of Splunk - Using the KVStore to Maintain App State (Stefan Sievert) Advanced Interac'ons Using SimpleXML (Mathew ElFng, Siegfried Puchbauer) Splunk Apps You Can't Miss, and the People Who Make Them (Hal RoAenberg) 15

THANK YOU