Elasticsearch for Lua Developers. Pablo Musa
|
|
- Clement Stokes
- 8 years ago
- Views:
Transcription
1 Elasticsearch for Lua Developers Pablo Musa
2 + +
3 Me Pablo Musa Educational Elastic
4 Which student? 5 interested students 3 very good proposals Key Points: - Background (Lua, Elasticsearch, Open Source) - Availability (how many hours per week) - Future (development likelihood after GSoC) 4
5 Active participant in CTFs. Also, a 3rd year computer science student at IIT Roorkee developer and challenge setter of Backdoor CTF. ( Mostly self-taught himself by reading books and online articles. Dhaval Kapil Passionate about networking, security and development
6 Agenda Elasticsearch Elasticsearch & Lua elasticsearch-lua WERpapers Elastic 6
7 Elasticsearch is...
8 Elasticsearch is... an open source, distributed, scalable, highly available, document-oriented, RESTful full text search engine with soft real-time search and analytics capabilities 8
9 Elasticsearch is... an open source, distributed, scalable, highly available, document-oriented, RESTful, full text search engine with real-time search and analytics capabilities Apache 2.0 License 9
10 Elasticsearch is... an open source, distributed, scalable, highly available, document-oriented, RESTful, full text search engine with real-time search and analytics capabilities 10
11 Elasticsearch is... an open source, distributed, scalable, highly available, document-oriented, RESTful, full text search engine with real-time search and analytics capabilities 11
12 Elasticsearch is... an open source, distributed, scalable, highly available, document-oriented, RESTful, full text search engine with real-time search and analytics capabilities 12
13 Elasticsearch is... an open source, distributed, scalable, highly available, document-oriented, RESTful, full text search engine with real-time search and analytics capabilities { } "name" : "Webinar" "geo" : { "city" : "Amsterdam", "lat" : 4.85, "lon" : } Source: 13
14 Elasticsearch is... an open source, distributed, scalable, highly available, document-oriented, RESTful, full text search engine with real-time search and analytics capabilities Source: 14
15 Elasticsearch is... an open source, distributed, scalable, highly available, document-oriented, RESTful, full text search engine with real-time search and analytics capabilities 15
16 Elasticsearch is... an open source, distributed, scalable, highly available, document-oriented, RESTful, full text search engine with real-time search and analytics capabilities 16
17 Getting up and running... is easy $ wget $ tar -zxf elasticsearch tar.gz $ cd elasticsearch $./bin/elasticsearch 17
18 Elasticsearch & Lua
19 About the Project 3-month project A "complete" Lua client to access the Elasticsearch REST API Influence of official clients Respect the student 19
20 Elasticsearch Clients Design Principles "Nobody should have a reason not to use the client" We have no opinions Respect the language
21 Much more than just a "JSON and HTTP wrapper" Distributed systems - sniffer - selector Add Features - reindex - common syntax 21
22 High Level Architecture Helper Client Parser cjson lua- yajl Transport Sniffer Connection Pool Connection luasocket other http lib Selector RoundRobin Random... 22
23 elasticsearch-lua
24 Lua vs JSON { } title = "A Requirements Elicitation Approach Based in Templates and Patterns", author = "A. Durán Toro", add_authors = { "B. Bernárdez Jiménez", "A. Ruiz Cortés", "M. Toro Bonilla" }, key_words = { "requirements engineering", "requirements elicitation" }, language = "en", conference = "WER99", { } "title" : "A Requirements Elicitation Approach Based in Templates and Patterns", "author" : "A. Durán Toro", "add_authors" : [ "B. Bernárdez Jiménez", "A. Ruiz Cortés", "M. Toro Bonilla" ], "key_words" : [ "requirements engineering", "requirements elicitation" ], "language" : "en", "conference" : "WER99", 24
25 CRUD local paper = { title = "A Requirements Elicitation Approach Based in Templates and Patterns",... } local elasticsearch = require"elasticsearch" local client = elasticsearch.client() -- use default configs local r, e = client:index({ index = "papers", type = "wer", id = 1, body = paper }) local r, e = client:get( { index = "papers", type = "wer", id = 1 } ) local r, e = client:update({ index = "papers", type = "wer", id = 1, body = { doc = { title = "Updated Title" } } }) local r, e = client:delete( { index = "papers", type = "wer", id = 1 } ) 25
26 Searching local r, e = client:search( { index = "papers", type = "wer", q = "requirements" } ) { local took r, e = = 1.0, client:search( timed_out = { false, index = "papers", type = "wer", q = "title:requirements"}) _shards = { total = 5.0, successful = 5.0, failed = 0.0 }, local hits r, e = = { client:search{ index total = "papers", = 1.0, max_score = , type = hits "wer", = [ { body = { _index = "papers", "_type": "wer", "_id": "1", query = _score { = , filtered _source = { = { query title = { = "A Requirements Elicitation Approach Based in Templates and Patterns", multi_match key_words = { "requirements engineering", "requirements elicitation" }, query year = "2013" "requirements", fields = { "key_words^10", "title^5", "abstract" } } }, } filter } = { range = { year = { gte = "now-3y" } } } } } }] } } } 26
27 Why Elasticsearch?
28 Workshop on Requirements Engineering Ibero-american community of researchers on Requirements Engineering 28
29 WERpapers Dedicated site to the WER publications Apache + CGILua + MySQL + Google Search 29
30 Not for full text search Slow No text analysis No scoring No auto-completion...
31
32 Elasticsearch Implement your own search result page Relevancy based on downloads, title, abstract, etc. Auto-complete, Highlighting, Geo, etc. Everything in Lua 32
33 Elasticsearch is not Alone
34 Elastic Elasticsearch Store, search, analyze Logstash Collect, parse and enrich data Kibana Visualize and explore data Found Elasticsearch as a Service 34 Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited
35
36 Community, community, community Forums for all products Issue tracker User groups & Meetups 36 Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited
37 Community, community, community Forums for all products Issue tracker User groups & Meetups 37 Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited
38 Community, community, community Forums for all products Issue tracker User groups & Meetups 38 Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited
39 Next Steps +Use cases +Real applications +Tests Automate Integration with Elasticsearch Make it an official client 39
40 Thanks for listening! Questions? Pablo Musa 40
41 References
Real-time Data Analytics mit Elasticsearch. Bernhard Pflugfelder inovex GmbH
Real-time Data Analytics mit Elasticsearch Bernhard Pflugfelder inovex GmbH Bernhard Pflugfelder Big Data Engineer @ inovex Fields of interest: search analytics big data bi Working with: Lucene Solr Elasticsearch
More informationInformation Retrieval Elasticsearch
Information Retrieval Elasticsearch IR Information retrieval (IR) is the activity of obtaining information resources relevant to an information need from a collection of information resources. Searches
More informationData Discovery and Systems Diagnostics with the ELK stack. Rittman Mead - BI Forum 2015, Brighton. Robin Moffatt, Principal Consultant Rittman Mead
Data Discovery and Systems Diagnostics with the ELK stack Rittman Mead - BI Forum 2015, Brighton Robin Moffatt, Principal Consultant Rittman Mead T : +44 (0) 1273 911 268 (UK) About Me Principal Consultant
More informationApril 8th - 10th, 2014 LUG14 LUG14. Lustre Log Analyzer. Kalpak Shah. DataDirect Networks. ddn.com. 2014 DataDirect Networks. All Rights Reserved.
April 8th - 10th, 2014 LUG14 LUG14 Lustre Log Analyzer Kalpak Shah DataDirect Networks Lustre Log Analysis Requirements Need scripts to parse Lustre debug logs Only way to effectively use the logs for
More informationThe Open Source Knowledge Discovery and Document Analysis Platform
Enabling Agile Intelligence through Open Analytics The Open Source Knowledge Discovery and Document Analysis Platform 17/10/2012 1 Agenda Introduction and Agenda Problem Definition Knowledge Discovery
More informationUsing Logstash and Elasticsearch analytics capabilities as a BI tool
Using Logstash and Elasticsearch analytics capabilities as a BI tool Pashalis Korosoglou, Pavlos Daoglou, Stefanos Laskaridis, Dimitris Daskopoulos Aristotle University of Thessaloniki, IT Center Outline
More informationLog management with Logstash and Elasticsearch. Matteo Dessalvi
Log management with Logstash and Elasticsearch Matteo Dessalvi HEPiX 2013 Outline Centralized logging. Logstash: what you can do with it. Logstash + Redis + Elasticsearch. Grok filtering. Elasticsearch
More informationEfficient Management of System Logs using a Cloud Radoslav Bodó, Daniel Kouřil CESNET. ISGC 2013, March 2013
Efficient Management of System Logs using a Cloud Radoslav Bodó, Daniel Kouřil CESNET ISGC 2013, March 2013 Agenda Introduction Collecting logs Log Processing Advanced analysis Resume Introduction Status
More informationAndrew Moore Amsterdam 2015
Andrew Moore Amsterdam 2015 Agenda Why log How to log Audit plugins Log analysis Demos Logs [timestamp]: [some useful data] Why log? Error Log Binary Log Slow Log General Log Why log? Why log? Why log?
More informationThe New RERO Statistics Services
The New RERO Statistics Services Invenio User Group Workshop 2015 Johnny Mariéthoz 2015/10/05 Introduction I institutions need statistics for reports and analysis as instance manager we need statistics
More informationImprove performance and availability of Banking Portal with HADOOP
Improve performance and availability of Banking Portal with HADOOP Our client is a leading U.S. company providing information management services in Finance Investment, and Banking. This company has a
More informationLog Analysis with the ELK Stack (Elasticsearch, Logstash and Kibana) Gary Smith, Pacific Northwest National Laboratory
Log Analysis with the ELK Stack (Elasticsearch, Logstash and Kibana) Gary Smith, Pacific Northwest National Laboratory A Little Context! The Five Golden Principles of Security! Know your system! Principle
More informationWho did what, when, where and how MySQL Audit Logging. Jeremy Glick & Andrew Moore 20/10/14
Who did what, when, where and how MySQL Audit Logging Jeremy Glick & Andrew Moore 20/10/14 Intro 2 Hello! Intro 3 Jeremy Glick MySQL DBA Head honcho of Chicago MySQL meetup 13 years industry experience
More informationSearch and Real-Time Analytics on Big Data
Search and Real-Time Analytics on Big Data Sewook Wee, Ryan Tabora, Jason Rutherglen Accenture & Think Big Analytics Strata New York October, 2012 Big Data: data becomes your core asset. It realizes its
More informationAnalyzing large flow data sets using. visualization tools. modern open-source data search and. FloCon 2014. Max Putas
Analyzing large flow data sets using modern open-source data search and visualization tools FloCon 2014 Max Putas About me Operations Engineer - DevOps BS, MS, and CAS in Telecommunications Work/research
More informationWHITE PAPER Redefining Monitoring for Today s Modern IT Infrastructures
WHITE PAPER Redefining Monitoring for Today s Modern IT Infrastructures Modern technologies in Zenoss Service Dynamics v5 enable IT organizations to scale out monitoring and scale back costs, avoid service
More informationCloudant Querying Options
Cloudant Querying Options Agenda Indexing Options Primary Views/MapReduce Search Geospatial Cloudant Query Review What s New Live Tutorial 5/20/15 2 Reading & Writing Basics POST / /_bulk_docs
More informationThe Search API in Drupal 8. Thomas Seidl (drunken monkey)
The Search API in Drupal 8 Thomas Seidl (drunken monkey) Disclaimer Everything shown here is still a work in progress. Details might change until 8.0 release. Basic architecture Server Index Views Technical
More informationLog Management with Open-Source Tools. Risto Vaarandi SEB Estonia
Log Management with Open-Source Tools Risto Vaarandi SEB Estonia Outline Why use open source tools for log management? Widely used logging protocols and recently introduced new standards Open-source syslog
More informationUsing NXLog with Elasticsearch and Kibana. Using NXLog with Elasticsearch and Kibana
Using NXLog with Elasticsearch and Kibana i Using NXLog with Elasticsearch and Kibana Using NXLog with Elasticsearch and Kibana ii Contents 1 Setting up Elasticsearch and Kibana 1 1.1 Installing Elasticsearch................................................
More informationGUJARAT TECHNOLOGICAL UNIVERSITY
GUJARAT TECHNOLOGICAL UNIVERSITY Seminar on Intrusion Detection for Hypervisor- Based Cloud Computing Infrastructure by Dr. Rajeev Agrawal, North Carolina A&T State University, USA GTU s PG Research Center
More informationLogging on a Shoestring Budget
UNIVERSITY OF NEBRASKA AT OMAHA Logging on a Shoestring Budget James Harr jharr@unomaha.edu Agenda The Tools ElasticSearch Logstash Kibana redis Composing a Log System Q&A, Conclusions, Lessons Learned
More informationthe missing log collector Treasure Data, Inc. Muga Nishizawa
the missing log collector Treasure Data, Inc. Muga Nishizawa Muga Nishizawa (@muga_nishizawa) Chief Software Architect, Treasure Data Treasure Data Overview Founded to deliver big data analytics in days
More informationFHWA Office of Operations (R&D) RDE Release 3.0 Potential Enhancements. 26 March 2014
FHWA Office of Operations (R&D) RDE Release 3.0 Potential Enhancements 26 March 2014 Overview of this Session Present categories for potential enhancements to the RDE over the next year (Release 3.0) or
More informationSearch Big Data with MySQL and Sphinx. Mindaugas Žukas www.ivinco.com
Search Big Data with MySQL and Sphinx Mindaugas Žukas www.ivinco.com Agenda Big Data Architecture Factors and Technologies MySQL and Big Data Sphinx Search Server overview Case study: building a Big Data
More informationitunes Store Publisher User Guide Version 1.1
itunes Store Publisher User Guide Version 1.1 Version Date Author 1.1 10/09/13 William Goff Table of Contents Table of Contents... 2 Introduction... 3 itunes Console Advantages... 3 Getting Started...
More informationLog Management with Open-Source Tools. Risto Vaarandi rvaarandi 4T Y4H00 D0T C0M
Log Management with Open-Source Tools Risto Vaarandi rvaarandi 4T Y4H00 D0T C0M Outline Why do we need log collection and management? Why use open source tools? Widely used logging protocols and recently
More informationBig Data for Satellite Business Intelligence
Big Data for Satellite Business Intelligence GSAW 2015 Loic COULET, Kratos ISE 2015 by Kratos ISE. Published by The Aerospace Corporation with permission. Who s talking? Computer Science Passionate Kratos
More informationIntroduction to Cassandra
Introduction to Cassandra DuyHai DOAN, Technical Advocate Agenda! Architecture cluster replication Data model last write win (LWW), CQL basics (CRUD, DDL, collections, clustering column) lightweight transactions
More informationHow To Use Elasticsearch
Elasticsearch, Logstash, and Kibana (ELK) Dwight Beaver dsbeaver@cert.org Sean Hutchison shutchison@cert.org January 2015 2014 Carnegie Mellon University This material is based upon work funded and supported
More informationFUJITSU Software ServerView Cloud Monitoring Manager V1 Introduction
FUJITSU Software ServerView Cloud Monitoring Manager V1 Introduction November 2015 Fujitsu Limited Product Overview 1 Why a Monitoring & Logging OpenStack Service? OpenStack systems are large, complex
More informationQlik REST Connector Installation and User Guide
Qlik REST Connector Installation and User Guide Qlik REST Connector Version 1.0 Newton, Massachusetts, November 2015 Authored by QlikTech International AB Copyright QlikTech International AB 2015, All
More informationData Driven Success. Comparing Log Analytics Tools: Flowerfire s Sawmill vs. Google Analytics (GA)
Data Driven Success Comparing Log Analytics Tools: Flowerfire s Sawmill vs. Google Analytics (GA) In business, data is everything. Regardless of the products or services you sell or the systems you support,
More informationOpenText Information Hub (ihub) 3.1 and 3.1.1
OpenText Information Hub (ihub) 3.1 and 3.1.1 OpenText Information Hub (ihub) 3.1.1 meets the growing demand for analytics-powered applications that deliver data and empower employees and customers to
More informationData Mailbox. support.ewon.biz. Reference Guide
Reference Guide RG 005-0-EN / Rev. 1.0 Data Mailbox The Data Mailbox is a Talk2M service that gathers ewon historical data and makes it available for third party applications in an easy way. support.ewon.biz
More informationBig data is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone
Big data is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it Dan Ariely MYSQL AND HBASE ECOSYSTEM
More informationNoSQL Roadshow Berlin Kai Spichale
Full-text Search with NoSQL Technologies NoSQL Roadshow Berlin Kai Spichale 25.04.2013 About me Kai Spichale Software Engineer at adesso AG Author in professional journals, conference speaker adesso is
More informationMongoDB: document-oriented database
MongoDB: document-oriented database Software Languages Team University of Koblenz-Landau Ralf Lämmel, Sebastian Jackel and Andrei Varanovich Motivation Need for a flexible schema High availability Scalability
More informationSolr Bridge Search Installation Guide
Solr Bridge Search Installation Guide Table of contents 1. Solr Installation 1.1 Tomcat6 (Web server) installation. 1.2 Apache Solr 3.3.0 installation. 1.3 Install SolrBirge package (preconfigured solr
More informationLog managing at PIC. A. Bruno Rodríguez Rodríguez. Port d informació científica Campus UAB, Bellaterra Barcelona. December 3, 2013
Log managing at PIC A. Bruno Rodríguez Rodríguez Port d informació científica Campus UAB, Bellaterra Barcelona December 3, 2013 Bruno Rodríguez (PIC) Log managing at PIC December 3, 2013 1 / 21 What will
More informationIntelligence. Productivity. Mobility. Unified Service. Predictive analytics: Offline mobile: Self, assisted & field service
Productivity Intelligence Mobility Unified Service Next generation productivity: Predictive analytics: Offline mobile: Self, assisted & field service Surface trending documents with Delve Immersive Excel
More informationStress Testing for Performance Tuning. Stress Testing for Performance Tuning
Stress Testing for Performance Tuning Stress Testing for Performance Tuning A t l o g y s T e c h n i c a l C o n s u l t i n g, R - 8, N e h r u P l a c e, N e w D e l h i Page 1 This Guide is a Sys Admin
More informationIntroduction to Cloud Computing on Amazon Web Services (AWS) with focus on EC2 and S3. Horst Lueck
Introduction to Cloud Computing on Amazon Web Services (AWS) with focus on EC2 and S3 Horst Lueck 2011-05-17 IT Pro Forum http://itproforum.org Thanks to Open Office Impress The Cloud the Name The 90s
More informationBuilding the Internet of Things Jim Green - CTO, Data & Analytics Business Group, Cisco Systems
Building the Internet of Things Jim Green - CTO, Data & Analytics Business Group, Cisco Systems Brian McCarson Sr. Principal Engineer & Sr. System Architect, Internet of Things Group, Intel Corp Mac Devine
More informationMobile Analytics. mit Elasticsearch und Kibana. Dominik Helleberg
Mobile Analytics mit Elasticsearch und Kibana Dominik Helleberg Speaker Dominik Helleberg Mobile Development Android / Embedded Tools http://dominik-helleberg.de/+ Mobile Analytics Warum? Server Software
More informationReal-time Streaming Analysis for Hadoop and Flume. Aaron Kimball odiago, inc. OSCON Data 2011
Real-time Streaming Analysis for Hadoop and Flume Aaron Kimball odiago, inc. OSCON Data 2011 The plan Background: Flume introduction The need for online analytics Introducing FlumeBase Demo! FlumeBase
More informationSTeP-IN SUMMIT 2013. June 18 21, 2013 at Bangalore, INDIA. Performance Testing of an IAAS Cloud Software (A CloudStack Use Case)
10 th International Conference on Software Testing June 18 21, 2013 at Bangalore, INDIA by Sowmya Krishnan, Senior Software QA Engineer, Citrix Copyright: STeP-IN Forum and Quality Solutions for Information
More informationJan 28 th, 2015 FREE Webinar by
Google Analytics Data Mining with R (includes 3 Real Applications) Jan 28 th, 2015 FREE Webinar by 1/28/2015 1 Our Speakers Kushan Shah Maintainer of RGoogleAnalytics Library & Web Analyst at Tatvic @
More informationHacettepe University Department Of Computer Engineering BBM 471 Database Management Systems Experiment
Hacettepe University Department Of Computer Engineering BBM 471 Database Management Systems Experiment Subject NoSQL Databases - MongoDB Submission Date 20.11.2013 Due Date 26.12.2013 Programming Environment
More informationPerformance Evaluation of NoSQL Systems Using YCSB in a resource Austere Environment
International Journal of Applied Information Systems (IJAIS) ISSN : 2249-868 Performance Evaluation of NoSQL Systems Using YCSB in a resource Austere Environment Yusuf Abubakar Department of Computer Science
More informationConfiguring CQ Security
Configuring CQ Security About Me CQ Architect for Inside Solutions http://inside-solutions.ch CQ Blog: http://cqblog.inside-solutions.ch Customer Projects with Adobe CQ Training Material on Adobe CQ Agenda
More informationCymon.io. Open Threat Intelligence. 29 October 2015 Copyright 2015 esentire, Inc. 1
Cymon.io Open Threat Intelligence 29 October 2015 Copyright 2015 esentire, Inc. 1 #> whoami» Roy Firestein» Senior Consultant» Doing Research & Development» Other work include:» docping.me» threatlab.io
More informationComparative Analysis of Open-Source Log Management Solutions for Security Monitoring and Network Forensics
Comparative Analysis of Open-Source Log Management Solutions for Security Monitoring and Network Forensics Risto Vaarandi, Paweł Niziski NATO Cooperative Cyber Defence Centre of Excellence, Tallinn, Estonia
More informationHadoop. MPDL-Frühstück 9. Dezember 2013 MPDL INTERN
Hadoop MPDL-Frühstück 9. Dezember 2013 MPDL INTERN Understanding Hadoop Understanding Hadoop What's Hadoop about? Apache Hadoop project (started 2008) downloadable open-source software library (current
More informationDatasheet FUJITSU Software ServerView Cloud Monitoring Manager V1.0
Datasheet FUJITSU Software ServerView Cloud Monitoring Manager V1.0 Datasheet FUJITSU Software ServerView Cloud Monitoring Manager V1.0 A Monitoring Cloud Service for Enterprise OpenStack Systems Cloud
More informationData Pipeline with Kafka
Data Pipeline with Kafka Peerapat Asoktummarungsri AGODA Senior Software Engineer Agoda.com Contributor Thai Java User Group (THJUG.com) Contributor Agile66 AGENDA Big Data & Data Pipeline Kafka Introduction
More informationDrupal CMS for marketing sites
Drupal CMS for marketing sites Intro Sample sites: End to End flow Folder Structure Project setup Content Folder Data Store (Drupal CMS) Importing/Exporting Content Database Migrations Backend Config Unit
More informationThe MongoDB Tutorial Introduction for MySQL Users. Stephane Combaudon April 1st, 2014
The MongoDB Tutorial Introduction for MySQL Users Stephane Combaudon April 1st, 2014 Agenda 2 Introduction Install & First Steps CRUD Aggregation Framework Performance Tuning Replication and High Availability
More informationEfficient Management of System Logs using a Cloud
, CESNET z.s.p.o.,zikova 4, 160 00 Praha 6, Czech Republic and University of West Bohemia,Univerzitní 8, 306 14 Pilsen, Czech Republic E-mail: bodik@civ.zcu.cz Daniel Kouřil, CESNET z.s.p.o.,zikova 4,
More informationWho? Wolfgang Ziegler (fago) Klaus Purer (klausi) Sebastian Gilits (sepgil) epiqo Austrian based Drupal company Drupal Austria user group
Who? Wolfgang Ziegler (fago) Klaus Purer (klausi) Sebastian Gilits (sepgil) epiqo Austrian based Drupal company Drupal Austria user group Rules!?!? Reaction rules or so called ECA-Rules Event-driven conditionally
More informationAn Open Source NoSQL solution for Internet Access Logs Analysis
An Open Source NoSQL solution for Internet Access Logs Analysis A practical case of why, what and how to use a NoSQL Database Management System instead of a relational one José Manuel Ciges Regueiro
More informationIndex. AdWords, 182 AJAX Cart, 129 Attribution, 174
Index A AdWords, 182 AJAX Cart, 129 Attribution, 174 B BigQuery, Big Data Analysis create reports, 238 GA-BigQuery integration, 238 GA data, 241 hierarchy structure, 238 query language (see also Data selection,
More informationProcessing millions of logs with Logstash
and integrating with Elasticsearch, Hadoop and Cassandra November 21, 2014 About me My name is Valentin Fischer-Mitoiu and I work for the University of Vienna. More specificaly in a group called Domainis
More informationTowards Smart and Intelligent SDN Controller
Towards Smart and Intelligent SDN Controller - Through the Generic, Extensible, and Elastic Time Series Data Repository (TSDR) YuLing Chen, Dell Inc. Rajesh Narayanan, Dell Inc. Sharon Aicler, Cisco Systems
More informationNetwork Activity D Developing and Maintaining Databases
Network Activity D - Developing and Maintaining Databases Report D3.2.2 User Interface implementation Patricia KELBERT MNHN Paris BGBM Berlin July 2006-1- Table of Contents 1 Introduction... 4 2 Material
More informationDeveloping an Application Tracing Utility for Mule ESB Application on EL (Elastic Search, Log stash) Stack Using AOP
Developing an Application Tracing Utility for Mule ESB Application on EL (Elastic Search, Log stash) Stack Using AOP Mohan Bandaru, Amarendra Kothalanka, Vikram Uppala Student, Department of Computer Science
More informationvcloud Air Platform Programmer's Guide
vcloud Air Platform Programmer's Guide vcloud Air OnDemand 5.7 This document supports the version of each product listed and supports all subsequent versions until the document is replaced by a new edition.
More informationApache Lucene. Searching the Web and Everything Else. Daniel Naber Mindquarry GmbH ID 380
Apache Lucene Searching the Web and Everything Else Daniel Naber Mindquarry GmbH ID 380 AGENDA 2 > What's a search engine > Lucene Java Features Code example > Solr Features Integration > Nutch Features
More informationModern Web development and operations practices. Grig Gheorghiu VP Tech Operations Nasty Gal Inc. @griggheo
Modern Web development and operations practices Grig Gheorghiu VP Tech Operations Nasty Gal Inc. @griggheo Modern Web stack Aim for horizontal scalability! Ruby/Python front-end servers (Sinatra/Padrino,
More informationEngagement Analytics API Reference Guide
Engagement Analytics API Reference Guide Rev: 2 April 2014 Sitecore 6.5 or later Engagement Analytics API Reference Guide A developer's reference guide to the Engagement Analytics API Table of Contents
More informationMonitis Project Proposals for AUA. September 2014, Yerevan, Armenia
Monitis Project Proposals for AUA September 2014, Yerevan, Armenia Distributed Log Collecting and Analysing Platform Project Specifications Category: Big Data and NoSQL Software Requirements: Apache Hadoop
More informationPowering Monitoring Analytics with ELK stack
Powering Monitoring Analytics with ELK stack Abdelkader Lahmadi, Frédéric Beck INRIA Nancy Grand Est, University of Lorraine, France 2015 (compiled on: June 23, 2015) References online Tutorials Elasticsearch
More informationHTCondor at the RAL Tier-1
HTCondor at the RAL Tier-1 Andrew Lahiff, Alastair Dewhurst, John Kelly, Ian Collier, James Adams STFC Rutherford Appleton Laboratory HTCondor Week 2014 Outline Overview of HTCondor at RAL Monitoring Multi-core
More informationFull-text Search in Intermediate Data Storage of FCART
Full-text Search in Intermediate Data Storage of FCART Alexey Neznanov, Andrey Parinov National Research University Higher School of Economics, 20 Myasnitskaya Ulitsa, Moscow, 101000, Russia ANeznanov@hse.ru,
More informationUsing Apache Solr for Ecommerce Search Applications
Using Apache Solr for Ecommerce Search Applications Rajani Maski Happiest Minds, IT Services SHARING. MINDFUL. INTEGRITY. LEARNING. EXCELLENCE. SOCIAL RESPONSIBILITY. 2 Copyright Information This document
More informationSOLR INSTALLATION & CONFIGURATION GUIDE FOR USE IN THE NTER SYSTEM
SOLR INSTALLATION & CONFIGURATION GUIDE FOR USE IN THE NTER SYSTEM Prepared By: Leigh Moulder, SRI International leigh.moulder@sri.com TABLE OF CONTENTS Table of Contents. 1 Document Change Log 2 Solr
More informationImplementing Search in Web, Mobile, and IOT Applications An Overview of DataStax Enterprise Search
Implementing Search in Web, Mobile, and IOT Applications An Overview of DataStax Enterprise Search Table of Contents Introduction... 3 Why Search?... 3 General Search Requirements... 3 Traditional Deployment
More informationHow To Use Big Data For Telco (For A Telco)
ON-LINE VIDEO ANALYTICS EMBRACING BIG DATA David Vanderfeesten, Bell Labs Belgium ANNO 2012 YOUR DATA IS MONEY BIG MONEY! Your click stream, your activity stream, your electricity consumption, your call
More informationA Year of HTCondor Monitoring. Lincoln Bryant Suchandra Thapa
A Year of HTCondor Monitoring Lincoln Bryant Suchandra Thapa HTCondor Week 2015 May 21, 2015 Analytics vs. Operations Two parallel tracks in mind: o Operations o Analytics Operations needs to: o Observe
More informationmdata from Mobile Commons enables organizations to make any data accessible to the public via text message, no programming required.
mdata Web Services mdata from Mobile Commons enables organizations to make any data accessible to the public via text message, no programming required. How it Works 1. A user sends a text message with
More informationWikimedia Architecture Doing More With Less. Asher Feldman <asher@wikimedia.org> Ryan Lane <ryan@wikimedia.org> Wikimedia Foundation Inc.
Wikimedia Architecture Doing More With Less Asher Feldman Ryan Lane Wikimedia Foundation Inc. Overview Intro Scale at WMF How We Work Architecture Dive Top Five
More informationElasticsearch on Cisco Unified Computing System: Optimizing your UCS infrastructure for Elasticsearch s analytics software stack
Elasticsearch on Cisco Unified Computing System: Optimizing your UCS infrastructure for Elasticsearch s analytics software stack HIGHLIGHTS Real-Time Results Elasticsearch on Cisco UCS enables a deeper
More informationBuilding a logging pipeline with Open Source tools. Iñigo Ortiz de Urbina Cazenave
Building a logging pipeline with Open Source tools Iñigo Ortiz de Urbina Cazenave NLUUG Utrecht - Netherlands 28 May 2015 whoami; 2 Iñigo Ortiz de Urbina Cazenave Systems Engineer whoami; groups; 3 Iñigo
More informationEvent Logging - Introduction
Event logging Event Logging Introduction Event Logging Listener Overview Log Listener OSGi Event Listener SAM Listener Event Logging Agent Introduction Agent Receiver part Agent Processing Part Event Logging
More informationEvaluation and implementation of CEP mechanisms to act upon infrastructure metrics monitored by Ganglia
Project report CERN Summer Student Programme Evaluation and implementation of CEP mechanisms to act upon infrastructure metrics monitored by Ganglia Author: Martin Adam Supervisors: Cristovao Cordeiro,
More informationOntology, NFV and the Future OSS September 2015
Ontology, NFV and the Future OSS September 2015 As NFV moves from labs and trials into production, the need to assure the services it delivers has become urgent, but legacy tools struggle to deliver. Ontology
More informationA New Approach to Network Visibility at UBC. Presented by the Network Management Centre and Wireless Infrastructure Teams
A New Approach to Network Visibility at UBC Presented by the Network Management Centre and Wireless Infrastructure Teams Agenda Business Drivers Technical Overview Network Packet Broker Tool Network Monitoring
More informationChina Bank BigData Usecase Huawei FusionInsight Solution
China Bank BigData Usecase Huawei FusionInsight Solution Steven Yuan, Director of Bigdata Research and Development Agenda 1. BigData Trend 2. China Bank Business Challenges & Requirements 3. Huawei s BigData
More informationNOT IN KANSAS ANY MORE
NOT IN KANSAS ANY MORE How we moved into Big Data Dan Taylor - JDSU Dan Taylor Dan Taylor: An Engineering Manager, Software Developer, data enthusiast and advocate of all things Agile. I m currently lucky
More informationNoSQL web apps. w/ MongoDB, Node.js, AngularJS. Dr. Gerd Jungbluth, NoSQL UG Cologne, 4.9.2013
NoSQL web apps w/ MongoDB, Node.js, AngularJS Dr. Gerd Jungbluth, NoSQL UG Cologne, 4.9.2013 About us Passionate (web) dev. since fallen in love with Sinclair ZX Spectrum Academic background in natural
More informationWhy API? Using the REST API in an education environment. JAMF Software, LLC
Why API? Using the REST API in an education environment. Brad Schmidt Technical Services Operations Manager Hopkins Public Schools Why API? Presentation agenda: Briefly - What is an API? What is the JSS
More informationPowerDNS dnsdist. NLUUG Najaarsconferentie 2015. Presentation is on: https://www.powerdns.com/nluug. http://dnsdist.org/
PowerDNS dnsdist NLUUG Najaarsconferentie 2015 Presentation is on: https://www.powerdns.com/nluug http://dnsdist.org/ bert.hubert@powerdns.com / @powerdns_bert Outline PowerDNS/Open-Xchange/Dovecot/Bert
More informationMobile App Analytics with Google Analytics
Mobile App Analytics with Google Analytics About InfoTrust and Your Presenter Web analytics consulting & product development 2,500+ sites analyzed and supported annually 30+ Digital marketing training
More informationWikimedia Infrastructure. Roan Kattouw
Wikimedia Infrastructure Roan Kattouw WikiWhat? Encyclopedia Wiki software Volunteer-run GPL No staff editors Volunteer and staff devs 3rd party users Movement WP + sister projects Non-profit foundation
More informationPL/JSON Reference Guide (version 1.0.4)
PL/JSON Reference Guide (version 1.0.4) For Oracle 10g and 11g Jonas Krogsbøll Contents 1 PURPOSE 2 2 DESCRIPTION 2 3 IN THE RELEASE 3 4 GETTING STARTED 3 5 TWEAKS 4 6 JSON PATH 6 7 BEHAVIOR & ERROR HANDLING
More informationDomain driven design, NoSQL and multi-model databases
Domain driven design, NoSQL and multi-model databases Java Meetup New York, 10 November 2014 Max Neunhöffer www.arangodb.com Max Neunhöffer I am a mathematician Earlier life : Research in Computer Algebra
More informationPulsar Realtime Analytics At Scale. Tony Ng April 14, 2015
Pulsar Realtime Analytics At Scale Tony Ng April 14, 2015 Big Data Trends Bigger data volumes More data sources DBs, logs, behavioral & business event streams, sensors Faster analysis Next day to hours
More informationIntroduction to Web Development with R
Introduction to Web Development with R moving to the cloud... Jeroen Ooms http://www.stat.ucla.edu/~jeroen UCLA Dept. of Statistics Revolution Analytics user 2010, Gaithersburg, Maryland, USA An example:
More informationTesting Automation for Distributed Applications By Isabel Drost-Fromm, Software Engineer, Elastic
Testing Automation for Distributed Applications By Isabel Drost-Fromm, Software Engineer, Elastic The challenge When building distributed, large-scale applications, quality assurance (QA) gets increasingly
More informationWelcome The webinar will begin shortly
Welcome The webinar will begin shortly Angela Chumley Angela.Chumley@crownpeak.com 08.18.15 Engagement Tip Mute Button Listen Actively Ask Questions 2 AGENDA Getting Started Web Content Management (WCMS)
More information