The Big Data Integration and Analytics Revolution in Agricultural Finance, Risk, and Insurance
|
|
- Beverly Lynch
- 8 years ago
- Views:
Transcription
1 The Big Data Integration and Analytics Revolution in Agricultural Finance, Risk, and Insurance Joshua D. Woodard Assistant Professor and Zaitz Faculty Fellow in Agribusiness and Finance Dyson School of Applied Economics and Management Cornell University IARFIC Keynote June 8th, 2015
2 Introduction Overview, society and data The data integration problem Ongoing system development efforts Challenges and considerations Purpose is to provide high level overview relevant to ag finance, risk and insurance field
3 The Data Integration Problem Analysts typically source data from many different government and nongovernment sources, different temporal and spatial resolution Relevant data spread over a wide variety of operational/transaction based databases, datamarts, unstructured text files, etc. Sources all have different data storage and formatting protocols, API s, different levels of temporal and spatial aggregation etc. Existing infrastructures can not be queried jointly, nor at all Not processed to scales appropriate for most uses Typical approach is to one off for every study, to do the following: At a point in time, download slices of data from several different sources, Then format (often by hand or copy/paste) individual data sets and mash together (may take days or weeks; not automated/replicable/documented) Perform one off analysis To expand analysis or update, entire process must be recreated by human
4 A Fairly Small Sampling
5 The Data Integration Problem This makes it impossible for most to conduct analysis of today s programs, and imposes large costs on agencies and others Results in duplication in effort, redundancy, error, and difficulty/waste in sourcing data Limits use and usability of data Pushes out many potential users Very difficult to recreate analyses and update Renders research and analysis less credible (see recent blunders) Data analysis versus management/integration/processing Yet, very little focus in the community on building such systems to date
6 Advantages of Data Warehousing and Integration Systems Acts as a clearinghouse for data to support policy makers, oversight, research & development, and business intelligence Not a transactional database, but rather used for informational/reporting/research purposes Integrated and centralized Subject oriented and optimized to give answers to diverse questions Data are processed in various ways for variety of uses, flexible Non-volatile, meaning data are never deleted, and is always growing Consistent data storage and formatting protocols within warehouse (reconciles source data)
7 AgDB Data Warehousing Overview Data Chunks External Databases, Datastores, Datastreams: RMA, USGS, NRCS, AMS, ERS, PRISMS, CME, NASA, NASS, FSA, FAS, etc. OLTP Data Scheduled Jobs To Download and Extract from Source Over Web Integration Services Filter/clean Prepocessing/ Aggregation/ Interpolation/ Transformation Load Data Auditing & Validation AgDB Data Warehouse Web Data Services, OLAP, Data Marts External Clients Web Decision Tools
8 Advantages of Data Warehousing and Integration Systems Resilient to change, additions, updates Data from different sources can be joined, integrated, and queried with low effort Wide degree of control and consistency in aggregating, interpolating, and cross referencing among and between different types of data Improved data integrity (auditing, cleaning, validation) Increases utility, usability, and access to data Results in lower costs and more reliability for analysts and policymakers who use the data Overall: Save time, save money, increase capabilities Facilitate user tools and access to data that farmers, researchers, and policy makes want
9 Current Efforts, AgDB Data Warehousing System at Cornell Genesis of system & motivation for Open Data Warehouse Pulls in data from disparate sources and consolidates in a single repository (primarily various USDA data, but also as others) Basic ETL: Data extraction and/or sourcing Extraction, Preprocessing and transformation before loading; Filter, transform, integrate, classify, aggregate, summarize DBMS: Microsoft SQL Server SSIS and other programs for transformation (Python, Matlab, ArcGIS, ArcPy, etc.) Built in spatial libraries, SSDT, etc. Other candidates: Oracle, PostgreSQL, MySQL, MongoDB, other custom Deployed on CIT servers at Cornell (moving to Azure this summer) Data Access: Web API: Virtually any language or stat program (e.g., Matlab, Python, Excel, STATA, Java) Point and click interfaces (also generate API calls for replicability) Others: RMA rating calculator API, spot/basis interpolation, Dairy Margin Tool, Grape Vine Cost tool, etc. See Forum for code samples Web interfaces in development, test site at: Some qualifiers
10 Abridged/Partial Summaries of Major Datasets/Sources Currently in AgDB Data Source and Item IPCC Climate Change Projections National Climatic Data Center Drought Data PRISMs Climate Group Chicago Mercantile Exchange Risk Management Agency (RMA) US Census Bureau USDA Economic Research Service (ERS) USDA Agricultural Marketing Service (AMS) USDA National Agricultural Statistics Service (NASS) USDA Foreign Agricultural Service USDA National Resource Conservation Service (NRCS) Description Future temperature and precipitation projections across different emission scenarios and percentiles of the 16 General Circulation Models (GCMs). Raw or spatially processed data. Monthly PDSI drought index data available at the climate district level aggregation. Data is available from 1895 to present, by NCDC District. Monthly and daily historical temperature and precipitation data, as well as GDD/HDD processed data. Monthly data is available from 1895 to present. Daily weather data is available from 1981 to present. 800 meter resolution (raw) and processed by FIPS, Township, and in certain cases CLU (pre-2008) available. Daily historical futures and options data for agricultural commodities from the Chicago Mercantile Exchange (CME), Chicago Board of Trade (CBOT), and Kansas City Board of Trade (KCBOT). Data is available from 1959 to present, updated daily. Agricultural insurance price and participation data available at the county level aggregation. Data is available from 1989 to present from Summary of Business. Other data also loaded from various unstructured text files (including historical discovery prices, GRIP yields, etc.) County-level and township level geographical coordinates, land area size, water area size, and population data. Annual farm structural and financial data available at state-level aggregation for the 15 Agricultural Resource Management Survey (ARMS) states. Data is available from 1996 to present. Other various datasets are also sourced from the ad hoc ERS tools and API s. Monthly data on the volume, pricing, and utilization of raw milk received by handlers regulated under Federal milk orders from dairy farmers. All tables in the Public MMO database. Census and survey data available at regional, state, and county level aggregation. The broad categories of data available are crops, animals and products, economics, demographics, and environmental. Data is available from 1926 to present. Obtained via FTP bulk download from QuickStats. CDL data processed against ready to map gssurgo NRCS data by crop also available (raw and county processed). Data on production, supply and distribution of agricultural commodities for the U.S. and key producing and consuming countries. Soil data for the continental US from gssurgo, raw and processed available at various levels of aggregation.
11 Applications & Accessing Data Applications: Virtually anything Insurance Conservation and Climate Change Policy Analysis and oversight Farm Bill Program Analysis Product Development Different tools for different users 1) Direct DB Connect (or bulk download for external users) Matlab, R, Python, or Web apps using, or standard SQL connections (ODBC, BCP, etc.) 2) Web API and data services for analysts 3) Interactive web tools for farmers, consumers of research Workflows for webtools Enables extension of research and tool dev
12
13
14
15
16 Soil Rating/Insurance 0.07 Soil Productivity Index Kernel Density among Common Land Units (CLU's), McLean County, Illinois (SSURGO Data) Soil Productivity Index (IL 810 Circular)
17 Ongoing Efforts and Priorities Recently received a Microsoft Azure Research Grant for use of Azure cloud platform, conversion to cloud platform in progress (early to mid-summer) Additional datasets, API s, tools (ongoing) Open Source launch (mid-summer) Upgraded data portal interface (late summer) for faster and more flexible cataloging/access Movement towards and incorporation of NoSQL platforms Identify various user needs, partners, and collaborators
18 Challenges, Policy Considerations, and Opportunities Technical and training Some degree of learning curve, but frankly minimal Technical limitations are eroding quickly, political ones are not Expanding purview Still inherently a public good, so without intervention will be underprovisioned Marginal cost curse Coordination within the community Improving access to government data (incentives and bandwidth) What data are made available or opened up How data are made available Privacy concerns Field is at an interesting vantage point compared to many others given mix of market, business, environmental and other natural systems data
19 Thank you Questions?
Ag-Analytics Documentation
Ag-Analytics Documentation Release 1.0 Joshua Woodard, Lin Xue February 11, 2016 Contents 1 1. What is Ag-Analytics? 3 2 2. Finding Data 5 2.1 2.1. Data Overview............................................
More informationFoundations of Business Intelligence: Databases and Information Management
Foundations of Business Intelligence: Databases and Information Management Content Problems of managing data resources in a traditional file environment Capabilities and value of a database management
More informationCourse 103402 MIS. Foundations of Business Intelligence
Oman College of Management and Technology Course 103402 MIS Topic 5 Foundations of Business Intelligence CS/MIS Department Organizing Data in a Traditional File Environment File organization concepts Database:
More informationFoundations of Business Intelligence: Databases and Information Management
Chapter 5 Foundations of Business Intelligence: Databases and Information Management 5.1 Copyright 2011 Pearson Education, Inc. Student Learning Objectives How does a relational database organize data,
More information5.5 Copyright 2011 Pearson Education, Inc. publishing as Prentice Hall. Figure 5-2
Class Announcements TIM 50 - Business Information Systems Lecture 15 Database Assignment 2 posted Due Tuesday 5/26 UC Santa Cruz May 19, 2015 Database: Collection of related files containing records on
More informationChapter 6 FOUNDATIONS OF BUSINESS INTELLIGENCE: DATABASES AND INFORMATION MANAGEMENT Learning Objectives
Chapter 6 FOUNDATIONS OF BUSINESS INTELLIGENCE: DATABASES AND INFORMATION MANAGEMENT Learning Objectives Describe how the problems of managing data resources in a traditional file environment are solved
More informationGEOG 482/582 : GIS Data Management. Lesson 10: Enterprise GIS Data Management Strategies GEOG 482/582 / My Course / University of Washington
GEOG 482/582 : GIS Data Management Lesson 10: Enterprise GIS Data Management Strategies Overview Learning Objective Questions: 1. What are challenges for multi-user database environments? 2. What is Enterprise
More informationChapter 6. Foundations of Business Intelligence: Databases and Information Management
Chapter 6 Foundations of Business Intelligence: Databases and Information Management VIDEO CASES Case 1a: City of Dubuque Uses Cloud Computing and Sensors to Build a Smarter, Sustainable City Case 1b:
More informationGlobal outlook on the perspectives of technologies like Power Hub
Power Hub January 2013 Global outlook on the perspectives of technologies like Power Hub Larry Cochrane, Microsoft Utilities Industry Technology Strategist & Architect Global outlook on the perspectives
More informationEmerging Technologies Shaping the Future of Data Warehouses & Business Intelligence
Emerging Technologies Shaping the Future of Data Warehouses & Business Intelligence Appliances and DW Architectures John O Brien President and Executive Architect Zukeran Technologies 1 TDWI 1 Agenda What
More informationDecoding the Big Data Deluge a Virtual Approach. Dan Luongo, Global Lead, Field Solution Engineering Data Virtualization Business Unit, Cisco
Decoding the Big Data Deluge a Virtual Approach Dan Luongo, Global Lead, Field Solution Engineering Data Virtualization Business Unit, Cisco High-volume, velocity and variety information assets that demand
More informationIntegrating Ingres in the Information System: An Open Source Approach
Integrating Ingres in the Information System: WHITE PAPER Table of Contents Ingres, a Business Open Source Database that needs Integration... 3 Scenario 1: Data Migration... 4 Scenario 2: e-business Application
More informationwww.ijreat.org Published by: PIONEER RESEARCH & DEVELOPMENT GROUP (www.prdg.org) 28
Data Warehousing - Essential Element To Support Decision- Making Process In Industries Ashima Bhasin 1, Mr Manoj Kumar 2 1 Computer Science Engineering Department, 2 Associate Professor, CSE Abstract SGT
More informationApplication Of Business Intelligence In Agriculture 2020 System to Improve Efficiency And Support Decision Making in Investments.
Application Of Business Intelligence In Agriculture 2020 System to Improve Efficiency And Support Decision Making in Investments Anuraj Gupta Department of Electronics and Communication Oriental Institute
More informationWhen to consider OLAP?
When to consider OLAP? Author: Prakash Kewalramani Organization: Evaltech, Inc. Evaltech Research Group, Data Warehousing Practice. Date: 03/10/08 Email: erg@evaltech.com Abstract: Do you need an OLAP
More informationOLAP and OLTP. AMIT KUMAR BINDAL Associate Professor M M U MULLANA
OLAP and OLTP AMIT KUMAR BINDAL Associate Professor Databases Databases are developed on the IDEA that DATA is one of the critical materials of the Information Age Information, which is created by data,
More informationAlexander Nikov. 5. Database Systems and Managing Data Resources. Learning Objectives. RR Donnelley Tries to Master Its Data
INFO 1500 Introduction to IT Fundamentals 5. Database Systems and Managing Data Resources Learning Objectives 1. Describe how the problems of managing data resources in a traditional file environment are
More informationLITERATURE SURVEY ON DATA WAREHOUSE AND ITS TECHNIQUES
LITERATURE SURVEY ON DATA WAREHOUSE AND ITS TECHNIQUES MUHAMMAD KHALEEL (0912125) SZABIST KARACHI CAMPUS Abstract. Data warehouse and online analytical processing (OLAP) both are core component for decision
More informationFoundations of Business Intelligence: Databases and Information Management
Foundations of Business Intelligence: Databases and Information Management Problem: HP s numerous systems unable to deliver the information needed for a complete picture of business operations, lack of
More informationData Warehousing and OLAP Technology for Knowledge Discovery
542 Data Warehousing and OLAP Technology for Knowledge Discovery Aparajita Suman Abstract Since time immemorial, libraries have been generating services using the knowledge stored in various repositories
More informationIntegrating data in the Information System An Open Source approach
WHITE PAPER Integrating data in the Information System An Open Source approach Table of Contents Most IT Deployments Require Integration... 3 Scenario 1: Data Migration... 4 Scenario 2: e-business Application
More informationBuilding a Web-Enabled Data Warehouse
Building a Web-Enabled Data Warehouse www.advancedatatools.com 4216 Evergreen Lane, Suite 136 Annandale, VA 22003 (703) 256-0267 or (800) 807-6732 Company Profile is dedicated to providing database consulting
More informationData Integration Checklist
The need for data integration tools exists in every company, small to large. Whether it is extracting data that exists in spreadsheets, packaged applications, databases, sensor networks or social media
More informationChapter 6 8/12/2015. Foundations of Business Intelligence: Databases and Information Management. Problem:
Foundations of Business Intelligence: Databases and Information Management VIDEO CASES Chapter 6 Case 1a: City of Dubuque Uses Cloud Computing and Sensors to Build a Smarter, Sustainable City Case 1b:
More informationReport Data Management in the Cloud: Limitations and Opportunities
Report Data Management in the Cloud: Limitations and Opportunities Article by Daniel J. Abadi [1] Report by Lukas Probst January 4, 2013 In this report I want to summarize Daniel J. Abadi's article [1]
More informationLEARNING SOLUTIONS website milner.com/learning email training@milner.com phone 800 875 5042
Course 20467A: Designing Business Intelligence Solutions with Microsoft SQL Server 2012 Length: 5 Days Published: December 21, 2012 Language(s): English Audience(s): IT Professionals Overview Level: 300
More informationSoil Data Warehouse Western Regional NCSS Conference Western Regional NCSS Conference
Soil Warehouse Western Regional NCSS Conference Telluride, Colorado July 9, 2002 Ken Harward, Project Manager NRCS Information Technology Center Fort Collins, Colorado Outline of Presentation Soil Warehouse
More informationThe ESB and Microsoft BI
Business Intelligence The ESB and Microsoft BI The role of the Enterprise Service Bus in Microsoft s BI Framework Gijsbert Gijs in t Veld CTO, BizTalk Server MVP gijs.intveld@motion10.com About motion10
More informationTechnology in Action. Alan Evans Kendall Martin Mary Anne Poatsy. Eleventh Edition. Copyright 2015 Pearson Education, Inc.
Copyright 2015 Pearson Education, Inc. Technology in Action Alan Evans Kendall Martin Mary Anne Poatsy Eleventh Edition Copyright 2015 Pearson Education, Inc. Technology in Action Chapter 9 Behind the
More informationMDM and Data Warehousing Complement Each Other
Master Management MDM and Warehousing Complement Each Other Greater business value from both 2011 IBM Corporation Executive Summary Master Management (MDM) and Warehousing (DW) complement each other There
More informationEditing Strategies for Enterprise Geodatabase
Federal GIS Conference February 9 10, 2015 Washington, DC Editing Strategies for Enterprise Geodatabase Ty Fabling Esri Solution Engineer A Complete Platform Enabling GIS Everywhere Desktop Web Device
More informationAzure Scalability Prescriptive Architecture using the Enzo Multitenant Framework
Azure Scalability Prescriptive Architecture using the Enzo Multitenant Framework Many corporations and Independent Software Vendors considering cloud computing adoption face a similar challenge: how should
More informationData Warehousing: A Technology Review and Update Vernon Hoffner, Ph.D., CCP EntreSoft Resouces, Inc.
Warehousing: A Technology Review and Update Vernon Hoffner, Ph.D., CCP EntreSoft Resouces, Inc. Introduction Abstract warehousing has been around for over a decade. Therefore, when you read the articles
More informationDeploy. Friction-free self-service BI solutions for everyone Scalable analytics on a modern architecture
Friction-free self-service BI solutions for everyone Scalable analytics on a modern architecture Apps and data source extensions with APIs Future white label, embed or integrate Power BI Deploy Intelligent
More informationSo What s the Big Deal?
So What s the Big Deal? Presentation Agenda Introduction What is Big Data? So What is the Big Deal? Big Data Technologies Identifying Big Data Opportunities Conducting a Big Data Proof of Concept Big Data
More informationOracle9i Data Warehouse Review. Robert F. Edwards Dulcian, Inc.
Oracle9i Data Warehouse Review Robert F. Edwards Dulcian, Inc. Agenda Oracle9i Server OLAP Server Analytical SQL Data Mining ETL Warehouse Builder 3i Oracle 9i Server Overview 9i Server = Data Warehouse
More informationStructure of the presentation
Integration of Legacy Data (SLIMS) and Laboratory Information Management System (LIMS) through Development of a Data Warehouse Presenter N. Chikobi 2011.06.29 Structure of the presentation Background Preliminary
More informationAgenda. Big Data & Hadoop ViPR HDFS Pivotal Big Data Suite & ViPR HDFS ViON Customer Feedback #EMCVIPR
1 Agenda Big Data & Hadoop ViPR HDFS Pivotal Big Data Suite & ViPR HDFS ViON Customer Feedback 2 A World of Connected Devices Need a new data management architecture for Internet of Things 21% the % of
More informationChapter 5 Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization
Turban, Aronson, and Liang Decision Support Systems and Intelligent Systems, Seventh Edition Chapter 5 Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization
More informationWorkday Big Data Analytics
Workday Big Data Analytics Today s fast-paced business climate demands that decision-makers stay informed. Having access to key information gives them the best insight into their business. However, many
More informationRelational Databases for the Business Analyst
Relational Databases for the Business Analyst Mark Kurtz Sr. Systems Consulting Quest Software, Inc. mark.kurtz@quest.com 2010 Quest Software, Inc. ALL RIGHTS RESERVED Agenda The RDBMS and its role in
More informationFoundations of Business Intelligence: Databases and Information Management
Chapter 6 Foundations of Business Intelligence: Databases and Information Management 6.1 2010 by Prentice Hall LEARNING OBJECTIVES Describe how the problems of managing data resources in a traditional
More informationSisense. Product Highlights. www.sisense.com
Sisense Product Highlights Introduction Sisense is a business intelligence solution that simplifies analytics for complex data by offering an end-to-end platform that lets users easily prepare and analyze
More informationSQL Server 2012 Business Intelligence Boot Camp
SQL Server 2012 Business Intelligence Boot Camp Length: 5 Days Technology: Microsoft SQL Server 2012 Delivery Method: Instructor-led (classroom) About this Course Data warehousing is a solution organizations
More informationCourse 10777A: Implementing a Data Warehouse with Microsoft SQL Server 2012
Course 10777A: Implementing a Data Warehouse with Microsoft SQL Server 2012 OVERVIEW About this Course Data warehousing is a solution organizations use to centralize business data for reporting and analysis.
More informationTurning your Warehouse Data into Business Intelligence: Reporting Trends and Visibility Michael Armanious; Vice President Sales and Marketing Datex,
Turning your Warehouse Data into Business Intelligence: Reporting Trends and Visibility Michael Armanious; Vice President Sales and Marketing Datex, Inc. Overview Introduction What is Business Intelligence?
More informationMicrosoft Business Intelligence
Microsoft Business Intelligence P L A T F O R M O V E R V I E W M A R C H 1 8 TH, 2 0 0 9 C H U C K R U S S E L L S E N I O R P A R T N E R C O L L E C T I V E I N T E L L I G E N C E I N C. C R U S S
More informationNext-Generation Cloud Analytics with Amazon Redshift
Next-Generation Cloud Analytics with Amazon Redshift What s inside Introduction Why Amazon Redshift is Great for Analytics Cloud Data Warehousing Strategies for Relational Databases Analyzing Fast, Transactional
More informationDatabricks. A Primer
Databricks A Primer Who is Databricks? Databricks was founded by the team behind Apache Spark, the most active open source project in the big data ecosystem today. Our mission at Databricks is to dramatically
More informationData Warehousing and Data Mining
Data Warehousing and Data Mining Part I: Data Warehousing Gao Cong gaocong@cs.aau.dk Slides adapted from Man Lung Yiu and Torben Bach Pedersen Course Structure Business intelligence: Extract knowledge
More informationImplementing a Data Warehouse with Microsoft SQL Server 2012
Course 10777A: Implementing a Data Warehouse with Microsoft SQL Server 2012 Length: Audience(s): 5 Days Level: 200 IT Professionals Technology: Microsoft SQL Server 2012 Type: Delivery Method: Course Instructor-led
More informationCourse Outline. Module 1: Introduction to Data Warehousing
Course Outline Module 1: Introduction to Data Warehousing This module provides an introduction to the key components of a data warehousing solution and the highlevel considerations you must take into account
More informationBussiness Intelligence and Data Warehouse. Tomas Bartos CIS 764, Kansas State University
Bussiness Intelligence and Data Warehouse Schedule Bussiness Intelligence (BI) BI tools Oracle vs. Microsoft Data warehouse History Tools Oracle vs. Others Discussion Business Intelligence (BI) Products
More informationChallenges and Success of Migrating to an Enterprise Database in York County, PA
Challenges and Success of Migrating to an Enterprise Database in York County, PA PA GIS Conference June 16, 2015 Wade Gobrecht York County Planning Commission Andrew Ross GeographIT Source: Can You Draw
More informationCopyright 2011 Sentry Data Systems, Inc. All Rights Reserved. No Unauthorized Reproduction.
The Datanex Platform is a healthcare focused cloud computing platform that allows solution providers to construct rich healthcare business intelligence applications that leverage the world s fastest and
More information<Insert Picture Here> Oracle BI Standard Edition One The Right BI Foundation for the Emerging Enterprise
Oracle BI Standard Edition One The Right BI Foundation for the Emerging Enterprise Business Intelligence is the #1 Priority the most important technology in 2007 is business intelligence
More informationCourse 803401 DSS. Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization
Oman College of Management and Technology Course 803401 DSS Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization CS/MIS Department Information Sharing
More informationBy Makesh Kannaiyan makesh.k@sonata-software.com 8/27/2011 1
Integration between SAP BusinessObjects and Netweaver By Makesh Kannaiyan makesh.k@sonata-software.com 8/27/2011 1 Agenda Evolution of BO Business Intelligence suite Integration Integration after 4.0 release
More informationImplementing a Data Warehouse with Microsoft SQL Server 2012
Course 10777 : Implementing a Data Warehouse with Microsoft SQL Server 2012 Page 1 of 8 Implementing a Data Warehouse with Microsoft SQL Server 2012 Course 10777: 4 days; Instructor-Led Introduction Data
More informationORACLE BUSINESS INTELLIGENCE SUITE ENTERPRISE EDITION PLUS
ORACLE BUSINESS INTELLIGENCE SUITE ENTERPRISE EDITION PLUS PRODUCT FACTS & FEATURES KEY FEATURES Comprehensive, best-of-breed capabilities 100 percent thin client interface Intelligence across multiple
More informationDeveloping Business Intelligence and Data Visualization Applications with Web Maps
Developing Business Intelligence and Data Visualization Applications with Web Maps Introduction Business Intelligence (BI) means different things to different organizations and users. BI often refers to
More informationIntroduction to Big Data! with Apache Spark" UC#BERKELEY#
Introduction to Big Data! with Apache Spark" UC#BERKELEY# So What is Data Science?" Doing Data Science" Data Preparation" Roles" This Lecture" What is Data Science?" Data Science aims to derive knowledge!
More informationORACLE BUSINESS INTELLIGENCE SUITE ENTERPRISE EDITION PLUS
Oracle Fusion editions of Oracle's Hyperion performance management products are currently available only on Microsoft Windows server platforms. The following is intended to outline our general product
More informationSAP Analytics Roadmap for Small and Midsize Companies. Kevin Chan, Director, Solutions Management @ SAP
SAP Analytics Roadmap for Small and Midsize Companies Kevin Chan, Director, Solutions Management @ SAP A WORLD OF ACCELERATING CHANGE An emerging middle class growing to 5B Data doubling every 18 months
More informationFoundations of Business Intelligence: Databases and Information Management
Foundations of Business Intelligence: Databases and Information Management Wienand Omta Fabiano Dalpiaz 1 drs. ing. Wienand Omta Learning Objectives Describe how the problems of managing data resources
More informationBIG DATA ANALYTICS REFERENCE ARCHITECTURES AND CASE STUDIES
BIG DATA ANALYTICS REFERENCE ARCHITECTURES AND CASE STUDIES Relational vs. Non-Relational Architecture Relational Non-Relational Rational Predictable Traditional Agile Flexible Modern 2 Agenda Big Data
More information<Insert Picture Here> Extending Hyperion BI with the Oracle BI Server
Extending Hyperion BI with the Oracle BI Server Mark Ostroff Sr. BI Solutions Consultant Agenda Hyperion BI versus Hyperion BI with OBI Server Benefits of using Hyperion BI with the
More informationEnterprise Solutions. Data Warehouse & Business Intelligence Chapter-8
Enterprise Solutions Data Warehouse & Business Intelligence Chapter-8 Learning Objectives Concepts of Data Warehouse Business Intelligence, Analytics & Big Data Tools for DWH & BI Concepts of Data Warehouse
More informationCourse 10977A: Updating Your SQL Server Skills to Microsoft SQL Server 2014
www.etidaho.com (208) 327-0768 Course 10977A: Updating Your SQL Server Skills to Microsoft SQL Server 2014 5 Days About this Course This five day instructor led course teaches students how to use the enhancements
More informationYour Data, Any Place, Any Time.
Your Data, Any Place, Any Time. Microsoft SQL Server 2008 provides a trusted, productive, and intelligent data platform that enables you to: Run your most demanding mission-critical applications. Reduce
More informationChapter 5. Warehousing, Data Acquisition, Data. Visualization
Decision Support Systems and Intelligent Systems, Seventh Edition Chapter 5 Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization 5-1 Learning Objectives
More informationBusiness Benefits From Microsoft SQL Server Business Intelligence Solutions How Can Business Intelligence Help You? PTR Associates Limited
Business Benefits From Microsoft SQL Server Business Intelligence Solutions How Can Business Intelligence Help You? www.ptr.co.uk Business Benefits From Microsoft SQL Server Business Intelligence (September
More informationDatabricks. A Primer
Databricks A Primer Who is Databricks? Databricks vision is to empower anyone to easily build and deploy advanced analytics solutions. The company was founded by the team who created Apache Spark, a powerful
More informationImplement a Data Warehouse with Microsoft SQL Server 20463C; 5 days
Lincoln Land Community College Capital City Training Center 130 West Mason Springfield, IL 62702 217-782-7436 www.llcc.edu/cctc Implement a Data Warehouse with Microsoft SQL Server 20463C; 5 days Course
More informationWednesday, 12 th November 2015 Presenter: Jon Lambert
PRODUCT BRIEF: MICROSOFT MOBILE APPLICATION DEVELOPMENT AUDIT AND COMPLIANCE SOLUTIONS Wednesday, 12 th November 2015 Presenter: Jon Lambert 12 November 2015 Commercial-in-Confidence Communications Design
More informationImplementing a SQL Data Warehouse 2016
Implementing a SQL Data Warehouse 2016 http://www.homnick.com marketing@homnick.com +1.561.988.0567 Boca Raton, Fl USA About this course This 4-day instructor led course describes how to implement a data
More informationG Cloud Services Definition Document. Compliance Service. Invigilatis Limited. Contents. Pages. Invigilatis Applications 1.
G Cloud Services Definition Document Compliance Service Invigilatis Limited Contents Pages Invigilatis Applications 1 Modules 2 Business Intelligence 3 Service Definition 4-6 Service Levels Access Upgrades
More informationOptimizing the Performance of the Oracle BI Applications using Oracle Datawarehousing Features and Oracle DAC 10.1.3.4.1
Optimizing the Performance of the Oracle BI Applications using Oracle Datawarehousing Features and Oracle DAC 10.1.3.4.1 Mark Rittman, Director, Rittman Mead Consulting for Collaborate 09, Florida, USA,
More informationIn-Database Analytics
Embedding Analytics in Decision Management Systems In-database analytics offer a powerful tool for embedding advanced analytics in a critical component of IT infrastructure. James Taylor CEO CONTENTS Introducing
More informationData Warehousing. Jens Teubner, TU Dortmund jens.teubner@cs.tu-dortmund.de. Winter 2015/16. Jens Teubner Data Warehousing Winter 2015/16 1
Jens Teubner Data Warehousing Winter 2015/16 1 Data Warehousing Jens Teubner, TU Dortmund jens.teubner@cs.tu-dortmund.de Winter 2015/16 Jens Teubner Data Warehousing Winter 2015/16 13 Part II Overview
More informationFoundations of Business Intelligence: Databases and Information Management
Chapter 5 Foundations of Business Intelligence: Databases and Information Management 5.1 See Markers-ORDER-DB Logically Related Tables Relational Approach: Physically Related Tables: The Relationship Screen
More informationRachael Narel Engagement Manager Chad Dotzenrod BI Practice Lead SWC Technology Partners
Rachael Narel Engagement Manager Chad Dotzenrod BI Practice Lead SWC Technology Partners #SWCEvents Agenda SWC Introduction Importance of Business Intelligence BI Sicknesses Causes & Symptoms Remedies
More informationDesigning a Dimensional Model
Designing a Dimensional Model Erik Veerman Atlanta MDF member SQL Server MVP, Microsoft MCT Mentor, Solid Quality Learning Definitions Data Warehousing A subject-oriented, integrated, time-variant, and
More informationMAD Skills: New Analysis Practices for Big Data
MAD Skills: New Analysis Practices for Big Data Jeffrey Cohen, Brian Dolan, Mark Dunlap Joseph M. Hellerstein, and Caleb Welton VLDB 2009 Presented by: Kristian Torp Overview Enterprise Data Warehouse
More informationDATA MINING AND WAREHOUSING CONCEPTS
CHAPTER 1 DATA MINING AND WAREHOUSING CONCEPTS 1.1 INTRODUCTION The past couple of decades have seen a dramatic increase in the amount of information or data being stored in electronic format. This accumulation
More informationCourse Outline: Course: Implementing a Data Warehouse with Microsoft SQL Server 2012 Learning Method: Instructor-led Classroom Learning
Course Outline: Course: Implementing a Data with Microsoft SQL Server 2012 Learning Method: Instructor-led Classroom Learning Duration: 5.00 Day(s)/ 40 hrs Overview: This 5-day instructor-led course describes
More informationAssignment # 1 (Cloud Computing Security)
Assignment # 1 (Cloud Computing Security) Group Members: Abdullah Abid Zeeshan Qaiser M. Umar Hayat Table of Contents Windows Azure Introduction... 4 Windows Azure Services... 4 1. Compute... 4 a) Virtual
More informationWhat s New with Informatica Data Services & PowerCenter Data Virtualization Edition
1 What s New with Informatica Data Services & PowerCenter Data Virtualization Edition Kevin Brady, Integration Team Lead Bonneville Power Wei Zheng, Product Management Informatica Ash Parikh, Product Marketing
More informationSAP and Hortonworks Reference Architecture
SAP and Hortonworks Reference Architecture Hortonworks. We Do Hadoop. June Page 1 2014 Hortonworks Inc. 2011 2014. All Rights Reserved A Modern Data Architecture With SAP DATA SYSTEMS APPLICATIO NS Statistical
More informationWeek 13: Data Warehousing. Warehousing
1 Week 13: Data Warehousing Warehousing Growing industry: $8 billion in 1998 Range from desktop to huge: Walmart: 900-CPU, 2,700 disk, 23TB Teradata system Lots of buzzwords, hype slice & dice, rollup,
More informationBuilding a BI Solution in the Cloud
Building a BI Solution in the Cloud Stacia Varga, Principal Consultant Email: stacia@datainspirations.com Twitter: @_StaciaV_ 2 SQLSaturday #467 Sponsors Stacia (Misner) Varga Over 30 years of IT experience,
More informationBusiness Intelligence for SUPRA. WHITE PAPER Cincom In-depth Analysis and Review
Business Intelligence for A Technical Overview WHITE PAPER Cincom In-depth Analysis and Review SIMPLIFICATION THROUGH INNOVATION Business Intelligence for A Technical Overview Table of Contents Complete
More informationHarnessing the Power of the Microsoft Cloud for Deep Data Analytics
1 Harnessing the Power of the Microsoft Cloud for Deep Data Analytics Today's Focus How you can operate your business more efficiently and effectively by tapping into Cloud based data analytics solutions
More informationImplementing a Data Warehouse with Microsoft SQL Server 2012
Implementing a Data Warehouse with Microsoft SQL Server 2012 Module 1: Introduction to Data Warehousing Describe data warehouse concepts and architecture considerations Considerations for a Data Warehouse
More informationLection 3-4 WAREHOUSING
Lection 3-4 DATA WAREHOUSING Learning Objectives Understand d the basic definitions iti and concepts of data warehouses Understand data warehousing architectures Describe the processes used in developing
More informationG Cloud Services Definition Document. Property Management Service. Invigilatis Limited. Contents. Pages. Invigilatis Applications 1.
G Cloud Services Definition Document Property Management Service Invigilatis Limited Contents Pages Invigilatis Applications 1 Modules 2 Business Intelligence 3 Service Definition 4-6 Service Levels Access
More informationScalable Enterprise Data Integration Your business agility depends on how fast you can access your complex data
Transforming Data into Intelligence Scalable Enterprise Data Integration Your business agility depends on how fast you can access your complex data Big Data Data Warehousing Data Governance and Quality
More informationData Virtualization and ETL. Denodo Technologies Architecture Brief
Data Virtualization and ETL Denodo Technologies Architecture Brief Contents Data Virtualization and ETL... 3 Summary... 3 Data Virtualization... 7 What is Data Virtualization good for?... 8 Applications
More informationData Warehousing and Data Mining in Business Applications
133 Data Warehousing and Data Mining in Business Applications Eesha Goel CSE Deptt. GZS-PTU Campus, Bathinda. Abstract Information technology is now required in all aspect of our lives that helps in business
More informationErnesto Ongaro BI Consultant February 19, 2013. The 5 Levels of Embedded BI
Ernesto Ongaro BI Consultant February 19, 2013 The 5 Levels of Embedded BI Saleforce.com CRM 2013 Jaspersoft Corporation. 2 Blogger 2013 Jaspersoft Corporation. 3 Linked In 2013 Jaspersoft Corporation.
More information