Querying MongoDB without programming using FUNQL

Transcription

1 Querying MongoDB without programming using FUNQL

2 FUNQL? Federated Unified Query Language

3 What does this mean? Federated - Integrates different independent stand alone data sources into one coherent view Unified One query for different database technologies: Relational, Document Oriented, Object Oriented, Key/Value (yes its possible!) Query Language Domain: Filtering, data transformation, aggregation. Can be executed interatively. No programming knowledge required.

4 When to Use Ad-Hoc queries When Map/Reduce is overkill When you don't want to get lost in $}': { syntax Data integration Business Intelligence Pre aggregation

5 Benefits for Mongo Easier use of MongoDB and quicker completion of projects. Standardizes the heterogeneous NoSQL Market. Protects customer investments by making MongoDB part of a larger market. Enlarges the pool of skilled users. Learning environment for non-technical users and SQL users. Co-existence with relational databases and migration made easy.

6 Benefits for developers Skill applicable to different database systems. Faster development. Easy testing and data cleaning. Larger market for developer skills. Protects your investment in learning NoSQL. Fast prototyping.

7 Benefits for developers Skill applicable to different database systems. Faster development. Easy testing and data cleaning. Larger market for developer skills. Protects your investment in learning NoSQL. Fast prototyping.

8 Example from collection where object.subobject.field = $parameter select field, { nested, fields, } group by field, aggregates, sub_groups where condition // on aggregate //= having into // File (json/xml) or other database/collection

9 From - Where - Select why it makes a big difference to SQL's select - from - where Seemingly unimportant syntactical sugar But it completely changes the execution model of a query. Simplifies it from declarative to data-flow Each statement is a pipeline, reading data from the previous statement and passing (modified) data to the next. Can easily be executed in parallel. Not invented here: Pioneered by Microsoft with Linq

10 Dealing with relational data The join operation fits into the same model. from table1 join table2 on table1.id = table2.foreign_id Is now just an pipeline stage, that adds the data from table2 into the dataflow. Tables can come from different systems. E.g one is from MongoDB, the other from DB2 Join grouped operator returns a 1-n relationship a a collection of nested documents.

11 Nested/Recursive Data Structures flatten operator transforms a (fixed) nested structure into one linear table. enumerate linearizes (walks through) an arbitrarily deeply nested tree. group turns a linear structure into a nested structure. Funql understands database references. You can use dot-notation for navigation.

12 Aggregation The model allows much more advanced aggregation operations than SQL Multi dimensional grouping Mixing of normal select and aggregation Analytics directly in the query Can leverage MongoDBs new aggregation operators

13 Funql and the Aggregation Framework Both use the same pipeline model select statement == mongos project flatten stetement ~~ mongo's unwind range == first, last Simple mapping, so Funql can operate at Mongo's native speed. db.zips.aggregate({$group:{_id: '$city', pop: {$sum: '$pop'}}}, {$sort: {pop: -1}}}) from zips group by city, sum pop as pop sort by pop descending

14 Status Formal language spec at reportsanywhere.com/pebble funql.org is coming up Reference implementation in Java currently for MongoDB Generic Json data Cassandra (coming) Relational Databases (coming) Fork it on Bitbucket (coming up)

15 Architecture Core Engine Can filter, select, join, sort, group, aggregate Extremely simple driver API: 5 Methods. open/close, iterate, lookup, range Not necessarily optimized Advanced driver API: Examines the syntax tree and invokes native operations.