Big Data Solutions. Portal Development with MongoDB and Liferay. Solutions

Transcription

1 Big Data Solutions Portal Development with MongoDB and Liferay Solutions

2 Introduction Companies have made huge investments in Business Intelligence and analytics to better understand their clients and their business. Reporting infrastructure, query tools, dashboards and data visualization have all helped companies visualize and use their data to make better decisions. Now companies are hearing about Big Data. More than just a popular tag, Big Data represents a remarkable move in the data available to companies: Much more data, of different types, coming at a dramatic pace. Companies of all sizes and in all fields wonder if Big Data is a game changer for them as it is for others. To address the different issues, companies need robust solution to face these challenges, to take Big Data and use it to deliver more customer value, faster. They need a new approach and a new platform, one focused on using Big Data analytics to drive customer value. Know Key Drivers of Big Data In 2010, the world produced over 1 zettabyte of data; by 2014, we will produce approximately 7 zettabyte a year. Much of this data surge is caused by a significant increase in devices located at the periphery of the network including embedded sensors, tablet, and smartphones. Not just Volume 85% users using it for 3 TB or less No need to have large data to leverage the benefits of Big Data's modern architecture and solution Dimensions of Data Volume Data volume is the primary attribute of big data. Compared to non-traditional data, machine-generated data is generated in much larger amounts. Enterprises are awash with ever-growing data of all kinds, simply accumulating terabytes-even petabytes-of information. Velocity One factor that actually makes big data gigantic is the different sources. The daily addition of millions of connected devices (tablets and smartphones) will upsurge not only volume but also velocity. More and more data is produced each day that has to be collected in shorter time frame, as data becomes out-of-date very swiftly. Variety From various sources such as text, video, audios, log files, click streams, etc. both structured and unstructured data exhibit a dizzying rate of change. As new services are integrated, or new marketing campaigns carried out, new data types becomes essential to grab the resulting information.

3 Complexity Big Data is an opportunity to discover insights in fresh and growing types of content and data, to make your business more agile, and to resolve issues that were earlier considered challenging. It has become vital to reduce the complexity to improve accessibility of the data as per customer needs. MongoDB over RDBMS Typical, so-called relational, databases are made for a world in which data is scattered, and therefore can be and will be curated properly. It is a community in which the questions one would like to answer using the data must be clear at the start, to ensure that the database is created to answer them - and only them - successfully.

4 Limitations of RDBMS Its support is limited to Terabyte only. It does not support Petabyte and Manages only Structured Data; doesn't support Semi Structured and Unstructured Data RDBMS doesn't scale inherently - Scale up/scale out (Load Balancing & Replication) Hard to Shard/Partition Unable to manage large Data Files Both Read/Write not possible - With large set of data, it becomes quite difficult to bring data from the data layer into the analytics layer to perform computation Relational database theory is not ideal for geospatial applications and decent relational geospatial databases are expensive and normally tough to maintain. But with a flexible document database like MongoDB it simply becomes easier to iterate toward Big Data insights. Open-source technology like MongoDB can be easily applied to the Big Data problems. MongoDB is a scalable, high-performance, open-source, document-oriented database. MongoDB is an open-source document database, and the leading NoSQL database, written in C++. As soon as one remembers that Big Data is a question of both storage and processing, it makes sense that the top operational data store would be MongoDB, given its flexibility and scalability; it rejects the orthodox approach of relational databases. Speed, Scalability, and Performance to Handling Unstructured Data is the major reason why MongoDB has become a complimentary to RDBMS. Furthermore, it's much simpler to code against MongoDB compared to relational databases for developers.

5 Good things about MongoDB Open source, written in: C++ Uses memory mapped files for data storage Main point: Retains some friendly properties of SQL. (Query, index) License: AGPL (Drivers: Apache) Performance over features Journaling (with --journal) is best turned on Protocol: Custom, binary (BSON) On 32bit systems, limited to ~2.5Gb Master/slave replication (auto failover with replica sets) An empty database takes up 192Mb Sharding built-in Queries are JavaScript expressions GridFS to store big data + metadata (not actually an FS) Has geospatial indexing Run arbitrary JavaScript functions server-side Better update-in-place than CouchDB Data center aware Document-oriented storage (Based on JSON documents)

6 NoSQL The Right Solution With an increasing amount of data, it has become inevitable to use the next generation database i.e. non-relational, distributed, open-source and horizontally scalable. NoSQL, often termed as 'not only sql', is a whole new way of thinking about a database. It is not a relational database and it does not provide full ACID (atomicity, consistency, isolation, durability) guarantees, but yet has a distributed and error tolerant architecture; and therefore, it is the best solution for all situations. Following characteristics relates to NoSQL: Chema-free Easy replication support Simple API Eventually consistent / BASE (not ACID) Gives high query performance Supports a huge amount of data and more Most NoSQL databases are open source

7 NoSQL Data Models 4 broad Data models 120+ variants available in the market Concept of Big Data Portal 90% of the data in this modern time has been created in the past two years alone and every day, we create 2.5 quintillion bytes of data that comes from everywhere: posts to social media sites, sensors used to collect weather information, digital images and videos, purchase transaction information, and mobile phone GPS signals to name a few. In such scenario, companies need a web based solution that integrates the robust presentation features of a portal such as rich user interfaces, collaboration, and secure access, with centralized & enormously scalable data storage as the back end, composing of different types of content (Images, Audio, Video, Documents, Metadata) in huge amount. Organizations are leveraging Big Data for its sophisticated architecture solution. Concept of Big Data Portal has been defined to harness the potential of Liferay & MongoDB.

8 Liferay Portal is a free and open source enterprise portal written in Java that allows users to set up features common to websites. Even though Liferay provides an advanced programming interface for developers, no programming expertise are required for basic website installation and management. MongoDB is a highly scalable, NoSQL and agile document-oriented database based on JSON-like document storage with dynamic schemas. The integration of Liferay and MongoDB result in a powerful, cost effective solution: i.e. "BIG DATA PORTAL" Big Data portal caters to the ever growing need to manage the huge amount of information available.

9 Benefits: a) How does MongoDB enhance Liferay? b) How does Liferay enhance MongoDB? How does MongoDB enhance Liferay? Scalability Leverages Auto sharding & replica set features Elasticity in scaling storage - go up or down Cost Effectiveness Commodity Hardware Eliminates need for high-end storage systems Agility & Performance Faster Development Easier Deployment Flexible & Schema less

10 Large Object Storage & Centralized Data Management GridFS enables large binary objects like images, Video or Audio Simplifies Management of data Single system to manage structured & unstructured data How does Liferay enhance MongoDB? Rich Front End Powerful websites consisting of Gadgets & Portlets - Portions of a Web page that may be Complete application Pages & Themes - Common, Consistent look & feel across multiple pages Navigation - Menu bar, Tabs, Links Secure Views to data Role based Site based Login status based Mobile Integration Data access on the go Different Themes for Mobile - HTML5, CSS3 Flexible Architecture and Lean Platform Use of Open standards, Web services and integration tools SOA Other Potential Enhancements Dynamic Data Lists End Users define custom entities to store into database, forms for capture and display Project Status Surveys/Polls Arbitrary, exensible, typed data entry CMS, Document Library Custom Content Types Huge volumes of unstructured, user-generated content

11 Solutions and features of web portal Portal provides secure user interface to manage all sorts of content such as texts, video and images. The portal provides administrators to configure fine grained access control to the site. The portal also provides a scalable & flexible data storage to scale for ever growing variety of content. Technical Architecture Portal provides secure user interface to manage all sorts of content such as texts, video and images. The portal provides administrators to configure fine grained access control to the site. The portal also provides a scalable & flexible data storage to scale for ever growing variety of content.

12 NoSQL is not always a replacement of RDBMS. Its rather complimentary to RDBMS. So it's important to understand your data first before you choose to implementating NoSQL. NoSQL is perfect when you need CMS, Document Library Handle unstructured data Schema-less persistence High performance Replication Low cost Conclusion MongoDB enables Portals for scalability (for huge volumes of data) and flexibility (schema-less content). Liferay's rich user interface, security, content management, social and mobile features complement MongoDB's powerful storage features. Big Data Portal with Liferay and MongoDB provide higher ROI to enterprises and lower TCO. About Softweb Solutions Software Solutions is a leading Software Development Company that serves clients worldwide of all sizes. We have a strong performance record in offering quality software development services demonstrated by ISO 9001:2008 certification and Microsoft Gold Certified partnership. We provide cutting edge Mobile App Development solutions for various platforms like Apple ios, Android OS, Blackberry and Windows Phone. We attempt to deal with constantly evolving business needs by incorporating all our strengths in our solutions. To know more about us and how we deliver mobility solutions to the organizations across the world. Contact our representative or us at info@softwebsolutions.com You can also visit our corporate website