WHAT HAPPENS WHEN BIG DATA MEETS THE CLOUD

Transcription

1 E-Guide WHAT HAPPENS WHEN BIG DATA MEETS THE CLOUD SearchData Management

2 B oth big data and the cloud are recent emergences in the IT world, and they have the ability to intersect in the most impactful way. In this e-guide, learn how to choose the best cloud service or cloud that best meets your organization s needs. Additionally, learn how Cazena, a startup company, was able to solve its pain points with managing and analyzing big data in the cloud. PAGE 2 OF 11

3 HOW TO CHOOSE THE BEST CLOUD BIG DATA PLATFORM Tom Nolle, CIMI Corporation Big data and the cloud are each revolutionary IT concepts, and their intersection is doubly so. Today, users and cloud providers alike are focusing on that intersection, planning applications and service offerings to exploit the technologies. To address the intersection of big data and the cloud, and to choose the, cloud planners need to decide on a database model, select between cloud database services and cloud database s, and review the features of each against their own special needs. There are three popular models for big data: the distributed Map Reduce model popularized by Hadoop; the NoSQL model used for non-relational, nontabular storage; and the SQL RDBMS model for relational tabular storage of structured data. You can use all three in the cloud, so in most cases it will be less cloud consideration and more database design and usage concerns that determine the model choice. Once you've identified a database model you can explore cloud options for the model selected. Most business transactions are best stored and accessed in RDBMS form, PAGE 3 OF 11

4 where SQL queries and tabular summarization can be supported easily. This is the model that both enterprise users and database architects are most likely to be familiar with, and a good rule of thumb is to assume SQL/RDBMS until you can prove another option is better. The most common "don't use SQL" determinant is that the data to be stored is object-structured rather than tabular in nature. Object data collects information for a given entity (the object) as a set of properties that may be relatively free-form within the object. Although the object typically has a unique "key," the properties may or may not relate among objects. If you can't visualize data as a set of tables with fixed fields and valuable field-to-field relationships, then SQL/RDBMS may be difficult to adopt, and one of the other options may be better. HADOOP AND NOSQL WORK WITH NON-STRUCTURED DATA Both the Hadoop and NoSQL options are easier to adapt to non-structured data. Hadoop is ideal for applications where unstructured data is stored in clusters distributed on a network. NoSQL can also be adapted to this cluster distribution, so in most cases the "object-structured" question will help determine the best choice between Hadoop and NoSQL models. If the database stores PAGE 4 OF 11

5 information about specific, identified things in the form of "thing-properties," then NoSQL is likely best. Data that has no natural structure, like free-form text, is likely better stored using Hadoop. Note that, generally, you can query SQL, NoSQL and Hadoop databases using SQL. The latter two may require an overlay product, and the lack of natural tabular organization may render query processing more time-consuming. If you expect most database activity to be in SQL form, you probably have tabular data and should be considering an SQL/RDBMS model. The second point to consider is whether to use a database package from a cloud provider or to host your own database in the cloud. Amazon, Rackspace, Microsoft and Google are well-known providers of cloud big data services. Joyent and Qubole are examples of lesser-known providers with strong big data credentials. Hadoop is usually available from major cloud providers, and other database models may also be supported as cloud services. TAKING A DO-IT-YOURSELF APPROACH If you can't get a public cloud big data service you still have the option of hosting your own big data application in the cloud using a public cloud infrastructure as a service or as a service and big data software you've selected. PAGE 5 OF 11

6 A do-it-yourself approach can offer advantages even when you can get big data services. First, it widens your options for cloud hosting because not all cloud providers will support big data as a service. Second, you can use multiple public clouds or switch between cloud providers with greater ease. Third, often you can create hybrid big data applications more easily if you can adopt the same big data software in the cloud and on-premises. The disadvantage, according to cloud buyers, is that creating in-cloud big data with your own tools is more complicated and sometimes more costly. Obviously, the best s for cloud big data depend on your database model. Top-rated Hadoop options include Apache Hadoop, SAP's HANA/Hadoop combination, Hortonworks, Hadapt and VMware's Cloud Foundry, as well as services provided by IBM, Microsoft and Oracle. For NoSQL, consider Cassandra, Hbase or MongoDB. IBM also offers NoSQL for the cloud, and there are plenty of other NoSQL providers. For the latter, check to be sure that they support the level of big data cloud scaling you expect. SQL big data in the cloud is probably most often supported by extending your on-premises SQL vendor offering. IBM, Oracle and Microsoft all offer SQL that's suitable, with some tuning, for big data cloud deployment. HP's Haven is a general big data architecture for the cloud that embraces both structured PAGE 6 OF 11

7 and unstructured data and that supports SQL queries. UNDERSTANDING YOUR NEEDS IS CRITICAL To choose the right cloud big data, you need to understand your needs so that you can evaluate how each supports your needs. Big data applications can be balanced between query and update, they can be mostly transactional, mostly analytic, etc. You may need to run tests to determine whether a given big data option is efficient for your specific mix of update and access. Be particularly careful about SQL queries against non-sql databases. Analytics that require extensive use of SQL can create major performance issues even with traditional RDBMS, and more so with other database models. Creative database design and careful use of JOINed databases may make things more efficient. You also should ensure that distributed big data clusters can be accessed efficiently for combined queries. This can be complicated with cloud-hosted data because users have only limited control over how the data is distributed. A combination of testing to determine optimum data distribution strategies and a contract to ensure data stays at least generally within those distribution guidelines is critical. PAGE 7 OF 11

8 Cloud big data hosting has considerable variables, so be prepared to gather a lot of operating data on quality of experience to ensure that workers are getting what they need, and that costs are managed. Otherwise you'll end up with something too costly to fix and to slow to accept. STARTUP CAZENA TALKS UP BIG DATA IN THE CLOUD AT AWS RE:INVENT Scot Petersen, Editorial Director The mantra was on everybody's lips at AWS 2014 this week: The cloud is no longer something enterprises are merely considering or poking around in. Now, said John Treadway, a senior vice president at Cloud Technology Partners Inc., organizations are asking, "How quickly, how fast, how do I manage it?" "The saying that the cloud genie is out of the bottle, so you'd better start thinking what to wish for, is now true," Treadway said. Still, as SearchDataManagement has been following closely of late, the PAGE 8 OF 11

9 forecast for big data in the cloud remains hazy, despite the fact that it's precisely the kind of business value big data can offer that makes it so tantalizing in combination with the virtually unlimited resources of s such as Amazon Web Services. An interesting case in point is Cazena, a stealth-mode startup in Waltham, Mass., that involves some of the founders and other former top executives of Netezza, a data warehousing company that is now part of IBM. Cazena executives, including CEO Prat Moghe, were at to discuss a new survey and report on -- what else? -- how enterprises will use the cloud for big data analytics. Cazena commissioned the report, which was co-written by Gigaom analysts Lynn Langit and Andrew Brust. The bottom line of the report is that yes, enterprises are getting on board with managing big data in the cloud, but on their terms. According to Moghe, most companies want new big data s but also want to continue to use the same types of traditional business intelligence and analytics tools that they're comfortable with. "They don't want sexy data science," he said in an interview, adding that the survey shows many organizations "are exploring bread-and-butter analytics in the cloud." It's a valid point. Users often can get lost in whiz-bang technology and PAGE 9 OF 11

10 forget what all the data is for -- to add value to the business and for customers. None of that should change simply because of where the data is located. According to the report on the survey, "standard BI tools (55%) and analytics tools (49%) are the preferred tooling types for analytics projects that have been migrated to the cloud." The preference for sticking with familiar BI and analytics software isn't surprising, the report said: "Consistent with the general tone of the results in this survey, enterprises value minimal disruption and prefer reusing skills, tools and techniques whenever possible." So what does Cazena plan to do? Prospective users will have to wait until early next year to find out for sure. The startup, which has raised $8 million in funding from venture capital firms Andreessen Horowitz and North Bridge Venture Partners, is vague on what its view of big data in the cloud will translate into, but the signs point to a "big data as a service" offering. "Our vision," Moghe said, "is to ultimately make big data available on demand to folks who need it." PAGE 10 OF 11

11 FREE RESOURCES FOR TECHNOLOGY PROFESSIONALS TechTarget publishes targeted technology media that address your need for information and resources for researching products, developing strategy and making cost-effective purchase decisions. Our network of technology-specific Web sites gives you access to industry experts, independent content and analysis and the Web s largest library of vendor-provided white papers, webcasts, podcasts, videos, virtual trade shows, research reports and more drawing on the rich R&D resources of technology providers to address market trends, challenges and solutions. Our live events and virtual seminars give you access to vendor neutral, expert commentary and advice on the issues and challenges you face daily. Our social community IT Knowledge Exchange allows you to share real world information in real time with peers and experts. WHAT MAKES TECHTARGET UNIQUE? TechTarget is squarely focused on the enterprise IT space. Our team of editors and network of industry experts provide the richest, most relevant content to IT professionals and management. We leverage the immediacy of the Web, the networking and face-to-face opportunities of events and virtual events, and the ability to interact with peers all to create compelling and actionable information for enterprise IT professionals across all industries and markets. PAGE 11 OF 11