Tables in the Cloud. By Larry Ng

Transcription

1 Tables in the Cloud By Larry Ng The Idea There has been much discussion about Big Data and the associated intricacies of how it can be mined, organized, stored, analyzed and visualized with the latest software, database and hardware mega infrastructures. While the dilemma of Big Data has captivated the attention of the industry, relatively little attention has been paid to the challenge of managing low-tech digital data. Why? Because the simple table, constructed of rows of data across columns of fields, remains the most popular tool to manage and use digital data. In this era of rapid technology evolution the ubiquitous simple table continues to withstand the test of time. The table is under appreciated. The humble table has been with us for millennia. We first learned it in grammar school. It is the foundation of spreadsheets and relational databases. Its power lies with its beauty and simplicity. It is understood by all and used by everyone. The simple, two dimensional table is the most common construct for organizing and managing data. This paper posits that a cloud based service to share tables from publishers to users across a broad range of content sets would be an interesting experiment in making digital data more readily accessible. Data Everywhere Most organizations have plenty of data. The problem is that this data is typically scattered across hundreds of systems, sub-systems and databases and thousands of files and spreadsheets. The information provided by this data can be enormously helpful to an organization if managed efficiently. Optimally, those who require the data should have easy access to it and those who create the data should possess the ability to easily publish and distribute it to others. Empowering and sharing data throughout the company speeds information flow, decision making and improves overall efficiency. The evolution of the Internet is a good case study of how this problem was solved in the past. Before the World Wide Web (www), the Internet was complicated and confined to universities and computer scientists. The invention of the browser and HTML (Hyper Text Markup Language) simplified the paradigm and made the Internet accessible to the masses. HTML allowed everyone to publish documents with some text, images and links to other documents with some basic formatting tags. Anyone can pick up a book, and in a few short hours, learn how to publish a document on the Internet. The browser would render these documents into visual appealing web pages. The beauty of this early Internet was that it allowed millions of people to easily create and publish content, without the help of computer experts. The browser allowed others to consume this content with an excellent user experience. The catalyst that ignited the tremendous success 1

2 of the Internet was its simplicity. Billions of people now consume content created by millions of people throughout the world. The democratization and availability of content everywhere required the beauty and elegance of a simple solution. An elegantly simple solution is perhaps the big idea for structured digital data. Tables in the Cloud Almost everyone with a personal computer knows how to use two very popular applications: word processors and spreadsheets. Word processors and text editors are the applications of choice for creating text documents. The spreadsheet is the application for undertaking data analysis. Digital data is a natural fit for the innately tabular spreadsheet. Once captured within a spreadsheet the data can be filtered, sorted and analyzed. Formulas and calculations can be easily applied on the dataset. Tabular data presents very well from a visual perspective, especially data containing numbers. Similarly tabular in design, the relational database is the most popular database architecture to store and manage digital data. A table, consisting of rows of data across columns of fields, is the basic building block of relational databases. Given that much of the world s digital content is either stored in spreadsheets or databases, it seems logical that the simple table should be an excellent mechanism for publishing and distributing content. Unlike XML (extended Markup Language), which appears complicated and abstract to non-techies, tables are easy to see, touch and understand. Tables have the following nice characteristics: Everyone is familiar with tables Most digital data is already stored within tables Tables are easy to create, visualize and use The idea is to create a platform whereby owners and creators of digital content can easily publish their data in tables for others to view and retrieve. The tables may be static or real-time streaming, public or private and updated using Excel, XML or an API. Anything that has rows of data across columns of fields may be stored in the cloud and be made readily accessible for consumption. Unlike many other forms of content, this structured digital data may be digested, managed and analyzed by both humans and computers. The rest of this paper discusses how such a platform may be created, borrowing on the ideas and successes of other content publishing platforms. The project is called Tables in the Cloud. 2

3 Users There are a few key factors that are minimally required to attract users. First and foremost, the content must be good. Without high quality content, users will get frustrated quickly, and will likely never revisit the site. Therefore, there must be a mechanism to measure content quality. Many successful sites have chosen to use the community to review the quality of the contributed content, and to provide feedback to the publishers. Rating systems, likes, comments and rankings have been effective in letting the community review, critique and provide feedback for the benefit of other users. The site must be extremely easy to use. The mantra is the design principle KISS keep it simple, stupid. The extreme case of this design was Google during the early days, which just had a logo and a search box. Though Google has much more capability today, the home page is still basically the same. Simpler is always better. A more recent requirement is that the content should be accessible by mobile devices and tablets. The trend towards mobile and portable devices is accelerating. People need content at work, at home or on the road, anytime, anywhere. This requirement drives the need to make the platform a cloud based service. Content should be stored centrally and be accessible from everywhere. Users like aggregation. They prefer the convenience of getting their content from a few sources. That is why there is typically one, or at most, a few large platforms that dominate various forms of content (e.g., Wikipedia, Amazon, itunes, AppStore, YouTube, WordPress). People prefer getting lots of stuff in one place as opposed to many places malls, box stores, supermarkets, multiplexes, museums, restaurant rows, Paris and Disney World. At first this aggregation seems unnatural to publishers, since they typically prefer to exhibit their content without having competitors nearby. But the strong consumer preference for aggregation convinces them that doing business alongside competitors is better than having no business at all. And lastly, the end-user value proposition and experience must exceed their expectations and be significantly better than that which has become customary to them. Though not necessarily requiring a paradigm shift, a little better will not cut it. To be successful and gain wide acceptance and adoption, the user experience must be substantially better. In summary, to have any chance of success with users, the site must: have high quality content be easy to use be widely accessible anytime, anywhere on any device aggregate content from multiple sources, including competitors offer a superior value proposition and user experience 3

4 Publishers Most of the successful content-driven sites provide publishers excellent and easy to use tools to publish their content to the platform. It is imperative for the site to be publisher friendly. Ideally, publishers should require a minimal amount of work to contribute their content. Secondly, publishers want flexibility and control over their content. They want to have control over the parameters of what, when, who and how. Though there should be some rules to restrict inappropriate or illegal content, the overall tenet should be that publishers are in charge of their own data and who is allowed to see and use it. Publishers must have the ability to package their content, charge fees and entitle those who are authorized to access it. Groups of users may be granted access to specific sets of content. At their discretion, publishers may choose to publish data to the public domain (that is, everyone on the Internet has access), to registered users (only those who have a user-name and password) or private users (those who are explicitly authorized by the publisher). There should also be good administrative and reporting tools for publishers to capture metrics and manage and track the usage and performance of their content. This feedback is invaluable for publishers to improve their content. In summary, publishers will minimally need: easy to use publishing tools flexibility and control of their content the ability to package, charge and authorize access to content the ability to publish to the public domain, registered users or private users the ability to track usage and performance 4

5 Features The following are the initial thoughts on key features for the platform. Tables the content should fit in a simple table of rows and columns. The beauty of simplicity is important during the early stages of getting this concept off the ground. Who would have thought 140 character text tweets on Twitter would unite people, stir movements and change the world. Excel and XML publishers should be able to easily publish content using the two most popular tools for structured data, Excel and XML. Basically, if it fits on a spreadsheet, they should be able to publish it with a minimal amount of effort. Real-time contributions will need to use a protocol or Excel add-in. Domain each publisher will be assigned a unique domain name as a publisher. All content on the network will be attributed to the publisher s name. Publishing Tools publishers will have access to a set of tools to manage their content and monitor its usage and performance. Snapshot and Real-time data may be published as either snapshot tables or streaming, real-time tables. Real-time support for content is a growing trend, and is especially important in some industry sectors such as the financial markets. Cloud the content should be stored in the cloud. Publishers should be able to publish once, and have their data available everywhere. Login publishers and users should use their address as their login id. It is unique, convenient and easy to remember. Mobility the content should be easily accessible on the web, tablets and mobile devices. Digital Data the data should be downloadable digitally either directly into Excel or an XML document. Also, advanced users may subscribe to a real-time feed using an API. The value proposition of digital data is that it is organized to be easily consumed electronically with no fuss or mess. Public, Registered or Private publishers should control whether their data is made available to the public domain (the entire Internet at large), to registered users ( user-name and password) or privately to a group of authorized users specified by the publisher. Transparency the platform should be transparent as far as usage statistics, ratings and rankings. Scalability the platform needs to be robust and highly scalable, to support two vectors of growth amount of content and number of users. 5

6 Conclusion Easily available structured, digital data is unchartered territory in the Internet content space. It seems cumbersome and complicated to most people. It is the domain of the techies and spreadsheet jocks. It parallels the feeling of computers before the PC or networks before the Internet. The democratization of digital data needs a simple cloud based solution. The humble and unassuming table may be the catalyst to unlock the value of organized data. Unlike freeform text, tabular data wants to be touched, manipulated and analyzed. Let s start a conversation, perhaps build a prototype, and see where it goes. 6