Current Data Security Issues of NoSQL Databases

Transcription

1 1 Current Data Security Issues of NoSQL Databases January 2014 PAGE 1 PAGE 1 1 Fidelis Cybersecurity 1601 Trapelo Road, Suite 270 Waltham, MA 02451

2 Abstract NoSQL databases, sometimes referred as Not-- Only-- SQL databases, have recently gained much attention and popularity because of their demonstrated high scalability and performance. The primary advantage of NoSQL databases is that they are designed to efficiently store significant amounts of unstructured data. Facing the "Big Data" problem that has challenged most traditional relational database management systems (RDBMS), major Web 2.0 companies have developed or adopted different flavors of NoSQL databases for their growing data and infrastructure needs, including Amazon (Dynamo), Google (BigTable), LinkedIn (Voldemort), Face book (Cassandra), etc. From their inception, NoSQL databases have been designed for solving the Big Data issue by utilizing distributed, collaborating hosts to achieve satisfactory performance in data storage and retrieval. Other equally important database requirements, such as data security and consistency, have not been fully addressed. Following a previous study published in 2011 [Ref.1] that identified several NoSQL security issues, this white paper summarizes an open-- source research on recent NoSQL improvements on data security issues, as dictated by the PCI-- DSS compliance. With the help of third-- party-- security solutions, some of current NoSQL databases seem able to achieve the PCI-- DSS compliance. However, the potential data inconsistency among replications may impede a wide acceptance of NoSQL by much less-- tolerable financial applications. It is generally conceived for the foreseeable future NoSQL and RDBMS are co-- deployed to process different data flows in the ways they are best designed to do. Potentially, NoSQL databases may replace RDBMS once they are continuously improved to provide sufficient data security. PAGE 2 PAGE 2

3 Relational and NoSQL Databases There are three basic requirements for databases management systems, confidentiality, integrity and availability. The stored data must be available when it is needed (availability), but only to authorized entities (confidentiality), and only modified by authorized entities (integrity). Traditional relational database management systems (RDBMS), like Oracle, SQL and MySQL, have been well-- developed to meet the three requirements. In addition, enterprise RDBMS are further required to have ACID properties, Atomic, Consistency, Isolation, and Durability, that guarantee that database transactions are processed reliably [Ref. 2]. With such desirable properties, RDBMS have been widely used as the dominant data storage choice. RDBMS now are facing major performance problems in processing exponential growth of unstructured data, such as documents, e-- mail, multi-- media or social media. Thus a new breed of non-- relational, cloud-- based distributed databases, called NoSQL, has emerged to satisfy the unprecedented needs for scalability, performance and storage. Currently there are about 150 different NoSQL databases available [Ref. 3]. They are designed to achieve the desired scalability and performance by sharing a BASE transaction concept (Basically Available, Soft state, and eventually consistent). Under this concept, committed transactions are not written to database immediately to achieve data consistency as in RDBMS. Instead, the database just needs to reach a consistent state eventually among the clustering hosts. Based on the data storage model, NoSQL databases generally can be categorized into the following four groups [Ref. 4 and 5], Key-- Values Databases: Store un-- interpreted arbitrary data values into a system that can be recalled later using a key (hash). This schema less data model allows for easy scaling and very simple APIs for implementations. Column Databases: Store data in a similar key-- value fashion, except the key is a combination of column, row, and/or timestamp, which points to one or multiple columns (Column Family). The column family used here is like a table commonly found in a relational database. Document Databases: Store documents that consist of one or more self-- contained named fields in each document, like JSON or BSON format. The structure of documents is dynamic that allows for free modification with the ability to add or remove fields of existing documents. Indexing on the named fields enables fast data retrieval. PAGE 3 PAGE 3

4 Graph Databases: Store data in a flexible graph model that scales across multiple machines. This model is suitable for data with relations that are best represented as a graph (elements interconnected with an undetermined number of relations between them), such as social relations, public transport links, road maps or network topologies. Illustrated in the diagram below are the characteristics of RDBMS and NoSQL databases being scaled up in both data size and data complexity. While RDBMS are limited in both aspects, NoSQL databases with simpler data models, e.g., key- - value and column are more easily scaled up in the data size. Fig. Relative scalability in data size and complexity of RDBMS and NoSQL The following table shows a sample list of well-- known companies that uses NoSQL databases for production needs. Young internet media and social network companies are more readily to accept NoSQL because of their needs for data flexibility and scalability. For example, Netflix in 2013 migrated completely its streaming services from Oracle to NoSQL (Cassandra) to improve availability [Ref. 6]. Other well-- established companies are relatively slower in transitioning to NoSQL, possibly burdened by legacy data storage and/or applications, in addition to still lingering concerns about NoSQL data security. PAGE 4 PAGE 4

5 Table1. Examples of major companies using NoSQL databases Company Name NoSQL Name NoSQL Storage Type Adobe HBase Column Amazon Dynamo SimpleDB Key-- Value Document BestBuy Riak Key-- Value ebay Cassandra MongoDB Column Document Facebook Cassandra Neo4j Column Graph Google BigTable Column LinkedIn Voldemort Key-- Value LotsOfWords CouchDB Document MongoHQ MongoDB Document Mozilla HBase Riak Column Key-- Value Netflix SimpleDB HBase Cassandra Document Column Column Twitter Cassandra Column Current NoSQL Data Security Issues Very recent data breaches occurred at MongoHQ (Oct 2013) [Ref. 7] and LinkedIn (July 2012) [Ref. 8] underscores the importance of NoSQL data security as more and more companies are bracing for the new family of products. Although the above two incidents are caused by weak encryption of passwords, and not directly linked to any known NoSQL vulnerability, they point to a fact that NoSQL are becoming targets of attackers who seek valuable information. NoSQL database may become even more susceptible to exploits once attackers overcome the learning curve, and are able to identify hidden security or software weaknesses. PAGE 5 PAGE 5

6 Okman et al in 2011 published a comprehensive study on security issues of NoSQL databases [Ref. 1], which discussed common security issues on two popular NoSQL, Cassandra (version 0.8) and MongoDB (version unknown). As most NoSQL databases are still work-- in-- progress products, three years later it is worthy to take another look and re-- evaluate their recent developments. The current version of Apache Cassandra is 2.0 (Enterprise version 3.2 is offered by DataStax), and it is version 2.4 for MongoDB. Since enhanced database security always comes at the expense of database performance, there is no surprise that most default security settings for Cassandra are set to either none or minimum [Ref. 9]; and MongoDB's current manual states "The most effective way to reduce risk for MongoDB is to run your entire MongoDB deployment in a trusted environment" [Ref. 10]. 1. Data at Rest -- [Cassandra] The latest Cassandra (Enterprise 3.2) provides an optional feature for Transparent Data Encryption (TDE) to protect data that is flushed out from the memory (memtable) and written to disk. To some extent, this feature can be enabled to protect sensitive data. However, since the encryption certificate is stored locally, a secured file system is necessary before TDE is turned on. In addition, the commit log of Cassandra, a file where committed data is appended to, is not encrypted at all. [MongoDB] Data files in MongoDB are never encrypted, and there is no method provided to accomplish this. If encryption is needed, the application layer should perform the data encryption before writing to database. Strong file system security is also recommended. [Third Party Tools] To help NoSQL databases solve the critical deficiency in data-- at-- rest security, a few third-- party tools have emerged to provide transparent data encryption and the associated key management, such as Gazzang [Ref. 11], Zettaset [Ref. 12] and IBM InfoSphere Guardium [Ref. 13]. The solutions provided by Gazzang and Zettaset are specifically targeted for distributed cloud-- based NoSQL and Hadoop systems. IBM InfoSphere Guardium, on the other hand, is suitable for a wide range of RDBMS and NoSQL databases. 2. Data in Motion (Client-- Node Communications) -- [Cassandra] By default, the client-- node communication is not encrypted. SSL can be turned on by editing the corresponding settings under client_encryption_options in the cassandra.yaml file after generating valid server certificates. PAGE 6 PAGE 6

7 [MongoDB] The default distribution of MongoDB does not support SSL client-- node communication. In order to use SSL, it is required to either recompile MongoDB with the "-- -- ssl" option, or use the MongoDB Enterprise version. Additional steps to generate keys are needed for configuring client/server for SSL communication. 3. Data in Motion (Inter-- Node Communications) -- [Cassandra] By default the inter-- node communication is not encrypted either. If needed, available SSL encryption options are "all" (all inter-- node), "dc" (between datacenters), and "rack" (between racks). Inter-- node SS communication can be configured by editing the corresponding settings under server_encryption_options in the cassandra.yaml file. [MongoDB] Inter-- node encryption communication is not supported in MongoDB. 4. Authentication -- [Cassandra] By default the authenticator setting of basic Cassandra is AllowAllAuthenticator, which means essentially there is no authentication. The other available option is PasswordAuthenticator, in which user names and passwords (hashed but unsalted) are stored in the system_auth.credentials table. Enterprise Cassandra can further provide Kerberos authentication, which requires setting up separate Kerberos servers and installing Kerberos client software on all joining Cassandra hosts. [MongoDB] Authentication is also disabled by default. Basic MongoDB does provide support for authentication on a per-- database level. Users exist in the context of a single logical database. MongoDB Enterprise supports an additional Kerberos service for authentication. 5. Authorization -- Due to the schema-- less nature of NoSQL's data models, fine-- grained data access controls at the row or column level, as provided by RDBMS like Oracle, are not available with current NoSQL databases. Some of them do implement some sort of authorization if needed. [Cassandra ] The default choice is AllowAllAuthorizer, which essentially provides no authorization and allows any action by any user. If CassandraAuthorizer is selected, then privileged administrators can grant any of the privileges (ALTER, AUTHORIZE, CREATE, DROP, MODIFY, SELECT) on any resources (ALL KEYSPACES, KEYSPACE, TABLE) to a selected user, by executing CQL (Cassandra Query Language) statements. PAGE 7 PAGE 7

8 [MongoDB] Disabled by default, MongoDB provisions authorization on a per-- database level by using a role-- based approach. Available roles are limited to the following, read, readwrite, dbadmin, useradmin, clusteradmin, readanydatabase, readwriteanydatabase, useradminanydatabase, and dbaadminanydatabase. 6. Audit Security logging and monitoring is also required by the PCI-DSS compliance (Requirement 10), to determine the "who, what, where and when" of users accessing a data processing resource, such as a database. [Cassandra] Auditing is available in Enterprise Cassandra as a log4j-- based integration, and a per-- node basis. To get the maximum audit information, turning on auditing on every node is recommended. Filters are available for logging, using a combination of the following categories, ADMIN, ALL, AUTH, DML, DDL, DCL, and QUERY. [MongoDB] MongoDB is far behind in implementing the desired security logging and monitoring. Most monitoring and reporting tools currently distributed with MongoDB are related to database performance, mainly for showing the running state of a MongoDB instance. There is an HTTP Console for each MongoDB instance to show information about the system and connecting clients. However, if security is not enabled for the MongoDB instance, which is by default, no authorization is needed to access this interface, resulting in a potential vulnerability. 7. Data Consistency -- Because of the shared BASE design among NoSQL databases, inherent data inconsistency among clustering nodes becomes possibly. This may explain why NoSQL databases have not well made their way into processing critical financial transactions. The potential data inconsistency can be shown in the following series of diagrams, PAGE 8 PAGE 8

9 Fig. 2 A user enters information into a social network site Fig. 3 Shortly the information is updated, but hasn't been consistently replicated PAGE 9 PAGE 9

10 Fig. 4 Read inconsistency could happen if stale data is retrieved Since NoSQL databases do not guarantee strong data consistency, it usually falls on developers to design applications that can work with the eventual consistency model, and to weigh tradeoffs between data consistency and performance impact. Cassandra does provide a range of configurable write and read consistency levels (CL) to meet particular application needs, as shown in Fig. 5 [Ref. 14]. PAGE 10 PAGE 10

11 Fig. 5 Configurable write and read consistency levels available in Cassandra 8. NoSQL Injection Exploits Just like their traditional RDMBS counterparts, NoSQL databases are susceptible for injection attacks, especially those heavily use server-- side JavaScript and PHP to enhance database performance. Take MongoDB for example, its internal operator "$where", designed to be used as a filter like the "where" clause in SQL, can also takes sophisticated JavaScript functions to filter data. An attacker thus can pass arbitrary code or commands into the $where operator as part of the query. Other vulnerable MongoDB operations include db.eval(), mapreduce, and group, which also permit to run arbitrary JavaScript expressions on the server. The next release of Open Web Application Security Project (OWASP) Test Guide (v4), currently still in draft, is to include new procedures for testing NoSQL injections [Ref. 15]. Although the draft test uses MongoDB as the target for example, other NoSQL databases that built upon JavaScript and/or PHP engines may possess similar vulnerabilities. Typically, NoSQL injection attacks will execute where the attack string is parsed, evaluated, or concatenated into a NoSQL API call. Attackers PAGE 11 PAGE 11

12 just need to be familiar with the syntax, data model, and underlying programming language of the target database in order to design specific exploits. The following examples demonstrate how JavaScript NoSQL injections can be crafted against a vulnerable MongoDB instance. [JavaScript NoSQL Injection #1] To demonstrate a potential NoSQL injection against MongoDB, consider the following two valid, equivalent JavaScript statements to retrieve a collection of data that meet the (credits < debits) condition, 1. db.mycollection.find( { $where: "this.credits < this.debits" } ); 2. db.mycollection.find( { $where: function() { return obj.credits - obj.debits < 0; } } ); If a dynamic threshold that takes a user input is desired, the second statement can be re-- written as follows, 3. db.mycollection.find( { $where: function() { return obj.credits - obj.debits < $userinput; } } ); This may expose a vulnerability where an attacker could overwrite the $userinput variable with arbitrary code, such as $userinput = "0;var date=new Date(); do{curdate = new Date();}while(curDate-date<10000)" If sanitization check fails to screen the $userinput value, now upon concatenation the third statement becomes the following form that could trigger a DOS attack and cause the MongoDB instance to execute at 100% CPU usage for 10 second! 4. db.mycollection.find( { $where: function() { return obj.credits - obj.debits < 0;var date=new Date(); do{curdate = new Date();}while(curDate-date<10000); } } ); [JavaScript NoSQL Injection #2] If developers are not careful enough in security coding, it is also possible for an attacker to pass malicious code directly from a malformed URL [Ref. 16]. The following is a generic JavaScript query function to perform a search based on a provided 'year' criterion, input_value. function() { PAGE 12 PAGE 12

13 var search_year = input_value; return this.publicationyear == search_year this.filmingyear == search_year this.recordingyear == search_year; } The application developer may code this application using PHP, and the source code that includes building the above function might look like the following, before being passed to a MongoDB instance, $query = 'function() {var search_year = \''. $_GET['year']. '\';'. 'return this.publicationyear == search_year '. ' this.filmingyear == search_year '. ' this.recordingyear == search_year;}'; $cursor = $collection->find(array('$where' => $query)); This code builds the function ad-- hoc by concatenating the value of the request parameter year, and then pass it to MongoDB. This code is vulnerable to a server-- side JavaScript injection attack. For example, an attacker could formulate the following URL call to cause an effective DoS attack against the system: Conclusions Based on this open-- source research, the following conclusions can be drawn; 1. NoSQL databases are desirable and popular among Web-- based companies, due to their demonstrated advantages in data flexibility, scalability and performance. 2. Security issues of NoSQL in general remain to be improved. There are only a few NoSQL (e.g., Cassandra) that currently meet the data security requirements of PCI-- DSS, e.g., data-- at-- rest and data-- in-- motion. However, enhanced security is expected to come at the expense of performance 3. More server-- side JavaScript injection vulnerabilities are expected from NoSQL databases, because many of them are running JavaScript engines to achieve high performance. PAGE 13 PAGE 13

14 4. Working with NoSQL databases, application developers have much greater responsibility in ensuring reliable transactions and data consistency. In addition, they also have to adhere more closely to the standards and practices of security coding. 5. Relational and NoSQL databases are best co-- deployed to process different data flows to achieve the optimal combined features from both families. PAGE 14 PAGE 14

15 Author Dr. Ming-- Shih Wong is a Senior Cyber Security Engineer for Incident Response & Forensics for Fidelis Cybersecurity Solutions. He has years of experience in conducting advanced projects for Air Force Intelligence, Surveillance and Reconnaissance (AFISR) and Defense Advanced Research Project Agency (DARPA). He has also participated in several major commercial data-- breach Incident Response and Remediation engagements that involve noteworthy companies in payment card and technology industries. As a member of the Fidelis First Response team, he collaborates with forensics and SOC experts in optimizing search strategies for investigation of key indicators of compromise (IOC), utilizing advanced tools like Splunk, ArcSight, open-- source log analyzers, or even customized databases. He has also extended his database management foundation to cover PCI security standards and compliance, including data security at rest and data security in motion. He provides expertise in the following data analytics and remediation areas, database security management, breach-- indicator investigation, PCI data security standards and compliance, network and host intrusion detections. References 1. L. Okman, N. Gal-- Oz, Y. Gonen, E. Gudes, and J. Abramov, "Security Issues in NoSQL Databases", 2011 International Joint Conference of IEEE TrustCom-- 11/IEEE ICESS-- 11/FACT database.org/ "The Four Categories of NoSQL", four-- categories-- of-- nosql-- databases. 6. "Netflix Relies on NoSQL", relies-- on-- nosql/ 7. "Hosting Service MongoDB Suffers Major Security Breach That Explains PAGE 15 PAGE 15

16 Buffer's Hack Over the Weekend", service-- mongohq-- suffers-- major-- security-- breach-- that-- explains-- buffers-- hack-- over- - the-- weekend/ 8. "LinkedIn Suffer Data Breach", us-- linkedin-- breach-- idusbre "DataStax Enterprise 3.2 Documentation", "MongoDB Security Introduction", introduction 11. "Data Encryption and Key Management for the Cloud", security 12. "Hadoop Strict Encryption for (Big) Data-- At-- Rest", strict-- encryption-- for-- big-- data-- at/ "NoSQL Does Not Have to Mean No Security -- Data security and compliance best practices for NoSQL data systems", DF 14. "Cassandra Replication & Consistency", to-- cassandra-- replicatio-- and-- consistency 15. "Testing for NoSQL Injection", injection 16. "Server-- Side JavaScript Injections", us-- 11/Sullivan/BH_US_11_Sullivan_Server_Side_WP.pdf PAGE 16 PAGE 16