Enabling SOX Compliance on DataStax Enterprise

Transcription

1 Enabling SOX Compliance on DataStax Enterprise

2 Table of Contents Table of Contents... 2 Introduction... 3 SOX Compliance and Requirements... 3 Who Must Comply with SOX?... 3 SOX Goals and Objectives... 3 DataStax Enterprise and SOX... 3 Addressing SOX Compliance... 4 Conclusion... 7 About DataStax... 7

3 Introduction The Sarbanes-Oxley Act of 2002 (SOX) was established to regulate financial practices and prevent accounting fraud. All public companies issuing securities, organizations providing auditing services whether they are domestic or foreign, and public accounting firms must comply with Sarbanes- Oxley. SOX came into existence as a result of the corporate financial scandals involving Arthur Andersen, Enron, WorldCom and Global Crossing, which resulted in billions of dollars in corporate and investor losses. SOX s main purpose is to review legislative audit requirements, strengthen corporate governance and protect investors by improving the accuracy and reliability of financial disclosures (Section 302). It is administered by Security and Exchange Council (SEC) and all public companies are required to implement and report internal financial audit controls to the SEC for compliance. Each company's external auditors are also required to audit and report on the reliability of management s assessment of internal control (Section 404), in addition to the company s financial statements. SOX Compliance and Requirements The scope of SOX compliance is quite broad and encompasses a company s management of its quality and security. But the main focus is on protecting the integrity of data thereby enhancing both audit quality and investor confidence. Segregation of Duties (SOD) and its management is another key component that external auditors look at while validating an organization s financial results. Compliance to SOX also deals with what records need to be stored and for how long. The new SOX legislation has placed significant compliance and audit trail demands on IT and financial departments within organizations; auditing application transactions in a database is one of the key requirements. Any unauthorized changes to application data might directly impact financial reporting or cause fraudulent transactions. Any revisions or modifications to these systems must be fully documented, as to what was changed and why, by whom and when. This being the case, it is imperative for these systems to be audited through secure and reliable means and the audit trail records maintained, as required by these mandates. Noncompliance can lead to severe penalties, lawsuits and negative publicity for these companies. IT and financial departments have to either build on their own or rely on access control and auditing mechanisms provided by third party vendors to track and report changes to key data elements stored in the database. This white paper provides general guidelines on how DataStax Enterprise can help organizations comply with SOX using its robust security features, including access control and auditing capabilities. Who Must Comply with SOX? Public companies issuing securities, organizations providing auditing services whether they are domestic or foreign, and public accounting firms must comply with Sarbanes-Oxley. SOX Goals and Objectives Organizations and firms expect the underlying database to be highly secure and also provide capabilities to track user database activities with authentication, as part of SOX compliance requirement. DataStax Enterprise and SOX DataStax is the leading provider of enterprise NoSQL database software products and services based on Apache Cassandra. DataStax helps drive the open source development of Cassandra by delivering DataStax Enterprise (DSE), an enterpriseclass NoSQL platform comprised of three components: DataStax Enterprise Server- includes production-certified Apache Cassandra, an in-memory option, and the ability to run analytics (via Apache Spark and Hadoop) and perform enterprise search operations (via Apache Solr) on Cassandra data. DSE includes enterprise security features like transparent data encryption, data auditing, internal and external authentication, clientto-node encryption and node-to-node encryption through SSL. OpsCenter a visual, browser-based solution for managing and monitoring Cassandra and DataStax Enterprise server.

4 Production Support full 24 x 7 x 365 support from the big data experts at DataStax. Addressing SOX Compliance Identifying areas and priorities for IT and financial management frameworks is an essential part of the compliance process. The following activities help guide the compliance process: Assessment - Gather data usage information Set and Enforce Controls - Define usage patterns for data to prevent unauthorized actions Monitor and Measure - Capture and report on activity The above iterative process helps IT and financial departments satisfy compliance requirements of auditors while ensuring robust security with welldefined controls. Assessment Identifying assets and knowing where data resides in an organization is the first critical component of the assessment process. Doing so drives effective protection policies to ensure best practices on security. Set and Enforce Controls Auditors specifically evaluate user privileges to critical data. Ensuring proper SOD controls for these privileged users are essential to maintain the integrity of the data. A good rule of thumb is to enforce the least powerful privileges and grant users only the privileges they need to do their job. DataStax Enterprise offers internal authentication that restricts access to databases containing critical data to a business-need-to-know basis. In this way organizations can reduce the number of users that can result in SOD violations. Internal Authentication Internal authentication ensures only authorized users get access to the Cassandra database system via the use of login accounts and passwords. Internal authentication works for the following Cassandra client drivers when you provide a user name and password to connect to the database: Astyanax Cassandra-cli Cqlsh DataStax Java Driver and DataStax C# Driver Hector Pycassa Internal authentication stores user names and bcrypt-hashed passwords in Cassandra s system_auth.credentials table. CQL (the Cassandra Query Language) supports the following statements for setting up and removing user accounts: ALTER USER CREATE USER DROP USER LIST USERS Current Internal Authentication Limitations in DSE Dsetool and Hadoop utilities do not support internal authentication. Access to Solr documents, excluding cached and indexed data, can be limited to users who have been granted access permissions. Password authentication pertains to connecting Spark wiht Cassandra, not authenticating Spark components between each other, and authenticating changes to the Shark configuration. Hadoop data on CFS is not supported by internal authentication. OpsCenter 5.0 comes with built-in granular security with role based access controls. User roles can be configured to perform database operations in the cluster. External Authentication DataStax Enterprise also supports external authentication through Kerberos and LDAP mechanism for Cassandra, Search and Analytics data. LDAP/Active Directory is a standardized way of storing security credentials in a centralized repository for a company s applications. Kerberos is a computer network authentication protocol that allows nodes communicating over a non-secure network to prove their identity to one another securely using tickets. As mentioned earlier, only authorized users will have access to Cassandra database system using external validation. Both Kerberos and SSL libraries provide integrity protection for all transmitted data, an essential part of SOX compliance. A unique ID can also be assigned for users accessing critical data using LDAP s or Kerberos s single sign on capability.

5 Object Permission Management Object permission management delivers granularbased control over who can modify data with grant and revoke statements. In Cassandra, superuser accounts grant initial permissions, and subsequently a user may or may not have grant/revoke permissions for a Cassandra keyspace or a table. Object permission management is independent of authentication (works with Kerberos or Cassandra). Cassandra s CQL interface supports the following authorization statements: GRANT LIST PERMISSIONS REVOKE Internal and external authentication along with object permission management can enforce security policies in real time by blocking access to unauthorized users and prevent fraudulent activity. This ensures data protection and integrity. Monitor and Measure It is important to track and monitor the activity of users who have highly privileged access to critical databases. This is typically done by maintaining an audit log for all activities including data modification by these users accounts as required by SOX section 404. DataStax Enterprise can help in monitoring the activities of these high privileged users through its auditing capabilities as explained below. Effective measures should also be taken in the event of data loss or corruption, with ways either to prevent or recover from such an occurrence. Data Auditing in DSE DataStax Enterprise creates detailed audit trails of activities that occur in a Cassandra cluster. It offers flexibility in storing audit events in a file (log4j-based) or in a Cassandra table. It also enables the tracking of user activities on the database, from login events to the creation, deletion, and alteration of database objects. All privileges such as Grant or Revoke to the user or a role within a database can be audited. DataStax also supports audit logging of queries and prepared statements submitted to the DataStax Java Driver, which uses the CQL binary protocol. Auditing is configured through a text file in the file system. The audit logger logs information on the node set up for logging. To get the maximum information from data auditing, the user can turn on data auditing on every node. DSE also provides an option to disable logging for specific keyspaces. Available audit levels: All - turns on auditing for all actions OFF - no auditing FATAL - severe errors causing premature termination ERROR - other runtime errors or unexpected conditions WARN - use of deprecated APIs, poor use of API, near errors, and other undesirable or unexpected runtime situations DEBUG - detailed information on the flow through the system TRACE - more detailed than DEBUG INFO - highlight the progress of the application at a coarse-grained level Datastax does not recommend using TRACE or DEBUG in production due to verbosity and performance. All DML, DCL or DDL statements are logged in DataStax. Audit information is stored in either log files or Cassandra tables. This provides the capability to do in-depth analysis via the DataStax Enterprise platform using Hadoop and Solr. SETTING ADMIN ALL AUTH DML DDL DCL QUERY LOGGING Describe schema versions, cluster name, version, ring, and other admin events DDL, DML, queries, and errors Login events Insert, update, delete and other DML events Object and user create, alter, drop, and other DDL events Grant, revoke, create user, drop user, and list users events Queries Measure Taking appropriate measures and reporting activity during/after the event of data corruption or data loss scenarios is crucial as part of SOX. It is also important to monitor such events and resolve as necessary. Data corruption can happen due to programming error, hardware failure or bad memory. The user should monitor logs (output, system logs) for any errors during such scenarios. In addition, corruption can be fixed via DSE s nodetool repair process available in open source Cassandra or via the automated repair service available in DataStax Enterprise. The nodetool scrub process is also helpful in rebuilding SSTables during such scenarios.

6 Data loss can happen due to human error, power failure or natural disasters such as hurricanes, typhoons, or earthquakes, and the like. Proper measures can to be taken early on in order to prevent certain data loss scenarios as explained below. Choosing a Proper Replication Factor Cassandra protects against data loss by storing multiple redundant copies of the data across different machines in the cluster. DataStax recommends three replicas (RF=3) within a single data center (DC) and guarantees against data loss due to single machine failure once the data has been persisted to at least two machines. Using Tunable Consistency Tunable consistency can be configured to manage response time versus data accuracy (in and across data centers). LOCAL_QUORUM is recommended when Cassandra is deployed across multiple data centers to protect against single machine failure. This however, does allow for a small window of data loss when writes are propagated to one data center that has experienced a catastrophic failure soon after this operation, but has not yet been replicated to remote data centers. QUORUM can be utilized, but if two data centers were simultaneously lost (with DC=3, RF=3), there is a small window where the data is not yet replicated to another data center and those writes will be lost. To mitigate any data loss, DataStax recommends using a consistency level of EACH_QUORUM. By utilizing EACH_QUORUM the user is guaranteed against single, or even multiple catastrophic simultaneous data center losses. Using the Rack Aware Switch A rack-aware snitch ensures that replicas of each row exist in multiple server racks. This allows a rack to be lost due to fire, mechanical problems, or routine maintenance without impacting availability. Using Hinted Handoff Remaining Cassandra nodes in a cluster will store up to three hours of information (hints/writes) by default during events such as when a network link between data centers is severed or a node becomes unavailable. If after three hours the link has not been restored, then the node or data center is considered permanently lost and mutations intended for the node(s) will be dropped. If after three hours a link to a down node or data center is reestablished, then you must run the nodetool repair process on each node that was down or use the repair service available in DataStax Enterprise. Using Snapshots and Backup Database backup and restore are best practice approaches as part of database maintenance during the event of data loss. Cassandra delivers a mechanism to easily snapshot or backup the state of the database on each node or via visual and scheduled backups through OpsCenter (available as part of DataStax Enterprise). These snapshots provide a read-only mechanism through which the state of the database on each node can be restored in the event of a data loss. If data loss due to programmer error or corruption were to occur, only mutations not yet persisted to disk would be impacted. In addition, the content of a snapshot on each machine can be copied to external media. Database files (SSTables and commit log) can also be copied from each node to external media. OpsCenter 5.1 supports backing up of data to the cloud (Amazon S3). In the event of catastrophic data loss, the files (either snapshot, or SSTables and commitlog) can be restored onto the same database cluster. The restore can also be done to a specific data/time in the past using point in time restore capability. While Cassandra does provide multiple mechanisms to mitigate the possibility of data loss, the risk cannot be entirely eliminated. If all replicas were lost simultaneously, the missing data would need to be restored via backup. There would be a small delta of information that existed on the nodes and had not yet been replicated to the backup media. If this were to happen, data within that window would be irrecoverable. By default, the commitlog syncs to disk every 10 seconds (periodic mode), by default. A window for data loss can exist if all replicas were lost simultaneously before data written to the commit log was not flushed to disk. This issue can be mitigated by enabling batch mode (in order to guard against this possibility), but note that this mode will negatively affect the write performance. For these reasons it is recommended that a significant number of nodes be deployed to decrease the statistical likelihood that all replicas of a row are lost. Organizations complying with SOX should have a solid database strategy to guard against such data loss/data corruption scenarios as mentioned above. This can be achieved with proper configuration of the DSE cluster, along with the right replication and backup strategy. Also actively monitoring and logging events through auditing capabilities of DSE and reporting during any such events are critical components of the SOX compliance process.

7 Conclusion The Sarbanes-Oxley Act of 2002 mandates that organizations have security and audit controls to ensure data integrity. DataStax Enterprise (DSE) delivers robust security features for enterprises looking to protect and audit critical data as part of SOX compliance. Segregation of Duties (SOD) and Enforcement of Control policies through DSE s internal and external authentication along with object permission management and granular role based security, and also monitoring database activities by user s action via auditing capabilities, can help organizations to remain SOX compliant. For more information about Cassandra, DataStax Enterprise and OpsCenter, visit For downloads of DataStax Enterprise and OpsCenter which may be freely used for development evaluation purposes visit About DataStax DataStax provides a massively scalable enterprise NoSQL platform to run modern online applications for some of the world s most innovative and dataintensive enterprises. Powered by the open source Apache Cassandra database, DataStax delivers a fully distributed, continuously available platform that is faster to deploy and less expensive to maintain than other database platforms. DataStax has more than 500 customers in 38 countries including leaders such as Netflix, Rackspace, Pearson Education, and Constant Contact, and spans verticals including web, financial services, telecommunications, logistics, and government. Based in San Mateo, Calif., DataStax is backed by industry-leading investors including Lightspeed Venture Partners, Meritech Capital, and Crosslink Capital.