Reference Data Collections

Transcription

1 Juniper Secure Analytics Release Juniper Networks, Inc North Mathilda Avenue Sunnyvale, CA USA Published:

2 Copyright Notice Copyright 2015 Juniper Networks, Inc. All rights reserved. Juniper Networks, Junos, Steel-Belted Radius, NetScreen, and ScreenOS are registered trademarks of Juniper Networks, Inc. in the United States and other countries. The Juniper Networks Logo, the Junos logo, and JunosE are trademarks of Juniper Networks, Inc. The following terms are trademarks or registered trademarks of other companies: Java TM and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates. All other trademarks, service marks, registered trademarks, or registered service marks are the property of their respective owners. All specifications are subject to change without notice. Juniper Networks assumes no responsibility for any inaccuracies in this document or for any obligation to update information in this document. Juniper Networks reserves the right to change, modify, transfer, or otherwise revise this publication without notice. FCC Statement The following information is for FCC compliance of Class A devices: This equipment has been tested and found to comply with the limits for a Class A digital device, pursuant to part 15 of the FCC rules. These limits are designed to provide reasonable protection against harmful interference when the equipment is operated in a commercial environment. The equipment generates, uses, and can radiate radio-frequency energy and, if not installed and used in accordance with the instruction manual, may cause harmful interference to radio communications. Operation of this equipment in a residential area is likely to cause harmful interference, in which case users will be required to correct the interference at their own expense. The following information is for FCC compliance of Class B devices: The equipment described in this manual generates and may radiate radio-frequency energy. If it is not installed in accordance with Juniper Networks installation instructions, it may cause interference with radio and television reception. This equipment has been tested and found to comply with the limits for a Class B digital device in accordance with the specifications in part 15 of the FCC rules. These specifications are designed to provide reasonable protection against such interference in a residential installation. However, there is no guarantee that interference will not occur in a particular installation. If this equipment does cause harmful interference to radio or television reception, which can be determined by turning the equipment off and on, the user is encouraged to try to correct the interference by one or more of the following measures: Reorient or relocate the receiving antenna. Increase the separation between the equipment and receiver. Consult the dealer or an experienced radio/tv technician for help. Connect the equipment to an outlet on a circuit different from that to which the receiver is connected. Caution: Changes or modifications to this product could void the user's warranty and authority to operate this device. Disclaimer THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT, SUBJECT TO THE MODIFICTAIONS SET FORTH BELOW ON THIS PAGE, ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR JUNIPER NETWORKS REPRESENTATIVE FOR A COPY. Release Copyright 2015, Juniper Networks, Inc. All rights reserved. Printed in USA. Revision History March 2015 The information in this document is current as of the date listed in the revision history. END USER LICENSE AGREEMENT The Juniper Networks product that is the subject of this technical documentation consists of (or is intended for use with) Juniper Networks software. Use of such software is subject to the terms and conditions of the End User License Agreement ( EULA ) posted at as modified by the following text, which shall be treated under the EULA as an Entitlement Document taking precedence over any conflicting provisions of such EULA as regards such software: As regards software accompanying the STRM products (the Program ), such software contains software licensed by Q1 Labs and is further accompanied by third-party software that is described in the applicable documentation or materials provided by Juniper Networks. 2

3 For the convenience of Licensee, the Program may be accompanied by a third party operating system. The operating system is not part of the Program, and is licensed directly by the operating system provider (e.g., Red Hat Inc., Novell Inc., etc.) to Licensee. Neither Juniper Networks nor Q1 Labs is a party to the license between Licensee and the third party operating system provider, and the Program includes the third party operating system AS IS, without representation or warranty, express or implied, including any implied warranty of merchantability, fitness for a particular purpose or non-infringement. For an installed Red Hat operating system, see the license file: /usr/share/doc/redhat-release-server-6server/eula. By downloading, installing or using such software, you agree to the terms and conditions of that EULA as so modified. 3

4 4

5 CONTENTS 1 REFERENCE DATA COLLECTIONS About Creating a Reference Data Collection Utility Command Reference

6

7 1 REFERENCE DATA COLLECTIONS Use the ReferenceDataUtil.sh utility to make complex reference data collections, such as a Reference Map, Reference Map of Sets, and Reference Map of Maps. This technical note applies to Juniper Secure Analytics (JSA) and Log Manager. About Reference Data Collections Reference data collections enable the storage, retrieval and testing of complex data structures. You can create the following reference data collection types: Reference Map - In a Reference Map, data is stored in records that map a key to a value. For example, to correlate user activity on your network, you can create a reference map that uses the Username parameter as a key and the user s global ID as a value. Reference Map of Sets - In a Reference Map of Sets, data is stored in records that map a key to multiple values. For example, to test for authorized access to a patent, you can create a Map of Sets that uses a custom event property for Patent ID as the key and the Username parameter as the value to populate a list of authorized users. Reference Map of Maps - In a Reference Map of Maps, data is stored in records that map one key to another key, which is then mapped to single value. For example, to test for network bandwidth violations, you can create a Map of Maps that uses the Source IP parameter as the first key, the Application parameter as the second key, and the Total Bytes parameter as the value. For each reference data collection, you specify what the element type is for the collection. You can specify the following element types: Alphanumeric Alphanumeric Ignore Case IP Numeric Port Date

8 8 REFERENCE DATA COLLECTIONS Creating a Reference Data Collection Using the ReferenceDataUtil.sh utility, you can create a reference data collection. Before you begin If you plan to load an external file containing data elements, ensure that the file is in Comma Separated Value (CSV) format. Also ensure that you have copied the file to your JSA system. The file must follow the format in the following examples reference data collections: Example 1 ReferenceMap key1,data key1,value1 key2,value2 Example 2 ReferenceMapOfSets key1,data key1,value1 key1,value2 Example 3 ReferenceMapOfMaps key1,key2,data map1,key1,value1 map1,key2,value2 The symbol in the first column indicates a comment line.the first non-comment line is the column header and identifies the column name (ie., key1, key2, data). Then each non-commented line after that is a data record that gets added to the map. Keys are alphanumeric strings. About this task See Utility Command Reference for a list of all commands and parameters you can use to manage your reference data collections. You can also type./referencedatautil.sh and press Enter to access a list these commands

9 Creating a Reference Data Collection 9 Step 1 Step 2 Step 3 Procedure Using SSH, log in to JSA as the root user: Username: root Password: <password> Create a reference data collection: a To change to the /opt/qradar/bin directory, type the following command: b cd /opt/qradar/bin To create the reference data collection, type the following command:./referencedatautil.sh create <name> <count> [MAP MAPofSETS MAPofMAPS] [ALN NUM IP PORT ALNic DATE] [timeout_type] [timetolive] To populate the map with data from an external file, type the following command:./referencedatautil.sh load <name> <filename> [-encoding=...] [-sdf="... "] What to do next Log in to the JSA user interface to create rules that add data to your reference data collections or rule tests that detect activity from elements in your reference data collection. For more information on creating rules and rule tests, see the Juniper Secure Analytics Users Guide.

10 10 REFERENCE DATA COLLECTIONS Utility Command Reference Use the following commands to manage your reference data collections: Table 1-1 Command Reference Command create Parameters <name> is the name of the reference data collection. <count> is the maximum number of elements that the reference data collection can contain. [timeout_type] specifies whether the timetolive is from the time the element was inserted (0) or last seen (1). [timetolive] specifies the amount of time reference data collection elements remain in the collection. [MAP MAPofSETS MAPofMAPS] specifies the type of reference data collection. [ALN NUM IP PORT DATE] is the type of data in the reference set, where: update add ALN specifies a reference data collection of alphanumeric values. This data type supports IPv4 and IPv6 addresses. ALNic specifies a reference data collection of alphanumeric values but tests ignore the case. This data type supports IPv4 and IPv6 addresses. NUM specifies a reference data collection of numeric values. IP specifies a reference data collection of IP addresses. This data type supports only IPv4 address. PORT specifies a reference data collection of PORT addresses. DATE specifies a reference data collection of DATE values. <name> is the name of the reference data collection. <count> is the maximum number of elements that the reference data collection can contain. [timeout_type] specifies whether the timetolive is from the time the element was inserted (0) or last seen (1). [timetolive] specifies the amount of time reference data collection elements remain in the collection. <name> is the name of the reference data collection. <value key1 [key2]> specifies the values you want to add. Key1 is required for MAP, MAPofSETS and key2 is required for MAPofMAPS. Keys are alphanumeric strings.

11 Utility Command Reference 11 Table 1-1 Command Reference Command delete remove purge get getall load Parameters <name> is the name of the reference data collection <value key1 [key2]> specifies the values you want to delete. Key1 are required for MAP, MAPofSETS and key2 is required for MAPofMAPS [-sdf="... "] specifies the Simple Date Format string used to parse the date data. <name> is the name of the reference data collection. <name> is the name of the reference data collection. <name> is the name of the reference data collection [loadelements] displays all elements in the specified reference data collection. [loadelements] displays all elements in all reference data collections. <name> is the name of the reference data collection <filename> is a fully qualified filename to be loaded, with each line in the file representing a record to be added to the reference data collection [-encoding=...] specifies encoding to use when reading the file. [-sdf="... "] specifies the Simple Date Format string used to parse the date data.