Software Design Document Logging/Audit

Software Design Document Logging/Audit FM 7.5 Version 0.2 (Draft) Please send comments to: dev@opensso.dev.java.net

Contents 1 Introduction......1 1.1 Document Status...1 1.2 Revision History...1 1.3 Summary...1 1.4 Scope...1 1.5 Context......1 1.6 Glossary...2 1.7 References...2 2 Overview...3 3 Design Considerations......4 3.1 Assumptions and Dependencies......4 3.2 Goals and Guidelines...4 3.3 Development Method...4 4 Architectural Strategies...5 5 System Architecture...6 6 Detailed System Architecture...7 7 Appendices...8 Copyright 2007 Sun Microsystems, Inc. All rights reserved. iii

1 Introduction 1.1 Document Status Project Name Federation Manager 7.5 Document Title Logging/Audit Date of Issue 02/22/07 Current Version 0.3 Author Burt Fujii Issuing Organization Sun Microsystems, Inc. Feedback E-mail dev@opensso.dev.java.net 1.2 Revision History Date Version Author Comments 02/22/07 0.1 Burt Fujii Initial draft 03/06/07 0.2 Burt Fujii Updates 03/14/07 0.3 Burt Fujii Added change to DB Date/Time column 1.3 Summary The purpose of this document is to provide a design for the features added in the Logging/Auditing services. The scope of this document details the implementation of the logging/auditing enhancements recently identified. The intended audience is the engineering and support groups, and those who would be in contact with the customers. 1.4 Scope The designs of the additions of the NameID field in the Log Record, and of the Dynamic Logging Level feature are detailed. The Complete Logs and Options To See Detailed XML Messages items mentioned in the Requirements Document are mentioned here, as a reminder for the components that must effect the changes. Copyright 2007 Sun Microsystems, Inc. All rights reserved. 1

0, Version 0 Introduction 1.5 Context This document should be read in the context of a Federation Manager administrator and/or developer using the Logging API. 1.6 Glossary Level As defined by java.util.logging.level 1.7 References [1] FM 7.5 Logging/Audit Software Requirement Specification 2 Copyright 2007 Sun Microsystems, Inc. All rights reserved.

2 Overview In the area of logging/auditing, several specific requirements have been identified: The Federation log records need to contain (in addition to the already included local userid and ContextID) an opaque handle providing the subject's NameID from the IDP. This field would be used to correlate IDP/SP log entries. Complete logs for all events; e.g., single sign-on, single-logout, etc. Options to see detailed XML messages between SPs and IDPs; e.g., whole assertion XML. Dynamic logging level setting, without having to restart the server. This document details implementation of the additional log record field (opaque handle for NameId) and dynamic logging level setting. The other two requirements are controlled by the components providing the logging information, so they should be covered in other documents. An additional change when specifically logging to a database is also detailed. The current VARCHAR data type definition makes it unnatural for writing SQL reports that query or sort on the Time column. Copyright 2007 Sun Microsystems, Inc. All rights reserved. 3

0, Version 0 Design Considerations 3 Design Considerations 3.1 Assumptions and Dependencies Inclusion of the new NameID log record field depends on the logging client providing that property and value. Logging of all events depends upon the components effecting those log records/requests. Options for logging detailed XML messages between SPs and IDPs can be effected by logging at appropriate logging levels (e.g., FINE, FINER, and FINEST corresponding to more detailed levels than the default INFO). Upgrade/Migration Upgrade/migration of the Logging service will be necessary if FM7.5 is applied to an existing OpenSSO (or AM7.1?) installation. Also, existing log files or tables should be saved or moved, then deleted before starting logging with the added NameID field, so that the log files will have consistent log records. 3.2 Goals and Guidelines System performance impacts should be as expected with increased logging load. It should be expected that more logging, particularly by specifying more detailed logging levels, may have an effect on the server's transaction throughput. Also, the change of the data type for the Data column (when logging to a database) from a VARCHAR(255) to a large object (CLOB in Oracle, LONGTEXT in MySQL) may have some effect on the database server's storage requirements. 3.3 Development Method <State development method if it differs from the OpenSSO development method> 4 Copyright 2007 Sun Microsystems, Inc. All rights reserved.

Architectural Strategies 0, Version 0 4 Architectural Strategies <State any existing technologies/libraries used.> <State any new strategy introduced.> Copyright 2007 Sun Microsystems, Inc. All rights reserved. 5

0, Version 0 System Architecture 5 System Architecture Currently, the log record fields logged are controlled by the "Configurable Log Fields" selections in the Logging Configuration page of the Administration Console. The logging service is dependent on the client creating the LogRecord to provide the values for the log record fields. The server's logging level is defaulted to INFO, though specific log files' logging level can be specified in the AMConfig.properties file. Also, overall logging can be inactivated with an entry in the AMConfig.properties file. 6 Copyright 2007 Sun Microsystems, Inc. All rights reserved.

Detailed System Architecture 0, Version 0 6 Detailed System Architecture 6.1 NameID Opaque Handle The attributes in the Administration Console's Logging configuration page reflect what is specified in the Logging service's amlogging.xml file. An addition for the nameid opaque handle must be made. Specifically, the addition will be to the iplanet-am-logging-logfields attribute in the amlogging.xml file. An associated addition to the amlogging.properties file will be made for internationalization purposes. The LogRecord class requires changes to handle the added nameid property. The LogRecord(Level level, String msg, Object token) method extracts properties from the SSOToken included. If the new property is not supplied in that manner, then each component must supply it using the LogRecord's addloginfo(string key, Object value) method. The authentication service utilizes the LogRecord(Level level, String msg, Hashtable properties) method. To provide the nameid property and value, it must be supplied in the Hashtable passed. If the nameid property and value are not supplied, the log record's nameid field will display "Not Available". Sample Log Records with the nameid not supplied (first) and supplied (second): #Fields: time Data ModuleName MessageID Domain ContextID LogLevel LoginID NameID IPAddr LoggedBy HostName "2007-02-22 17:34:50" "this is the new test log message" MyModule "Not Available" dc=opensso,dc=java,dc=net 786d16c9d25bfbb601 WARNING id=amadmin,ou=user,dc=opensso,dc=java,dc=net "Not Available" 192.18.120.170 id=amadmin,ou=user,dc=opensso,dc=java,dc=net alisun31 "2007-02-22 17:37:35 "this is the second test log message" MyModule "Not Available" dc=opensso,dc=java,dc=net 6e1d5b606a277b9901 SEVERE id=amadmin,ou=user,dc=opensso,dc=java,dc=net JoseJ 192.18.120.170 id=amadmin,ou=user,dc=opensso,dc=java,dc=net alisun31 Note that the logging level for the first record was specified as "WARNING", and "SEVERE" for the second. The code snippet for specifying the NameID property to the LogRecord uses the LogRecord's addloginfo method: LogRecord logrecord = new LogRecord(logLevel, logargs[messageindex], usertoken); logrecord.addloginfo("modulename", "MyModule"); logrecord.addloginfo("nameid", logargs[nameidindex]); Copyright 2007 Sun Microsystems, Inc. All rights reserved. 7

0, Version 0 Detailed System Architecture Logger logger = (Logger)Logger.getLogger(logArgs[LogNameIndex]); logger.log(logrecord, loggedbytoken); In the Administration Console's Logging configuration page, the Configurable Log Fields attribute gets the added NAMEID selection (not accurately depicted): Configurable Log Fields: CONTEXTID DOMAIN HOSTNAME IPADDR LOGGEDBY LOGINID LOGLEVEL MESSAGEID MODULENAME NAMEID 6.2 Dynamic Logging Level To effect dynamic setting of the server's logging level, an addition to the Administration Console's Logging configuration page will be made. Likewise with the nameid opaque handle, an addition to the amlogging.xml and amlogging.properties files are made. An attribute (sun-am-log-level) is added with ChoiceValues corresponding to the java.util.logging.level values (SEVERE, WARNING, INFO, CONFIG, FINE, FINER, FINEST, OFF), with default INFO. The Logging Level setting applies to all logs, except as noted below. Logging levels for individual log files could be provided, but then the Logging configuration page would get very long. Currently, the sms.dtd supports a radio uitype, but a pulldown type would allow for a more compact display. Another drawback of listing individual log names is the difficulty in supporting additional log names. Currently, logging levels for individual logs can be statically set by adding properties to the AMConfig.properties file of the form: iplanet-am-logging.<logname>.level=finer where <logname> is the log's full name, e.g., amsaml.access. Note that the Liberty, SAML, and Federation are the only components that implement any levels other than INFO. In order to observe AMConfig.properties logging level additions, Logging configuration page changes will apply to all logs other than those specified in AMConfig.properties. Also note that the AMConfig.properties entries are only read once, then cached by the SystemProperties class, so adding or changing logging level properties in that file after server startup will have no effect until the next server restart. Example amlogging.xml addition: <AttributeSchema name="sun-am-log-level" type="single_choice" syntax="string" 8 Copyright 2007 Sun Microsystems, Inc. All rights reserved.

Detailed System Architecture 0, Version 0 i18nkey="a130"> <ChoiceValues> <ChoiceValue i18nkey="choicelogsevere">severe</choicevalue> <ChoiceValue i18nkey="choicelogwarning">warning</choicevalue> <ChoiceValue i18nkey="choiceloginfo">info</choicevalue> <ChoiceValue i18nkey="choicelogconfig">config</choicevalue> <ChoiceValue i18nkey="choicelogfine">fine</choicevalue> <ChoiceValue i18nkey="choicelogfiner">finer</choicevalue> <ChoiceValue i18nkey="choicelogfinest">finest</choicevalue> <ChoiceValue i18nkey="choicelogoff">off</choicevalue> </ChoiceValues> <DefaultValues> <Value>INFO</Value> </DefaultValues> </AttributeSchema> and associated amlogging.properties additions: a130=logging Level choicelogsevere=severe choicelogwarning=warning choiceloginfo=info choicelogconfig=config choicelogfine=fine choicelogfiner=finer choicelogfinest=finest choicelogoff=off The resulting Logging service configuration addition would look like (not accurately depicted): Logging Level: CONFIG FINE FINER FINEST INFO OFF SEVERE WARNING Copyright 2007 Sun Microsystems, Inc. All rights reserved. 9

0, Version 0 Detailed System Architecture 6.3 Detailed XML Messages When logging to flatfiles, there should be no issue with the log record's Data field ( column in database parlance) having a massive (e.g., something like up to 10KBytes) amount of characters. There may be some issue trying to read the records with a simple utility such as vi, but that is not addressed here. When logging to a database (supported ones are Oracle and MySQL), the Data column has been defined as a VARCHAR(255), or a character data type of up to 255 bytes. The maximum for (the now deprecated) Oracle VARCHAR type is 4KBytes, and for the MySQL VARCHAR type is 65,532 bytes (or 255 bytes for MySQL 3.x/4.0/4.1). In order to support a Data column of up to 10KBytes, the Oracle CLOB (Character Large Object) data type should be used; and for MySQL, the LONGTEXT data type, with its 4GB (4294967296) maximum should be used. The maximum size for LONGTEXT is 4GB for MySQL 3.x through 5.0, so using it rather than VARCHAR would allow for more universal support for the large Data column, even if there is a statement of support for MySQL 4.2 and later. There may be some issue with using a larger data type for the Data column for all logging. The bulk of logging by the server does not need more than the currently allotted 255 bytes. The database's storage requirements may be affected by the change to the larger data type. A not-so-attractive alternative is to have the data type switchable between VARCHAR2/VARCHAR(255) and CLOB/LONGTEXT, but the existing tables would all have to be deleted, and the opensso server restarted. Incidentally, the VARCHAR2 data type will replace the VARCHAR type when defining the other columns when logging to an Oracle database. The new large object type for the Data column makes existing AM log tables incompatible with the new log tables. Existing tables would need to be archived and deleted. 6.4 Time Column The current data type for the Time (actually containing date and time) column is VARCHAR for both Oracle and MySQL tables. To use the appropriate data type for the respective databases, the DATE type should be used in Oracle and DATETIME type in MySQL. Changing the Time column from a VARCHAR to DATE/DATETIME type makes existing AM log tables incompatible with the new log tables. Existing tables would need to be archived and deleted. 10 Copyright 2007 Sun Microsystems, Inc. All rights reserved.