Review Manager Guide

Review Manager Guide v8.0 May 10, 2013 For the most recent version of this document, visit kcura's Documentation Site.

Table of Contents 1 Review Manager overview 4 1.1 Review Manager life cycle 4 1.2 The Review Manager console 4 1.3 Review Manager console options 5 1.4 Installation and upgrade 5 1.4.1 Selecting from the Application Library 5 1.4.2 Importing from file 6 1.4.3 Schema import 6 1.4.4 Review Manager configuration 6 1.4.5 Security configuration 7 1.4.6 Upgrade 8 1.5 Accessing Review Manager 8 1.6 Forecasts 8 1.6.1 Working in the Review Manager forecasts 9 1.6.2 Adding and editing a forecast 9 1.7 Forecast fields 9 1.8 Tiers 10 1.8.1 Adding and editing a tier 10 1.9 Tier fields 10 1.10 Roles 12 1.10.1 Adding a new role 12 1.11 Role fields 12 1.11.1 Linking roles and reviewer user 13 1.11.2 Linking roles and reviewer groups 14 1.12 Reports 14 1.12.1 Summary reports 15 1.12.2 Forecast summary report 15 1.12.3 Generating report PDFs 16 1.12.4 Optimizing a summary report 16 1.12.5 Insight summary report 17 1.12.6 Detail summary reports: reviewers insight, roles insight, and document type insight 18 1.13 Detail report input fields 22 1.14 Resources 22 Relativity Review Manager Guide - 2

1.14.1 Prior installations of Review Manager 22 Relativity Review Manager Guide - 3

1 Review Manager overview The Review Manager application allows you to generate forecasts, insights, and optimizations to help track the time and cost of review. Review Manager s graphical reports of key review metrics help streamline workflow and implement best practices. 1.1 Review Manager life cycle Review Manager generates forecasts to predict and measure the time, cost, and efforts required for a document review project. It then displays these metrics in graphical reports for trend recognition and other analysis. The following are examples of how Review Manager can assist with review workflow. Forecasts o Calculate time and cost estimates for document review projects o Provide graphical timelines of forecasted data to enhance early project assessment Insights o Provide real-time review metrics for trend identification analysis o Display information graphically for quick comprehension of project status, distribution and decision-making assistance Optimizations o o Present suggestions for increasing review efficiency and deadline integrity Offer course corrections prior to or during a review 1.2 The Review Manager console Similar to other consoles in Relativity, the Review Manager console appears on the right side of the layout once you've created a forecast. The Review Manager console runs the progression of your review forecast, as indicated by the different colored buttons. See Review Manager console options on the next page for details. You can reference the console to navigate to the next step while setting up your forecast, or run a report if your review is already in progress. Relativity Review Manager Guide - 4

1.3 Review Manager console options The Review Manager console contains the following options in the review forecast process: Forecast Home - displays forecast, tier, and role information. Used to navigate back to main layout of the forecast. Refresh Data - navigates to the data refresh layout. Before running Refresh Data, understand that this option can be system intensive and may take a few hours. We recommend running a Refresh Data during off-peak hours: o o o To avoid negatively impacting system performance if you're working in an existing case If criteria have changed in your saved searches If you're running the option to perform an incremental data refresh If you encounter a "Failed" message in the status field, contact your system administrator. Start Refresh - on-demand refresh. This button only appears in the console on the data refresh layout. We recommend to refresh data during non-peak hours due to the performance impact on the system. Cancel - cancels the requested data refresh process. This button only appears in the console on the data refresh layout. Forecast Summary - displays forecasted time and cost information in a graphical report Insight Summary - displays actual progress against forecasted time and cost information in a graphical report Reviewers Insight Detail - displays hourly and daily review metrics report by reviewer Roles Insight Details - displays hourly and daily review metrics report by role Document Type Insight Detail - displays hourly and daily review metrics report by document type Show Optimizations - displays suggestions for optimal number of reviewers, number of hours, and review rate display along with forecast information on the summary and insights reports Generate PDF - creates and saves a PDF of the report currently in view. A link to the PDF is stored below the report. Refresh Groups/Users - queries the latest group and user information into Review Manager 1.4 Installation and upgrade The Review Manager Schema import is executed entirely via the Relativity Desktop Client. Only a user that is a Relativity Administrator and Script Administrator can import application schema. If this is the first install of Review Manager, we recommend to import the application into an empty workspace as a test. This identifies any issues with configuration or naming conflicts in isolation. Once successfully installed to an empty workspace, the application can be imported to existing cases as needed. Follow the steps below to install Review Manager on your current Relativity instance. 1.4.1 Selecting from the Application Library Follow the steps below to import Review Manager using the Application Library. 1. Select the Relativity Applications tab. 2. Click the New Relativity Application button. 3. Click Select from Application Library for Application Type. Relativity Review Manager Guide - 5

4. Click the and select Review Manager, then click OK. The following categories on the form should appear in gray: Application Artifacts, and Map Fields. 5. Click Import. If the install is successful, the Review Manager tab is visible and Review Manager is listed on the All Relativity Applications view. 1.4.2 Importing from file Follow the steps below to import Review Manager using the Applications tab. 1. Select the Relativity Applications tab. 2. Click the New Relativity Application button. 3. Select Import from File for Application Type. 4. Click the, navigate to the place of your Review Manager Schema and select it. The following categories on the form should appear in gray: Application Information, Application Artifacts, and Map Fields. 5. Click Import. If the install is successful, the Review Manager tab is visible and Relativity Manager is listed on the All Relativity Applications view. 1.4.3 Schema import Follow the steps below to import Review Manager using the Relativity Desktop Client. 1. Open the Relativity Desktop Client and enter your login credentials. 2. After login, you are presented with a list of workspaces. Select the specific workspace to which you would like to install Review Manager. 3. From the object type drop-down under Administration, select Relativity Application. 4. In the RDC, navigate to Tools > Import > Application. 5. Select the Review Manager Schema by clicking the next to the Application File field. The Review Manager Schema is an.xml file included in the Review Manager Installation Package.zip file. 6. Click Application in the Tools > Import menu. a. Select the Application Schema and click Import. b. Import is kicked off. This may take several minutes. c. If successful, a table is populated and all values in the Status column read Created or Updated. If errors are present, contact support@kcura.com. 1.4.4 Review Manager configuration Upon installing Review Manager for the first time to an instance of Relativity, a Relativity System Administrator must configure the application. To configure Review Manager: 1. Enter the workspace to which you have installed Review Manager. 2. Select the Configuration sub-tab under the Review Manager tab. 3. Click New Review Manager Configuration. 4. Enter the Relativity Services API user name, password, domain name, and protocol (http or https). Use a Relativity System Admin account for the application access since all report activity is logged against this account. Relativity Review Manager Guide - 6

For example, the Services API domain should follow this format as noted below: <Your Machine Name>/Relativity.Services/ArtifactManager.svc a. Steps 3 and 4 are only necessary the first time Review Manager is installed to an instance of Relativity. They aren't necessary when copying the application between workspaces. b. Upon save, the Relativity Services API user name and Relativity Services API password are configured. The fields are view-only and read Configured By Relativity System Admin. To make any subsequent updates, edit the fields with proper account ID and password information. 5. Contact support@kcura.com with any questions. 1.4.5 Security configuration You must configure certain permissions for proper functionality of Review Manager. In addition, two layouts should be restricted to only the System Administrator. To configure these security settings: 1. Securing the Review Manager Configuration Basic layout for System Administrator viewing/editing only is done in the layout tab. a. Security must be added to the Layout tab view. b. Click the padlock icon on each of the configuration layouts to secure. c. Select Overwrite Inherited Security. d. Add Admin groups to the Review Manager Configuration layout only. Please make sure the non-admin groups are not present. 2. Create users and add them to a group for configuration with non-admin permissions. 3. Navigate to the Administration tab > Workspace Details. 4. Click Edit Permissions. 5. Click Add Group to add the group to which your non-admin users belong to in this workspace. 6. Select the desired group and click Set Permissions. 7. The following tabs must be checked: a. Review Manager b. Forecast c. Configuration 8. The following Mass Actions must be checked: a. Mass Copy 9. The following Security Permissions must be checked with Delete and Add status: a. Forecast b. ReviewManagerConfiguration c. ReviewerGroup d. ReviewerUser e. ForecastReport Relativity Review Manager Guide - 7

f. Role g. ForcastJob h. ForecastJobItem i. Tier j. ReviewManagerNativeType k. DetailReportRole l. RelativityTimeZone 10. Save group permissions and close. 11. All other Relativity permissions behave in standard fashion within Review Manager. Review Manager is now successfully installed to Relativity. 1.4.6 Upgrade Review Manager patches are released concurrently with Relativity patches. Therefore, information about Review Manager patches can be found in the Relativity patch release notes. To upgrade the application use one of the following methods: 1. Upgrade to the latest Relativity patch - Upgrading to the latest Relativity path will automatically upgrade Review Manager to the latest patched version. 2. Upgrade Review Manager only - Download the newly released schema from the Customer Portal and import it into the Application Library. 1.5 Accessing Review Manager To access Review Manager, log in to Relativity and select a case workspace. A Review Manager tab is available if Review Manager was installed on that specific workspace. Note: Review Manager must be installed on each workspace and users must have rights to see Review Manager tab. 1.6 Forecasts In Review Manager, the forecast is your most important component in managing your review. The forecast is the starting point of your review and the required fields in orange must be filled out before adding Tiers or Roles. Review Manager allows you to update your forecast throughout the review process; however, the more information you enter up front to define your forecast, the more complete and accurate your forecast is. Note: There can be only one active forecast per workspace at a time. You can quickly see which forecasts are active by clicking on Review Manager > Forecast and noting whether the Active field is set to Yes or No. Relativity Review Manager Guide - 8

1.6.1 Working in the Review Manager forecasts This guide illustrates screen content based on the Review Manager sub-tab. The data entry screens and practices for the Forecast tab are similar to those in Relativity. 1.6.2 Adding and editing a forecast 1. To add a new forecast, select the Review Manager > Forecast tab. 2. Select the New Forecast button. 3. To edit an existing forecast, select the edit link next to the forecast name. 4. You can add or edit information pertaining to the forecast on the Forecast screen. See Forecast fields below for details. 5. Click Save. 1.7 Forecast fields The Forecast layout contains the following fields: Relativity Review Manager Guide - 9

Name - displays the name of the forecast Review Start Date - the date when the review began or is expected to begin Deadline Date - the targeted completion date of a review (manually entered) Calculated Review End Date - system calculated end date base on forecasted number or documents, number of reviewers, and review rates Total Documents - total number of documents or expected number of documents for review Billable Unit - the unit for what billing method is to be employed. This gives you the flexibility to keep track of cost from different perspectives. For example, you may calculate cost by Reviewer Hour or by Document e.g. $100 per hour. The available Billable Units are: o Document o Gigabyte o Page (documents must be imaged) o Reviewer Hour Description - a field describing Forecast in greater detail Case - case title or case number Active Forecast - denotes an active Forecast when checked and enables Data Refresh functionality for current forecast. Only one forecast in a workspace can be active. Data Last Refreshed On - the date and time the forecast data was last modified Comments - additional information, mental impressions, or messages to colleagues related to the forecast or review 1.8 Tiers A tier describes a sub-set of work to be done as part of the overall review process. During tier creation, details are entered about the work to be done, the number of documents affected, and the time anticipated to complete the specified work. 1.8.1 Adding and editing a tier Once a forecast is created, you can add a new tier. 1. To add a new tier, click New Tier. 2. To edit an existing tier, click the edit link next to the tier name. 3. You can add or edit information to the tier on the Tier Forecast screen. See Tier fields below for details. 4. Click Save. 1.9 Tier fields The Tier Forecast layout contains the following fields: Relativity Review Manager Guide - 10

Forecast Name - the name of the forecast. This value is auto-populated with the forecast name. Tier Name - the descriptive name of the tier. For example, First Pass, QC Pass, Privilege Pass. Projected Total Documents - estimated number of documents that a given tier is anticipated to review Data Volume (GB or Pages) - the volume of your data set, either in GB or pages Tier Order - the logical sequence in time of a given tier. Tier Order is important because this controls how the tiers are sequenced and displayed graphically. Start Date Method - calculation technique for estimating the start date of a tier. You can select a specific start date or use a % overlap calculation. Set Calendar Date - date selected by the user for the forecasted start of a tier. This is used when Set Specific Calendar Date is chosen in the Start Date Method drop down. % Overlap with Previous Tier - overlap in start times of tiers for a forecast. This is used to estimate start dates of tiers when the % Overlap with Previous Tier is selected in the Start Date Method field. Working Days - days of the week in which review activity is conducted Working Hours Per Shift - number of hours in a shift. Shifts cannot exceed 24 hours. Relativity Review Manager Guide - 11

Actual Begin Date - actual date on which activity for a tier begins. This is determined by the user. From this point forward, actual audit data appears in the application. Available Document Saved Search - a saved search of the total documents to be reviewed in this tier. This is set up by the user on the Documents tab that holds the criteria for a document to be included in a given tier of review. For example, "designation is not set" could be the criteria for a first level review tier. Completed Document Saved Search - a saved search of the total documents where the review is complete in this tier. This is set up by the user on the Documents tab that holds the criteria for a document to be included in a given tier of review and considered. For example, "designation is not set" could be the criteria for a first level review tier. Available Document Total - the resulting document total from the Available Document saved search Completed Document Total - the resulting document total from the Completed Document saved search Note: Tying the tier to saved searches identifies the queries that Review Manager utilizes to relate case document metrics to a specific tier. By performing a standard saved search, Review Manager is able to query document totals from the case workspace and have them analyzed as part of this specific tier. Designating reviewers to a tier is necessary in order to take into account the actual reviewers and their working hours. See the Admin Guide for more information on creating saved searches. 1.10 Roles Once a forecast has been created, it's necessary to apply at least one role to each tier. A role is a designation for the type of reviewer that is actually be working on a given tier. Note that a tier is descriptive of the work to be done, whereas the role is descriptive of the worker selected to perform the work. There can be more than one role per tier. Reviewers are linked to the tiers via role; you must link a reviewer to a role, and then a role to the tier. Note that only reviewers in the Relativity case workspace as users are available from the list to associate to the role. Additionally, selecting the Click Refresh Data button on the console in the Role layout updates your user list with the most recent Relativity case workspace user data. 1.10.1 Adding a new role 1. To add a new role, select the New button next to the Role heading on the main Forecast layout. 2. You can add or edit information pertaining to the role on the Role layout screen. See Role fields below for more details. 3. Click Save. 4. You can now link reviewers to the role. 1.11 Role fields The Role layout contains the following fields: Relativity Review Manager Guide - 12

Forecast Name - the name of the current forecast in which the user is operating Role - the descriptive name of the role as determined by the user # of Reviewers - the number of reviewers to be included in a role as estimated by the user Billable Unit - the unit by which cost is being calculated. This field should already be auto-populated with the option selected on the forecast. For example, cost per document where document is the billable unit. This is chosen on the forecast home layout and persists to all roles. Billable Rate - the amount in terms of currency to be applied to the billable unit. For example, $2.00 per document (for document billable unit) or $150.00 per hour (for review hours billable unit). Review Rate - the speed of review in documents per hour as estimated by the user Tier - the role to tier association. Note that there can be multiple roles associated with one tier, therefore, as you define your second or third roles, you can select the same tier if you desire. 1.11.1 Linking roles and reviewer user After saving your new role, you can now link users (reviewers) to your role. 1. To link a reviewer to a role, notice the three options next to the Reviewer User heading: New, Link, and Unlink. 2. Click Link. Relativity Review Manager Guide - 13

3. From the pop-up window, select the checkboxes next to the user(s) you want to link to the role. Note: Only Relativity users that have access to the current workspace via their existing Relativity group permissions are displayed in the pop-up window. To refresh the Relativity user list, click on the Refresh Groups/Users button in the console (only System Administrators can perform this function). After clicking Refresh Groups/Users, any user that has access to Review Manager is able to see all users and groups for that workspace. 4. Click Add, then click Set. 1.11.2 Linking roles and reviewer groups Another way of linking users (reviewers) to your role is to associate a Reviewer Group to a role. 1. To link a Reviewer Group to a role, notice the three options next to the Reviewer Group heading: New, Link, and Unlink. 2. Click Link. 3. From the pop-up window, select the checkboxes next to the group(s) you want to link to the role. Note: Only Relativity groups that have access to the current workspace via their existing Relativity permissions are displayed in the pop-up window. To refresh the Relativity group list click on the Refresh Groups/Users button in the console (only System Administrators can perform this function). After clicking Refresh Groups/Users, any user that has access to Review Manager is able to see all users and groups for that workspace. 4. Click Add, then click Set. 1.12 Reports Review Manager generates key metric reports to track your review progress. These reports provide detailed and summary metrics for both forecasted and actual data. Reports are accessed via the Review Manager console, and can be exported as an Excel file via Relativity s Export to Excel feature. Please note that a completed forecast must be present in order to generate a summary report. An easy way to assess whether a forecast is complete is to verify that the Projected Total Documents field has been populated. Relativity Review Manager Guide - 14

1.12.1 Summary reports Review Manager summary reports allow you to view forecast time and cost information, as well as actual review progress as it compares to your forecast estimates. The Forecast Summary Report only displays your forecast estimates whereas the Insight Summary Report displays both forecasted and actual values. All summary report information is displayed graphically and in tables. 1.12.2 Forecast summary report The Forecast Summary report provides a graphic representation of your forecast inputs and calculations. The graphs show aggregations of all tiers and roles for Review Rate, Time, and Cost. In addition to graphically representing data, tables display numeric representations of aggregate data by tier. The summary report allows for quick comparison of review structure to budget and deadlines. In these reports, the Forecast Data is indicated by the light blue color, the Actual Results are indicated in blue, and the Results Beyond Projections are indicated in red. Relativity Review Manager Guide - 15

1.12.3 Generating report PDFs It is possible to create a PDF of all reports available in Review Manager. To do this, click Generate PDF on the console. Having PDFs of reports is a great way to quickly distribute this information via email or physical copy to folks outside of the Relativity user community. This functionality also allows users to save a snapshot in time so you can asses the evolution of the review. You also have the ability to remove all cost information from the PDF of the report if you so choose. To do this, navigate to the Report Configuration layout in the layout drop-down. A link to each PDF generated is created below the report. At the bottom of the layout is the newly generated report along with all previously generated reports. These reports can be opened and saved locally. You may delete the report PDFs from Relativity if you have the permission to do so. 1.12.4 Optimizing a summary report After reviewing your Forecast, adjustments might be necessary in order to put the project back on track. In these situations, Review Manager is able to provide suggested optimizations by means of the Show Optimizations feature, activated through the Review Manager Console. This option breaks each Tier down by number of reviewers, number of hours, and review rate. Review Manager then displays the differences between the Forecasted data and what is recommended in order to bring your project back within its desired parameters. These suggestions could mean adding additional resources or possibly leveraging different technologies to increase your review rates, such as mass tagging or Assisted Review. If you elect to employ one or more of Review Manager s optimization suggestions, you may also elect to go back and revise your Forecast (or create a new one) to reflect the adjustments. Relativity Review Manager Guide - 16

1.12.5 Insight summary report Once the review begins, the Insight Summary Report compares Forecasted data to your actual progress. Data from all Tiers is displayed, allowing you to see the status of your review as compared to your Forecasted information. Like the Forecast Summary report, this information appears graphically and in tables. Optimization suggestions are available for aligning your actual data with your Forecasted data. Relativity Review Manager Guide - 17

1.12.6 Detail summary reports: reviewers insight, roles insight, and document type insight Review Manager can report and get insights from Reviewers, Roles, and Document Types during a review. Using this information, you can identify behavior driving trends observed in the review. Perhaps these trends are weaknesses that can be improved upon to influence the outcome of your review. Alternatively, you may notice a trend that seems to be speeding up the review, and wish to apply it to other Tiers. The detailed summary reports provide quick insight into where deviations are occurring from your original Forecast. Monitoring this information in real time allows you to be proactive in your review management decisions. See Detail report input fields on page 22 for more details. 1.12.6.1 Reviewers insight detail report The Reviewers Insight Detail Report shows trends by reviewer for specific metrics. All of the detail reports can show data over an hourly or daily interval for various date ranges. For example, you can view the Average Total Edits per Hour over the course of a week. Distinct Edits, Total Views, Distinct Views, and Active Time can also be tracked and analyzed. The reports always display the first 10 selections alphabetically on the graph, but all selections are displayed in the table below. The graph shows two standard deviations above the average value for any chosen metric (a horizontal dashed line denotes the average and gray banding denotes one standard deviation). Users must be associated with Roles for their data to reflect in these reports. Also, data must be included in the most recent data refresh to be reflected in the report. Relativity Review Manager Guide - 18

Relativity Review Manager Guide - 19

1.12.6.2 Roles insight detail report The Role Insight Detail report allows for the comparison of trends among the different Roles that have been created in Review Manager. All of the Roles that have been created workspace wide are available in this report. Only activity from reviewers associated with Roles appears in the report. 1.12.6.3 Document type insight detail report Just like the Reviewer Insight Detail report and the Role Insight Detail report, information can be viewed for the different document native types that have were loaded into Relativity. Relativity Review Manager Guide - 20

Relativity Review Manager Guide - 21

1.13 Detail report input fields Below are the descriptions of the Detail report input fields: Detail Report Native Types - native file types to be displayed on the detail report. Choices determined by load file Detail Report Reviewers - reviewers to be displayed on the detail report. Only reviewers that have been added to the application are available Detail Report Roles - roles to be displayed on the detail report. All Roles from the workspace are available, but only Roles in current Forecast should be selected Name - descriptive name to be given to the report Report End Date - ending date selection to be displayed on the report. (Report scales to include first and last data point within the selected range only.) Report Interval - the interval for the x-axis of the detail report. Average hourly units or a standard daily/weekly/monthly interval can be chosen Report Start Date - starting date selection to be displayed on the report. (Report scales to include first and last data point within the selected range only.) 1.14 Resources The following additional Review Manager resources are available on kcura.com: Relativity Review Manager Overview Video o Located on the Review Manager page of the kcura Ecosystem Relativity Review Manager Tutorials o Located on the Review Manager page of the kcura Ecosystem 1.14.1 Prior installations of Review Manager For installations of Review Manager on Relativity 7.4 and earlier, locate the following resources on the Relativity Customer Portal: Installation Guide o Provided in the Installation Package.zip file Reference Guide o Provided in the Installation Package.zip file Relativity Review Manager Guide - 22

Proprietary Rights This documentation ( Documentation ) and the software to which it relates ( Software ) belongs to kcura Corporation and/or kcura s third party software vendors. kcura grants written license agreements which contain restrictions. All parties accessing the Documentation or Software must: respect proprietary rights of kcura and third parties; comply with your organization s license agreement, including but not limited to license restrictions on use, copying, modifications, reverse engineering, and derivative products; and refrain from any misuse or misappropriation of this Documentation or Software in whole or in part. The Software and Documentation is protected by the Copyright Act of 1976, as amended, and the Software code is protected by the Illinois Trade Secrets Act. Violations can involve substantial civil liabilities, exemplary damages, and criminal penalties, including fines and possible imprisonment. 2013. kcura Corporation. All rights reserved. Relativity and kcura are registered trademarks of kcura Corporation. Relativity Review Manager Guide - 23