CatDV-StorNext Archive Additions: Installation and Configuration Guide

Transcription

1 CatDV-StorNext Archive Additions: Installation and Configuration Guide Quad Logic Systems Ltd Unit 3 Anglo Office Park Clarence Road Speedwell Bristol BS15 1NT Tel. +44 (0) Copyright & Trade Mark Acknowledgement Statements CatDV SquareBox StorNext Quantum -

2 CatDV-StorNext Archive Additions: Installation and Configuration Guide 1 CatDV-StorNext Archive Solution Overview System Requirements Example Deployment Architecture CatDV Pro (for Designated Archive Operators only) OSX and Windows CatDV Worker (One Macintosh Node Required) CatDV Server OSX or Windows StorNext Storage Manager RHEL5/ Pre-Requisites Configuring StorNext SN API Password Policy Folder Storage Policy Worker Preparation Install Archive Additions Create a Worker Folder Share CatDV StorNext Archive Additions: Server Configuration License Files Settings Deploying Worker Actions and Testing Communication Additional Configuration Settings CatDV-StorNext Archive Additions Folder Structure Install the CatDV Client Additions CatDV StorNext Archive Additions Worker Actions CatDV-StorNext Archive CatDV-StorNext Retrieve Troubleshooting Page 2 of 26

3 1 CatDV- StorNext Archive Solution Overview Quantum StorNext provides a SAN File System delivering high-performance file creation, storage and sharing of data. The addition of Storage Manager offers an intelligent means of data life cycle management. The CatDV StorNext Archive Additions solution leverages the power of CatDV Enterprise Server and StorNext Storage Manager to provide an intuitive, archival user interface. CatDV StorNext Archive Additions manages media location (whether online or committed to tape for long-term storage). Typically a small percentage of archived files will be needed for future events but protection of all assets makes sure that media can be made available when needed. Considering the cost of high bandwidth storage, the archive solution may also be employed for edit with proxy workflows where the high definition media is required for final conform tasks. CatDV provides first class media asset management and is the essential control component for your metadata, including file location and historical events relating to archive operations. CatDV StorNext Archive Additions moves media to and from the StorNext archive when requested. CatDV Server receives tracking metadata detailing the media location at any point in time. Please ensure you have a means of periodic back up of your CatDV Server database, as the information contained in iti is critical for management of the StoNext archive. The user interface for CatDV-SN Archive consists of three actions to empower nominated operators responsible for archive tasks. The product aims to be simple in use but to incorporate routines for validity checking and reporting to keep the user aware of archive related events. The CatDV StorNext Archive Additions comprises several additional programs over a standard CatDV Enterprise Server deployment: CatDV clients are used to control archive functions the CatDV Worker is used for the queuing, querying and StorNext communication. The user requests are queued for operation on the CatDV Worker node. Some initial checks during job submission ensure the receipt of valid operations e.g. media location and duplicate requests, which will provide early feedback to the operator. CatDV StorNext Archive Additions itself is a set of programs to handle data relocation and StorNext communication. StorNext Storage Manager is also, clearly a critical component Each component has its own installer 2 System Requirements The implementation has been tested and is supported with the following component versions Page 3 of 26

4 StorNext Storage Manager Red Hat Enterprise Server 5/6 (Linux is required for Storage Manager) StorNext and above CatDV-SN Worker Additions System requirements are as CatDV Worker Mac recommendations: OSX 10.6 and above CatDV Worker 4.x or Worker 5.x CatDV-SN Archivist Additions CatDV 9 v9 or later (desktop versions: CatDV Standard, CatDV Pro, CatDV Enterprise, CatDV Pegasus) System requirements are as CatDV Pro recommendations. CatDV Server At least CatDV Server 6.6. System requirements are as CatDV Server recommendations. Page 4 of 26

5 3 Example Deployment Architecture Figure 1 CatDV_SN Archive Software Distribution Page 5 of 26

6 3.1 CatDV Pro (for Designated Archive Operators only) OSX and Windows The user application, CatDV Pro, receives additional tools for archive operations. This only applies to staff designated as archive operators; there are no modifications to existing CatDV Pro users unless this functionality is required. Archive operators are provided with extra menu options to initiate archive or retrieval requests and to access historic performance data. Installation of CatDV-SN Archivist Additions is required on Archivists workstations. 3.2 CatDV Worker (One Macintosh Node Required) The CatDV Worker is employed to detect specific attribute changes and launch additional programs related to the task defined. If you have deployed multiple Worker nodes then only one is required to receive the additional archive components and scripts. Installation of CatDV-SN Worker Additions is required on this machine. 3.3 CatDV Server OSX or Windows There are no additions to CatDV server in the CatDV-SN Archive Solution unless the server is a Macintosh and you wish to run the Worker archiving actions on this machine. 3.4 StorNext Storage Manager RHEL5/6 Please see StorNext Storage Manager documentation for OS requirements. There are no additional programs for installation to the StorNext Server. The administrator will need to configure a Storage Manager policy that provides the relocation of media and truncation to comply with the desired workflow. Example configurations will be presented later in the documentation. Page 6 of 26

7 4 Pre- Requisites This section describes the items that are required before an installation should be attempted. 1. A CatDV server (v6.6 or above) and clients (v9 or above) installed (on either Mac or Windows) 2. A CatDV worker (v4 or above) installed and running on a Macintosh OSX 10.6 or above 3. CatDV running on appropriate hardware according to CatDV System requirements. 4. Appropriate CatDV path mappings if network paths on the CatDV clients (Pro, Enterprise or Pegasus), Server and Worker are different. 5. StorNext Storage Manager is already functioning and accessible on the network. 6. Appropriate networking so that CatDV clients, the CatDV Server, the CatDV Worker and StorNext can see the footage on the chosen storage. The CatDV Worker and Server may be on separate machines or potentially on the same machine if it s a Mac depending on the CatDV deployment. The Worker requires access to the project media in order to manipulate files to and from StorNext. This may consist of fiber or Ethernet accessible storage with SAN agents providing connectivity. CatDV StorNext Archive Additions communicates using addresses to make installation and configuration simpler. It is therefore important to maintain a static addressing scheme (which is normally the case in the SAN environment). The CatDV StorNext Archive Additions configuration program runs on the nominated CatDV Worker node machine and must be aware of these IP address targets, particularly the metadata controller. User accounts and related passwords will also be required to permit component communication. It may be useful to perform the following steps to verify the set up above has completed successfully: Test storage connectivity to ensure the volumes holding high-resolution media are accessible by the Worker and Archivist workstation. Archiving and retrieval operations require file movement so the highest bandwidth medium is preferred. Consider path mappings and thoroughly test if you need to support Windows and Mac clients as clips imported to CatDV from Windows users will require path mappings in some Worker actions. CatDV supports the use of proxies, which are not affected by archive or backup operations. Ensure the CatDV Worker is installed and licensed. It is recommended that the Worker folder be installed to the /Applications folder of the Mac. Configure the Worker to connect to CatDV Server where the login account has sufficient permissions to access/modify any clip in the system considered for archive or Page 7 of 26

8 retrieval. Please refer to CatDV Worker documentation for more information on how to install and configure the product. You may wish to place your own Worker scripts for other tasks on the same machine; this can be carried out at any time, as CatDV-StorNext Archive Additions will add archive tasks to the worker during installation. 5 Configuring StorNext A Storage Manager policy must be created to provide a folder as the target for archive operations. This policy folder will contain stub files relating to the original media. StorNext Manager is responsible for committing the selected files to longterm storage and releasing the space occupied on the SAN. CatDV-SN Archive tracks the location of media through disk to tape by updating the CatDV Server records. CatDV-SN Archive manages the retrieval process from request to restoration to the CatDV recorded path (or path equivalent) with the original file permissions. The policy folder is best located on a volume that is not normally accessed by users as this will help prevent unintended restore operations that can occur if stub files are accessed. A policy folder name that begins with a full stop. Can also assist as the folder is not visible to standard Mac clients. A StorNext configuration example will be shown and should act as a guide for your deployment. Whilst this section accounts for a large proportion of this manual, it is prerequisite to a functioning archive system. 5.1 SN API Password The StorNext Web Service is used for a number of media operations. This service is protected by a unique password, and saved to a file on the StorNext metadata controller. To check if the file exists, open 'Computer from the 'Places' menu heading. Continue opening 'Filesystem' to the path /usr/adic/ - select 'View"-'Show Hidden Files' in this directory and look for a file named.snapipassword You may open this file in a text editor to read any password set, or type in your preference. If the.snapipassword file does not exist on your system, the following guide will help in creating the entry. Log on to the Red Hat Server metadata controller as root and launch Text Editor from Applications Accessories menu. Page 8 of 26

9 Figure 2 Open Text Editor Type in your preferred password and save the text file as.snapipassword in /usr/adic. The password you have discovered or created is needed later by the CatDV-SN Archive, so make a note. If your StorNext installation incorporates high availability (HA) then this modification can be made to the partner server, however CatDV-SN Archive will not automatically failover and follow controller actions, please contact support if the failover situation is prolonged. 5.2 Policy Folder It is assumed that a tape library has been configured for use with the StorNext system and tapes allocated for archive operations. Please refer to StorNext documentation for help on this topic. A StorNext policy folder is required as a target for archive operations, there may be such a folder already configured but the following may help in revealing or creating a policy. The StorNext policy folder is prerequisite to any configuration. This is achieved in the StorNext web manger, which is normally accessible from any LAN workstation. Open a web browser and use the address or IP_address. The default login credentials are User: admin Password: password The following system has a tape library configured Page 9 of 26

10 Figure 3 Storage Destination Tape Library A view of Reports Media. Tapes are available 5.3 Storage Policy Figure 4 Media Reports Select Configuration Storage Policies (A default backup policy exists) Figure 5 Storage Manager Policies Select New and enter a name of catdv. The following screen shots show Page 10 of 26

11 suggested settings. Please refer to StorNext documentation for a more detailed explanation of the options available. Configuration settings entered for the catdv policy Note 5 Minutes is the minimum allowed. Figure 6 Storage Manager Policy General Tab The next step is to associate a directory with the policy, select the Associated Directories tab. This example uses a folder named.archive_catdv, created in volume store3, which must be a StorNext managed volume. The path used here is /stornext/store3/.archive_catdv (Note the preceding. to the folder name) This renders the folder invisible in the Finder of Macintosh clients. This is helpful in preventing inadvertent user actions on a system sensitive area. Page 11 of 26

12 The policy is applied to the folder you specify here. If the folder has been created on the file system then select and Add, otherwise use the Create Directory option and Add. Figure 7 Storage Manager Policy Associated Directories Tab Make a note of the policy folder path as it will be required during the configuration of CatDV-SN Archive. This completes the StorNext preparation, now is the time to choose the CatDV Worker node to handle CatDV-SN Archive requests. 6 Worker Preparation The Worker may exist on the CatDV Server (Macintosh) or a connected Macintosh client. It will need access to the media for relocation purposes and fiber interfaces provide the fastest performance. Install CatDV Worker following the default recommendations and apply your license. Configure the connection to CatDV Server, remember to use an account with sufficient permissions to read/write all clips that will Page 12 of 26

13 be subject to archive functions. 6.1 Install Archive Additions Install CatDV-SN Worker Additions, administrator rights will be required during installation and a user logout will occur. The installation will create a folder named SN_PROD421 at the root of the system disk containing programs, configuration and log files for the archive solution. 6.2 Create a Worker Folder Share Archivists have a command named StorNext Status in CatDV Pro Tools menu that will display historic throughput of archive related events. The Worker hosts the application and data that must be accessible from the Archivists workstations. The following guide assumes OSX workstation is in use; please refer to OSX Server documentation for file sharing if your operating system differs. Create a new standard user account on the Worker named SNArchive with a password of SNArchive. This will be used to restrict access to Archivists. A folder named /SN_PROD421/Results exists in the Worker system disk, which requires sharing over AFP - SMB is also required if any CatDV Pro clients are running Windows. Open system preferences Sharing and select File Sharing. Add a new share and browse to /SN_PROD421/Results Add the user SNArchive to the Results share and give read access. Select Options to modify protocols. 7 CatDV StorNext Archive Additions: Server Configuration 7.1 License Files CatDV-StorNext Archive Additions requires a licence for operation. The customer receives one or more files which must be copied to /SN_PROD421/Licence folder on the Worker to activate the archiving components. The license details may then be viewed in the License tab window of the CatDV-SN Archive Settings utility. The Licence tab displays the CatDV-SN Archive product license. Page 13 of 26

14 Figure 8 CatDV_SN Archive Licence Tab Note the above displays an expiry date which is ignored as the license Status is Full. A Status of Demo will be displayed for time limited licenses. 7.2 Settings Locate and launch /SN_PROD421/bin/Settings on the CatDV Worker. This utility is used to configure the product to suit your environment and view the installed product licence. The following screen shots are taken from a working deployment, with descriptions of the required information and use. Page 14 of 26

15 StorNext Web Service url Figure 9 CatDV_SN Archive Settings Tab E.g. Substitute your active StorNext controller IP address for StorNext_controller_ip_address Web Service pwd Enter the characters you discovered or created in /usr/adic/.snapipassword Policy Folder Enter the policy folder path you created in the form: /SAN_volume_name/policy folder name e.g. /store3/.catdv_archive CatDV Executable Enter the path to your CatDV Worker folder in use followed by /catdv e.g. /Applications/CatDV Worker 5.0.3/catdv Async WS Delay This should remain at the default of 600 SSH IP Address Page 15 of 26

16 SSH User ID Enter your active StorNext controller IP address root SSH Password Enter the root account password Days to keep logs The archive solution produces several log files, grouped on a daily basis. Items older than the value you enter here will be discarded. Worker Location Enter the path to the CatDV Worker in use. Any changes made in the Settings window are not committed until Save is selected under the File menu. 7.3 Deploying Worker Actions and Testing Communication The tab labelled Pre Flight is used to verify communication based on your settings and to deploy the required Worker Actions for archive functions. Note the Pre Flight is run against a Saved configuration in the Settings tab so do not make a change to Settings without saving or Pre Flight may not give the results expected. Figure 10 CatDV_SN Archive Pre Flight Tab Web Service and SSH communication to StorNext is attempted when Run Pre Flight is selected. If success is not reported, please revise your entries on the Settings tab and save prior to re-launching Pre Flight. The Worker Actions are queried for the existence of CatDV-SN Archive functions and inserted if not found. This utility may be run at anytime to verify program integrity or apply changes, please remember to Save any changes made in Settings before running a Pre Flight. Page 16 of 26

17 Launch CatDV Worker and verify that Worker actions named SN_Archive and SN_Retrieve exist. You may elect to add more Worker tasks but take care not to accidently modify SN_Archive and SN_Retrieve when viewing. Please refer to appendix A for details of the Worker scripts, which may be useful for future comparison. CatDV stores media path information as metadata and is used throughout the archive and retrieve processes. If your workflow requires the support of Windows media paths, CatDV-SN Archive supports the path equivalence model used by CatDV. This is best achieved on a CatDV Pro client to establish the required path mappings by testing prior to inserting into the SN_Archive action. Note that SN_Retrieve does not require any path mapping entries. 7.4 Additional Configuration Settings When configuration changes are Saved with the Settings utility, they are committed to a file named stornext.cfg which resides in /SN_PROD421/config folder. The Setting GUI does not expose all of the entries that exist in stornext.cfg When volumes are created in StorNext, the user may elect to not use the offered default mount point which starts with /stornext and a manual change to stornext.cfg will be necessary if this is the case in your deployment. TextEdit may be used on the Mac Worker to change line 13 (/stornext) to your specific mount point and save the file. Please do not modify any other settings or remove blank lines as errors will result in program flow. 7.5 CatDV- StorNext Archive Additions Folder Structure The installation of CatDV-SN Worker Additions created a folder named SN_PROD421 at the root of the Worker system disk. The folder structure and contents should not be re-located, as archive operation will be affected. /SN_PROD421/bin and Lib sub directory This folder contains executable routines to support archive functions. The Settings application is placed here, please create an alias if you wish to launch the utility from any other location. /SN_PROD421/config A file named stornext.cfg exists which is updated by changes made in the Settings utility. Do not modify this file directly. /SN_PROD421/licence You will normally receive a product licence as a zip attachment to an . Extract the contents of the zip file to this folder in order to activate CatDV-SN Archive. Page 17 of 26

18 /SN_PROD421/Report This folder contains an application named results.jar and a data file named results.xml. This folder should be shared to archivists for access to the StorNext Status utility. Please refer to Create a Worker Folder Share for details. /SN_PROD421/logs Log files are produced relating to phases of operation in archive functions, the files are closed at midnight and a new log is started. Some log entries contain debugging information but many report actions attempted with success or failure. The log file naming has the following format where log_type is a description of the activity. Log_typeyear-month-day.log archiveyear-month-day.log example archive log This log reports activity of the Worker detection process for archive requests. snretrieveyear-month-day.log example snretrieve log This log reports activity of the Worker detection process for restore requests. checkarchiveyear-month-day.log example checkarchive log Background process monitor for archive. checkretrieveyear-month-day.log example checkretrieve log Background process monitor for retrieve. execyear-month-day.log example exec log This log reports activity of the post restore process. Page 18 of 26

19 8 Install the CatDV Client Additions An installation of CatDV-StorNext Archive Additions is required on the CatDV Pro workstations that require archive functions. Installers are supplied for Mac and Windows platforms that will add archive tools to an existing CatDV Pro application. Launch CatDV Pro after installing CatDV-SN Archive Additions (aka Archivist Additions) and the Tools menu will contain extra selections similar to the following: Please refer to CatDV-StorNext User Guide for details on use. Page 19 of 26

20 9 CatDV StorNext Archive Additions Worker Actions This section describes the worker actions deployed as part of the Archive Additions 9.1 CatDV- StorNext Archive Two actions are automatically added during the Pre Flight process and details of settings are in the following screen shots. Pre Flight only checks for the existence of the scripts, not their accuracy, so the following may be useful in case of accidental modification. Only areas which require selection or input are shown unless otherwise stated. The worker actions are described below: General Server Query Figure 11 SN_Archive General Tab Figure 12 SN_Archive Server Query Tab Note: Text entered is case sensitive unless the 'Ignore case' check box is ticked Page 20 of 26

21 Parameters Figure 13 SN_Archive Parameters Tab Note if Windows paths are to be supported, enter mappings into Path Mappings Conversions Figure 14 SN_Archive Conversions Tab The text for Execute command 1: is as follows and is case sensitive java -jar /SN_PROD421/SNmovefile.jar $b $I Page 21 of 26

22 Post Processing Figure 15 SN_Archive Post-Processing Tab Note: The Publish to server must be set as 'Don't save changes' Select OK to save the action and create the second named SN_Retrieve 9.2 CatDV- StorNext Retrieve The worker action is described below General Figure 16 SN_Retrieve General Tab Page 22 of 26

23 Server Query Figure 17 SN_Retrieve Server Query Tab Note: Text entered is case sensitive unless the 'Ignore case' check box is ticked Parameters Figure 18 SN_Retrieve Parameters Tab Note: The Check interval may be lowered for testing purposes Conversions Figure 19 SN_Retrieve Conversions Tab The text for Execute command 1: is as follows and is case sensitive java -jar /SN_PROD421/SNretrieve.jar $b $l ${MS} Page 23 of 26

24 Post-Processing Figure 20 SN_Retrieve Post-Processing Tab Note: The Publish to server must be set as 'Don't save changes' Page 24 of 26

25 10 Troubleshooting All services must be functional to process archive functions. CatDV Server must accept the archive/retrieve requests so server login is required. The Worker Actions must be running, if metadata fields do not appear when a requested clip record is refreshed then check the Worker status. The Pre-Flight utility may be used to provide some guidance on communication or configuration problems. There is little room for error in the CatDV Archive Settings entered and committed during the Pre Flight process. If Pre Flight does not report errors, it is worthwhile submitting an archive of test assets to the intended archive via the StorNext Digital Archive User Interface. Whilst this action bypasses a lot of automation, it will convince you that StorNext has access to the media and submissions to your archive are successful. Check StorNext for any problems with this request. Remember to Save any modifications you make in Settings before running Pre Flight. When the CatDV Worker triggers a backup or archive action, the file paths are determined by path mapping associated with the event. Each entry relates to the path to be used by the Worker and entered into Archive Status. /Applications/CatDVArchive/Logs/Service.log recycles at 5MB and a new one is generated. Logs are maintained up to a count of twenty. The installation of CatDV Archivist Additions places additional files into a general Java extensions folder of the CatDV Pro client. This should allow for future upgrades to CatDV. As the archive function includes a final deletion of the online file, the Worker must have the ability to delete items on the storage location indicated by CatDV media path (or equivalent path if in Windows format). If the Worker has the volume mounted as read only then the file deletion will not be successful. This can occur in some SAN storage environments where read/write access is only given to a single user at any one time. CatDV Pro on the Windows platform may exhibit a file lock by merely having the clip selected. This can be avoided by modifying CatDV Preferences to Avoid pre-loading movie as shown. Page 25 of 26

26 CatDV StorNext Archive Additions does not prevent subsequent backup/archive requests, if so desired. The validity checks will report if an operation cannot be completed due to the current archive status of the clips involved. E.g. it is not possible to restore clips, which have not yet been archived, or attempting to archive a clip that is currently offline. If media is imported to CatDV via a Macintosh but archived by a Windows client, (or vice versa) a special case exists where this condition is discovered during media checks: The archivist may Submit the request as normal providing path equivalence mappings are valid in Worker tasks ( Move To Archive and Copy To Archive ). CatDV StorNext Status requires the archivist to enter the Worker IP address and credentials at the initial launch. Subsequent use of the tool relies upon a locally saved settings file in ~/Library/Application Support/CatDVArchive (Mac) or ~\appdata\local\catdvarchive (PC). This file should be deleted if the target Worker or its IP address ever has to change. Page 26 of 26