The Storage Resource Manager Interface Specification Version 2.2

Transcription

1 GWD-R-XXXX Grid Strage Resurce Management Editrs: A. Sim A. Shshani 9/5/2007 The Strage Resurce Manager Interface Specificatin Versin 2.2 Status f this Mem This dcument prvides infrmatin t the Grid cmmunity regarding the specificatin f the Strage Resurce Management. Distributin f this dcument is unlimited. Cpyright Ntice Cpyright Open Grid Frum (2007). All Rights Reserved. Abstract Strage management is ne f the mst imprtant enabling technlgies fr large-scale scientific investigatins. Having t deal with multiple hetergeneus strage and file systems is ne f the majr bttlenecks in managing, replicating, and accessing files in distributed envirnments. Strage Resurce Managers (SRMs), named after their web services prtcl, prvide the technlgy needed t manage the rapidly grwing distributed data vlumes, as a result f faster and larger cmputatinal facilities. SRMs are Grid strage services prviding erfaces t strage resurces, as well as advanced functinality such as dynamic space allcatin and file management n shared strage systems. They call n transprt services t bring files their space transparently and prvide effective sharing f files. SRMs are based n a cmmn specificatin that emerged ver time and evlved an ernatinal cllabratin. This apprach f an pen specificatin that can be used by varius institutins t adapt t their wn strage systems has prven t be a remarkable success the challenge has been t prvide a cnsistent hmgeneus erface t the Grid, while allwing sites t have diverse infrastructures. In particular, ne f the main gals t the SRM web service is t supprt ptinal features while preserving erperability.

2 Table f Cntents Intrductin... 4 Meaning f terms Cmmn Type Definitins File Strage Type File Type Retentin Plicy Access Latency Permissin Mde Permissin Type Request Type Overwrite Mde File Lcality Access Pattern Cnnectin Type Status Cdes Retentin Plicy Inf Request Tken User Permissin Grup Permissin Size in Bytes UTC Time Time in Secnds (Lifetime and RequestTime) SURL TURL Return Status Return Status fr SURL File MetaData Space MetaData Directry Optin Extra Inf Transfer Parameters File Request fr srmpreparetget File Request fr srmpreparetput File Request fr srmcpy Return File Status fr srmpreparetget Return File Status fr srmbringonline Return File Status fr srmpreparetput Return File Status fr srmcpy Request Summary Return Status fr SURL Return File Permissins Return Permissins n SURL Return Request Tkens Supprted File Transfer Prtcl Space Management Functins srmreservespace srmstatusofreservespacerequest srmreleasespace srmupdatespace srmstatusofupdatespacerequest srmgetspacemetadata srmchangespacefrfiles gsm-wg@gf.rg - 2 -

3 2.8. srmstatusofchangespacefrfilesrequest srmextendfilelifetimeinspace srmpurgefrmspace srmgetspacetkens Permissin Functins srmsetpermissin srmcheckpermissin srmgetpermissin Directry Functins srmmkdir srmrmdir srmrm srmls srmstatusoflsrequest srmmv Data Transfer Functins srmpreparetget srmstatusofgetrequest srmbringonline srmstatusofbringonlinerequest srmpreparetput srmstatusofputrequest srmcpy srmstatusofcpyrequest srmreleasefiles srmputdne srmabrtrequest srmabrtfiles srmsuspendrequest srmresumerequest srmgetrequestsummary srmextendfilelifetime srmgetrequesttkens Discvery Functins srmgettransferprtcls srmping Appendix : Strage Resurce Managers Cncepts Summary Overview The Basic Cncepts Additinal cncepts rduced with v Current SRM Implementatins Based n V2.2 specificatin Security Cnsideratins Cntributrs Editrs infrmatin Cntributrs Acknwledgement Intellectual Prperty Statement Disclaimer Full Cpyright Ntice References gsm-wg@gf.rg - 3 -

4 Intrductin This dcument cntains the erface specificatin f SRM 2.2. It incrprates the functinality f SRM 2.0 and SRM 2.1, but is much expanded t include additinal functinality, especially in the area f dynamic strage space reservatin and directry functinality in client-acquired strage spaces. This dcument reflects the discussins and cnclusins f a 2-day meeting in May 2006 at Fermilab, which as fllwed by a 3-day meeting in September 2006 at CERN, as well as crrespndence and cnference calls. The purpse f this activity is t agree n the functinality and standardize the erface f Strage Resurce Managers (SRMs) a Grid middleware cmpnent. The dcument is rganized in fur sectins. The first, called Defined Structures cntain all the type definitins used t define the functins (r methds). The next 5 sectins cntain the specificatin f Space Management Functins, Permissin Functins, Directry Functins, Data Transfer Functins and Discvery Functins. All the Discvery Functins are newly added functins. The Appendix describes the main cncepts f SRMs as a standard middleware specificatin fr varius strage systems. It is ended t supprt the same erface t simple files systems, as well as sphisticated strage system that include multiple disk caches, rbtic tape systems, and parallel file systems. The appendix als lists several implementatins f SRM v2.2 arund the wrld, and their deplyment in varius sites. Fr peple nt familiar with SRM cncepts, It is advisable t read the Appendix first. Fr peple familiar with previus versins f SRM specificatins, it is advisable t read the dcument SRM.v2.2.changes.dc psted at befre reading this specificatin. Meaning f terms By https we mean http prtcl with GSI authenticatin. It may be represented as httpg. At this time, any implementatin f http with GSI authenticatin culd be used. It is advisable that the implementatin is cmpatible with Glbus Tlkit 3.2 r later versins. Primitive types used belw are cnsistent with XML build-in schema types: i.e. lng is 64bit: (+/-) is 32 bit: (+/-) shrt is 16 bit: (+/-) unsignedlng ranges (inclusive): 0 t unsignedint ranges (inclusive): 0 t unsignedshrt ranges (inclusive): 0 t The definitin f the type anyuri used belw is cmpliant with the XML standard. See It is defined as: "The lexical space f anyuri is finite-length character sequences which, when the algrithm defined in Sectin 5.4 f [XML Linking Language] is applied t them, result in s which are legal URIs accrding t [RFC 2396], as amended by [RFC 2732]". In lcalsurl, we mean lcal t the SRM that is prcessing the request. authrizatinid : frm the SASL RFC 2222 During the authenticatin prtcl exchange, the mechanism perfrms authenticatin, transmits an authrizatin identity (frequently knwn as a userid) frm the client t server. The transmitted authrizatin identity may be different than the identity in the client s authenticatin credentials. This permits agents such as prxy servers t authenticate using gsm-wg@gf.rg - 4 -

5 their wn credentials, yet request the access privileges f the identity fr which they are prxying. With any mechanism, transmitting an authrizatin identity f the empty directs the server t derive an authrizatin identity frm the client s authenticatin credentials. Regarding file sharing by the SRM, it is a lcal implementatin decisin. An SRM can chse t share files by prving multiple users access t the same physical file, r by cpying a file anther user s space. Either way, if an SRM chses t share a file (that is, t avid reading a file ver again frm the surce site) the SRM shuld check with the surce site whether the user has a read/write permissin. Only if permissin is granted, the file can be shared. The wrd pinning is limited t the cpies r states f SURLs and the Transfer URLs (TURLs). Fr each functin, status cdes are defined with basic meanings fr the functin. Only thse status cdes are valid fr the functin. Specific cases are nt stated fr each status cde. If ther status cdes need t be defined fr a specific functin, send an t the cllabratin t discuss the usage. gsm-wg@gf.rg - 5 -

6 1. Cmmn Type Definitins Namespace SRM Ntatin: underlined attributes are REQUIRED File Strage Type enum TFileStrageType {VOLATILE, DURABLE, PERMANENT} Vlatile file has an expiratin time and the strage may delete all traces f the file when it expires. Permanent file has n expiratin time. Durable file has an expiratin time, but the strage may nt delete the file, and shuld raise errr cnditin instead File Type enum TFileType {FILE, DIRECTORY, LINK} 1.3. Retentin Plicy enum TRetentinPlicy { REPLICA, OUTPUT, CUSTODIAL } Quality f Retentin (Strage Class) is a kind f Quality f Service. It refers t the prbability that the strage system lse a file. Numeric prbabilities are selfassigned. Replica quality has the highest prbability f lss, but is apprpriate fr data that can be replaced because ther cpies can be accessed in a timely fashin. Output quality is an ermediate level and refers t the data which can be replaced by lengthy r effrt-full prcesses. Custdial quality prvides lw prbability f lss. The type will be used t describe retentin plicy assigned t the files in the strage system, at the mments when the files are written the desired destinatin in the strage system. It will be used as a prperty f space allcated thrugh the space reservatin functin. Once the retentin plicy is assigned t a space, the files put in the reserved space will autmatically be assigned the retentin plicy f the space. The assigned retentin plicy n the file can be fund thught the TMetaDataPathDetail structure returned by the srmls functin Access Latency enum TAccessLatency { ONLINE, NEARLINE } Files may be Online, Nearline r Offline. These terms are used t describe hw latency t access a file is imprvable. Latency is imprved by strage systems replicating a file such that its access latency is nline. The ONLINE cache f a strage system is the part f the strage system which prvides file with nline latencies. ONLINE has the lwest latency pssible. N further latency imprvements are applied t nline files. NEARLINE file can have their latency imprved t nline latency autmatically by staging the file t nline cache. Fr cmpleteness, we als describe OFFLINE here. OFFLINE files need a human t be invlved t achieve nline latency. Fr the SRM we nly keep ONLINE and NEARLINE. gsm-wg@gf.rg - 6 -

7 The type will be used t describe a space prperty that access latency can be requested at the time f space reservatin. The cntent f the space, files may have the same r lesser access latency as the space Permissin Mde enum TPermissinMde {NONE, X, W, WX, R, RX, RW, RWX} 1.6. Permissin Type enum TPermissinType {ADD, REMOVE, CHANGE} 1.7. Request Type enum TRequestType { PREPARE_TO_GET, PREPARE_TO_PUT, COPY, BRING_ONLINE, RESERVE_SPACE, UPDATE_SPACE, CHANGE_SPACE_FOR_FILES, LS } 1.8. Overwrite Mde enum TOverwriteMde {NEVER, ALWAYS, WHEN_FILES_ARE_DIFFERENT} Use case fr WHEN_FILES_ARE_DIFFERENT can be that files are different when the declared size fr an SURL is different frm the actual ne, r that the checksum f an SURL is different frm the actual ne. Overwrite mde n a file is cnsidered higher pririty than pinning a file. Where applicable, it allws t mark a valid Transfer URL t becme invalid when the wner f the SURL issues an verwrite request File Lcality enum TFileLcality { ONLINE, NEARLINE, ONLINE_AND_NEARLINE, LOST, NONE. UNAVAILABLE } Files may be lcated nline, nearline r bth. This indicates if the file is nline r nt, r if the file reached t nearline r nt. It als indicates if there are nline and nearline cpies f the file. The ONLINE indicates that there is a file n nline cache f a strage system which is the part f the strage system, and the file may be accessed with nline latencies. The NEARLINE indicates that the file is lcated n nearline strage system, and the file may be accessed with nearline latencies. The ONLINE_AND_NEARLINE indicates that the file is lcated n nline cache f a strage system as well as n nearline strage system. The LOST indicates when the file is lst because f the permanent hardware failure. The NONE value shall be used if the file is empty (zer size). gsm-wg@gf.rg - 7 -

8 The UNAVAILABLE indicates that the file is unavailable due t the temprary hardware failure. The type will be used t describe a file prperty that indicates the current lcatin r status in the strage system Access Pattern enum TAccessPattern { TRANSFER_MODE, PROCESSING_MODE } TAccessPattern will be passed as an input parameter t the srmpreparetget and srmbringonline functins. It will make a h frm the client t SRM hw the Transfer URL (TURL) prduced by SRM is ging t be used. If the parameter value is PrcessingMde, the system may expect that client applicatin will perfrm sme prcessing f the partially read data, fllwed by mre partial reads and a frequent use f the prtcl specific seek peratin. This will allw ptimizatins by allcating files n disks with small buffer sizes. If the value is TransferMde the file will be read at the highest speed allwed by the cnnectin between the server and a client Cnnectin Type enum TCnnectinType { WAN, LAN } TCnnectinType indicates if the client is cnnected thugh a lcal r wide area netwrk. SRM may ptimize the access parameters t achieve maximum thrughput fr the cnnectin type. This will be passed as an input t the srmpreparetget, srmpreparetput and srmbringonline functins Status Cdes enum TStatusCde {,, SRM_AUTHENTICATION_FAILURE,, SRM_INVALID_REQUEST, SRM_INVALID_PATH, SRM_FILE_LIFETIME_EXPIRED, SRM_SPACE_LIFETIME_EXPIRED, SRM_EXCEED_ALLOCATION, SRM_NO_USER_SPACE, SRM_NO_FREE_SPACE, SRM_DUPLICATION_ERROR, SRM_NON_EMPTY_DIRECTORY, SRM_TOO_MANY_RESULTS, SRM_INTERNAL_ERROR, SRM_FATAL_INTERNAL_ERROR, SRM_NOT_SUPPORTED, SRM_REQUEST_QUEUED, SRM_REQUEST_INPROGRESS, SRM_REQUEST_SUSPENDED, SRM_ABORTED, SRM_RELEASED, SRM_FILE_PINNED, SRM_FILE_IN_CACHE, SRM_SPACE_AVAILABLE, SRM_LOWER_SPACE_GRANTED, gsm-wg@gf.rg - 8 -

9 SRM_DONE, SRM_PARTIAL_SUCCESS, SRM_REQUEST_TIMED_OUT, SRM_LAST_COPY, SRM_FILE_BUSY, SRM_FILE_LOST, SRM_FILE_UNAVAILABLE, SRM_CUSTOM_STATUS } SRM_NOT_SUPPORTED is used, in general If a server des nt supprt a methd If a server des nt supprt particular ptinal input parameters Retentin Plicy Inf typedef struct { TRetentinPlicy retentinplicy, TAccessLatency accesslatency } TRetentinPlicyInf TRetentinPlicyInf is a cmbined structure t indicate hw the file needs t be stred. When bth retentin plicy and access latency are prvided, their cmbinatin needs t match what SRM supprts. Otherwise request will be rejected Request Tken The Request Tken assigned by SRM is unique and immutable (nn-reusable). Fr example, if the date:time is part f the request tken it will be immutable. Request tkens are case-sensitive. Request tken is valid until the request is cmpleted. Hwever, SRM server may chse t keep the request tkens fr a shrt perid f time after the request is cmpleted, and the time perid depends n the SRM servers User Permissin typedef struct { userid, TPermissinMde mde } TUserPermissin userid may represent the assciated client s Distinguished Name (DN) instead f unix style lgin name. VOMS rle may be included Grup Permissin typedef struct { grupid, TPermissinMde mde } TGrupPermissin grupid may represent the assciated client s Distinguished Name (DN) instead f unix style lgin name. VOMS rle may be included Size in Bytes Size in bytes is represented in unsigned lng. gsm-wg@gf.rg - 9 -

10 1.18. UTC Time Time is represented in datetime. Frmerly TGMTTime in SRM v2.1 date and time in Crdinated Universal Time (UTC, frmerly GMT) with n lcal time extentin. Frmat is same as in XML datetime type, except n lcal time extensin is allwed. E.g T13:20:00 is k (fr 1999 May 31st, 13:20PM, UTC) but T13:20:00-5:00 is nt Time in Secnds (Lifetime and RequestTime) Time (lifetime and request time) in secnds is represented in eger. 0 (zer) indicates the site defined default time. A negative value (-1) indicates infinite (indefinite) time. Exceptins: Any remaining times may have zer (0) secnd when n time is left. Sme special meaning f negative time is defined when needed depending n the peratin. E.g. remainingttalrequesttime in srmstatusofgetrequest SURL The type definitin SURL is represented as anyuri and used fr bth site URL and the Strage File Name (stfn). This was dne in rder t simplify the ntatin. Recall that stfn is the file path/name f the ended strage lcatin when a file is put (r cpied) an SRM cntrlled space. Thus, a stfn can be thught f a special case f an SURL, where the prtcl is assumed t be srm and the machine:prt is assumed t be lcal t the SRM. Fr example, when the request srmcpy is made as a pulling case, the surce file is specified by a site URL, and the target lcatin can be ptinally specified as a stfn. By cnsidering the stfn a special case f an SURL, a srmcpy takes SURLs as bth the surce and target parameters TURL TURL is represented in anyuri Return Status typedef struct {TStatusCde statuscde, explanatin } TReturnStatus Return Status fr SURL typedef struct {anyuri surl, TReturnStatus status } TSURLReturnStatus File MetaData typedef struct { path, // abslute dir and file path TReturnStatus status, gsm-wg@gf.rg

11 unsigned lng size, // 0 if directry datetime createdattime, datetime lastmdificatintime, TFileStrageType filestragetype, TRetentinPlicyInf retentinplicyinf, TFileLcality filelcality, [] arrayofspacetkens, TFileType type, // Directry r File lifetimeassigned, lifetimeleft, // n the SURL TUserPermissin wnerpermissin, TGrupPermissin gruppermissin, TPermissinMde therpermissin, checksumtype, checksumvalue, TMetaDataPathDetail[] arrayofsubpaths // ptinal recursive } TMetaDataPathDetail The TMetaDataPathDetail describes the prperties f a file. It is used as an utput parameter in srmls. retentinplicyinf indicates the assigned retentin plicy. filelcality indicates where the file is lcated currently in the system. arrayofspacetkens as an array f indicates where the file is currently lcated fr the client. Only space tkens that the client has authrized t access t read the file must be returned. Permissins n the SURL represent unix-like permissins: e.g. rwxr--r--. wnerpermissin describes the wner ID and wner permissin n the SURL. gruppermissin describes the grup permissin with grup identifier n the SURL. therpermissin describes the ther permissin n the SURL. Fr ACL-like permissins, srmgetpermissin must be used. lifetimeassigned is the ttal lifetime that is assigned n the SURL. It includes all SURL lifetime extensins if extended. lifetimeleft is the remaining lifetime n the SURL frm the current time until expiratin. A negative value (-1) indicates indefinite lifetime. Zer (0) indicates that the file is expired Space MetaData typedef struct { spacetken, TReturnStatus status, TRetentinPlicyInf retentinplicyinf, wner, unsigned lng ttalsize, // best effrt unsigned lng guaranteedsize, unsigned lng unusedsize, lifetimeassigned, lifetimeleft } TMetaDataSpace TMetaDataSpace is used t describe prperties f a space, and is used as an utput parameter in srmgetspacemetadata. retentinplicyinf indicates the infrmatin abut retentin plicy and access latency that the space is assigned. retentinplicyinf is requested and assigned at the gsm-wg@gf.rg

12 time f space reservatin thrugh srmreservespace and srmstatusofreservespacerequest. TMetaDataSpace refers t a single space with retentin plicy. It des nt include the extra space needed t hld the directry structures, if there is any. lifetimeassigned is the ttal lifetime that is assigned t the space. It includes all space lifetime extensins if extended. lifetimeleft is the remaining lifetime that is left n the space Directry Optin typedef struct { Blean issurceadirectry, Blean alllevelrecursive, // default = false numoflevels // default = 1 } TDirOptin Extra Inf typedef struct { key, value } TExtraInf TExtraInf is used where additinal infrmatin is needed, such as fr additinal infrmatin fr transfer prtcls f TURLs in srmping, srmgettransferprtcls, srmstatusofgetrequest, and srmstatusofputrequest. Fr example, when it is used fr additinal infrmatin fr transfer prtcls, the keys may specify access speed, available number f parallelism, and ther transfer prtcl prperties. It is als used where additinal infrmatin t the underlying strage system is needed, such as fr additinal infrmatin, but nt limited t, fr strage device, strage lgin ID, strage lgin authrizatin. Frmerly, it was TStrageSystemInf Transfer Parameters typedef struct { TAccessPattern accesspattern, TCnnectinType cnnectintype, [] arrayofclientnetwrks [] arrayoftransferprtcls } TTransferParameters TTransferParameters is used where arrayoftransferprtcls was used previusly in SRM v2.1. TTransferParameters may be prvided ptinally in the methds such as srmpreparetget, srmbringonline, srmpreparetput and srmreservespace. Optinal input parameters in TTransferParameters may cllide with the characteristics f the space specified. In this case, TTransferParameters as an input parameter must be ignred. File transfer prtcls are specified in a preferred rder n all SRM transfer functins. arrayofclientnetwrks is a h f the client IPs that SRM/dCache can use fr ptimizatin f its ernal strage systems based n the client s accessible IP addresses File Request fr srmpreparetget typedef struct { anyuri surcesurl, TDirOptin diroptin, } TGetFileRequest File Request fr srmpreparetput gsm-wg@gf.rg

13 typedef struct { anyuri targetsurl, // lcal t SRM unsigned lng expectedfilesize } TPutFileRequest File Request fr srmcpy typedef struct { anyuri surcesurl, anyuri targetsurl, TDirOptin diroptin } TCpyFileRequest Return File Status fr srmpreparetget typedef struct { anyuri surcesurl, TReturnStatus status, unsigned lng filesize, estimatedwaittime, remainingpintime, anyuri transferurl, TExtraInf[] transferprtclinf } TGetRequestFileStatus transferprtclinf f type TExtraInf is added t the TGetRequestFileStatus. This utput parameter can be used t prvide mre infrmatin abut the transfer prtcl s that client can access the TURL efficiently. estimatedwaittime t be negative value, -1, fr unknwn. remainingpintime is the lifetime n the TURL, and 0 means it expired. If a TURL has an indefinite lifetime, then negative value, -1, may be used Return File Status fr srmbringonline typedef struct { anyuri surcesurl, TReturnStatus status, unsigned lng filesize, estimatedwaittime, remainingpintime, } TBringOnlineRequestFileStatus estimatedwaittime t be negative value, -1, fr unknwn. remainingpintime is the lifetime n the TURL, and 0 means it expired. If a TURL has an indefinite lifetime, then negative value, -1, may be used Return File Status fr srmpreparetput typedef struct { anyuri SURL, TReturnStatus status, unsigned lng filesize, estimatedwaittime, remainingpinlifetime // n TURL remainingfilelifetime // n SURL anyuri transferurl, TExtraInf[] transferprtclinf } TPutRequestFileStatus transferprtclinf f type TExtraInf is added t the TPutRequestFileStatus t give clients mre infrmatin abut the prepared transfer prtcl s that client may use gsm-wg@gf.rg

14 the infrmatin t make an efficient access t the prepared TURL thrugh the transfer prtcl. estimatedwaittime t be negative value, -1, fr unknwn. remainingpintime is the lifetime n the TURL, and 0 means it expired. If a TURL has indefinite lifetime, then negative value, -1, may be used. remainingfilelifetime is the lifetime n the SURL, and 0 means it expired. If SURL has an indefinite lifetime, then negative value, -1, may be used Return File Status fr srmcpy typedef struct { anyuri surcesurl, anyuri targetsurl, TReturnStatus status, unsigned lng filesize, estimatedwaittime, remainingfilelifetime // n target SURL } TCpyRequestFileStatus estimatedwaittime t be negative value, -1, fr unknwn. remainingfilelifetime is the lifetime n the SURL, and 0 means it expired. If SURL has an indefinite lifetime, then negative value, -1, may be used Request Summary typedef struct { requesttken, TReturnStatus status, TRequestType requesttype, ttalnumfilesinrequest, numofcmpletedfiles, numofwaitingfiles, numoffailedfiles } TRequestSummary numofwaitingfiles describes the number f files n the queue. numoffailedfiles describes the number f failed files and abrted files. numofcmpletedfiles describes the number f successfully cmpleted files, number f failed files and number f abrted files. ttalnumfilesinrequest describes the numofwaitingfiles, numofcmpletedfiles, numoffailedfiles and number f files in prgress Return Status fr SURL typedef struct { anyuri surl, TReturnStatus status filelifetime, pinlifetime, } TSURLLifetimeReturnStatus filelifetime describes the file lifetime n SURL. pinlifetime describes the pin lifetime n TURL, if applicable Return File Permissins typedef struct {anyuri surl, TReturnStatus status, TPermissinMde permissin } TSURLPermissinReturn gsm-wg@gf.rg

15 1.39. Return Permissins n SURL typedef struct { anyuri surl, // bth dir and file TReturnStatus status, wner, TPermissinMde wnerpermissin, TUserPermissin[] arrayofuserpermissins, TGrupPermissin[] arrayofgruppermissins, TPermissinMde therpermissin } TPermissinReturn The TPermissinReturn describes the permissin prperties f a file. It is used as an utput parameter in srmgetpermissin Return Request Tkens typedef struct { requesttken, datetime createdattime } TRequestTkenReturn Supprted File Transfer Prtcl typedef struct { transferprtcl, TExtraInf[] attributes } TSupprtedTransferPrtcl transferprtcl (required): Supprted transfer prtcl. Fr example, gsiftp, http. attributes: Infrmatinal hs fr the paired transfer prtcl, such hw many number f parallel streams can be used, desired buffer size, etc. gsm-wg@gf.rg

16 2. Space Management Functins summary: srmreservespace srmstatusofreservespacerequest srmreleasespace srmupdatespace srmgetspacemetadata srmchangespacefrfiles srmstatusofchangespacefrfilesrequest srmextendfilelifetimeinspace srmpurgefrmspace srmgetspacetkens 2.1. srmreservespace This functin is used t reserve a space in advance fr the upcming requests t get sme guarantee n the file management. Asynchrnus space reservatin may be necessary fr sme SRMs t serve many cncurrent requests Parameters In: authrizatinid, String userspacetkendescriptin, TRetentinPlicyInf retentinplicyinf, unsigned lng desiredsizeofttalspace, unsigned lng desiredsizeofguaranteedspace, desiredlifetimeofreservedspace, unsigned lng [] arrayofexpectedfilesizes, TExtraInf[] stragesysteminf, TTransferParameters transferparameters Out: TReturnStatus returnstatus, requesttken, estimatedprcessingtime, TRetentinPlicyInf retentinplicyinf, unsigned lng sizeofttalreservedspace, // best effrt unsigned lng sizeofguaranteedreservedspace, lifetimeofreservedspace, spacetken Ntes n the Behavir a) Input parameter userspacetkendescriptin is case-sensitive. SRM server is expected t keep it as client prvides. It can be reused by the client. srmgetspacetkens will return all the space tkens that have the userspacetkendescriptin. b) If the input parameter desiredlifetimeofreservedspace is nt prvided, the lifetime f the reserved space may be set t infinite (indefinite) by default. c) If particular values f the input parameter retentinplicyinf cannt be satisfied by the SRM server, SRM_NOT_SUPPORTED r SRM_NO_FREE_SPACE must be returned. d) Asynchrnus space reservatin may be necessary fr sme SRMs t serve many cncurrent requests. In such case, request tken must be returned, and space tken must nt be assigned and returned until space reservatin is cmpleted, t prevent the usage gsm-wg@gf.rg

17 f the space tken in ther erfaces befre the space reservatin is cmpleted. If the space reservatin can be dne immediately, request tken must nt be returned. e) When asynchrnus space reservatin is necessary, the returned status cde shuld be SRM_REQUEST_QUEUED. f) Input parameter arrayofexpectedfilesize is a h that SRM server can use t reserve cnsecutive strage sizes fr the request. At the time f space reservatin, if space accunting is dne nly at the level f the ttal size, this h wuld nt help. In such case, the expected file size at the time f srmpreparetput will describe hw much cnsecutive strage size is needed fr the file. Hwever, sme SRMs may get benefits frm these hs t make a decisin t allcate sme blcks in sme specific devices. g) Optinal input parameter stragesysteminf is needed in case the underlying strage system requires additinal security infrmatin. h) SRM may return its default space size and lifetime if nt requested by the client. SRM may return SRM_INVALID_REQUEST if SRM des nt supprt default space sizes. i) If input parameter desiredsizeofttalspace is nt specified, the SRM will return its default space size. j) Output parameter estimateprcessingtime is used t indicate the estimatin time t cmplete the space reservatin request, when knwn. k) Output parameter sizeofttalreservedspace is in best effrt bases. Fr guaranteed space size, sizeofguaranteedreservedspace shuld be checked. These tw numbers may match, depending n the strage systems. l) Output parameter spacetken is a reference handle f the reserved space. m) If an peratin is successful ( r SRM_LOWER_SPACE_GRANTED), sizeofguaranteedreservedspace, lifetimeofreservedspace and spacetken are required t return t the client. n) Optinal input parameters in TTransferParameters may cllide with the characteristics f the space specified. In this case, TTransferParameters as an input parameter must be ignred Return Status Cde successful request cmpletin. Space is reserved successfully as the client requested. SRM_REQUEST_QUEUED successful request submissin and acceptance. Request tken must be returned, and space tken must nt be assigned and returned. SRM_REQUEST_INPROGRESS the request is being prcessed. SRM_LOWER_SPACE_GRANTED successful request cmpletin, but lwer space size is allcated than what the client requested SRM_AUTHENTICATION_FAILURE SRM server failed t authenticate the client client is nt authrized t reserve space SRM_INVALID_REQUEST If space size r lifetime is nt requested by the client, and SRM des nt supprt default values fr space size r lifetime. input parameters are invalid. SRM_NO_USER_SPACE SRM server des nt have enugh user space fr the client fr client t request t reserve. SRM_NO_FREE_SPACE SRM server des nt have enugh free space fr client t request t reserve. gsm-wg@gf.rg

18 SRM server des nt have enugh free space fr a particular retentinplicyinf SRM_EXCEED_ALLOCATION SRM server des nt have enugh space fr the client t fulfill the request because the client request needs mre than the allcated space quta fr the client. SRM_INTERNAL_ERROR SRM has an ernal transient errr, and client may try again. any ther request failure. Explanatin needs t be filled fr details. SRM_NOT_SUPPORTED functin is nt supprted in the SRM server specific values f the input parameter retentinplicyinf is nt supprted by the SRM any input parameter is nt supprted in the SRM server a particular type f an input parameter is nt supprted in the SRM server 2.2. srmstatusofreservespacerequest This functin is used t check the status f the previus request t srmreservespace, when asynchrnus space reservatin was necessary with the SRM. Request tken must have been prvided in respnse t the srmreservespace Parameters In: authrizatinid, requesttken Out: TReturnStatus returnstatus, estimatedprcessingtime, TRetentinPlicyInf retentinplicyinf, unsigned lng sizeofttalreservedspace, unsigned lng sizeofguaranteedreservedspace, lifetimeofreservedspace, spacetken Ntes n the Behavir a) If the space reservatin is nt cmpleted yet, estimateprcessingtime is returned when knwn. The returned status cde in such case shuld be SRM_REQUEST_QUEUED. b) See ntes fr srmreservespace fr descriptins fr utput parameters. c) If an peratin is successful ( r SRM_LOWER_SPACE_GRANTED), sizeofguaranteedreservedspace, lifetimeofreservedspace and spacetken are required t return t the client Return Status Cde SRM_REQUEST_QUEUED successful request submissin and the request is still n the queue t be served. SRM_REQUEST_INPROGRESS the request is being prcessed. SRM_LOWER_SPACE_GRANTED gsm-wg@gf.rg

19 successful request cmpletin, but lwer space size is allcated than what the client requested successful request cmpletin. Space is reserved successfully as the client requested. SRM_AUTHENTICATION_FAILURE SRM server failed t authenticate the client client is nt authrized t reserve space SRM_INVALID_REQUEST requesttken des nt refer t an existing knwn request in the SRM server. SRM_EXCEED_ALLOCATION SRM server des nt have enugh space fr the client t fulfill the request because the client request needs mre than the allcated space fr the client. SRM_NO_USER_SPACE SRM server des nt have enugh user space fr the client fr the client fr client t request t reserve. SRM_NO_FREE_SPACE SRM server des nt have enugh free space fr the client fr client t request t reserve. SRM server des nt have enugh free space fr a particular retentinplicyinf SRM_REQUEST_SUSPENDED request is suspended. SRM_INTERNAL_ERROR SRM has an ernal transient errr, and client may try again. any ther request failure. Explanatin needs t be filled fr details. SRM_NOT_SUPPORTED functin is nt supprted in the SRM server any input parameter is nt supprted in the SRM server a particular type f an input parameter is nt supprted in the SRM server 2.3. srmreleasespace srmreleasespace() releases an ccupied space Parameters In: authrizatinid, spacetken, TExtraInf[] stragesysteminf, Blean frcefilerelease Out: TReturnStatus returnstatus Ntes n the Behavir a) frcefilerelease is false by default. This means that the space will nt be released if it has files that are still pinned in the space. T release the space regardless f the files it cntains and their status frcefilerelease must be specified t be true. gsm-wg@gf.rg

20 b) When space is releasable and frcefilerelease is true, all the files in the space are released, even in OUTPUT r CUSTODIAL retentin quality space. c) srmreleasespace may nt cmplete right away because f the lifetime f existing files in the space. When space is released, the files in that space are treated accrding t their types: If file strage types are permanent, keep them until further peratin such as srmrm is issued by the client. If file strage types are durable, perfrm necessary actins at the end f their lifetime. If file strage types are vlatile, release thse files at the end f their lifetime. d) If space is being released with frcefilerelease ptin while SURLs are being created with srmpreparetput r srmcpy, the file is remved and SRM_INVALID_PATH must be returned by the srmputdne, srmstatusofputrequest, r srmstatusofcpyrequest when the file is vlatile. If the file is permanent type, the file is mved t the default space, and the space wuld be successfully released. The subsequent srmputdne, srmstatusofputrequest, r srmstatusofcpyrequest wuld be successful. e) If space is being released withut frcefilerelease ptin while SURLs are being created with srmpreparetput r srmcpy, must be returned in srmreleasespace. f) When a "replica" quality space is expired n its lifetime, all files inside must be expired (by definitin, file lifetimes are less than and equal t the remaining lifetime f the space). After the space is expired, the space that is assciated with the space tken n lnger exists, alng with all files inside - meaning their SURLs disappear frm the file system r reflect the expired lifetime Return Status Cde successful request cmpletin. Space is successfully released. SRM_AUTHENTICATION_FAILURE SRM server failed t authenticate the client client is nt authrized t release the space that is assciated with the spacetken SRM_INVALID_REQUEST spacetken des nt refer t an existing knwn space in the SRM server. SRM_INTERNAL_ERROR SRM has an ernal transient errr, and client may try again. SRM_NOT_SUPPORTED frcefilerelease is nt supprted functin is nt supprted space still cntains pinned files. space assciated with space is already released. any ther request failure. Explanatin needs t be filled fr details srmupdatespace srmupdatespace is t resize the space and/r extend the lifetime f a space. Asynchrnus peratin may be necessary fr sme SRMs t serve many cncurrent requests Parameters In: authrizatinid, spacetken, unsigned lng newsizeofttalspacedesired, unsigned lng newsizeofguaranteedspacedesired, gsm-wg@gf.rg

21 TExtraInf[] newlifetime, stragesysteminf Out: TReturnStatus returnstatus, requesttken, unsigned lng sizeofttalspace, // best effrt unsigned lng sizeofguaranteedspace, lifetimegranted Ntes n the Behavir a) If neither size nr lifetime is prvided in the input parameters, then the request will be failed, and SRM_INVALID_REQUEST must be returned. The existing values must nt be changed. b) newsize is the new actual size f the space. c) newlifetime is the new lifetime requested regardless f the previus lifetime. It might even be shrter than the remaining lifetime at the time f the call. It is relative t the calling time. Lifetime will be set frm the calling time fr the specified perid. d) Output parameter, lifetimegranted is the new lifetime granted regardless f the previus lifetime. It might even be shrter than the previus lifetime. It is relative t the calling time Return Status Cde successful request cmpletin. Space is successfully updated as the client requested. SRM_ REQUEST_QUEUED successful request submissin and acceptance. Request tken must be returned. SRM_LOWER_SPACE_GRANTED successful request cmpletin, but lwer space size is allcated than what the client requested SRM_AUTHENTICATION_FAILURE SRM server failed t authenticate the client client is nt authrized t update the space that is assciated with the spacetken SRM_SPACE_LIFETIME_EXPIRED lifetime f the space that is assciated with the spacetken is already expired. SRM_INVALID_REQUEST spacetken des nt refer t an existing knwn space in the SRM server. input parameter size r time is nt prvided. SRM_EXCEED_ALLOCATION SRM server des nt have enugh space fr the client t fulfill the request because the client request has mre than the allcated space fr the client. SRM_NO_USER_SPACE SRM server des nt have enugh space fr the client t fulfill the request SRM_NO_FREE_SPACE SRM server des nt have enugh free space t fulfill the request SRM_INTERNAL_ERROR SRM has an ernal transient errr, and client may try again. New requested size is less than currently used space. any ther request failure. Explanatin needs t be filled fr details. gsm-wg@gf.rg

22 SRM_NOT_SUPPORTED functin is nt supprted 2.5. srmstatusofupdatespacerequest This functin is used t check the status f the previus request t srmupdatespace, when asynchrnus space update was necessary with the SRM. Request tken must have been prvided in respnse t the srmupdatespace Parameters In: authrizatinid, requesttken Out: TReturnStatus returnstatus, unsigned lng sizeofttalspace, // best effrt unsigned lng sizeofguaranteedspace, lifetimegranted Ntes n the Behavir a) Output parameters fr sew sizes are the new actual sizes f the space. b) Output parameter, lifetimegranted is the new lifetime granted regardless f the previus lifetime. It might even be shrter than the previus lifetime. It is relative t the calling time Return Status Cde SRM_REQUEST_QUEUED successful request submissin and the request is still n the queue t be served. SRM_REQUEST_INPROGRESS the request is being prcessed. successful request cmpletin. Space is successfully updated as the client requested. SRM_LOWER_SPACE_GRANTED successful request cmpletin, but lwer space size is allcated than what the client requested SRM_AUTHENTICATION_FAILURE SRM server failed t authenticate the client client is nt authrized t update the space that is assciated with the spacetken SRM_SPACE_LIFETIME_EXPIRED lifetime f the space that is assciated with the spacetken is already expired. SRM_INVALID_REQUEST spacetken des nt refer t an existing knwn space in the SRM server. input parameter size r time is nt prvided. SRM_EXCEED_ALLOCATION SRM server des nt have enugh space fr the client t fulfill the request because the client request has mre than the allcated space fr the client. SRM_NO_USER_SPACE SRM server des nt have enugh space fr the client t fulfill the request gsm-wg@gf.rg

23 SRM_NO_FREE_SPACE SRM server des nt have enugh free space t fulfill the request SRM_REQUEST_SUSPENDED request is suspended. SRM_INTERNAL_ERROR SRM has an ernal transient errr, and client may try again. New requested size is less than currently used space. any ther request failure. Explanatin needs t be filled fr details. SRM_NOT_SUPPORTED functin is nt supprted 2.6. srmgetspacemetadata This functin is used t get infrmatin f a space. Space tken must be prvided, and space tkens are returned upn a cmpletin f a space reservatin thrugh srmreservespace r srmstatusofreservespacerequest Parameters In: authrizatinid, [] arrayofspacetkens Out: TReturnStatus returnstatus, TMetaDataSpace[] arrayofspacedetails Ntes n the Behavir a) Output parameters unusedsize in TMetaDataSpace returns 0 if there is n space left in the allcated space. b) When clients use mre space than allcated, clients get warned t accmmdate their files in the spaces r update the space befre running ut. SRM Return Status Cde Fr request level return Status, successful request cmpletin. Infrmatin f all requested spaces are returned successfully. SRM_PARTIAL_SUCCESS Request is cmpleted. Infrmatin f sme requested spaces are returned successfully, and sme are failed t be returned. SRM_AUTHENTICATION_FAILURE SRM server failed t authenticate the client client is nt authrized t request space infrmatin SRM_TOO_MANY_RESULTS Request prduced t many results that SRM server cannt handle. SRM_INVALID_REQUEST arrayofspacetken is empty. SRM_INTERNAL_ERROR SRM has an ernal transient errr, and client may try again. All space requests are failed. gsm-wg@gf.rg

24 any ther request failure. Explanatin needs t be filled fr details. SRM_NOT_SUPPORTED functin is nt supprted in the SRM server Fr space level return Status, successful request cmpletin fr the spacetken. Space infrmatin is successfully returned. client is nt authrized t request infrmatin n the space that is assciated with the spacetken SRM_INVALID_REQUEST spacetken des nt refer t an existing knwn space in the SRM server. SRM_SPACE_LIFETIME_EXPIRED The life time n the space that is assciated with the spacetken has expired SRM_EXCEED_ALLOCATION Space that is assciated with spacetken has n mre space left. any ther request failure. Explanatin needs t be filled fr details srmchangespacefrfiles This functin is used t change the space prperty f files t anther space prperty by specifying target space tkens. All files specified by SURLs will have a new space tken. SURLs must nt be changed. New space tken may be acquired frm srmreservespace. Asynchrnus peratin may be necessary fr sme SRMs, and in such case, request tken is returned fr later status inquiry. There is n default behavir when target space tken is nt prvided. In such case, the request will be rejected, and the return status must be SRM_INVALID_REQUEST Parameters In: authrizatinid, anyuri [] arrayofsurls, targetspacetken, TExtraInf[] stragesysteminf Out: TReturnStatus returnstatus, requesttken, estimatedprcessingtime, TSURLReturnStatus [] arrayoffilestatuses Ntes n the Behavir a) When space transitin is cmpleted successfully, must be returned fr each SURL. b) Fr any frbidden transitin by the SRM implementatin, SRM_INVALID_REQUEST must be returned. It includes changing spaces n SURLs that statuses are SRM_FILE_BUSY. c) Asynchrnus peratin may be necessary fr sme SRMs t serve many cncurrent requests. In such case, request tken must be returned. If the request can be cmpleted immediately, request tken must nt be returned. gsm-wg@gf.rg

25 d) When asynchrnus peratin is necessary, the returned status cde shuld be SRM_REQUEST_QUEUED, and arrayoffilestatuses may nt be filled and returned. e) All files specified in arrayofsurls will be mved t the space assciated with targetspacetken. f) When target space tken is used, space allcatin fr a new space tken must be dne explicitly by the client befre using this functin. g) If a directry path is prvided, then the effect is recursive fr all files in the directry. h) Space de-allcatin may be necessary in sme cases, and it must be dne by the client explicitly after this peratin cmpletes. The status can be checked by srmstatusofchangespacefrfilesrequest. i) When a space is successfully changed fr a file frm ne space t anther, it will either retain its remaining lifetime, r the lifetime will be reduced t that f the target space, whichever is the lesser. j) If the target space is nly large enugh t transfer a subset f the files, the request will cntinue taking place until the target space cannt hld any mre files, and the request must be failed. The status f the request must return an errr f SRM_EXCEED_ALLOCATION in such case Return Status Cde Fr request level return status, All file requests are successfully cmpleted. All SURLs have new targetspacetken. SRM_PARTIAL_SUCCESS All requests are cmpleted. Sme SURL requests have new targetspacetken, and sme SURL requests are failed t have new targetspacetken. Details are n the files status. SRM_ REQUEST_QUEUED request is submitted and accepted. requesttken must be returned. The status can be checked by srmstatusofchangespacefrfilesrequest. SRM_ REQUEST_INPROGRESS The request is being prcessed. Sme files are still queued, and sme files are cmpleted in space transitin. SRM_AUTHENTICATION_FAILURE SRM server failed t authenticate the client client is nt authrized t change the file types SRM_INVALID_REQUEST SURL is empty. targetspacetken is empty. targetspacetken des nt refer t an existing space in the SRM server. targetspacetken refers t a frbidden transitin by the SRM implementatin. SRM_SPACE_LIFETIME_EXPIRED target space that is assciated with targetspacetken has an expired lifetime. SRM_EXCEED_ALLOCATION target space that is assciated with targetspacetken is nt enugh t hld all SURLs. SRM_INTERNAL_ERROR SRM has an ernal transient errr, and client may try again. any ther request failure. Explanatin needs t be filled fr details. SRM_NOT_SUPPORTED functin is nt supprted in the SRM gsm-wg@gf.rg

26 any input parameter is nt supprted in the SRM server a particular type f an input parameter is nt supprted in the SRM server Fr file level return status, successful request cmpletin fr the SURL. The SURL has a new targetspacetken. SRM_ REQUEST_QUEUED file request is n the queue. SRM_ REQUEST_INPROGRESS file request is being prcessed. SRM_INVALID_PATH SURL des nt refer t an existing file client is nt authrized t change the space fr the file that is assciated with the SURL SRM_INVALID_REQUEST targetspacetken refers t a frbidden transitin fr the particular SURL by the SRM implementatin. The status f SURL is SRM_FILE_BUSY. SRM_EXCEED_ALLOCATION target space that is assciated with targetspacetken is nt enugh t hld SURL. SRM_FILE_LOST the requested file with the SURL is permanently lst. SRM_FILE_BUSY client requests fr files which there is an active srmpreparetput (n srmputdne is nt yet called) fr. The requested file with the SURL is being used by ther clients. SRM_FILE_UNAVAILABLE the requested file with the SURL is temprarily unavailable. All file requests are failed. any ther request failure. Explanatin needs t be filled fr details srmstatusofchangespacefrfilesrequest This functin is used t check the status f the previus request t srmchangespacefrfiles, when asynchrnus peratin was necessary in the SRM. Request tken must have been prvided in respnse t the srmchangespacefrfiles Parameters In: authrizatinid, requesttken Out: TReturnStatus returnstatus estimatedprcessingtime, TSURLReturnStatus [] arrayoffilestatuses Ntes n the Behavir gsm-wg@gf.rg

27 a) When space transitin is cmpleted successfully, must be returned fr each SURL. b) If changing space is nt cmpleted, estimateprcessingtime is returned when knwn. c) If all files are still in the queue and nne f the files are cmpleted in changing space, the returned status cde shuld be SRM_REQUEST_QUEUED. d) If sme files are queued, and sme files are cmpleted in changing space, SRM_REQUEST_INPROGRESS must be returned as the return status cde. Each file shuld have its wn status cde. e) If the target space is nly large enugh t transfer a subset f the files, the request will cntinue taking place until the target space cannt hld any mre files, and the request must be failed. The status f the request must return an errr f SRM_EXCEED_ALLOCATION in such case Return Status Cde Fr request level return status, All file requests are successfully cmpleted. All SURLs have new targetspacetken. SRM_PARTIAL_SUCCESS All requests are cmpleted. Sme SURL requests have new targetspacetken, and sme SURL requests are failed t have new targetspacetken. Details are n the files status. SRM_ REQUEST_QUEUED Request submissin was successful and the entire request is still n the queue. SRM_ REQUEST_INPROGRESS Sme files are still queued, and sme files are cmpleted in space transitin. SRM_AUTHENTICATION_FAILURE SRM server failed t authenticate the client client is nt authrized t change the file types SRM_INVALID_REQUEST requesttken des nt refer t an existing knwn request in the SRM server. targetspacetken refers t a frbidden transitin by the SRM implementatin. SRM_SPACE_LIFETIME_EXPIRED target space that is assciated with targetspacetken has an expired lifetime. SRM_EXCEED_ALLOCATION target space that is assciated with targetspacetken is nt enugh t hld SURLs. SRM_REQUEST_SUSPENDED request is suspended. SRM_INTERNAL_ERROR SRM has an ernal transient errr, and client may try again. All file requests are failed. any ther request failure. Explanatin needs t be filled fr details. SRM_NOT_SUPPORTED functin is nt supprted in the SRM any input parameter is nt supprted in the SRM server a particular type f an input parameter is nt supprted in the SRM server Fr file level return status, gsm-wg@gf.rg