EPrintsHybridStorageTraining TableofContents 1Introduction EPrintsHybridStorage ManagementandConfiguration 1Introduction... 1 2PowerfulStorage... 2 3Viewingyourstorageserviceusages... 3 4ManagingyourStoragePolicy(Exercises)... 5 4.1View/EdittheStoragePolicy...5 4.2UnderstandingthedefaultStoragePolicy...5 4.3Exercise1:VolatileFiles...5 4.4Exercise2:MultipleStorageLocations...6 4.5Exercise3:Storagepolicybaseduponrepositorymetadata...6 4.6ExampleSolution...8 EPrints3.2introducesanabstractedstoragelayerwhichprovidestheabilityfordatahostingsolutionssuchasAmazonS3tobeutilisedasa storagebackendtoeprints.theadvantageofthisisthatyoucan"plug in"tomultiplestorageservicesatthesametimeandcontrolthesewitha local"storagepolicy". InthistutorialwelookatthesomeofthestorageinterfacesthatEPrintscanuse,andalsohowtomodifythestoragepoliciestosuittheneedsof amodernrepository. 1
EPrintsHybridStorageTraining 2PowerfulStorage TheEPrintsteamhavebeenworkingonanumberofstorageplug inswhichareavailableboththroughfiles.eprints.organdwillbereleasedin EPrints3.2.Inthissectionweintroduceafewofthem: Name PluginID Description Local Local Savestothelocalharddisk. Local Compressed Sun Honeycomb LocalCompress HoneyComb Ausefullittleadditiontothelocalplug inthatcompressesfilestransparently(nochangeisseenin theuserinterface).goodforsavinglocaldiskspaceandarchivingoldandrarelyusedobjects. SupportsSun'sHoneycombplatform(nolongercommerciallyavailable).Thisplug insupportsthe APIusedbythisclassofhighlyrobustarchivalstorage. AmazonS3 AmazonS3 Plug intosupportthemostwidelyknowncloudstorageprovider.aswellassupportingbasicstorage functionalitytheplug inalsosupportsamazoncloudfrontfordirectlocaliseddeliveryofresources. SunCloud Storage SunCSS VerysimilartotheAmazonS3offerings.NotethatthisisabetaservicesandEPrintsisatesting partner. 2
EPrintsHybridStorageTraining 3Viewingyourstorageserviceusages TheStorageManagerscreencanbefoundundertheConfigToolstaboftheadminscreen. 3
EPrintsHybridStorageTraining ThefigurebelowshowstheStorageManager.Fromthisscreenyoucaneasilyviewwhereyourobjectsareandhowmuchspacetheyconsume. Youcanalsomovethembetweenstorageplatformswithasingleclick. 4
EPrintsHybridStorageTraining 4ManagingyourStoragePolicy(Exercises) 4.1View/EdittheStoragePolicy TheEPrintsStorageControllerismanagedbyapolicydefinedinanxmlfile.Thisconfigfile(storage/default.xml)canbeeditedbyclickingthe ViewConfigurationbuttonavailablefromtheConfigToolstaboftheadmininterface.storage/default.xmlislocatednearthebottomofthe availablelistoffiles.byclickingonthisfileyoucanviewandedititinyourbrowser. 4.2UnderstandingthedefaultStoragePolicy LikemanyconfigurationfilesinEPrints,thestoragepolicyisdefinedinxmlusingtheEPrintsControllanguage/namespace(epc)todefine decisionsinanxsltlikefashion. Belowyoucanseeacopyofthecurrentdefaultstoragepolicy.Theannotationsshouldhelpexplainwhateachlinedoes.Notethatbydefault, allfilesarestoredlocally. <store> <epc:when test="datasetid = 'document'"> </store> Start Policies Begin Choice Section If current object is document Use Local plug-in End document condition Otherwise Use Local plug-in End otherwise condition End Choice Section End Policies 4.3Exercise1:VolatileFiles Agoodstartingpointformanagingstorageservicesistodecidewhathappenstovolatilefiles.Thesearefilesthataregeneratedbythe repositoryforinternaluse(e.g.imagepreviews).assuchitisunlikelythatthesefileswillneedtobestoredoffsiteorpreserved. Volatilefilesarepartofthedocumentdatasetandwecandifferentiatethesefromotherfilesbylookingforarelationwhichexistsbetweenthe twotypesoffiles. 5
EPrintsHybridStorageTraining Editthedefaultstoragepolicyandinsertthefollowingcodetohandlevolatileandnon volatiledocumentfilesdifferently.thecodereplacesthe <pluginname= Local /> lineinsidethe epc:when sectionofthecode. <epc:when test="$parent{relation_type} = 'http://eprints.org/relation/isvolatileversionof'"> <plugin name="localcompress"/> AfterchangingthepolicyyouwillneedtoaddanewEPrinttotherepository.UploadaPDF,JPEGorGIFtotheEPrint,thenviewtheStorage Managerscreentoverifythattherearefilesinmorethanonelocation. 4.4Exercise2:MultipleStorageLocations Withtheabovedone,itshouldnowbeeasytoaddasecondlocationtostorenon volatiledocumentsin.thiscanbedonebysimplyaddinga new<plugin>tagtotherelevantsection. EPrintshandlesmultiplestoragelocationsforbothstorageanddeliverybysimplyprocessingthemintheordertheyappearinthestorage policy.forstorage,filesarestoredinalllocations.fordelivery,thefileisservedfromthefirstlocationintheconfigfile.intheeventthatthisis notavailable,itmovesontothesecond,andsoon. 4.5Exercise3:Storagepolicybaseduponrepositorymetadata Inthissectionyouwillusetheepclanguagetoaccessfilemetadata.Thismetatdatawillthenbeusedtocontrolthestorageoftheitem. Eachobjectwehandlewiththestoragecontrollerisa"file"objectineprintsandthusthefollowingpiecesofmetadataarejustafewthatare directlyavailable: 6
EPrintsHybridStorageTraining filename Nameofthefile mime_type Typeofthefile(e.g.PDF,JPEG,etc...) filesize Thesizeofthefile Aconditionalcanbeinsertedintothepolicyfiletousethisdatatomakedecisions: <epc:when test="mime_type = 'application/pdf'"> <plugin name="storagepluginname"/> AddtheaboveruletostorePDFfiles(application/pdf)inadifferentlocationfromeverythingelsestoredintherepository. NotethatEPrintsandUsermetadataisalsoavailable.Itispossibletomakeadecisionbasedon(forexample)whouploadedtheitemorwhich subjectareathepublicationrecordisassociatedwith. 7
EPrintsHybridStorageTraining 4.6ExampleSolution Thefollowingcodewillsolveallexercisescontainedinthisdocument. <store xmlns="http://eprints.org/ep3/storage" xmlns:epc="http://eprints.org/ep3/control"> <epc:when test="datasetid = 'document'"> <epc:when test="$parent{relation_type} = 'http://eprints.org/relation/isvolatileversionof'"> <epc:when test="mime_type = 'application/pdf'"> <plugin name="suncss"/> <plugin name="localcompress"/> </store> 8