EPrints Hybrid Storage Management and Configuration

Transcription

1 EPrintsHybridStorageTraining TableofContents 1Introduction EPrintsHybridStorage ManagementandConfiguration 1Introduction PowerfulStorage Viewingyourstorageserviceusages ManagingyourStoragePolicy(Exercises) View/EdittheStoragePolicy UnderstandingthedefaultStoragePolicy Exercise1:VolatileFiles Exercise2:MultipleStorageLocations Exercise3:Storagepolicybaseduponrepositorymetadata ExampleSolution...8 EPrints3.2introducesanabstractedstoragelayerwhichprovidestheabilityfordatahostingsolutionssuchasAmazonS3tobeutilisedasa storagebackendtoeprints.theadvantageofthisisthatyoucan"plug in"tomultiplestorageservicesatthesametimeandcontrolthesewitha local"storagepolicy". InthistutorialwelookatthesomeofthestorageinterfacesthatEPrintscanuse,andalsohowtomodifythestoragepoliciestosuittheneedsof amodernrepository. 1

2 EPrintsHybridStorageTraining 2PowerfulStorage TheEPrintsteamhavebeenworkingonanumberofstorageplug inswhichareavailableboththroughfiles.eprints.organdwillbereleasedin EPrints3.2.Inthissectionweintroduceafewofthem: Name PluginID Description Local Local Savestothelocalharddisk. Local Compressed Sun Honeycomb LocalCompress HoneyComb Ausefullittleadditiontothelocalplug inthatcompressesfilestransparently(nochangeisseenin theuserinterface).goodforsavinglocaldiskspaceandarchivingoldandrarelyusedobjects. SupportsSun'sHoneycombplatform(nolongercommerciallyavailable).Thisplug insupportsthe APIusedbythisclassofhighlyrobustarchivalstorage. AmazonS3 AmazonS3 Plug intosupportthemostwidelyknowncloudstorageprovider.aswellassupportingbasicstorage functionalitytheplug inalsosupportsamazoncloudfrontfordirectlocaliseddeliveryofresources. SunCloud Storage SunCSS VerysimilartotheAmazonS3offerings.NotethatthisisabetaservicesandEPrintsisatesting partner. 2

3 EPrintsHybridStorageTraining 3Viewingyourstorageserviceusages TheStorageManagerscreencanbefoundundertheConfigToolstaboftheadminscreen. 3

4 EPrintsHybridStorageTraining ThefigurebelowshowstheStorageManager.Fromthisscreenyoucaneasilyviewwhereyourobjectsareandhowmuchspacetheyconsume. Youcanalsomovethembetweenstorageplatformswithasingleclick. 4

5 EPrintsHybridStorageTraining 4ManagingyourStoragePolicy(Exercises) 4.1View/EdittheStoragePolicy TheEPrintsStorageControllerismanagedbyapolicydefinedinanxmlfile.Thisconfigfile(storage/default.xml)canbeeditedbyclickingthe ViewConfigurationbuttonavailablefromtheConfigToolstaboftheadmininterface.storage/default.xmlislocatednearthebottomofthe availablelistoffiles.byclickingonthisfileyoucanviewandedititinyourbrowser. 4.2UnderstandingthedefaultStoragePolicy LikemanyconfigurationfilesinEPrints,thestoragepolicyisdefinedinxmlusingtheEPrintsControllanguage/namespace(epc)todefine decisionsinanxsltlikefashion. Belowyoucanseeacopyofthecurrentdefaultstoragepolicy.Theannotationsshouldhelpexplainwhateachlinedoes.Notethatbydefault, allfilesarestoredlocally. <store> <epc:when test="datasetid = 'document'"> </store> Start Policies Begin Choice Section If current object is document Use Local plug-in End document condition Otherwise Use Local plug-in End otherwise condition End Choice Section End Policies 4.3Exercise1:VolatileFiles Agoodstartingpointformanagingstorageservicesistodecidewhathappenstovolatilefiles.Thesearefilesthataregeneratedbythe repositoryforinternaluse(e.g.imagepreviews).assuchitisunlikelythatthesefileswillneedtobestoredoffsiteorpreserved. Volatilefilesarepartofthedocumentdatasetandwecandifferentiatethesefromotherfilesbylookingforarelationwhichexistsbetweenthe twotypesoffiles. 5

6 EPrintsHybridStorageTraining Editthedefaultstoragepolicyandinsertthefollowingcodetohandlevolatileandnon volatiledocumentfilesdifferently.thecodereplacesthe <pluginname= Local /> lineinsidethe epc:when sectionofthecode. <epc:when test="$parent{relation_type} = ' <plugin name="localcompress"/> AfterchangingthepolicyyouwillneedtoaddanewEPrinttotherepository.UploadaPDF,JPEGorGIFtotheEPrint,thenviewtheStorage Managerscreentoverifythattherearefilesinmorethanonelocation. 4.4Exercise2:MultipleStorageLocations Withtheabovedone,itshouldnowbeeasytoaddasecondlocationtostorenon volatiledocumentsin.thiscanbedonebysimplyaddinga new<plugin>tagtotherelevantsection. EPrintshandlesmultiplestoragelocationsforbothstorageanddeliverybysimplyprocessingthemintheordertheyappearinthestorage policy.forstorage,filesarestoredinalllocations.fordelivery,thefileisservedfromthefirstlocationintheconfigfile.intheeventthatthisis notavailable,itmovesontothesecond,andsoon. 4.5Exercise3:Storagepolicybaseduponrepositorymetadata Inthissectionyouwillusetheepclanguagetoaccessfilemetadata.Thismetatdatawillthenbeusedtocontrolthestorageoftheitem. Eachobjectwehandlewiththestoragecontrollerisa"file"objectineprintsandthusthefollowingpiecesofmetadataarejustafewthatare directlyavailable: 6

7 EPrintsHybridStorageTraining filename Nameofthefile mime_type Typeofthefile(e.g.PDF,JPEG,etc...) filesize Thesizeofthefile Aconditionalcanbeinsertedintothepolicyfiletousethisdatatomakedecisions: <epc:when test="mime_type = 'application/pdf'"> <plugin name="storagepluginname"/> AddtheaboveruletostorePDFfiles(application/pdf)inadifferentlocationfromeverythingelsestoredintherepository. NotethatEPrintsandUsermetadataisalsoavailable.Itispossibletomakeadecisionbasedon(forexample)whouploadedtheitemorwhich subjectareathepublicationrecordisassociatedwith. 7

8 EPrintsHybridStorageTraining 4.6ExampleSolution Thefollowingcodewillsolveallexercisescontainedinthisdocument. <store xmlns=" xmlns:epc=" <epc:when test="datasetid = 'document'"> <epc:when test="$parent{relation_type} = ' <epc:when test="mime_type = 'application/pdf'"> <plugin name="suncss"/> <plugin name="localcompress"/> </store> 8