!!! White!Paper!! NexentaStor!Solutions!Guide!for!! Mirantis!OpenStack! Version!1.0!! October!2014!

Transcription

1 WhitePaper NexentaStorSolutionsGuidefor MirantisOpenStack Version1.0 October2014

2 TableofContents Preface'...'3 About'this'document'...'3 Audience'...'3 External'References'...'4 Document'History'...'4 Overview'...'5 Reference'Architecture'...'6 Node'configuration'...'7 High'Availability'configurations'...'7 Network'topologies'...'7 Installation'and'Configuration'...'8 Software'requirements'...'8 Prerequisites'...'8 Configure'OpenStack'Glance'to'store'Images'on'NexentaStor'...'9 Export'the'newly'created'folder'through'NFS'...'10 Configure'OS'to'mount'NexentaStor'file'system'...'11 Updating'the'Glance'configuration'...'11 Configure'OpenStack'Cinder'to'use'ISCSI'from'NexentaStor'for'block'device'storage'...'12 Configure'Cinder'to'use'the'NexentaStor'ISCSI'backend'driver'...'12 Configure'OpenStack'Cinder'to'use'NFS'from'NexentaStor'for'block'device'storage'...'13 Export'the'newly'created'folder'through'NFS'...'14 Update'the'Cinder'driver'for'NexentaStor'4.0.x'...'15 Configure'Cinder'to'use'NexentaStor'NFS'Driver'...'15 Validating'the'installation'...'17 Copyright 2014NexentaAllRightsReserved 2

3 Preface Aboutthisdocument ThisreferencearchitecturedescribeshowtointegrateMirantisOpenStack5.1(usingOpenStackIcehouse) withnexentastor4.0.x,utilizingnexentastorasabackendstorageforbothgoldenimagesaswellasvirtual machinestorage. Audience ThisdocumentisintendedforsystemadministratorsthatwanttointegrateNexentaStorwithanexisting OpenStackinstallation. ItisexpectedthatthereaderisexperiencedwithOpenStackandhasbasicknowledgeofNexentaStor Copyright 2014NexentaAllRightsReserved 3

4 ExternalReferences Foradditionalinformation,pleaseseethefollowingdocuments: Reference Nexentawebsite Mirantiswebsite OpenStackwebsite NexentaStorDownloadandDocumentation MirantisOpenStack5.1download OpenStackCinderDriverReference documentationfornexentadriver NexentaISCSIdriverconfigurationoptions NexentaNFSdriverconfigurationoptions GitHubrepositoryforNexentaCinder Drivers DocumentHistory Location reference/content/nexentavdriver.html reference/content/nexentaviscsivdrivervoptions.html reference/content/nexentavnfsvdrivervoptions.html me/drivers/nexenta Version Date Changes 1.0 October,2014 Initialversion Copyright 2014NexentaAllRightsReserved 4

5 Overview NexentaStorisaSoftwareVDefinedVStorageplatformthatdeliversunifiedfileandblockstorageservices scalableintothepetabyterangeusingindustrystandardhardware,andincludesadvanceddatamanagement capabilities. WhendeployedinOpenStackenvironmentsNexentaStorprovideacosteffectiveandscalablebackendfor storingbothvirtualmachineimagesandapplicationdata. TheintegrationdriversbetweenNexentaStorandOpenStackhavebeenapartoftheupstreamreleasesof OpenStacksincetheEssexreleasein2012.WiththeintegrationintheupstreamCinderdriversNexentaStor works outofthebox withallmajordistributionsofopenstack,includingmirantisopenstack5.1 NexentaStorcanprovidestorageforOpenStackinanumberofways: StoragefortheImageRepositoryservice(Glance)throughNFS BlockStoragefortheCinderVolumeservicethroughbothISCSIand/orNFS ApplicationdatastoragethroughbothISCSIandNFS BelowisaconceptualarchitecturedrawingofhowNexentaStorprovideblockandimageservicesinan OpenStackenvironment.ThisdocumentprovidedetailedstepsofhowanexistingNexentaStorconfiguration canbeintegratedintomirantisopenstack. Copyright 2014NexentaAllRightsReserved 5

6 ReferenceArchitecture BothMirantisOpenStackandNexentaStorcanbeconfiguredtoprovideservicesinavarietyofways.To ensurethatthebestpossibleendresultisachievedtheguidelinesandbestpracticesformirantisopenstack shouldbefollowedtoconfigureopenstack. ForBestPracticesforNexentaStoritissuggestedthattheadministratorfollowtheguidelinesoutlinedinthe NexentaStorUserGuide,andconfigureadual/redundantcontrollersetupasdescribedintheNexentaStor HighAvailabilityGuide. DuringthecertificationandfunctionalverificationofNexentaStorandMirantisOpenStack5.1thefollowing configurationwasused: AMirantisFuelMasternode OneOpenStackControllernode OneOpenStackCindernode 5OpenStackComputenodes Figure1Verificationsetup Copyright 2014NexentaAllRightsReserved 6

7 Nodeconfiguration WhenNexentaStorisusedinanenvironmentthatismanagedbyMirantisOpenStack5.1allnode deploymentsandosconfigurationmustbeperformedthroughmirantisfuel. NexentaStorsupportdeploymentusingeitherCentos6.5orUbuntu12.04LTSwhenusingOpenStackIcehouse basedreleases. Note:ThisdocumentassumesCentOSisbeingusedwhenreferencingcommandVlineutilitiesand/orOSlevel configurationfilesandtools. HighAvailabilityconfigurations NexentaStorsupportMirantisOpenStackrunningineitherHighAvailabilitymode,orin nonvha mode.as bestpraxisnexentastoritselfshouldalwaysbeconfiguredinhighavailabilitymodewithtworedundant controllersasdocumentedinthenexentastorhauserguide. Networktopologies ThetraditionalNovaNetworkingcomponentswhereusedtoverifythesolutionforthisdocument.However, NexentaStorwillworkwithbothNovaNetworkandNeutronmodesaslongastheCompute,Glanceand CindernodescanaccesstheNexentaStorserver(s). Itishighlyrecommendedthatthestoragenetworkisbrokenoutaseitheraphysicalnetwork,orasaseparate VLAN,tokeepthetrafficsegregatedfromothertraffictypes. Figure2Networkusedforfunctionaltesting Copyright 2014NexentaAllRightsReserved 7

8 InstallationandConfiguration WhenNexentaStorisusedasabackendstoragesolutionforOpenStacktheguidelinesandbestpractices publishedonbymirantisapply.thedeploymentofmirantisopenstackshouldbedonethroughfuel,andthe deploymentshouldpassallautomatedhealthchecks. AfterinstallinganOpenStackenvironmentusingMirantisFuelanumberofconfigurationchangesare requiredtousenexentastorasbackendstorageforcinderandglance: ItisidealthatadedicatedCinderserverisusedforNexentaStor,butitispossibletoconfigure NexentaStorasapartofamultiVbackendconfigurationtogetherwithLVMorotherbackends.Please seetheopenstackreferencedocumentationfordetailsonhowtoconfiguremultivbackendsolutions. NexentaStormustbeconfiguredwithaseparateNFSshareforGlanceImages IftheNexentaStorCinderNFSdriversisusedaseparateNFSsharemustbeconfiguredforCinder storage TousetheNexentaStorCinderISCSIbackenditissuggestedthataseparatedataset/volumeiscreated tokeepthecindervolumesseparatedfromotherfilesystems IftheCinderisconfiguredtousetheNexentaStorNFSbackendonaNexentaStor4.0.xappliancea separatepatchmustbeappliedtothecinderdriverfromupstreamgithubrepositories.thisisto correctchangesinbehaviorbetweennexentastor3.1.xand4.0.xthatoccurredafterthereleaseof OpenStackIcehouse Softwarerequirements MirantisOpenStack5.1(OpenStackIcehouse).WhileNexentaStorissupportedasabackendstorefor CinderfromOpenStackEssexandonwardsthisguidefocusonprovidingdetailsbasedonOpenStack Icehouse OpenStackcanbeinstalledonanyLinuxdistributionsupportedbyMirantisOpenStack5.1. Prerequisites Thisguideassumesthatthefollowingbaserequirementsaresatisfied: 1. NexentaStor4.0.3isinstalledandconfiguredonsupportedhardware 2. MirantisOpenStack5.1isusedandMirantisFUELisusedtodeploy/manageservers.Technicallyall versionssinceopenstackessexaresupported,butthisdocumentisspecifictomirantisopenstack5.1 andicehouse 3. TheenvironmentisrunningCentOS7.x.Again,technicallyanyLinuxversionthatsupportMirantis OpenStack5.1orOpenStackIcehousearesupported,butthedocumentationisspecifictoCentOS7.x. Copyright 2014NexentaAllRightsReserved 8

9 ConfigureOpenStackGlancetostoreImagesonNexentaStor TheOpenStackimagerepositoryservicecanbeconfiguredtostoreimagesonNexentaStorbyusingNFS (NetworkFileSystem).TheGlanceImageCacheshouldremainlocalontheGlanceserverforbest performance. CreatinganewshareinNexentaStorisdonefromeitherfromtheconsoleoftheappliancethroughan intelligentcommandlineshell,orbythenexentamanagementconsole. BelowisascreenshotfromtheNexentaStorManagementConsolethatdetailhowtocreateasuitableshare forstoringglanceimages.asanexamplewearecreatinganewsharecalled glance_images inthe p1 volume. Figure4CreateNFSshareforGlanceImageRepository Figure3CreatefoldertostoreGlanceImages Copyright 2014NexentaAllRightsReserved 9

10 ExportthenewlycreatedfolderthroughNFS ToenabletheGlanceservertoaccessthenewfolderoverNFSitneedstobeshared.Thisisdonethroughthe folderoverviewinthenexentastormanagementconsole(seebelow). IfspecificpermissionsareneededtoenabletheGlanceusertoaccesstheGlanceImagestheadvanced permissionstabcanbereachedbyclickingeditnexttothenfscheckbox. Figure5ExportGlancefolderthroughNFS Copyright 2014NexentaAllRightsReserved 10

11 ConfigureOStomountNexentaStorfilesystem ToconfiguretheGlanceservertoutilizethenewsharefromtheNexentaStorapplianceitmustbemounted ontheglanceserver,andtheglanceconfigurationfileneedstobeupdatedtousethenewlocationasa storagelocation. Toutilizethenewfilesystemcreateamountpoint,forexample/var/lib/glanceVnexentastorandthenaddthe followinglinesto/etc/fstabontheglanceservertoensureitismountedatboot: Replace <NS> withthenameofthenexentastorapplianceand <VOL> withthenameofthevolume whereyoucreatedtheglance_imagefolderabove.inourcasetheentrywouldlooklikethis: Finally,confirmthattheNexenataStorfilesystemiscorrectlyconfiguredbymanuallymountingthefilesystem byrunning mount/var/lib/glancevnexentastor ontheglanceserver.itshouldreturnwithoutanyerrorsand thefilesystemshouldnowbemountedunder/var/lib/glancevnexentastor. Note:ItispossiblethattheCentOSOSimagesdoesnothavetherequiredpackagesformountinganNFSfile shareprevinstalled.ifthemountcommandfailitmayberequiredtofirstinstallthepackagenfsvutilsby running yuminstallnfsvutils. UpdatingtheGlanceconfiguration IntheGlanceconfigurationfile/etc/glance/glanceVapi.confthefollowinglineneedstobechanged: sothatitpointstothenewimagerepositoryonnexentastor IftheenvironmentalreadyhadGlanceimagesstoredunder/var/lib/glance/imagesthesecannowbecopied to/var/lib/glancevnexentastor/.itissuggestedthatacopyoperationisusedtoallowglancetocontinueto functionduringtheoperation. RestartGlancewhenallimageshavebeentransferredtoNexentaStor. AftertherestarthascompletedGlancewillnowutilizeNexentaStorforimagerepositoryandtheoldimages canberemoved. # Mount Glance Image Repository from NexentaStor <NS>:/volumes/<VOL>/glance_images /var/lib/glance-nexentastor nfs rw 0 0 # Mount Glance Image Repository from NexentaStor nexentastor:/volumes/p1/glance_images /var/lib/glance-nexentastor nfs rw 0 0 filesystem_store_datadir=/var/lib/glance/images/ filesystem_store_datadir=/var/lib/glance-nexentastor/ # service openstack-glance-api restart Stopping openstack-glance-api: [ OK ] Starting openstack-glance-api: [ OK ] Copyright 2014NexentaAllRightsReserved 11

12 ConfigureOpenStackCindertouseISCSIfromNexentaStorforblockdevicestorage NexentaStorcanbeusedasblockdevicestorageforOpenStackCinderusingbothISCSIandNFS.WhenISCSIis usedthecindervolumesarecreatedaszvols,blockdevices,onthenexentastorappliance. AsbestpracticetheZVOLsshouldbestoredinadataset/volumethatisuniqueperCinderbackendtokeep themseparatedandensurethatthebackenddriversdonotinterfere. Intheexamplesbelowweareusingthedataset/volume p1 tostorethezvolscreatedbythecinderiscsi driver. ConfigureCindertousetheNexentaStorISCSIbackenddriver ToenabletheCinderdriverforNexentaStorISCSIdriverthefollowingadditionsshouldbedonetotheCinder configurationfile/etc/cinder/cinder.conf: ReplacetheHOSTNAME,USERandPASSWORDwiththefullyqualifiedhostnameorIPaddress,and administrativecredentials.thecinder.conffileshouldhaveappropriatefilepermissionssothatonlythe Cinderdriverandrootcanreadthefile(Ifunsurerun chmod640/etc/cinder/cinder.conf ).Thevaluefor nexenta_volumeshouldbesettothenameofthedataset/volumewherethevolumesshouldbestored. ToenabletheNexentaStorbackendconfiguredaboveensurethatitisenabledinthecinder.conffile: Note:ItispossiblethattheCentOSOSimagedoesnothavetherequiredpackagesforusingISCSIpreV installed.pleaseensurethattheiscsivinitiatorvutilspackageisinstalledandifitisnot,installitusing yum installiscsivinitiatorvutils [nexentaiscsi-1] volume_driver=cinder.volume.drivers.nexenta.iscsi.nexentaiscsidriver volume_backend_name=nexenta_iscsi nexenta_rest_port=8457 nexenta_host=<hostname> nexenta_user=<user> nexenta_password=<password> nexenta_volume=<volume> enabled_backends=nexentaiscsi-1 WhentheconfigurationhasbeencompletedCinderVolumedriverneedstoberestartedforthechangesto takeeffect: # /etc/init.d/opestack-cinder-volume restart Stopping openstack-cinder-volume: [ OK ] Starting openstack-cinder-volume: [ OK ] AftertherestarthascompletedCinderwillnowutilizethenewNexentaStorISCSIbackendtocreatenew CinderVolumesbasedontheschedulingandweightingrules. Copyright 2014NexentaAllRightsReserved 12

13 ConfigureOpenStackCindertouseNFSfromNexentaStorforblockdevicestorage NexentaStorcanbeusedasblockdevicestorageforOpenStackCinderusingbothISCSIandNFS.WhenNFSis usedthecindervolumesarecreatedasregularorsparsefilesonthenexentastorappliance. AsbestpracticethesefilesshouldbestoredinonefolderperCinderbackendtokeepthemseparatedand ensurethatthebackenddriversdonotinterfere.iftheconfigurationutilizemultiplecinderbackendsthese mustallhaveseparatenfsshares/folders. BelowisascreenshotfromtheNexentaStorManagementConsolethatdetailhowtocreateasuitableshare forstoringthecindervolumesusedbythenfsdriver.asanexamplewearecreatinganewsharecalled cinder_nfs inthe p1 volume. Figure6CreatingafolderforstoringCinderVolumesoverNFS Copyright 2014NexentaAllRightsReserved 13

14 ExportthenewlycreatedfolderthroughNFS ToenabletheCinderVolumeservertoaccessthenewfolderoverNFSitneedstobeshared.Thisisdone throughthefolderoverviewinthenexentastormanagementconsole(seebelow). IfspecificpermissionsareneededtoenabletheCinderusertoaccesstheCinderVolumestheadvanced permissionstabcanbereachedbyclickingeditnexttothenfscheckbox. Figure7SharefolderforCinderVolumestoragethroughNFS Copyright 2014NexentaAllRightsReserved 14

15 UpdatetheCinderdriverforNexentaStor4.0.x OpenStackIcehousewasreleasedbeforeNexentaStor4.0.xandtheNexentaNFSdriversthatshipwith IcehousearenotcompatiblewithNexentaStor4.0.x.TheupstreamdriversinGitHubwerefixedduringthe releaseofnexentastor4.0.1andthiswillnotbeaproblemwiththejunorelease.however,foricehousean updatednfsdriverisrequired. TodownloadandinstalltheupdatedNFSdriverrunthefollowingcommandsontheCinderVolumeserver: ThisdownloadsthepatchedNFSdriverfromtheupstreamGitHubrepositorytoenableIcehousetoworkwith NexentaStor4.0.x. ConfigureCindertouseNexentaStorNFSDriver ToenabletheCinderdriverforNexentaStorNFSthefollowingadditionsshouldbedonetotheCinder configurationfile/etc/cinder/cinder.conf: ToenabletheNexentaStorbackendconfiguredaboveensurethatitisenabledinthecinder.conffile: # cd /usr/lib/python2.6/site-packages/cinder/volume/drivers/nexenta # mv nfs.py nfs.py.orig # wget \ f1c9cc5/cinder/volume/drivers/nexenta/nfs.py [nexentanfs-1] volume_driver = cinder.volume.drivers.nexenta.nfs.nexentanfsdriver volume_backend_name=nexenta_nfs nexenta_shares_config=/etc/cinder/nexenta_shares enabled_backends=nexentanfs-1 Copyright 2014NexentaAllRightsReserved 15

16 TheNexentaStorNFSfilesystemaccessisconfiguredthroughthefiledefinedinnexenta_shares_config.This filecontainsalltheinformationthecinderdriverrequirestocommunicatewiththenexentastorappliance. Thesharedefinitionfilehasthefollowingformat: ReplacetheHOSTNAME,USERandPASSWORDwiththefullyqualifiedhostnameorIPaddress,and administrativecredentials.thesharedefinitionfileshouldhaveappropriatefilepermissionssothatonlythe Cinderdriverandrootcanreadthefile(Ifunsurerun chmod640/etc/cinder/nexenta_shares ). Examplenexenta_sharedefinition: WhentheNexentaStordriverisconfiguredtheCinderdriverwillautomaticallymounttheNFSexportonaasV neededbasis. Note:ItispossiblethattheCentOSOSimagesdoesnothavetherequiredpackagesformountinganNFSfile shareprevinstalled.ifthemountcommandfailitmayberequiredtofirstinstallthepackagenfsvutilsby running yuminstallnfsvutils. <HOSTNAME>:/volumes/<VOLUME>/<FOLDER> nexentastor:/volumes/p1/cinder_nfs WhentheconfigurationhasbeencompletedCinderVolumedriverneedstoberestartedforthechangesto takeeffect: # /etc/init.d/opestack-cinder-volume restart Stopping openstack-cinder-volume: [ OK ] Starting openstack-cinder-volume: [ OK ] AftertherestarthascompletedCinderwillnowutilizethenewNexentaStorNFSbackendtocreatenew CinderVolumesbasedontheschedulingandweightingrules. Copyright 2014NexentaAllRightsReserved 16

17 Validatingtheinstallation Aftertheconfigurationhasbeencompleteditshouldbevalidatedusingtheautomatedhealthcheck capabilitiesofmirantisfuel.doingthiswillcatchmosterrorsbeforetryingtodeployproductionworkloads. AllGlanceandCinderrelatedtestsshouldpasswithnoerrors. TheHealthCheckisinitiatedfromtheMirantisFuelconsole(withinthecontextoftherelevantOpenStack cloud).allofthesanitytestsshouldpass,anditisimportantthatthe CreateVolume relatedfunctional Testsalsopass. Ifanyofthesebasictestsfailthecauseshouldbedeterminedandcorrectedbeforeproceedingtodeploya workloadonthesesystems. Figure8RunMirantisFuelHealthCheck Copyright 2014NexentaAllRightsReserved 17