How to ext Puppet using Ruby Miguel Di Ciurcio Filho miguel@instruct.com.br http://localhost:9090/onepage 1/43
What is Puppet? Puppet Architecture Facts Functions Resource Types Hiera, Faces and Reports Aga http://localhost:9090/onepage 2/43
About Puppet Labs Developer of IT automation software for system administrators First open source product release in 2005 First commercial product release in 2011 3,000+ Community members 50,000+ Nodes managed in the largest deployments Support for Red Hat, CentOS, Ubuntu, Debian, SUSE, Solaris, AIX, Mac OS X, Windows, etc. http://localhost:9090/onepage 3/43
Infraestructure administration and operation Authorized Puppet Labs Partner since 2012 Puppet Open Source and Enterprise consulting Offers all official training courses in Brazil: Puppet Fundamentals Puppet Practitioner Puppet Architect About Instruct http://localhost:9090/onepage 4/43
What is Puppet? Puppet is IT automation software that defines and enforces the state of your infrastructure. Frees SysAdmins from writing one off, fragile scripts and other manual tasks. Ensures consistency across your infrastructure. Puppet uses a declarative language for modeling your configuration, meaning you tell Puppet what results you want, rather than how to get there. Fully written in Ruby : ) http://localhost:9090/onepage 5/43
How Puppet Works http://localhost:9090/onepage 6/43
Data Flow How Puppet manages data flow from Individual Nodes http://localhost:9090/onepage 7/43
Declarative Modeling Language Model the desired state, Puppet figure out how to enforce it. Imperative Shell Code Comparison if [ 0 -ne $(getent passwd elmo > /dev/null)$? ] then useradd elmo --gid sysadmin -n fi Declarative Puppet C user { 'elmo': ensure => presen gid => 'sysadm } GID=`getent passwd elmo awk -F: '{print $4}'` GROUP=`getent group $GID awk -F: '{print $1}'` if [ "$GROUP"!= "$GID" ] && [ "$GROUP"!= "sysadmin" ] then usermod --gid $GROUP $USER fi if [ "`getent group sysadmin awk -F: '{print $1}'`" == "" ] then groupadd sysadmin fi group { 'sysadmin': ensure => presen } http://localhost:9090/onepage 8/43
Puppet enforces resources in an idempotent way. Idempotency It is our job as developers to teach Puppet how to do this for resource types we develop so that our users don't have to. # First Puppet Run notice: /Group[sysadmin]/ensure: created notice: /User[elmo]/ensure: created notice: Finished catalog run in 0.08 seconds # Second Puppet Run notice: Finished catalog run in 0.03 seconds Idempotence: The property of certain operations in mathematics or computer science in that they can be applied multiple times without further changing the result beyond the initial application. Notes: Idempotent able to be applied multiple times with the same outcome. Puppet resources are idempotent, since they describe a desired final state rather than a series of steps to follow. Source: Puppet Docs http://docs.puppetlabs.com/references/glossary.html#idempotent http://localhost:9090/onepage 9/43
Describe the desired state. Let Puppet maintain it. State Model http://localhost:9090/onepage 10/43
Puppet Resources Resources are building blocks. They can be combined to make larger components. Together they can model the expected state of your system. http://localhost:9090/onepage 11/43
Puppet Demo http://localhost:9090/onepage 12/43
Demo summary Modules: directories that contain your configuration. Encapsulates all of the components related to a given configuration in a single directory hierarchy that enables the following: Auto loading of classes File serving for templates and files Auto delivery of custom Puppet extensions Easy sharing with others Facts: key/value pairs generated by facterabout a given node. Classes: collection of resources that are managed together as a single unit. http://localhost:9090/onepage 13/43
Why use Ruby to ext Puppet? Avoid clumsy and polutted manifests when using Puppet Domain Specific Language is not adequate. More control of the underlying platform. Better error and exception handling. Puppet is modular and extable by design. http://localhost:9090/onepage 14/43
Custom Facts To create your own custom facts, add ruby files in: <modulepath>/<modulename>/lib/facter /etc/puppetlabs/puppet/modules/custom_facts/ lib facter gem_count.rb ## $::gem_count # /etc/puppetlabs/puppet/modules/custom_facts/lib/facter/gem_count.rb Facter.add('gem_count') do setcode do IO.popen('gemlist').readlines.length.to_s Facter.addwill create a fact with the given name. You should pass a block to Facter.addthat calls the setcodemethod. The block you pass to the setcodemethod is the body of your fact. http://localhost:9090/onepage 15/43
Fact Values Fact values are simple strings. The fact is the value returned from the setcodeblock. This value will be cast to a string if necessary. If a nilor empty string is returned, the fact will not be set. Puppet will populate global variables with all facts, including custom facts. Built in fact: $::osfamily Custom fact: $::gem_count Every manifest on the master can use every fact. http://localhost:9090/onepage 16/43
Custom Facts Demo http://localhost:9090/onepage 17/43
Functions Functions are modular snippets of Ruby code. Functions run during compilation. Functions run on the puppet master. Provide additional functionality to the Puppet DSL. http://localhost:9090/onepage 18/43
A function is either a :statementor an :rvalue. Types of Functions :statement Code that just executes. May perform an action, such as raising a warning to be logged in the report. This is the default. :rvalue Code that executes and returns a value. Assigned to a variable. Assigned to a resource parameter. http://localhost:9090/onepage 19/43
Adding a Function To create your own custom functions, add ruby files in: <modulepath>/<modulename>/lib/puppet/parser/functions The name of the function and the filename must match and unlike facts, a file can contain only a single function. /etc/puppetlabs/puppet/modules/custom_facts/ lib puppet parser functions myfunc.rb ## myfunc() A custom :statementfunction: Puppet::Parser::Functions.newfunction(:myfunc) do #... A custom :rvaluefunction: Puppet::Parser::Functions.newfunction(:myfunc2, :type => :rvalue) do #... http://localhost:9090/onepage 20/43
Arguments in Functions Arguments are passed to Puppet functions as a single array. Note that arguments from Puppet are converted to Strings. Puppet::Parser::Functions.newfunction(:multiply, :type => :rvalue) do args Integer(args[0]) * Integer(args[1]) Using a function in a Puppet manifest: # do_some_math.pp $output = multiply(2, 3) notice("theansweris${output}") [root@training ~]# puppet apply do_some_math.pp notice: Scope(Class[main]): The answer is 6 notice: Finished catalog run in 0.05 seconds Using a function via inline execution: [root@training ~]# puppet apply e 'notice(multiply(2,3))' notice: Scope(Class[main]): 6 notice: Finished catalog run in 0.05 seconds http://localhost:9090/onepage 21/43
Custom Functions Demo http://localhost:9090/onepage 22/43
What is a Resource Type? Resource Types abstract a physical resource: specify only the interface of a resource allow Puppet users to declare a resource in terms of attributes: When developing a Type in Ruby, you can specify: attributes documentation implicit relationships http://localhost:9090/onepage 23/43
Resource Abstraction Layer Provides a consistent model across supported platforms. http://localhost:9090/onepage 24/43
Resource Type Each resource type has one or more providers. http://localhost:9090/onepage 25/43
What is a Provider? Resource types dep on providers to apply changes to the underlying operating system. They translate specification into implementation. Several providers are typically available for a given resource type to provide cross platform support. http://localhost:9090/onepage 26/43
Providers The interface between the resource and the OS. http://localhost:9090/onepage 27/43
package {'vim': ensure => present, } Providers Providers translate specification into implementation. This will use the default provider for the package type to ensure that vimis installed. This may run yumor apt getcommands, for example. http://localhost:9090/onepage 28/43
Developing Types To create your own custom types, add ruby files in: <modulepath>/<modulename>/lib/puppet/type /etc/puppetlabs/puppet/modules/site/ lib puppet type media.rb # /etc/puppetlabs/puppet/modules/site/lib/puppet/type/media.rb Puppet::Type.newtype(:media) do desc "Thisisanexampletypetosyncmediafiles." #... Puppet::Type.newtypewill create a type with the given name. The name of the type and the filename must match. The block you pass to the newtypemethod is the body of your type. The descmethod adds a type description. Notes: Descriptions for built in and custom types can be accessed with puppet describe <type>. http://localhost:9090/onepage 29/43
Adding Providers to your Type To create providers for a type, add ruby files in: <modulepath>/<modulename>/lib/puppet/provider/<type>/ /etc/puppetlabs/puppet/modules/site/ lib puppet provider media http.rb # /etc/puppetlabs/puppet/modules/site/lib/puppet/provider/media/http.rb Puppet::Type.type(:media).provide(:http) do desc "HTTPproviderforthemediatype." #... The providemethod of Puppet::Type.type()will create a provider for a type of the given name. The name of the type and the path must match. The name of the provider and the filename must match. The block you pass to the providemethod is the body of your provider. The descmethod adds a type description. http://localhost:9090/onepage 30/43
Custom Type and Provider Demo http://localhost:9090/onepage 31/43
Hiera, Faces and Reports http://localhost:9090/onepage 32/43
Hiera Hiera works as a data lookup tool. Benefits of retrieving configuration data from Hiera: Easier to configure your own nodes. Easier to reuse public Puppet modules. Easier to design your modules for reuse. Easier to publish your own modules for collaboration. Easier to ensure that all nodes affected by changes in configuration data are updated. http://localhost:9090/onepage 33/43
Puppet Integration Ships with Puppet by default Configured via /etc/puppetlabs/puppet/hiera.yaml Facts and other variables in scope are used for data resolution. YAML back provided by default. Puppet DSL lookup functions included. # /etc/puppetlabs/puppet/hiera.yaml :backs: yaml :yaml: :datadir: '/etc/puppetlabs/puppet/hieradata' :hierarchy: %{environment}/%{osfamily} %{osfamily} common With this configuration, Hiera will look up values based on environment& osfamilyfirst, then just osfamily, before defaulting to returning common values. http://localhost:9090/onepage 34/43
Using the YAML back to retrieve data. Hiera YAML back cat /etc/puppetlabs/puppet/hieradata/defaults.yaml message: "This is a sample variable that came from Hiera" # puppet apply e "notice(hiera('message'))" Notice: Scope(Class[main]): This is a sample variable that came from Hiera Notice: Finished catalog run in 0.18 seconds Other backs available: json MySQL PostgreSQL eyaml (Encrypted) http://localhost:9090/onepage 35/43
Adding a Hiera Back To create your own Hiera backs, add ruby files in: <modulepath>/<modulename>/lib/hiera/back /etc/puppetlabs/puppet/modules/custom_hiera/ lib hiera back custom_back.rb # /etc/puppetlabs/puppet/modules/custom_hiera/lib/hiera/back/custom_back.rb class Hiera module Back class Custom_back def initialize Hiera.debug("[CustomBack]:Initialized") def lookup(key, scope, order_override, resolution_type) Hiera.debug("[CustomBack]:Lookingup'#{key}'") http://localhost:9090/onepage 36/43
Built in Faces Much of Puppet is exposed through built in Faces. You can see the full list with puppet help. % puppet help Usage: puppet <subcommand> [options] <action> [options] Available subcommands, from Puppet Faces: ca Local Puppet Certificate Authority management. catalog Compile, save, view, and convert catalogs. certificate Provide access to the CA for certificate management. config Interact with Puppet's configuration options. configurer Like agent example A short example facts Retrieve and store facts. file Retrieve and store files in a filebucket help Display Puppet help. key Create, save, and remove certificate keys. man Display Puppet manual pages. module Creates, installs and searches for modules on the Puppet Forge. node View and manage node definitions. parser Interact directly with the parser. plugin Interact with the Puppet plugin system. report Create, display, and submit reports. resource API only: interact directly with resources via the RAL. resource_type View classes, defined resource types, and nodes from all manifests. secret_agent Mimics puppet agent. status View puppet server status. http://localhost:9090/onepage 37/43
Invoking a Face puppet [FACE] [ACTION] [ARGUMENTS] [OPTIONS] % puppet example hello Hello there FACE The face to run. ACTION The action that should be invoked on the face. ARGUMENTS Arguments passed to the action. OPTIONS Options to the command. http://localhost:9090/onepage 38/43
Creating Faces Place Ruby files in <module>/lib/puppet/{application,face} lib puppet application example.rb face example.rb An application provides the subcommand: require 'puppet/application/face_base' class Puppet::Application::Example <Puppet::Application::FaceBase A face provides the logic: Puppet::Face.define(:example, '0.0.1') do summary "Thisdoesn'tdomuchyet" #... action :hello do when_invoked do options "Hellothere" Face definitions must include the version using semantic versioning Notes: You may read about semantic versioning at http://semver.org. Learn more about Puppet Faces versioning at http://puppetlabs.com/blog/puppet faces what the heck are faces/. http://localhost:9090/onepage 39/43
Puppet Reporting Interface Each Puppet run on a client generates a report containing: every action taken during the run log output from the client metrics showing the performance of the run Some example default report handlers provided by Puppet: Saving reports in the Console database Sing emails based on changed resources Puppet also provides an interface for designing custom report handlers. http://localhost:9090/onepage 40/43
Developing Report Handlers To create your own custom report handlers, add ruby files in: <modulepath>/<modulename>/lib/puppet/reports /etc/puppetlabs/puppet/modules/custom_reports/ lib puppet reports sample.rb # /etc/puppetlabs/puppet/modules/custom_reports/lib/puppet/reports/sample.rb require 'puppet' Puppet::Reports.register_report(:sample) do desc "documentthereport" def process... The filename should match the name of the report handler. Register a report with Puppet's report handler. Document the report with the descmethod. Reporting logic should be implemented in the processmethod. http://localhost:9090/onepage 41/43
Processing a Report # /etc/puppetlabs/puppet/modules/custom_reports/lib/puppet/reports/sample.rb require 'puppet' Puppet::Reports.register_report(:sample) do desc "documentthereport" def process # report is available as the self object if self.status == :failed self.logs.each do log # do something with log The report itself is available as selfwithin the processmethod. Access data members as: self.host self.status self.time self.logs etc. Notes: The selfobject is actually redundant here. It is not required in the ruby language. It is used on this slide simply to make it clear where these data members are coming from. http://localhost:9090/onepage 42/43
Next Steps & Questions Brazillian Community: http://puppet br.org Instruct: http://instruct.com.br Puppet training in Brazil http://instruct.com.br/calario.html Learn more! Download Puppet Enterprise manage 10 nodes for free http://puppetlabs.com/download puppet enterprise Learning Puppet Tutorials http://docs.puppetlabs.com/learning/ Download the Learning Puppet VM http://info.puppetlabs.com/download learning puppet VM.html Puppet Docs http://docs.puppetlabs.com/ http://localhost:9090/onepage 43/43