Puppet CA: Certificates explained Thomas Gelf - PuppetCamp Düsseldorf 2014
Thomas Gelf, nice to meet you! joined NETWAYS in 2010 formerly more than ten years of... web (application) development routing/switching: bank/isp backbones ISP: Mail, Hosting, SIP-Carrier, IPv6...
Origins nationality: Italian mother tongue: German kind of. SOUTH TYROLEAN!!!
Me and Puppet first Puppet steps with 0.24 talks, articles, blog posts trainer, consultant over-certified
Me @ PuppetConf 2014 Had a great time, the conference was awesome! PuppetConf 2015 will be in Portland - see you there :)
NETWAYS
Netways and Puppet German Puppet Labs Training Partner Trainings Consulting Workshops
http://www.netways.de/training Puppet Trainings
What this talk is all about certificates puppet certificates REST API distributed environments security issues and their consequences certificate lifecyle
WHY SHOULD I CARE?
Running Puppet Enterprise?
CERTIFICATES
Public Key Infrastructure - PKI everybody has it's own private key signs or encrypts a message verification/decryption uses public key algorithms: RSA, DSA...
PKI - Wikipedia
X.509 describes how our Puppet PKI works https:// - you use it every day ITU-T standard defines a strict hierarchy a tree instead of a "web of trust" X509v3: allows extensions
Certificate structure (distinguished) name serial number algorythm issuer validity: FROM - TO...
The distinguished name: DN just a string often a DNS name could also be "CA: puppet master" something you should care about!
The revocation list allows to invalidate certificates does so based on serial numbers important if you "loose" certificates
.csr: certificate signing request, Base64 Filename extensions -----BEGIN CERTIFICATE REQUEST-----... -----END CERTIFICATE REQUEST-----.pem: a certificate, Base64 -----BEGIN CERTIFICATE----- Puppet uses.pem also for private keys: -----BEGIN RSA PRIVATE KEY-----
PUPPET CERTIFICATES
Puppet certificates: archeology Want to see a fresh new Puppet CA? Try it out! mkdir /tmp/ssltest puppet master --no-daemonize --verbose \ --ssldir /tmp/ssltest \ --certname test.example.com
Puppet certificates: archeology A fresh new Puppet CA!
Puppet certificates: archeology ls -l /tmp/ssltest
Same thing for the agent puppet agent --test \ --ssldir /tmp/sslagent \ --certname test.example.com
We all know the basics puppet cert list puppet cert list --all puppet cert sign test.example.com puppet cert revoke test.example.com puppet cert clean test.example.com find./ -name 'test.example.com*' --delete
SSL directories puppet master --configprint ssldir puppet agent --configprint ssldir manual configuration makes sense think about user permissions ~/.puppet, /var/lib/puppet master and agent on the same host passenger VS debug (--no-daemonize)
Let's dump a certificate openssl x509 -in testexample.com.pem -noout -text puppet cert print test.example.com
Custom data in your certificates https://docs.puppetlabs.com/puppet/latest/reference /ssl_attributes_extensions.html /etc/puppet/csr_attributes.yaml custom attributes in your CSR
MCollective
Study security guidelines! Study security guidelines! Study security guidelines! STUDY SECURITY GUIDELINES! puppetlabs.com/mcollective/security-overview
Get inspired by existing modules make sure you understood them or write your own ones re-use Puppet certificates read about trust and STUDY THE SECURITY GUIDELINES!
THE REST API
It's a web application! <VirtualHost *:8140> SSLEngine on SSLProtocol SSLCipherSuite SSLHonorCipherOrder ALL -SSLv2 -SSLv3 EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+... on SSLCertificateFile $ssldir/certs/$fqdn.pem SSLCertificateKeyFile $ssldir/private_keys/$fqdn.pem SSLCertificateChainFile $ssldir/ca/ca_crt.pem SSLCACertificateFile $ssldir/ca/ca_crt.pem SSLCARevocationFile $ssldir/ca/ca_crl.pem SSLVerifyClient optional SSLVerifyDepth 1 SSLOptions +StdEnvVars +ExportCertData
The Rest API # http://docs.puppetlabs.com/guides/rest_api.html https://master:8140/{environment}/{resource}/{key} available on puppet master and on VERY ancient agents (listen=true)
Puppet REST API URI examples GET /{environment}/catalog/{node certificate name} GET /{environment}/file_bucket_file/md5/{checksum} GET /{environment}/facts/{node certname}
Permissions http://docs.puppetlabs.com/guides/rest_auth_conf.html # auth.conf # allow all nodes to store their own reports path ~ ^/report/([^/]+)$ method save allow $1
SSL-enabled curl example Use your certificates and discover the API: curl \ --cert /var/lib/puppet/ssl/certs/host.pem \ --key /var/lib/puppet/ssl/private_keys/host.pem \ --cacert /var/lib/puppet/ssl/ca/ca_crt.pem \ -k -H "Accept: yaml" \ https://master:8140/production/facts/somehostname
DISTRIBUTED SETUP
Puppet Advanced* Training
One CA is more than enough: Configuration for such a setup [master] ca = false [agent] ca_server = ca.example.com Optionally, still experimental: DNS SRV records
Chain of trust Since 3.2.1 you can use intermediate CAs to delegate trust # http://docs.puppetlabs.com/puppet/3/reference\ # /config_ssl_external_ca.html [agent] ssl_client_ca_auth = $certdir/issuer.pem Tell Apache about your chain: SSLCertificateChainFile "/path/to/ca_bundle.pem"
It could look like this +------------------------+ Root self-signed CA +------+----------+------+ +----------+ +------------+ v v +-----------------+ +----------------+ Master CA Agent CA +--------+--------+ +--------+-------+ v v +-----------------+ +----------------+ Master SSL Cert Agent SSL Cert +-----------------+ +----------------+
SSL Professional? integrate it in your existing hierarchy use your own toolchain ship signed certificates (carefully)
SECURITY
Puppet and security issues Read Security Disclosures! https://puppetlabs.com/security/
Thank Heartbleed! docs.puppetlabs.com/trouble_remediate _heartbleed_overview.html docs.puppetlabs.com/latest/reference /ssl_regenerate_certificates.html
A specific security problem Very interesting and worth to read: CVE-2011-3872 "In versions prior to 2.6.12 and 2.7.6, the Puppet CA will improperly insert any certdnsnames values into agent certificates as well as master certificates. This bug was introduced in Puppet 0.24.0." puppet master --configprint certdnsnames puppet, puppet.example.com
Study it! http://links.puppetlabs.com/cve20113872_remediation Have a look at the remediation toolkit And to be on the safe side, check your agent certs: openssl x509 -in test.example.com.pem -noout -text \ grep 'Subject Alt' -A 1 X509v3 Subject Alternative Name: DNS:test.example.com, DNS:puppet, DNS:puppet.example.com
WARNING "upgrading" doesn't fix a mess like this old certificates would remain valid you have to switch to a new CA......and this leads us to the next topic
CA LIFECYCLE MANAGEMENT
Bad news Puppet should allow for automatic resigning of SSL certs http://projects.puppetlabs.com/issues/7272 There is no such thing in Puppet "...will be available with Puppet Sites"
YOU ARE ON YOUR OWN
One way of replacing a CA stop all agents throw away their certificates create a new CA with a new name start your agents sign their new CSRs
CA... master: rm -rf $(puppet master --configprint ssldir) agents: rm -rf $(puppet agent --configprint ssldir) # default ca_name: "Puppet CA: <master certname>" CERTNAME=$(puppet master --configprint certname) TS=$(date +%Y-%m-%d) puppet cert --generate \ --ca_name "Puppet CA: $CERTNAME <$TS>" $CERTNAME \ --dns_alt_names puppet,puppet.example.com puppet cert --allow-dns-alt-names sign $CERTNAME
You could also get inspired by the remediation kits write your own SSH loop fix it with MCollective (carefully!) open new feature requests
Don't like trouble? Before generating your CA: [master] ca_ttl = 20y Leave your company in time NB: expiration > 2038-01-19 == bad idea
BTW: WE ARE HIRING ;-)
Thank you for your attention!
Questions? class puppetcamp { } package { 'questions': ensure => answered }