--- date: 2016-01-05 title: Using EYAML with Puppet 4 category: devops --- Happy 2016 all This weekend I finally got round to adding eyaml support to Puppet in my lab. What is on earth am I talking about? Puppet can use a thing called *Hiera* as a data source, think of it as a database for configuraion. In an ideal world, your manifests will be completely generic - in fact your control repo could consist of nothing but a `Puppetfile` with a list of modules to install (if any one lives in that ideal world, you are better than me). Hiera in turn can have different backends for describing this data, such as: - JSON - YAML - eYAML or *encrypted* YAML So how do we use it? On my workstation (where I do my Puppet development), I also the eyaml gem: gem install hiera-eyaml It should be noted that I run [RVM](https://rvm.io) and run a Ruby environment and Gemset just for Puppet development (https://gogs.chriscowley.me.uk/chriscowley/puppet/src/production/.rvmrc). My Gemfile includes the *hiera-eyaml* gem, so I simply run `bundle install`. Next, you need to create some keys: eyaml createkeys This creates your key pair in `./keys`. Move this somewhere more sensible and configure eyaml to look there. mkdir -pv ~/.eyaml mv -v keys ~/.eyaml/ cat > ~/.eyaml/config.yaml < EOF --- pkcs7_private_key: '/home/chris/.eyaml/keys/private_key.pkcs7.pem' pkcs7_public_key: '/home/chris/.eyaml/keyspublic_key.pkcs7.pem' EOF Now you can test it by running `eyaml encrypt -l msql_root_password -s "correcthorsebatterystaple"` which will output something like: [hiera-eyaml-core] Loaded config from /home/mmzv6833/.eyaml/config.yaml msql_root_password: ENC[PKCS7,MIIBiQYJKoZIhvcNAQcDoIIBejCCAXYCAQAxggEhMIIBHQIBADAFMAACAQEwDQYJKoZIhvcNAQEBBQAEggEAPvmib/bFce7ArK/FSMHX9DVsqDo38tL/Xpc9XtWCPlqvfkfwBFPRD0qM2qbEL3JchRSmirb/yBy/20HFk7vX84PIy7IfSYEt+u2RkVUuWgSHfjnKVnJc5wul8IqHdeWIoFT5/D6dsrBP94qD6CwbIzKRRzSijuxPMbXhQQecwPBBSQAtNHWAVsw4U7lv7tVP+OoZSSnP0zqJOp2Pt6x4ivj/Wha4hPcF8KvUNKLR7ZcebHbJslJUTYqg1cMwRPMuccbXS3JvGdoFiACAPEjghbAmK6UgaZ2nTxuVGJ4B81Q6Nnsk3Ir/JVjFCKI+x+bZoVn+y1coLBPy52RE5OdPoDBMBgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBCrcffFAXvzkNnYGpjcIVr2gCBpSG4Q9HZRDT07Yz0ijDb+3RlbLnRzlMvsP2O4phTOig==] OR msql_root_password: > ENC[PKCS7,MIIBiQYJKoZIhvcNAQcDoIIBejCCAXYCAQAxggEhMIIBHQIBADAFMAACAQEw DQYJKoZIhvcNAQEBBQAEggEAPvmib/bFce7ArK/FSMHX9DVsqDo38tL/Xpc9 XtWCPlqvfkfwBFPRD0qM2qbEL3JchRSmirb/yBy/20HFk7vX84PIy7IfSYEt +u2RkVUuWgSHfjnKVnJc5wul8IqHdeWIoFT5/D6dsrBP94qD6CwbIzKRRzSi juxPMbXhQQecwPBBSQAtNHWAVsw4U7lv7tVP+OoZSSnP0zqJOp2Pt6x4ivj/ Wha4hPcF8KvUNKLR7ZcebHbJslJUTYqg1cMwRPMuccbXS3JvGdoFiACAPEjg hbAmK6UgaZ2nTxuVGJ4B81Q6Nnsk3Ir/JVjFCKI+x+bZoVn+y1coLBPy52RE 5OdPoDBMBgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBCrcffFAXvzkNnYGpjc IVr2gCBpSG4Q9HZRDT07Yz0ijDb+3RlbLnRzlMvsP2O4phTOig==] This is all well and good for you dev environment, but utterly useless for the Puppetmaster as it has no idea about the key. Even if it did, out-of-the-box it will not look for the encrypted data anyway. So, copy the keys to your Puppetmaster: scp -r ~/.eyaml/keys user@puppetmaster:~/ Notice I user `scp`: at least the private key is extremely sensitive; never transport it in clear text, make sure you store it securely, etc, etc. You have been warned. Now put them where your Puppetserver can get to them and install the Gem. sudo mkdir -pv /etc/puppetlabs/puppet/secure sudo mv -v keys /etc/puppetlabs/puppet/secure/ sudo chown -Rv puppet:puppet /etc/puppetlabs/puppet/secure sudo chmod -Rv 550 /etc/puppetlabs/puppet/secure sudo puppetserver gem install hiera-eyaml Now you need to modify your hiera config file (`/etc/puppetlabs/code/hiera.yaml`) to look something like: --- :backends: - eyaml - yaml :hierarchy: - %{::clientcert} - %{::environment} - "virtual_%{::is_virtual}" - common :yaml: :datadir: '/etc/puppetlabs/code/hieradata' :eyaml: :datadir: '/etc/puppetlabs/code/hieradata' :pkcs7_private_key: /etc/puppetlabs/secure/keys/private_key.pkcs7.pem :pkcs7_public_key: /etc/puppetlabs/secure/keys/public_key.pkcs7.pem A couple of things here: - I do not keep my hiera data in my repo, I like to have it completely separate. - It is truly a database for my Puppet code - I can use the same data across environments - I keep my encrypted data separate too, as is default, in .eyaml files. That is the Puppetmaster ready to use, we just need to actually put some encrypted data in there for it to collect. Think back to Puppet's excellent [Hiera docs](http://docs.puppetlabs.com/hiera/latest/hierarchy.html) ![Hierarchy](http://docs.puppetlabs.com/hiera/latest/images/hierarchy1.png) Now the host _db01_ will have potentially 2 files specific to it: `db01.example.com.yaml` and `db01.example.com.eyaml`. So to install Mysql on that (naturally using the [Puppetlabs module](https://forge.puppetlabs.com/puppetlabs/mysql)) `db01.example.com.yaml` will contain: --- classes: - mysql::server mysql::server::remove_default_accounts: true then in `db01.example.com.eyaml` put: --- mysql::server::root_password: > ENC[PKCS7,MIIBiQYJKoZIhvcNAQcDoIIBejCCAXYCAQAxggEhMIIBHQIBADAFMAACAQEw DQYJKoZIhvcNAQEBBQAEggEAhWgxLsgvtUzALxqE23nrcgy8xR+UbV5b45Vo joRLq4QLDhLKuwAsoaQ3MbYfrbJ5RQ2PTFlwB+Cp7X2uLQ0YYfisABT/dwaK 9iYZoXkvsSvt8iqZkVNP9HZLf/X1EkLfljbsEx7vigMyWu8ApDt5aGCxqGA6 NTGZkeOoUhfRM9KuzRvkIQB0eutuIx420EgKI0gdCVPv1Y51UdEMl7rClwz3 4ATlPmL0F2NVNifZC+KdWGei+PYSYM394JvS0ZBxuNWLowlmR2SgbzSCpWZn mB1jolaG7nXv7Y1OnvZraA3EIUcwKiILlsC1vlXuVc6xdKBvhb70j6p30SzB 6eF2IzBMBgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBCH5GYGDeLdZniZdkCt Xe7bgCAaY8TVUw4NaHc2ARbCSAsZSH91UPaDMAWaC6wrYorLEw==] When the agent on _db01_ connects, the Puppetmaster will work its way down through the hierachy populating each variable as it goes. As the _eyaml_ backend is defined first in the hierachy, then it will take priority (for simple paramters, hashes and arrays are more complex). This means that if you were to leave an old value for `mysql::server::root_password` in the plain yaml file, it will be ignored. I hope this is helpful for someone, what it means is that you can share all the code for your infrastructure with the world. I will certainly be doing this, I just need to refacter things a little to put my defined types into Hiera as I use them to create databases. This is not complex and will be the subject of a future post.