Mar 20, 2014

Few best practices on setting up Puppet 3 master/agent environment

Puppet is a configurations management tool like Chef and CFEngine. This tool is to manage configurations of large dynamically changing infrastructures like clouds efficiently. Puppet 3 is the latest release from PuppetLabs but still some operating system distributions does not include those packages in their repositories. So we need to some manual things to install puppet 3.

In this post I will explain few best practices to follow when installing a puppet master - agent environment. I have configure puppet master and agent environments several times and came across with this sequence and I think this a good way of doing this. But please note this not "the" best way of doing it and not recommended to use it as it is in a production environment. And also this will not describe about best practises of writing puppet manifests/modules.

Set a domain name for the environment
First of all use a domain name for your environment. Think that you are going to set up a puppet environment for ABC company, you can set the domain for that as 'abc.com' or 'dc1.abc.com' (data center 1 of ABC company). If you are doing it for testing purposes its advisory to use 'example.com'. 'example.com' is a reserved domain name for documentation and example purposes and no one can register that domain, so it will avoid many DNS resolution issues.

Give a proper FQDN for each hosts hostname.
Set a fully qualified domain name (FQDN) to each and every host within the puppet environment including the puppet master node. It will reduce lots of SSL related issues. It is not enough to just to give a hostname because most systems adds a domain (via DHCP) that will introduce some issues. Run 'hostname' and ' hostname -f ' and see the difference.

Use 'puppet' as a prefix as the puppet masters hostname. So it would be like;

    puppet.abc.com or
    puppet.cd1.adc.com or
    puppet.example.com

And for the puppet agents;

    8976712.apache.abc.com or
    8976712.apache.dc1.abc.com or
    8976712.apache.example.com

Or

    8976712.node001.abc.com or
    8976712.node002.dc1.abc.com or
    8976712.node003.example.com

Use a UUID when creating the hostnames for puppet agents. Then give the service name (apache,mysql) or the node number (node002 - if using multiple services in a single server). That name must match the node definitions in the 'site.pp' (or 'nodes.pp').

Use the 'hostname' command and edit the '/etc/hostname' configurations file to change the hostname. You can do it like this, assuming that the host is '8976712.node001.abc.com'

# hostname 8976712.node001.abc.com
# echo '8976712.node001.abc.com' >/etc/hostname

Give and IP address to each FQDN.
It is a must to give an appropriate IP addresses to each hostname/FQDN. At least, the system should be able to refer to the '/etc/hosts' file and resolve the IP address of the relevant FQDN and should have following entries in the '/etc/hosts' file.

    127.0.0.1 localhost
    127.0.0.1 < local fqdn >
    < puppet master ip > < puppet master fqdn >

For an example, if you take '8976712.node001.abc.com' node, its '/etc/hosts' file should like this.

    127.0.0.1 localhost
    127.0.0.1 8976712.node001.abc.com
    192.168.1.100 puppet.abc.com

Check the system time and timezone information
Both puppet master and agents should have same system time and time zone on both systems. Use 'date' command to check the system time and time zone. Synchronize the system time with a well known time server. Commands are bit different from one distribution to another.

Download and install puppet repositories from PuppetLabs website
PuppetLabs provide an apt and a yum repository. Most distributions does not support puppet 3 for the moment therefore, we need to add those manually.

Please refer to "Using the Puppet Labs Package Repositories" article and install the appropriate repository for your system. Then update your repository lists.

Install puppet master 
After completing all above steps, then try to install puppet master using a package management system (apt/yum).

It's better to go ahead with default setting. But you need to do few changes to some configuration files to make it work as a master-agent environment puppet master server. Use a 'autosign.conf' file to automatically sign agents SSL requests. But avoid using ' * ' in that. Better to use it like this;

*.abc.com

It's better to add the 'server=puppet.< domain >' in the 'puppet.conf  's 'main' section. On Debian based distros change the 'start' option in to 'yes' to start the puppet master. After configuring all restart the puppet master service. Open port 8140 from the system firewall specially check that if you are using any RedHat distribution.

Track changes
Use a version controlling system like git or subversion to track changes to puppet manifests. Use branching, versioning/tagging features to do it effectively.

Install puppet agent
First of all it is better to have puppet master installed. Then check the hostname and DNS resolutions for the hostname and puppet master. Then try to install puppet agent using a package management system.

You have to do few changes to connect to the puppet master server. Edit the '/etc/puppet/puppet.conf ' and add 'server=puppet.< domain >' to the 'main' section. Change the 'start' option to 'yes' in '/etc/default/puppet' configuration file in debian based distros. Then restart the puppet agent.

Test the system
Add this into your puppet masters '/etc/puppet/manifests/site.pp' file.
node default {
    file { '/tmp/mytestfile.t':
        owner   => 'root',
        group   => 'root',
        content => "This file was created by puppet.\n",
        ensure  => present,
    }
Then run 'puppet agent -vt ' on the agent and check the '/tmp ' directory.

Automated script
I wrote a script to automate this and you can get it from here on github. It support Debian, RedHat and SLES distributions. If you have any issues please report those to this.