Mar 20, 2014

Few best practices on setting up Puppet 3 master/agent environment

Puppet is a configurations management tool like Chef and CFEngine. This tool is to manage configurations of large dynamically changing infrastructures like clouds efficiently. Puppet 3 is the latest release from PuppetLabs but still some operating system distributions does not include those packages in their repositories. So we need to some manual things to install puppet 3.

In this post I will explain few best practices to follow when installing a puppet master - agent environment. I have configure puppet master and agent environments several times and came across with this sequence and I think this a good way of doing this. But please note this not "the" best way of doing it and not recommended to use it as it is in a production environment. And also this will not describe about best practises of writing puppet manifests/modules.

Set a domain name for the environment
First of all use a domain name for your environment. Think that you are going to set up a puppet environment for ABC company, you can set the domain for that as 'abc.com' or 'dc1.abc.com' (data center 1 of ABC company). If you are doing it for testing purposes its advisory to use 'example.com'. 'example.com' is a reserved domain name for documentation and example purposes and no one can register that domain, so it will avoid many DNS resolution issues.

Give a proper FQDN for each hosts hostname.
Set a fully qualified domain name (FQDN) to each and every host within the puppet environment including the puppet master node. It will reduce lots of SSL related issues. It is not enough to just to give a hostname because most systems adds a domain (via DHCP) that will introduce some issues. Run 'hostname' and ' hostname -f ' and see the difference.

Use 'puppet' as a prefix as the puppet masters hostname. So it would be like;

    puppet.abc.com or
    puppet.cd1.adc.com or
    puppet.example.com

And for the puppet agents;

    8976712.apache.abc.com or
    8976712.apache.dc1.abc.com or
    8976712.apache.example.com

Or

    8976712.node001.abc.com or
    8976712.node002.dc1.abc.com or
    8976712.node003.example.com

Use a UUID when creating the hostnames for puppet agents. Then give the service name (apache,mysql) or the node number (node002 - if using multiple services in a single server). That name must match the node definitions in the 'site.pp' (or 'nodes.pp').

Use the 'hostname' command and edit the '/etc/hostname' configurations file to change the hostname. You can do it like this, assuming that the host is '8976712.node001.abc.com'

# hostname 8976712.node001.abc.com
# echo '8976712.node001.abc.com' >/etc/hostname

Give and IP address to each FQDN.
It is a must to give an appropriate IP addresses to each hostname/FQDN. At least, the system should be able to refer to the '/etc/hosts' file and resolve the IP address of the relevant FQDN and should have following entries in the '/etc/hosts' file.

    127.0.0.1 localhost
    127.0.0.1 < local fqdn >
    < puppet master ip > < puppet master fqdn >

For an example, if you take '8976712.node001.abc.com' node, its '/etc/hosts' file should like this.

    127.0.0.1 localhost
    127.0.0.1 8976712.node001.abc.com
    192.168.1.100 puppet.abc.com

Check the system time and timezone information
Both puppet master and agents should have same system time and time zone on both systems. Use 'date' command to check the system time and time zone. Synchronize the system time with a well known time server. Commands are bit different from one distribution to another.

Download and install puppet repositories from PuppetLabs website
PuppetLabs provide an apt and a yum repository. Most distributions does not support puppet 3 for the moment therefore, we need to add those manually.

Please refer to "Using the Puppet Labs Package Repositories" article and install the appropriate repository for your system. Then update your repository lists.

Install puppet master 
After completing all above steps, then try to install puppet master using a package management system (apt/yum).

It's better to go ahead with default setting. But you need to do few changes to some configuration files to make it work as a master-agent environment puppet master server. Use a 'autosign.conf' file to automatically sign agents SSL requests. But avoid using ' * ' in that. Better to use it like this;

*.abc.com

It's better to add the 'server=puppet.< domain >' in the 'puppet.conf  's 'main' section. On Debian based distros change the 'start' option in to 'yes' to start the puppet master. After configuring all restart the puppet master service. Open port 8140 from the system firewall specially check that if you are using any RedHat distribution.

Track changes
Use a version controlling system like git or subversion to track changes to puppet manifests. Use branching, versioning/tagging features to do it effectively.

Install puppet agent
First of all it is better to have puppet master installed. Then check the hostname and DNS resolutions for the hostname and puppet master. Then try to install puppet agent using a package management system.

You have to do few changes to connect to the puppet master server. Edit the '/etc/puppet/puppet.conf ' and add 'server=puppet.< domain >' to the 'main' section. Change the 'start' option to 'yes' in '/etc/default/puppet' configuration file in debian based distros. Then restart the puppet agent.

Test the system
Add this into your puppet masters '/etc/puppet/manifests/site.pp' file.
node default {
    file { '/tmp/mytestfile.t':
        owner   => 'root',
        group   => 'root',
        content => "This file was created by puppet.\n",
        ensure  => present,
    }
Then run 'puppet agent -vt ' on the agent and check the '/tmp ' directory.

Automated script
I wrote a script to automate this and you can get it from here on github. It support Debian, RedHat and SLES distributions. If you have any issues please report those to this.

Feb 4, 2014

KVM with Virt-Manager as a virtualization tool for Linux

I have use several operating system (OS) level virtualization tools like VMware Workstation, VMware Player, Oracle VirtualBox, Microsoft VirtualPC and KVM for many years.

Overall VMware Workstation is the best tool for me. But to use that we need to purchase a license.  As an alternative to VMware Workstation we can use VMware player (a stripdown version of WMware Workstation) for free but only for non-commercial use. Also you can't run multiple guest operating systems concurrently using that.

As an alternative to VMware products most of linux people use Oracle VirtualBox. I had some issues when I try to NAT a virtual instance (guest OS) on Ubuntu 12.10 (host OS) machine. As a solution for this most blogs forums suggest to change the virtual network option into Bridge. But most networks (including my home wifi network) doesn't allow this option because we do MAC address filtering.

Obviously you can not install Microsoft VirtualPC on a Linux host (even with vine). Truly I haven't use Xen, so I can't give any opinions on that.

kernel-based virtual machine (KVM) is another tool that we can use to do OS level virtualization. I will explain how to install KVM and Virt-Manager the graphical user interface which can use to interact with KVM on Ubuntu (Check this to install KVM on CentOS).

Installing KVM and Virt-manager on Ubuntu


Update your repository list
sudo apt-get update
Install packages and dependencies
sudo apt-get install kvm virt-manager
After completing the installation you can search for "Virtual machine Manager" on the search of Ubuntu Unity. Give the sudo password in the popup window.

Or else you can use the following command to start the virt-manager GUI from the terminal.
sudo virt-manager
In the first attempt it will prompt the following user interface. Use the default settings and click on "connect".

KVM Virt-Manager add connection

Installing guest operating system in KVM


Open virt-manager by clicking on the icon from unity search or using command. Then click on the left most icon of the GUI as follows. It will open a wizard to create a new virtual machine.

Create new virtual machine


Give a proper name, that name will appear on the virt-manager virtual machine list.

There are several ways to install an operating system and this tool also support few of them too. Usually we use a boot-able CD/DVD to install an operating system to a new machine/laptop. Also we can directly give the ISO image of an operating system.  Therefore, I will go ahead with the "Local installation media" option and click on "Forward"


In this I'm going to give an Ubuntu 12.04 desktop ISO image. Select the "Use ISO image" option and click on "Browse".


It will open another window which will list down all the image files of your virtual machines. Click on the bottom "Browse Local" button and browse to the ISO image that you need to install. Give the OS type and version in relevant fields. Then continue the wizard.


You can download those ISO images from relevant websites other than Windows.

In this step you need to set the guest machines memory and CPU. As I'm going to install a Ubuntu Desktop with the user interface (UI), I'm going to give 1024 MB of RAM and a single CPU will work for this.


Now we need to set the disk space. It is enough to give 8-10GB of disk space for a virtual (guest) machine.

Specially, remove the default check on the "Allocate entire disk now" option. If you do so, KVM will not allocate full 10GB (or what you set) from your host machines hard disk. It will only use the real data capacity used in the installation and only when you add data to the guest system it will grow. So this save lot of disk space.


In the last step you will get a confirmation page.

Few best practices when using KVM

  • To run virtual machine manager you need sudo permission. So you can create an alias for this.
alias virt-manager='sudo virt-manager'
sudo visudo
  • Add the line with your username
username ALL= NOPASSWD: ALL
  • Try to use text only installations of operating systems. It will reduce resources (RAM/disk space) usage. Most server edition operating systems by default install the text only (run level 3) environment.
  • Use base images
Create some base operating system installations in the system. If you need a virtual machine you can get a clone of the base installation and use. In this figure I have create three base images (virtual machines)
a CentOS, RHEL and a Ubuntu.

Other three machines are clones of a Ubuntu base machine which I used to simulate a Ubuntu base network. After the simulation I can delete those virtual machines with used virtual hard disk (.img file).


Install your favorite commonly used tools like vim/emacs, tree, htop, telnet, git, subversion, oracle JDK, links, debug tools and custom scripts.

Install additional repositories like EPEL, RPMForge,  repos on RedHat base distributions. Puppet repositories on all distributions.

So how to clone a virtual machine? Right click on the virtual machine (should be on power off state) and select "Clone" option. It will give the following window. Give a proper name for that and continue it. It will take few minutes to copy and the time will depend on your base image size.


When you need delete a virtual machine, select and right click on that and select "Delete" option. It will prompt another window as follows. Select the "Delete associated storage files" option and it will enable the list of storages which you need to delete in order to save your disk space. Keep in mind not to delete any iso images if those appear on this list.


May 5, 2013

Connect to a Cisco VPN via VPNC using .pcf configurations file

If you want to connect to a Cisco VPN from a Linux host, its better to use the Cisco AnyConnect VPN client for Linux. It is a free client tool but you need to login to the Cisco website.

VPNC is a free VPN client which is capable of connecting to Cisco VPNs (3000).

Use yum to install vpnc.

# yum install vpnc NetworkManager-vpnc vpnc-consoleuser -y

It will install all necessary packages. In the installation it create a default configurations file in the configurations directory. You need to backup that.

# mv  /etc/vpnc/default.conf  /etc/vpnc/default.conf.bak

Then you need to convert the .pcf file into a vpnc configurations file. To do that you need to run a script.

# perl   /usr/share/doc/vpnc-*/pcf2vpnc   your.pcf   /etc/vpnc/default.conf


After creating the configurations file you can try to connect to the VPN by;

# vpnc

It will ask for the user password;

# Enter password for youusername@your.vpn.server :

Give the password. If that succeed you can see a message like this.

# VPNC started in background (pid: 18957)

If you want to disconnect from the VPN, simply type ;

# vpnc-disconnect

Then it will try to disconnect your session from the VPN and will give a message like this.

# Terminating vpnc daemon (pid: 18957)