Feb 4, 2017

Granting dbadmin privileges to a user in MongoDB cluster

We need to grant 'dbadmin' privileges to a user called 'store_db_user' to their mondo database in a 4 node cluster.

First we need to connect to the primary database of the cluster with super.

# mongo -u supperuser -p password -h node1.mongo.local

If you connect to the primary replica it will change the shell prompt to something like this;

mongoreplicas:PRIMARY>

Then you can list down the databases using following command.

mongoreplicas:PRIMARY>show dbs
admin     0.078GB
local     2.077GB
store_db  0.078GB

Then switch to the relevant database;

mongoreplicas:PRIMARY>use store_db

And grant permissions;

mongoreplicas:PRIMARY>db.grantRolesToUser(
  "store_db_user",
  [
    { role: "dbOwner", db: "store_db" },
  ]
)

Exit from the admin user and login to the cluster as the database user.

# mongo -u store_db_user -p store_passwd -h node1.mongo.local/store_db

Validate the change.

mongoreplicas:PRIMARY>show users
{
	"_id" : "store_db.store_db_user",
	"user" : "store_db_user",
	"db" : "store_db",
	"roles" : [
		{
			"role" : "dbOwner",
			"db" : "store_db"
		},
		{
			"role" : "readWrite",
			"db" : "store_db"
		}
	]
}

Apr 27, 2016

Running your WordPress blog on WSO2 App Cloud

WSO2 App Cloud is now supporting Docker base PHP applications. In this blog post I will describe how to install a WordPress blog in this environment. In order to setup a WordPress environment we need to have two things;

  1. Web server with PHP support
  2. MySQL database

If we have both of these we can start setting up WordPress. In WSO2 App Cloud we can use the PHP application as the WordPress hosting web server which is a PHP enabled Apache web server docker image. Also it provides a database service where you can easily create and manage MySQL databases via the App Cloud user interface (UI).

Note:- 
For the moment WSO2 App Cloud is on beta therefore these docker images will have only 12h of lifetime with no data persistence in the file storage level. Data on MySQL databases will be safe unless you override. If you need more help don't hesitate to contact Cloud support.

Creating PHP application

Signup or signin to WSO2 App Cloud via http://wso2.com/cloud. Then click on the "App Cloud beta" section.
Then it will redirect you to the App Cloud user interface. Click on 'Add New Application' button on the left hand corner.
This will prompt you to several available applications. Select 'PHP Web Application' box and continue.
Then it will prompt you a wizard. In that give a proper name and a version to your application. Name and version will be use to generate the domain name for your application.

There are several options that you can use to upload PHP content to this application. For the moment I will download the wordpress-X.X.X.zip file from the wordpress site and upload it to application.
In the below sections of the UI you can set the run time and container specification. Give the highest Apache version as the runtime and use minimal container speck as wordpress does not require much processing and memory.
If the things are all set and the file upload is complete click on 'Create' button. You will get the following status pop-up when you click the create button and it will redirect you to the application when its complete.
In the application UI note the URL. Now you can click on the 'Launch App' button so that it will redirect you to your PHP application.
Newly installed WordPress site will be like this.
Now we need to provide database details to it. Therefore, we need to create database and a user.

Creating database

Go back to the Application UI and click on 'Back to listing' button.
In that UI you can see a button in the top left hand corner called 'Create database'. Click on that.
In the create database UI give a database name, database user name and a password . Password need to pass the password policy so you can click on 'Generate password' to generate a secure password easily. By the way of you use generate password option make sure you copy the generated password before you proceed with database creation. Otherwise you may need to reset the password.

Also note that database name and database user name will append tenant domain and random string accordingly to the end of both. Therefore, those fields will only get few number of input characters.
If all set then click on 'Create database' button to proceed. After successfully creating the database it will redirect you to a database management user interface like following.
Now you can use those details to login to the newly create mysql database as follows;
$ mysql -h mysql.storage.cloud.wso2.com -p'' -u
eg :-
$ mysql -h mysql.storage.cloud.wso2.com -p'XXXXXXXXX' -u admin_LeWvxS3l wpdb_thilina 
Configuring WordPress

If the database creation is successful and you can login to it without any issue we can use those details to configure WordPress.

Go back to the WordPress UI and click on 'let's go' button. It will prompt to a database configuration wizard. Fill those fields with the details that we got from the previous section.
If WordPress application can successfully establish a connection with the database using your inputs it will prompt you to a UI as follows.
On that click on 'Run the install'. Then WordPress will start populating database tables and insert initial data to the given database.

When its complete it will ask for some basic configurations like the site title, admin user name and passwords.
Click on 'Install WordPress' after filling those information. Then it will redirect you to the WordPress admin console login page. Login to that using the username and password gave in the previous section.
So now WordPress is ready to use. But the existing URL is not very attractive. If you have a domain you can use it as the base URL of this application.

Setting custom domain (Optional)

IN the application UI click on the top left three lines button shown in the following image.
It will show some advance configuration that we can use. In that list select the last one 'Custom URL' option.
It will prompt you following user interface. Enter the domain name that you are willing to use.
But before you validate make sure you add a DNS CNAME to that domain pointing to you application launch URL.

Following is the wizard that I got when adding the CNAME via Godaddy. This user interface and adding CNAME options will be different for you DNS provider.
You can validate the CNAME by running 'dig' command in Linux or nslookup in windows.
If the CNAME is working click on 'Update'.
 If that is successful you will get the above notification and if you access that domain name it will show your newly created WordPress blog.

Apr 2, 2016

Add Let's Encrypt free SSL certificates to WSO2 API Cloud

Let's encrypt is a free and open certificate authority runs for the public benefit. This service is provided by the Internet Security Research Group and there are lots of companies working with them to make the Internet secure. People who have a domain name can get free SSL certificate for their websites using this service for three months. I they need to use for more than that three months we need to renew the certificate and its also for free. But the best thing is that this certificate is accepted by most of the new web browsers and systems by default. So you don't need to add CA certs to you browsers any more.

In this article I will explain how we can use that service to get a free SSL certificate and add that to WSO2 API Cloud. So that you can have your own API store like;

https://store.thilina.piyasundara.org

In order to do that you need to have following things in hand.
  • Domain name.
  • Rights to add/delete/modify DNS A records and CNAMEs.
  • Publicly accessible webserver with root access or a home router with port forwarding capabilities. 

Step 1

If you have a publicly accessible webserver you can skip this step.If you don't have a publicly accessible webserver you can make your home PC/Laptop a temporary webserver if you can do port forwarding/NATing in you home router. I will show how I did that with my ADSL router. You can get help on port forwarding information by referring to this website http://portforward.com.

a. Add a port forwarding rule in your home router.

Get your local (laptop) IP (by running ifconfig/ip addr) and put that as the backend server in your router for. Set the WAN port as 80 and LAN port as 80.


After adding the rule it will be like this.

b. Start a webserver in your laptop. We can use the simple Python server for this. Make sure to check the IPTable rules/Firewall rules.

mkdir /tmp/www
cd /tmp/www/
echo 'This is my home PC :)' > index.html
sudo python3 -m http.server 80

c. Get the public IP of your router. Go to this link : http://checkip.dyndns.org it will give the public IP address. This IP is changing time-to-time so no worries.


d. Try to access that IP from a browser.
If it is giving the expected output you have a publicly accessible webserver.


Step 2

Now we need to update a DNS entry. My expectation is to have a single SSL certificate for both domains 'store.thilina.piyasundara.org' and 'api.thilina.piyasundara.org'.

a. Go to your DNS provides console and add an A record for both domain names to point to the public IP of your webserver (or the IP that we got from the previous step).


b. Try to access both via a browser and if its giving the expected out put you can proceed to the next step.


Step 3

I'm follow the instruction in the 'let's encrypt' guide. As I'm using the python server I need to use the 'certonly' option when running the command to generate the certs.

a. Get the git clone of the letsencrypt project.

git clone https://github.com/letsencrypt/letsencrypt
cd letsencrypt

b. Run cert generation command. (this requires root/sudo access)

./letsencrypt-auto certonly --webroot -w /tmp/www/ -d store.thilina.piyasundara.org -d api.thilina.piyasundara.org

If this succeed you can find the SSL keys and certs in '/etc/letsencrypt/live/store.thilina.piyasundara.org' location.

Step 4

Check the content of the certs. (Be root before you try to 'ls' that directory)

openssl x509 -in cert.pem -text -noout

Step 5

Create an API in WSO2 API Cloud if you don't have one. Else start on adding a custom domain to your tenant.

a. Remove both A records and add CNAME records to those two domains. Both should point to the domain 'customdns.api.cloud.wso2.com'.


b. Now click on the 'Configure' option in the top options bar and select the 'Custom URL' option.


c. Make ready you SSL certs. Copy 'cert.pem', 'chain.pem' and 'privkey.pem' to you home directory.

d. Modify API store domain. Click on the modify button, add the domain name click on verify. It will take few seconds. If that succeed you have correctly configured the CNAME to point to WSO2 cloud.

e. Add cert files to the API Cloud. The order should be the certificate (cert.pem), private key (privatekey.pem) and the CAs chain file (chain.pem). Again it will take sometime to verify uploaded details.


f. Update the gateway domain same as the previous.

Now if you go the API Store it will show something like this.



g. Same way you can use the gateway domain when you need to invoke APIs.

curl -X GET --header 'Accept: application/json' --header 'Authorization: Bearer ' 'https://gateway.api.cloud.wso2.com:8243/t/thilina/gituser/1.0.0/thilinapiy'

Now you don't need '-k' option. If not make sure you operating system (CA list) is up to date.

Step 6

Make sure to remove port forwarding in you home router if you use that and any changes that you make while obtaining the SSL certificates.

May 7, 2014

Run WSO2 products in a Docker container

Docker is an open-source project to easily create lightweight, portable, self-sufficient containers from any application. There are two ways to run docker container;

1. Run a pre-build docker image.
2. Build your own docker image and use it.

In the first option you can use a base image like Ubuntu, CentOS or an image built by someone else like thilina/ubuntu_puppetmaster. You can find these images index.docker.io

In the second option you can build the image using a "Dockerfile". In this approach we can do customizations to the container by editing this file.

When creating a docker container for WSO2 products option 2 is the best. I have wrote a sample Dockerfile on github. It describes how to build a Docker container for WSO2 API manager single node implementation. For the moment docker have some limitations like unable to edit the '/etc/hosts' file, etc. If you need to create a clusters of WSO2 products (an API manager cluster in this case) you need to do some additional things like setting up a DNS server, etc.

How to build an API manager docker container?


Get a git clone of the build repository.
git clone https://github.com/thilinapiy/dockerfiles
Download Oracle JDK 7 tar.gz (not JDK 8) and place it in '/dockerfiles/base/dist/'
mv /jdk-7u55-linux-x64.tar.gz /dockerfiles/base/dist/
Download WSO2 API manager and place that in '/dockerfiles/base/dist/'
mv /wso2am-1.6.0.zip /dockerfiles/base/dist/
Change directory to '/dockerfiles/base/'.
cd dockerfiles/base/
Run docker command to build image.
docker build -t apim_image .

How to start API manager from the build image?


Start in interactive mode
docker run -i -t --name apim_test apim_image
Start in daemon mode
docker run -d    --name apim_test apim_image
Other options that can use when starting a docker image
--dns  < dns server address >
--host < hostname of the container >

Major disadvantages in docker (for the moment)

  • Can't edit the '/etc/hosts' file in the container.
  • Can't edit the '/etc/hostname' file. --host option can use to set a hostname when starting.
  • Can't change DNS server settings in '/etc/resolve.conf'. --dns option can use to set DNS servers. Therefore, if you need to create a WSO2 product cluster you need to setup a  DNS server too.

Read more about WSO2 API manager : Getting Started with API Manager


May 6, 2014

Python 3 appindicator example script for Ubuntu 14.04

On Ubuntu 14.04 Python 3 is the default python version. Therefor If you try to run previous appindicator scripts on Ubuntu 14.04 those will not work. This script is done using Python 3 and relevant libraries.

Loading ....

Mar 20, 2014

Few best practices on setting up Puppet 3 master/agent environment

Puppet is a configurations management tool like Chef and CFEngine. This tool is to manage configurations of large dynamically changing infrastructures like clouds efficiently. Puppet 3 is the latest release from PuppetLabs but still some operating system distributions does not include those packages in their repositories. So we need to some manual things to install puppet 3.

In this post I will explain few best practices to follow when installing a puppet master - agent environment. I have configure puppet master and agent environments several times and came across with this sequence and I think this a good way of doing this. But please note this not "the" best way of doing it and not recommended to use it as it is in a production environment. And also this will not describe about best practises of writing puppet manifests/modules.

Set a domain name for the environment
First of all use a domain name for your environment. Think that you are going to set up a puppet environment for ABC company, you can set the domain for that as 'abc.com' or 'dc1.abc.com' (data center 1 of ABC company). If you are doing it for testing purposes its advisory to use 'example.com'. 'example.com' is a reserved domain name for documentation and example purposes and no one can register that domain, so it will avoid many DNS resolution issues.

Give a proper FQDN for each hosts hostname.
Set a fully qualified domain name (FQDN) to each and every host within the puppet environment including the puppet master node. It will reduce lots of SSL related issues. It is not enough to just to give a hostname because most systems adds a domain (via DHCP) that will introduce some issues. Run 'hostname' and ' hostname -f ' and see the difference.

Use 'puppet' as a prefix as the puppet masters hostname. So it would be like;

    puppet.abc.com or
    puppet.cd1.adc.com or
    puppet.example.com

And for the puppet agents;

    8976712.apache.abc.com or
    8976712.apache.dc1.abc.com or
    8976712.apache.example.com

Or

    8976712.node001.abc.com or
    8976712.node002.dc1.abc.com or
    8976712.node003.example.com

Use a UUID when creating the hostnames for puppet agents. Then give the service name (apache,mysql) or the node number (node002 - if using multiple services in a single server). That name must match the node definitions in the 'site.pp' (or 'nodes.pp').

Use the 'hostname' command and edit the '/etc/hostname' configurations file to change the hostname. You can do it like this, assuming that the host is '8976712.node001.abc.com'

# hostname 8976712.node001.abc.com
# echo '8976712.node001.abc.com' >/etc/hostname

Give and IP address to each FQDN.
It is a must to give an appropriate IP addresses to each hostname/FQDN. At least, the system should be able to refer to the '/etc/hosts' file and resolve the IP address of the relevant FQDN and should have following entries in the '/etc/hosts' file.

    127.0.0.1 localhost
    127.0.0.1 < local fqdn >
    < puppet master ip > < puppet master fqdn >

For an example, if you take '8976712.node001.abc.com' node, its '/etc/hosts' file should like this.

    127.0.0.1 localhost
    127.0.0.1 8976712.node001.abc.com
    192.168.1.100 puppet.abc.com

Check the system time and timezone information
Both puppet master and agents should have same system time and time zone on both systems. Use 'date' command to check the system time and time zone. Synchronize the system time with a well known time server. Commands are bit different from one distribution to another.

Download and install puppet repositories from PuppetLabs website
PuppetLabs provide an apt and a yum repository. Most distributions does not support puppet 3 for the moment therefore, we need to add those manually.

Please refer to "Using the Puppet Labs Package Repositories" article and install the appropriate repository for your system. Then update your repository lists.

Install puppet master 
After completing all above steps, then try to install puppet master using a package management system (apt/yum).

It's better to go ahead with default setting. But you need to do few changes to some configuration files to make it work as a master-agent environment puppet master server. Use a 'autosign.conf' file to automatically sign agents SSL requests. But avoid using ' * ' in that. Better to use it like this;

*.abc.com

It's better to add the 'server=puppet.< domain >' in the 'puppet.conf  's 'main' section. On Debian based distros change the 'start' option in to 'yes' to start the puppet master. After configuring all restart the puppet master service. Open port 8140 from the system firewall specially check that if you are using any RedHat distribution.

Track changes
Use a version controlling system like git or subversion to track changes to puppet manifests. Use branching, versioning/tagging features to do it effectively.

Install puppet agent
First of all it is better to have puppet master installed. Then check the hostname and DNS resolutions for the hostname and puppet master. Then try to install puppet agent using a package management system.

You have to do few changes to connect to the puppet master server. Edit the '/etc/puppet/puppet.conf ' and add 'server=puppet.< domain >' to the 'main' section. Change the 'start' option to 'yes' in '/etc/default/puppet' configuration file in debian based distros. Then restart the puppet agent.

Test the system
Add this into your puppet masters '/etc/puppet/manifests/site.pp' file.
node default {
    file { '/tmp/mytestfile.t':
        owner   => 'root',
        group   => 'root',
        content => "This file was created by puppet.\n",
        ensure  => present,
    }
Then run 'puppet agent -vt ' on the agent and check the '/tmp ' directory.

Automated script
I wrote a script to automate this and you can get it from here on github. It support Debian, RedHat and SLES distributions. If you have any issues please report those to this.