Kick start on Openstack with Devstack {Folsom}

Recently I was told to install openstack controller-compute node set ... and my first impression on it was piece of cake!! because you get couple of scripts and all you need to just run the script.. !! I WAS SOO WRONG!! yes it is just couple of script but you need to know how things work and there are soo many places where it can totally messed up your machine and your set up. Meaning you can easily get messed up!!! If you don't go in the right path it will take ages .. and you might even have to format your machine several times to get things clean and start all over again which is very time consuming.. So I ended up installing openstack for almost a week spending day and night on my precious development time!!!

So I decided to write a post on focusing on HOW TO INSTALL OPENSTACK? So other people want do the same mistake I did or take ages to get things done. If you know what you are doing then installing devstack only take one hour's time all you need to do is make your configuration right and run the scrip!!!

So before we begin I would like to give a small introduction to OpenStack to make things more clearer.

What is OpenStack ?

In simple terms OpenStack is an opensource cloud operating system ... It is like the fabric of the cloud. OpenStack was initially founded by rackspace and NASA in 2010 and its being growing ever since. The best way to get what OpenStack is by reading their mission which is very simple and direct

"To produce the ubiquitous Open Source Cloud Computing platform that will meet the needs of public and private clouds regardless of size, by being simple to implement and massively scalable."

OpenStack is written in purely written in python, true it use other systems like MYSQL and rabbitMQ but the core is written in python which makes things very efficient and fast. And also initially it had two main component 1) Nova - which is the compute side of it and the other one 2) swift - the data storage side of it.

Current OpenStack stable version {Folsom} have few core components

• Openstack object storage (swift)
• OpenStack image registry and delivery (Glance)
• Openstack compute (nova)
• Horizon (web UI)
• Keystone (authentication)
• Quantum/Melange (Network)

In future OpenStack is going to provide other compoents such Cellometer (Billing service), Cinder (block storage) and load balancer services. So that's enough about OpenStack lets see how we can install it. I will be installing OpenStack using devstack..

What is devstack ?

Well devstack is openstack development project (there are quite a few :) ) but I think devstack is the easiest :).  Devstack provides a documented shell script (which is written in bash) to build a complete OpenStack development environments. Originally created by Rackspace now maintained by OpenStack community. This supports both ubuntu and fedora (but today we will be focusing how we can install openstack in ubuntu 12.04).

Installing OpenStack  

In this installation I will be explaining how we can install OpenStack in Multi-Node environment.

Prerequisites 

Install ubuntu 12.04 and

Install git
     apt-get install -y git sudo

Network setting on each node

    You need to have a stack ip and also decide on what your Floating IP range (public IP range ) and Fixed IP range (private IP range) are.

This is my network configurations

  Static IP (controller ) - 10.100.0.30

  Floating (Public) IP range - 10.100.0.128/25

  Fixed (Private) IP range  - 192.168.16.0/25 

Go to /etc/network/interfaces and configure your network accordingly.

Controller node 

auto eth0
iface eth0 inet static
address 10.100.0.30
netmask 255.255.252.0
gateway 10.100.1.254

Compute Node

auto eth0
iface eth0 inet static
address 10.100.0.29
netmask 255.255.252.0
gateway 10.100.1.254



Before we begin we need to create  separate user to do things in a cleaner way.


groupadd stack
useradd -g stack -s /bin/bash -d /opt/stack -m stack
echo "stack ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers

From here onward we will be doing all the operation from the stack user.

Once you create the user log out and login from the the stack user and download openstack using git. When you are downloading make sure you download the stable version (in my case its folsom) so you will not have any surprises. :)


sudo su stack # login from the user
cd # go to user home
git clone git://github.com/openstack-dev/devstack.git -b stable/folsom #download devstack stable version

Once you download devstack other things are pretty straight foward. Go to devstack folder and create localrc  file and add the configuration accordingly in each node.

Configuring and Installing OpenStack in Controller Node

vi localrc

HOST_IP=10.100.0.30
FLAT_INTERFACE=eth0
FIXED_RANGE=192.168.16.0/25 
FIXED_NETWORK_SIZE=126
FLOATING_RANGE=10.100.0.128/25
MULTI_HOST=1
LOGFILE=/opt/stack/logs/stack.sh.log
ADMIN_PASSWORD=openstack
MYSQL_PASSWORD=root
RABBIT_PASSWORD=rabbitmq
SERVICE_PASSWORD=password
SERVICE_TOKEN=poohdedoo

Once you create the file you can run the setup script using the following command.

./stack.sh

After a successful  installation you will get the following output


Horizon is now available at http://10.100.0.30/
Keystone is serving at http://10.100.0.30:5000/v2.0/
Examples on using novaclient command line is in exercise.sh
The default users are: admin and demo
The password: openstack
This is your host ip: 10.100.0.30
stack.sh completed in 3443 seconds.

Configuring and Installing OpenStack in Compute Node

Just like the controller node you need to have the basic infrastructure in order to install the computer. Only change is there is the content of the localrc file in the compute node.


vi localrc

SERVICE_HOST=10.100.0.30
HOST_IP=10.100.0.29
FLAT_INTERFACE=eth0
FIXED_RANGE=192.168.16.0/25 
FIXED_NETWORK_SIZE=126
FLOATING_RANGE=10.100.0.128/25
MULTI_HOST=1
LOGFILE=/opt/stack/logs/stack.sh.log
ADMIN_PASSWORD=openstack
MYSQL_PASSWORD=root
RABBIT_PASSWORD=rabbitmq
SERVICE_PASSWORD=password
SERVICE_TOKEN=poohdedoo
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292
Q_HOST=$SERVICE_HOST
ENABLED_SERVICES=n-cpu,n-net,n-api,c-sch,c-api,c-vol,rabbit


That's it !!. Just like the controller node you need to run the setup script to execute the installation.

./stack.sh

Once you successfully install openstack in both nodes you can login to openstack using the horizon dashboard

 http://10.100.0.30/ and login according to the user credentials you have provided in your setup configuration.

You can run the exercise.sh run provided by devstack in order to test your set up which includes uploading/launching instances and creating user groups/keypairs and user rules. 

If you want to upload images manually using the script you can use the following commands

glance image-create --name mysql-cartridge-amd64  --is-public true --container-format ami --disk-format ami < /home/wso2/wso2s2-openstack-1.0.0/cartridges/mysql-cartridge-amd64.img

To see what you have uploaded 

source openrc demo demo

glance image-list

You can also see these images using the horizon dashboard and further do task like launching the instances, assigning public IPs create key pairs and assign rules to security groups ect. I am not going to details since its a very straight forward dashboard and you can easily do everything by following the dashboard.

But I would like to proudly attach my openstack dashboad :) for references sake :)

On a final note restarting openstack is pretty tricky however, you can use the devstack script to restart openstack.

Restarting OpenStack using Devstack

Run the following in order to own the shell


script /dev/null 

And execute the following script
./rejoin-stack.sh

inside devstack folder in order to restart openStack.

To get out of the screen use ctrl-a , ctrl-d

Thats it folks !!
Once you set it up its like having a data center in your basement you can test load balancers and you can do all those caching stuff and experiment new stuff its really a cool thing to have. All the things you do which was in a private cloud and you can point it to a public cloud very easily!!!

Carbon Logging framework - Conversion Characters for TenantAwarePatternLayout

Usually when users use a logging framework one of their primary requirement is to customize the log output format. In Log4j framework this is done using the layout components.

There are couple of predefined layouts comming from the log4j api however, if you want more customize layout with extra set of attributes you need to write your own custom layout extending the org.apache.log4j.PatternLayout class.

In WSO2 Products we do not use the default PatternLyout we have written our own layout to make sure our log format is service aware and multi-tenanted. Therefore we have our own set of conversion characters defined in our TenantAwarePatternLayout class.

Below table explains characters used in the TenantAwarePatternLayout and all other characters (which are comming from PatternLayout) which you can use in your custom pattern.

Conversion Character	Description
Coming from custom carbon layout
D	used to output the name of the tenant domain which is currently logged in to the server (super tenant domain is carbon.super)
T	used to output the tenant id - to uniquely identify each tenant tenant id can be used in the layout (super tenant id is 0)
S	used to output the server key - to uniquely identify each carbon server a server key can be used (ie as,esb,dss )
U	used to output the user name - current user who is logged into the server
A	used to output the application name - service name or web application name of the current running applicaiton
H	used to output the host name
I	used to output the carbon instance id- uuid which is generated for each carbon instance to uniquely identify each instance
Coming from the log4j patternlayout
c	used to output category(logger); a.b.c → %c{2} = a.b (this can be also used as fully qualified name)
d	used to output the date of the logging event. for example, %d{hh:mm:ss,sss} or %d{dd mmm yyyy hh:mm:ss,sss}.
F	used to output the file name where the logging request was issued.
l	used to output location information of the caller which generated the logging event.
L	used to output the line number from where the logging request was issued.
m	used to output the application supplied message associated with the logging event.
M	used to output the method name where the logging request was issued.
n	used to output the platform dependent line separator character or characters.
p	used to output the priority of the logging event.
r	used to output the number of milliseconds elapsed from the construction of the layout until the creation of the logging event.
t	used to output the name of the thread that generated the logging event.

Please note %C, %F, %l, %L, %M slow down program run!

To create your own ConversionPattern you can go to the log4j.properties file and edit ConversionPattern of the appender of your choice.

log4j.appender.CARBON_LOGFILE.layout.ConversionPattern=TID: [%T] [%S] [%d] %P%5p {%c} - %x %m {%c}%n

Relational Storage Solution using WSO2 Storage as a Service

 Lets say you want to create a relational database on the cloud and you want to expose these data to other services?
Its very easy simple and very straight forward. All you need to have is a stratoslive account and then login to WSO2 Storage Server.

WSO2 Stroage Server provides almost all the data administrator level functionality when it comes to creating and manipulating data storage. Basically it provides features such as creating and manipulating data storage for relational data bases, Cassandra data storage and hadoop/hdfs file system.

Wondering How to create databases in the cloud ??

It only takes 4 clicks ... to Create and secure your database on the Cloud

In this article I am going to demonstrate on how you can create a database, and assign users and special privileges to that data base within few clicks

Step 1 - Creating Users

Assuming you already have a stratoslive account and logged in to storage as a server, lets first try to create a user for the database we are going to create.

Under Manage -> RSS Manager (relational storage service manager), you have the three basic functions to manipulate relational data bases.

To Create users click on Database Users. This will show the list of users (if you have not created users before you will not see any)  along with  "Add Database User " 

Select Database Server Instance Name as WSO2_RSS (which is the default server), and give the user credentials as your choice and click save to save the user.

Step 2 - Creating Privilege Group

Once you create the user you can create a template which separate out set of functions which can be performed on a given database,  so you can assign roles.

Go to RSS Manager ->  Privilege Templates -> Add Database Privilege Template. 

Lets say you only want this template to have   INSERT UPDATE DELETE CREATE DROP. Select the above fields along with a suitable name and save.

Step 3 - Creating Database

Now lets create a database in the cloud and assign the created users and privilege template to that database.

Go to RSS Manager -> Databases  -> Add Database 

Give the RSS instance name,  database name and save.

Once you create the database it will go under Databases list, where you have the option to manage databases by adding users and privileges to that database.

Go to RSS Manager -> Databases  -> database name -> Manage -> Attach Database User

Assign the user and role we created in step1 and step2 to the database as shown below.

Once you attach this user and role you can use this database with the user and the jdbc url shown in the databases page.

That's It .. now you have created your very own database in the cloud!!!

If you want view this database using a mysql client.. you can use WSO2 Data Services Server DB explorer.

Go to  WSO2 Data Services Server -> Tools -> Database Explorer.

Give the database information accordingly along with our newly created database and user.  And log in to your database.

And you can create your tables and manage data accordingly

If you want to expose your data as services you can use the following guide which explains how to expose your data as a service within few clicks.