Openstack Heat try-out

Leave a comment

August 8, 2013 by kimizhang

What’s Heat ?

Heat is a service to orchestrate multiple composite cloud applications using the AWS CloudFormation template format, it’s Openstack’s implementation of CloudFormation and CloudWatch.

Heat integrates other core components of OpenStack into a one-file template system. The templates allow creation of most OpenStack resource types, the main functions of Heat cover:

-vAPP stack creation/update/deletion (suspend/resume will be added in Havana)
-High Availability
-Manual scale in/out(in Havana)

How to install Heat and make it ready to use?


->Heat has been included in RDO repo, installation can be done by Yum:

yum install openstack-heat-api openstack-heat-api-cfn openstack-heat-api-cloudwatch openstack-heat-cli openstack-heat-common openstack-heat-engine python-heatclient python-heatclient-doc heat-cfntools heat-jeos

->Comment out following line since we use RabbitMQ instead of QPID as message queue. Do this for heat-api-cfn.conf, heat-api-cloudwatch.conf, heat-api.conf, heat-engine.conf under /etc/heat.


->Configure [filter:authtoken] section of heat-api-cfn-paste.ini  heat-api-cloudwatch-paste.ini  heat-api-paste.ini under /etc/heat

paste.filter_factory = heat.common.auth_token:filter_factory
service_protocol = http
service_host = api-vip
service_port = 5000
auth_host = api-vip
auth_port = 35357
auth_protocol = http
auth_uri = http://api-vip:35357/v2.0
admin_tenant_name = service
admin_user = heat
admin_password = password

->Configure [filter:ec2authtoken] section of  heat-api-cfn-paste.ini  heat-api-cloudwatch-paste.ini under /etc/heat

paste.filter_factory =
auth_uri = http://api-vip:5000/v2.0
keystone_ec2_uri = http://api-vip:5000/v2.0/ec2tokens

>Update /etc/heat/heat-engine.conf as below:

heat_metadata_server_url =
heat_waitcondition_server_url =
heat_watch_server_url =
sql_connection = mysql://heat:heat@db-vip/heat

#The IP for metadata, watchcondition and watch_server should be the host IP of heat services are running on, and should be reachable from VM instance.

->Setup database for heat:

heat-db-setup rpm -y -r <root password of mysql>

->Setup keystone entries for heat:


->Start heat services:

service openstack-heat-api start
service openstack-heat-api-cfn start
service openstack-heat-api-cloudwatch start
service openstack-heat-engine start

2.Prepare JEOS(Just enough OS)

JEOS image is basic OS image with cloud-init  and heat-cfn-tools pre-installed, they are the keys for Heat and VM exchanging information and doing post and regular jobs.

Pre-built JEOS of Fedora 17 and 18 can be downloaded from:

To make our own JEOS on RHEL, following the steps:

a.Assume we have a RHEL 6.3 image registered in Glance already, launch a VM instance from it.
b.Log into the VM, install cloud-init and heat-cfn-tools

yum -y update
export http_proxy=<http proxy server IP>   #This is needed if lab has no direct internet access
curl -O
rpm -Uvh epel-release-6-7.noarch.rpm
yum -y install perl python python-setuptools cloud-init python-pip
pip-python install argparse ‘boto==2.5.2’ heat-cfntools
cfn-create-aws-symlinks –source /usr/bin
rm -rf /etc/udev/rules.d/70-persistent-net.rules

c.From Horizon GUI or CLI, make a snapshot for this VM, name it as “rhel63-x86_64-cfntools”.  This snapshot actually becomes a valid JEOS image.

Use cases

1.Deploy a single WordPress blog server with a floating IP associated.

This templates deploy a single VM instance with a associated floating IP, then install mysql-server, httpd and wordpress packages, initial database, integrate each other and finally start up wordpress web service.
->Download the template file:
->Source a tenant credential
source /root/nceprc
>Create a key-pair
nova  keypair-add ncepkey > ncepkey.pem
chmod 400 ncepkey.pem
->Deploy this template by heat
heat stack-create wordpress  -f WordPress_Single_Instance_With_EIP.template -P “InstanceType=m1.medium;DBUsername=wordpress;DBPassword=worldpress;KeyName=ncepkey;LinuxDistribution=RHEL-6.3”
#Parameters followed after “-P” are needed for instance type selection, DB access setup, key-pair selection and image selection.
->Check stack creation process
Once the stack-create is issued, we should see the stack in “CREATE_IN_PROGRESS” status from output of “heat list”
Once creation is complete, we can see the status will change to “CREATE_COMPLETE”.

[root@controller-1 heat(keystone_ncep)]# heat list
| ID | Name | Status | Created |
| a6740599-2c98-41b8-a55e-490f28f6f6f8 | wordpress | CREATE_COMPLETE | 2013-08-05T06:57:40Z |

We can also see detailed resource status in the stack by:

[root@controller-1 heat(keystone_ncep)]# heat resource-list wordpress
| Name | Type | Status | Updated |
| IPAddress | AWS::EC2::EIP | CREATE_COMPLETE | 2013-08-05T06:57:40Z |
| WebServerSecurityGroup | AWS::EC2::SecurityGroup | CREATE_COMPLETE | 2013-08-05T06:57:41Z |
| IPAssoc | AWS::EC2::EIPAssociation | CREATE_COMPLETE | 2013-08-05T06:57:52Z |
| WebServer | AWS::EC2::Instance | CREATE_COMPLETE | 2013-08-05T06:57:52Z |

Also heat stack-show wordpress can be used to show detailed information of the stack

->Try to access the wordpress web UI

From nova list, we could see a VM named “wordpress.WebServer” is created with internal IP and floating IP:

[root@controller-1 heat(keystone_ncep)]# nova list
| ID | Name | Status | Networks |
| 02da7185-a0a6-44cf-8076-875754163dac | wordpress.WebServer | ACTIVE | ncep-net=, |

Try to access the wordpress website by its floating IP

You should see the login page in the browser!!!

2.Deploy wordpress blog server with 2 VM instances(front-end + back-end topology)

This template deploy a 2 VM wordpress solution, one VM running web server, 2nd VM running Mysql DB.

->Download the template:


->Deploy the template by heat

heat stack-create wp-2-vm  -f WordPress_2_Instances.template -P “InstanceType=m1.medium;DBUsername=wordpress;DBPassword=worldpress;KeyName=ncepkey;LinuxDistribution=RHEL-6.3”

>From heat resource-list wp-2-vm, we can see 2 VM instances are created, also a floating IP and association with Webserver is there

[root@controller-1 heat(keystone_ncep)]# heat resource-list wp-2-vm
| Name | Type | Status | Updated |
| DatabaseServer | AWS::EC2::Instance | CREATE_COMPLETE | 2013-08-05T07:14:04Z |
| IPAddress | AWS::EC2::EIP | CREATE_COMPLETE | 2013-08-05T07:14:13Z |
| WebServer | AWS::EC2::Instance | CREATE_COMPLETE | 2013-08-05T07:14:13Z |
| IPAssoc | AWS::EC2::EIPAssociation | CREATE_COMPLETE | 2013-08-05T07:14:14Z |

->Try wordpress website with floating IP of WebServer VM instance

[root@controller-1 heat(keystone_ncep)]# nova list
| ID | Name | Status | Networks |
| 4879bb03-bc20-4c33-91e2-d4a7e197d2e5 | wp-2-vm.DatabaseServer | ACTIVE | ncep-net= |
| 8e1dc2c7-bc34-4e5d-b833-b64dc8a12c9b | wp-2-vm.WebServer | ACTIVE | ncep-net=, |

3.Deploy a wordpress blog server with EBS volume as mysql DB storage

This template deploy a VM instance with httpd, wordpress and mysql-server installed, a EBS volume is attached to instance as Mysql DB storage point.
->Download the template
heat stack-create wp-ebs -f WordPress_Single_Instance_With_EBS_EIP.template -P “VolumeSize=5;InstanceType=m1.medium;DBUsername=wordpress;DBPassword=worldpress;KeyName=ncepkey;LinuxDistribution=RHEL-6.3”
#Here we need use parameter “VolumeSize” to specify the volume size we intend to use as DB storage.
->From resource list , we can see it has volume and volume attachment there
[root@controller-1 heat(keystone_ncep)]# heat resource-list wp-ebs
| Name | Type | Status | Updated |
| DataVolume | AWS::EC2::Volume | CREATE_COMPLETE | 2013-08-05T07:35:48Z |
| WikiServerSecurityGroup | AWS::EC2::SecurityGroup | CREATE_COMPLETE | 2013-08-05T07:35:48Z |
| IPAddress | AWS::EC2::EIP | CREATE_COMPLETE | 2013-08-05T07:35:59Z |
| WikiServer | AWS::EC2::Instance | CREATE_COMPLETE | 2013-08-05T07:35:59Z |
| IPAssoc | AWS::EC2::EIPAssociation | CREATE_COMPLETE | 2013-08-05T07:36:00Z |
| MountPoint | AWS::EC2::VolumeAttachment | CREATE_COMPLETE | 2013-08-05T07:36:02Z |
->Check volume and attachment

[root@controller-1 heat(keystone_ncep)]# cinder list
| ID | Status | Display Name | Size | Volume Type | Bootable | Attached to |
| 16c9c84f-9bf3-43c7-853d-453ec161b170 | in-use | wp-ebs.DataVolume | 5 | None | false | 0738154d-8d58-4a84-a5fc-71774228105d |

Inside vm, we can check mounts:

[root@wp-ebs ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/vda1 5.0G 1.6G 3.1G 34% /
tmpfs 1.9G 0 1.9G 0% /dev/shm
/dev/vdb1 938M 39M 853M 5% /var/lib/mysql

->Try to access wordpress by floating IP.

4.Deploy a wordpress blog server with HA protection enabled

This template deploy a VM instance with wordpress server installed, and HA is enabled to monitor httpd/mysqld services to make sure they are already running.
->Download template:
->Assign admin role to the tenant user by ” keystone user-role-add” command.
To use HA, waitcondition resource is needed, which needs to create an ec2signed url and to do that it creates a restricted user in keystone. To create any user in keystone we need admin rights:(
This issue has been addressed by a bug ticket:
 keystone user-role-add –user <ncep user id> –role <admin role id>  –tenant-id <ncep tenant id>
heat stack-create wp-ha -f WordPress_Single_Instance_With_HA.template -P “HupPollInterval=1;InstanceType=m1.medium;DBUsername=wordpress;DBPassword=worldpress;KeyName=ncepkey;LinuxDistribution=RHEL-6.3”
#Parameter “HupPollInterval” is needed to specify the cfn-hup process running interval in minutes, cfn-hup is to check services status, and restart them if return code of “service <service name> status” is not “0”.
->Check resource

[root@controller-1 heat(keystone_ncep)]# heat resource-list wp-ha
| Name | Type | Status | Updated |
| CfnUser | AWS::IAM::User | CREATE_COMPLETE | 2013-08-05T07:53:30Z |
| WaitHandle | AWS::CloudFormation::WaitConditionHandle | CREATE_COMPLETE | 2013-08-05T07:53:30Z |
| WebServerKeys | AWS::IAM::AccessKey | CREATE_COMPLETE | 2013-08-05T07:53:30Z |
| WaitCondition | AWS::CloudFormation::WaitCondition | IN_PROGRESS | 2013-08-05T07:53:41Z |
| WikiDatabase | AWS::EC2::Instance | CREATE_COMPLETE | 2013-08-05T07:53:41Z |

Here “WaitCondition” actually is waiting VM instance is properly started and configured, then goes to next step

Whole resource list after create_complete:

[root@controller-1 heat(keystone_ncep)]# heat resource-list wp-ha
| Name | Type | Status | Updated |
| CfnUser | AWS::IAM::User | CREATE_COMPLETE | 2013-08-05T07:53:30Z |
| WebServerKeys | AWS::IAM::AccessKey | CREATE_COMPLETE | 2013-08-05T07:53:30Z |
| WaitHandle | AWS::CloudFormation::WaitConditionHandle | CREATE_COMPLETE | 2013-08-05T07:54:41Z |
| WikiDatabase | AWS::EC2::Instance | CREATE_COMPLETE | 2013-08-05T07:54:41Z |
| HttpFailureAlarm | AWS::CloudWatch::Alarm | CREATE_COMPLETE | 2013-08-05T07:54:47Z |
| WaitCondition | AWS::CloudFormation::WaitCondition | CREATE_COMPLETE | 2013-08-05T07:54:47Z |
| WebServerRestartPolicy | OS::Heat::HARestarter | CREATE_COMPLETE | 2013-08-05T07:54:47Z |

Resource “OS::Heat::HARestart” takes care of service HA.

->Try to access wordpress web by floating IP

Login page should show up.

->Try HA by stopping httpd service

In the VM, do service httpd stop.

Check cfn-hup.log

[root@wp-ha ~]# tail -f /var/log/cfn-hup.log

DEBUG [2013-08-05 16:28:02,552] Running command: /sbin/service httpd status
DEBUG [2013-08-05 16:28:02,609] Return code of 3 after executing: ‘[‘su’, ‘root’, ‘-c’, u’/sbin/service httpd status’]’
WARNING [2013-08-05 16:28:02,610] Restarting service httpd
DEBUG [2013-08-05 16:28:02,610] Running command: /sbin/service httpd start
DEBUG [2013-08-05 16:28:02,732] Running command: /etc/cfn/notify-on-httpd-restarted
DEBUG [2013-08-05 16:28:03,634] Running command: /sbin/service mysqld status
DEBUG [2013-08-05 16:28:03,714] Running command: /sbin/service crond status

We can see once cfn-hup found httpd service has problem, it tried to start httpd service.

->Check wordpress web access again, it should work again.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: