How to inject file/meta/ssh key/root password/userdata/config drive to a VM during nova boot

6

March 18, 2014 by kimizhang

There are possibilities to inject something into the VM during nova boot, let’s go through them one by one:

The test above is done on Openstack Havana 2012.2.1 @CentOS 6.5 x86_64.

1. Pre-condition

To achieve injection,  libguestfs is needed on nova compute hosts.

yum install libguestfs python-libguestfs libguestfs-tools-c

Configure nova.conf

libvirt_inject_password=true
libvirt_inject_key=true
libvirt_inject_partition=-1

Restart nova-compute

service openstack-nova-compute restart

2. File injection

Let’s try to inject /root/keystonerc into VM as /fileinject

#cat /root/keystonerc
export OS_TENANT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=admin_pass
export OS_AUTH_URL=”http://169.254.0.10:5000/v2.0/”

nova boot  –flavor 1 –image cirros  –nic net-id=d58bbcac-1908-4cda-a9da-a13cfbbf4e77  –file /fileinject=/root/keystonerc vm-file-inject

#Loging to VM to check

#ip netns exec qrouter-d667f653-c087-45b8-8c9c-e353f88a2c3d ssh cirros@10.0.0.2
$ cat /fileinject

export OS_TENANT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=admin_pass
export OS_AUTH_URL=”http://169.254.0.10:5000/v2.0/”

3. Meta injection

Let’s try to inject 2 metadata pairs, key1=test, key2=hello

nova boot  –flavor 1 –image cirros  –nic net-id=8b052b4a-840c-4b45-b96e-7980f7fa4a74  –meta key1=test –meta key2=hello    vm-meta-inject

#Loging to VM to check

1. ip netns exec qrouter-d667f653-c087-45b8-8c9c-e353f88a2c3d ssh cirros@10.0.0.4

$ cat /meta.js
{“key2”: “hello”, “key1”: “test”}

We can see the metadata key-value pairs are stored in /meta.js in the VM.

4. Root password injection

You can do this either by libvirt password injection or by cloud-init + Nova metadata service

4.1 Libvirt password injection.

This way needs libvirt_inject_password=true is configured in nova.conf.

By default, a random password for root is created and injected to VM, from nova boot output, we can see this root password:

# nova boot –flavor 1 –image cirros –nic net-id=8b052b4a-840c-4b45-b96e-7980f7fa4a74 –meta key1=test –meta key2=hello vm-meta-inject
+————————————–+—————————————+
| Property | Value |
+————————————–+—————————————+
| OS-EXT-STS:task_state | scheduling |
| image | cirros |
| OS-EXT-STS:vm_state | building |
| OS-EXT-SRV-ATTR:instance_name | instance-0000001d |
| OS-SRV-USG:launched_at | None |
| flavor | m1.tiny |
| id | e82bf7a2-176e-4f9f-83d5-c3542a7ed48e |
| security_groups | [{u’name’: u’default’}] |
| user_id | 25b9f5570f034feebf16e5f85e0fcc6b |
| OS-DCF:diskConfig | MANUAL |
| accessIPv4 | |
| accessIPv6 | |
| progress | 0 |
| OS-EXT-STS:power_state | 0 |
| OS-EXT-AZ:availability_zone | nova |
| config_drive | |
| status | BUILD |
| updated | 2014-03-14T07:00:48Z |
| hostId | |
| OS-EXT-SRV-ATTR:host | None |
| OS-SRV-USG:terminated_at | None |
| key_name | None |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None |
| name | vm-meta-inject |
| adminPass | aPduEQ56Yu3t |
| tenant_id | 1e888eccf99845f8bcf9a9730c83a669 |
| created | 2014-03-14T07:00:48Z |
| os-extended-volumes:volumes_attached | [] |
| metadata | {u’key2′: u’hello’, u’key1′: u’test’} |
+————————————–+—————————————+

The random generated password can be retrieved from config drive as well, this will be mentioned later in this post.

We can also freely set root password on Horizon GUI when launching the VM.

We need enable setting admin password in horizon local_settings.py

OPENSTACK_HYPERVISOR_FEATURES = {
…
‘can_set_password’: True,
}

Note: It seems Nova CLI does not support setting root password during nova boot, let’s if Heat support or not later.

4.2 Cloud-init root password injection

If you set libvirt_inject_password=false, root password can be stilled injected by cloud-init + Nova metadata service. This needs the image has cloud-init pre-installed.

Still if you don’t specify the root password, a random password is generated, and showed from the nova boot output.

5. SSH key injection

Two ways to inject SSH key, by hypervisor or  metadata service + cloud-init  service.

NOTE: Only one way from two should be used, they should not be used at the same time

5.1 SSH key injection by hypervisor

This way works if you don’t have cloud-init in the VM.

Create a keypair:

# nova keypair-add sshkey

Launch a VM with the keypair created above:

 nova boot –flavor 2 –image centos2 –nic net-id=8b052b4a-840c-4b45-b96e-7980f7fa4a74 –key-name sshkey  centos-vm

Now try to login the VM with SSH key:

[root@lcc-controller-1 ~]# ip netns exec qrouter-d667f653-c087-45b8-8c9c-e353f88a2c3d ssh -i sshkey.pem 10.0.0.9
Last login: Fri Mar 14 15:36:25 2014 from 10.0.0.1
[root@centos65cloudimage ~]#

5.2 SSH key injection by metadata service + cloudinit

To make this work, you need disable SSH key injection from nova.conf, and have cloud-init installed in VM.

Launch a VM with the keypair:

nova boot –flavor 2 –image centos2 –nic net-id=8b052b4a-840c-4b45-b96e-7980f7fa4a74  –key-name sshkey  centos-metadata

Try to login to the VM with SSH key, since cloud-init only insert ssh key to user “cloud-user”, we need to use this user to login:

[root@lcc-controller-1 ~]# ip netns exec qrouter-d667f653-c087-45b8-8c9c-e353f88a2c3d ssh -i sshkey.pem cloud-user@10.0.0.10
[cloud-user@centos-metadata ~]$

Actually the SSH key is retrieved from Nova metadata service:

[cloud-user@centos-metadata ~]$ curl http://169.254.169.254/2009-04-04/meta-data/public-keys/0/openssh-key
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAvc9IUAlGukTLzaXgUMOZElGR8nMZlQ1TFppNrStQSMHd2rpqq/h2ZYHv/3Cz09zFTadO0QTWlqA9ZPnGgSWytjwvdfIQJb47jh/RZzyo1mJZYCkneE5wpviZ7Txe2BTFFNX8qUQksvg8plR4tGZFmWXc1SceMpwBoOdRFEcdjIXHOPPfU4J4NvpOJPrN9xEDMQmLsU5Sun7t1Gkg8UTnVZTweg79QXQfmXewTMx2LPwxXtsthS1hDoXYLulyxFpyVhZPOMZOipYcmr5wumpQhdBAPHRx0eocT5+6bkPtg2pBd1aKyl4oYUpuIW4b6md7TUINHE3qlqD3JNi49C1Oqw== Generated by Nova

Compare with ~/.ssh/authorized_keys, they are the same:

[cloud-user@centos-metadata ~]$ cat .ssh/authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAvc9IUAlGukTLzaXgUMOZElGR8nMZlQ1TFppNrStQSMHd2rpqq/h2ZYHv/3Cz09zFTadO0QTWlqA9ZPnGgSWytjwvdfIQJb47jh/RZzyo1mJZYCkneE5wpviZ7Txe2BTFFNX8qUQksvg8plR4tGZFmWXc1SceMpwBoOdRFEcdjIXHOPPfU4J4NvpOJPrN9xEDMQmLsU5Sun7t1Gkg8UTnVZTweg79QXQfmXewTMx2LPwxXtsthS1hDoXYLulyxFpyVhZPOMZOipYcmr5wumpQhdBAPHRx0eocT5+6bkPtg2pBd1aKyl4oYUpuIW4b6md7TUINHE3qlqD3JNi49C1Oqw== Generated by Nova

6. Userdata injection

Userdata injection in also done by Nova metadata service

Launch a VM with a userdata file:

nova boot –flavor 2 –image centos2 –nic net-id=8b052b4a-840c-4b45-b96e-7980f7fa4a74  –user-data /root/keystonerc  centos-userdata

Inside VM, retrieve the userdata by Nova metadata service:

[cloud-user@centos-user-data ~]$ curl http://169.254.169.254/2009-04-04/user-data
export OS_TENANT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=admin_pass
export OS_AUTH_URL=”http://169.254.0.10:5000/v2.0/”

With help of cloud-init or Heat, the userdata can be used in many ways to achieve some post-config steps when a VM is launched.

7. Config drive as alternative of metadata service

Config drive is an alternative of Nova metadata service, the metadata contents are written on a special drive attached to VM, inside VM, you can mount and read metadata from it.

Launch a VM with config drive:

nova boot –flavor 2 –image centos2 –nic net-id=8b052b4a-840c-4b45-b96e-7980f7fa4a74  –user-data /root/keystonerc –meta key1=value1 –meta key2=value2 –file /root/fileinject-ks-pre.log=/root/ks-pre.log  –key-name sshkey –config-drive true  centos-configdrive

Login to VM, mount the config drive

#Check the config drive ID:

[root@centos-configdrive ~]# blkid
/dev/sr0: LABEL=”config-2″ TYPE=”iso9660″
/dev/vda1: UUID=”59b7c317-c842-4e4b-88fc-97ca78f733cf” TYPE=”ext4″
/dev/vda2: UUID=”gEZmmF-veNb-Tzzw-6cjq-XxZa-S4Dk-BmWw56″ TYPE=”LVM2_member”
/dev/mapper/vg_osclone-lv_root: UUID=”40d88bae-21b2-48da-8b8d-00cbe8f4944f” TYPE=”ext4″
/dev/mapper/vg_osclone-lv_swap: UUID=”8760f3c0-1630-4770-bfe0-027ac3189d66″ TYPE=”swap”

#Config drive is labeled as “config-2”, mount it to a directory

[root@centos-configdrive ~]# mount /dev/sr0 /mnt/

From the config drive, we can find same information as metadata service, plus the injected file.

#Check whole “meta” data set of the VM, including root password, SSH key, meta key/value pairs.

[root@centos-configdrive mnt]# cat /mnt/openstack/latest/meta_data.json
{“files”: [{“path”: “/root/fileinject-ks-pre.log”, “content_path”: “/content/0000”}],
“admin_pass”: “sx2dGQot5z7J”,
“random_seed”:”iXm+6uR8kVZ4nn01+bYPtQ2+QFUU541nye4Ni4XxSMEQOtUQJzjqT2Ylneu5hRY2Wd30d8ub34BFUKOOZKzIyfdylYzKLbue3DBzDbG1henyYcbAnjfLDF9W5IK8ediMsjWU9QyiRUlAUFEq9vkBJVFd3153iurcdk/ZUX3SeFoICU2ZgmPtOTr8H7OM00nRXE2c+zST2GpsEMQr5KC91pCmshE9LXyya1nSsgpk5ZR+idZlysudQ1ofzaeqoEiBSQm7vBYjZX1hUIP9nm1TZ8MWrIQptUFuz6TJ54Qzo43H+pNWGPvVPDkCQ+L+veCEky6/ADNan28jJndOLmN+8M+NT3m/IOSPxc0chQxYe2dRAXkwHbwvBTf0UalA6mIaklbfRr7Tb8iLV8Xhr6lNSpomJHKh7ti2geWb4BhshwdfzpIG7NfSJrEzlRTxE+dHUsdTZRZI4vsSv7cOLBkdu3ASQPrzY27Xd9DgUqkyFuuTuwJ75kdJBOiiexWsfriOu29ZcWFVJ5st2RJ7Ka3wRy9rFnT+QkY1iBBlpBR7UX/eeT/Jp+/M0euTdYTRaLIL5mSPkkmOmVKDFovhHv6BCzmoBTCM6xb7/FU3H/U8jBvLhqXZGk9IjhjdlG9FOsmEJvU2QCVSriLEvrHXRtV0QACe6XTh+pE5EDWerq6hSQw=”,
“uuid”: “22fd49a3-5087-4dae-9b52-39f737ebf0a1”,
“availability_zone”: “nova”,
“hostname”: “centos-configdrive.novalocal”,
“launch_index”: 0,
“meta”: {“key2”: “value2”, “key1”: “value1”},
“public_keys”: {“sshkey”: “ssh-rsa AAB3NzaC1yc2EAAAABIwAAAQEAvc9IUAlGukTLzaXgUMOZElGR8nMZlQ1TFppNrStQSMHd2rpqq/h2ZYHv/3Cz09zFTadO0QTWlqA9ZPnGgSWytjwvdfIQJb47jh/RZzyo1mJZYCkneE5wpviZ7Txe2BTFFNX8qUQksvg8plR4tGZFmWXc1SceMpwBoOdRFEcdjIXHOPPfU4J4NvpOJPrN9xEDMQmLsU5Sun7t1Gkg8UTnVZTweg79QXQfmXewTMx2LPwxXtsthS1hDoXYLulyxFpyVhZPOMZOipYcmr5wumpQhdBAPHRx0eocT5+6bkPtg2pBd1aKyl4oYUpuIW4b6md7TUINHE3qlqD3JNi49C1Oqw== Generated by Nova\n”},
“name”: “centos-configdrive”}

#Check userdata

[root@centos-configdrive mnt]# cat /mnt/openstack/latest/user_data
export OS_TENANT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=admin_pass
export OS_AUTH_URL=”http://169.254.0.10:5000/v2.0/”

#Check injected file
#The first injected file is placed as /mnt/openstack/content/0000, the 2nd is 0001, and so on.

[root@centos-configdrive ~]# ls -l /mnt/openstack/content/
total 1
-r–r–r–. 1 root root 582 Mar 17 16:55 0000