Nova Migration Type

The migration types are:

Non-live migration (sometimes referred to simply as ‘migration’)
The instance is shut down for a period of time to be moved to another hypervisor. In this case, the instance recognizes that it was rebooted.

Live migration (or ‘true live migration’)
Almost no instance downtime. Useful when the instances must be kept running during the migration.
The different types of live migration are:

2-1. Shared storage-based live migration.

Both hypervisors have access to shared storage.

2-2. Block live migration

No shared storage is required. Incompatible with read-only devices such as CD-ROMs and Configuration Drive (config_drive).

2-3. Volume-backed live migration

Instances are backed by volumes rather than ephemeral disk, no shared storage is required, and migration is supported (currently only available for libvirt-based hypervisors).

What's this?

2-3. Volume-backed live migration の Nova instance の Migration

Nova instance (running 状態) を別の Hypervisor (Compute node) へ移動
Nova instance の Boot Volume は Cinder に格納
Cinder は Shared Storage

[Case]

Cinder volume を接続していない場合
Cinder volume を接続している場合

[Case1] Migrate Nova instance (w/o Cinder volume)

Initial state

Hypervisor (Compute node)

# nova hypervisor-list
+----+-----------------------+
| ID | Hypervisor hostname   |
+----+-----------------------+
| 6  | node-27               |
| 9  | node-28               |
| 12 | node-29               |
+----+-----------------------+

AZ

# nova availability-zone-list | egrep -A 10 "^\| nova"
| nova                     | available                              |
| |- node-27               |                                        |
| | |- nova-compute        | enabled :-) 2015-02-13T02:24:11.000000 |
| |- node-28               |                                        |
| | |- nova-compute        | enabled :-) 2015-02-13T02:24:09.000000 |
| nova2                    | available                              |
| |- node-29               |                                        |
| | |- nova-compute        | enabled :-) 2015-02-13T02:24:10.000000 |
+--------------------------+----------------------------------------+

Nova instance is on node-28

# nova list --field name,status,,OS-EXT-STS:vm_state,OS-EXT-AZ:availability_zone,OS-EXT-SRV-ATTR:hypervisor_hostname
+--------------------------------------+------------+--------+--+----------------------+------------------------------+--------------------------------------+
| ID                                   | Name       | Status |  | OS-EXT-STS: Vm State | OS-EXT-AZ: Availability Zone | OS-EXT-SRV-ATTR: Hypervisor Hostname |
+--------------------------------------+------------+--------+--+----------------------+------------------------------+--------------------------------------+
| 6235d670-a5f0-4658-aef5-1b9549613c38 | admin_vm01 | ACTIVE |  | active               | nova                         | node-28                              |
+--------------------------------------+------------+--------+--+----------------------+------------------------------+--------------------------------------+

Boot volume of Nova instance is in Cinder

# nova show admin_vm01 | egrep "Property|image|os-extended-volumes:volumes_attached"
| Property                             | Value                                                    |
| image                                | Attempt to boot from volume - no image supplied          |
| os-extended-volumes:volumes_attached | [{"id": "c9e5313d-217b-4694-b304-c7ad3b2551cf"}]         |

Cinder list

# cinder list
+--------------------------------------+-----------+--------------+------+-------------+----------+--------------------------------------+
|                  ID                  |   Status  | Display Name | Size | Volume Type | Bootable |             Attached to              |
+--------------------------------------+-----------+--------------+------+-------------+----------+--------------------------------------+
| c9e5313d-217b-4694-b304-c7ad3b2551cf |   in-use  | admin_vol01  |  1   |     None    |   true   | 178b3b3d-c1cf-460c-903c-9b065ced5d4a |
+--------------------------------------+-----------+--------------+------+-------------+----------+--------------------------------------+

Migrate Nova instance

Migration from node-28 to node-27

# nova live-migration admin_vm01 node-27

Check the migration result

Migrating...

# nova list --field name,status,,OS-EXT-STS:vm_state,OS-EXT-AZ:availability_zone,OS-EXT-SRV-ATTR:hypervisor_hostname
+--------------------------------------+------------+-----------+--+----------------------+------------------------------+--------------------------------------+
| ID                                   | Name       | Status    |  | OS-EXT-STS: Vm State | OS-EXT-AZ: Availability Zone | OS-EXT-SRV-ATTR: Hypervisor Hostname |
+--------------------------------------+------------+-----------+--+----------------------+------------------------------+--------------------------------------+
| 6235d670-a5f0-4658-aef5-1b9549613c38 | admin_vm01 | MIGRATING |  | active               | nova                         | node-28                              |
+--------------------------------------+------------+-----------+--+----------------------+------------------------------+--------------------------------------+

Nova instance is moved from node-28 to node-27

# nova list --field name,status,,OS-EXT-STS:vm_state,OS-EXT-AZ:availability_zone,OS-EXT-SRV-ATTR:hypervisor_hostname
+--------------------------------------+------------+--------+--+----------------------+------------------------------+--------------------------------------+
| ID                                   | Name       | Status |  | OS-EXT-STS: Vm State | OS-EXT-AZ: Availability Zone | OS-EXT-SRV-ATTR: Hypervisor Hostname |
+--------------------------------------+------------+--------+--+----------------------+------------------------------+--------------------------------------+
| 6235d670-a5f0-4658-aef5-1b9549613c38 | admin_vm01 | ACTIVE |  | active               | nova                         | node-27                              |
+--------------------------------------+------------+--------+--+----------------------+------------------------------+--------------------------------------+

[Case2] Migrate Nova instance (w/ Cinder volume)

Initial State

Hypervisor (Compute node)

# nova hypervisor-list
+----+-----------------------+
| ID | Hypervisor hostname   |
+----+-----------------------+
| 6  | node-27               |
| 9  | node-28               |
| 12 | node-29               |
+----+-----------------------+

AZ

# nova availability-zone-list | egrep -A 10 "^\| nova"
| nova                     | available                              |
| |- node-27               |                                        |
| | |- nova-compute        | enabled :-) 2015-02-13T02:24:11.000000 |
| |- node-28               |                                        |
| | |- nova-compute        | enabled :-) 2015-02-13T02:24:09.000000 |
| nova2                    | available                              |
| |- node-29               |                                        |
| | |- nova-compute        | enabled :-) 2015-02-13T02:24:10.000000 |
+--------------------------+----------------------------------------+

Nova instance is on node-27

# nova list --field name,status,,OS-EXT-STS:vm_state,OS-EXT-AZ:availability_zone,OS-EXT-SRV-ATTR:hypervisor_hostname
+--------------------------------------+------------+--------+--+----------------------+------------------------------+--------------------------------------+
| ID                                   | Name       | Status |  | OS-EXT-STS: Vm State | OS-EXT-AZ: Availability Zone | OS-EXT-SRV-ATTR: Hypervisor Hostname |
+--------------------------------------+------------+--------+--+----------------------+------------------------------+--------------------------------------+
| f7bf58f3-5292-4eb1-80d8-627fe747a6f8 | admin_vm01 | ACTIVE |  | active               | nova                         | node-27                              |
+--------------------------------------+------------+--------+--+----------------------+------------------------------+--------------------------------------+

Boot volume of Nova instance is in Cinder
Nova instance has one additional Cinder volume

# nova show admin_vm01 | egrep "Property|image|os-extended-volumes:volumes_attached"
| Property                             | Value                                                                                            |
| image                                | Attempt to boot from volume - no image supplied                                                  |
| os-extended-volumes:volumes_attached | [{"id": "c9e5313d-217b-4694-b304-c7ad3b2551cf"}, {"id": "ebf7bb32-cf5a-4c45-8908-194ccf355435"}] |

Cinder list

# cinder list
+--------------------------------------+-----------+--------------+------+-------------+----------+--------------------------------------+
|                  ID                  |   Status  | Display Name | Size | Volume Type | Bootable |             Attached to              |
+--------------------------------------+-----------+--------------+------+-------------+----------+--------------------------------------+
| c9e5313d-217b-4694-b304-c7ad3b2551cf |   in-use  | admin_vol01  |  1   |     None    |   true   | f7bf58f3-5292-4eb1-80d8-627fe747a6f8 |
| ebf7bb32-cf5a-4c45-8908-194ccf355435 |   in-use  | admin_vol11  |  1   |     None    |  false   | f7bf58f3-5292-4eb1-80d8-627fe747a6f8 |
+--------------------------------------+-----------+--------------+------+-------------+----------+--------------------------------------+

Instance

$ ssh cirros@<INSTANCE> lsblk
NAME   MAJ:MIN RM    SIZE RO TYPE MOUNTPOINT
vda    253:0    0      1G  0 disk
`-vda1 253:1    0 1011.9M  0 part /
vdb    253:16   0      1G  0 disk
`-vdb1 253:17   0   1023M  0 part /mnt/vol01

$ ssh cirros@<INSTANCE> df -h /mnt/vol01/
Filesystem                Size      Used Available Use% Mounted on
/dev/vdb1              1006.9M     17.3M    938.5M   2% /mnt/vol01

Migrate Nova instance

Migration from node-27 to node-28

# nova live-migration admin_vm01 node-28

Check the migration result

Nova instance is moved from node-27 to node-28

# nova list --field name,status,,OS-EXT-STS:vm_state,OS-EXT-AZ:availability_zone,OS-EXT-SRV-ATTR:hypervisor_hostname
+--------------------------------------+------------+--------+--+----------------------+------------------------------+--------------------------------------+
| ID                                   | Name       | Status |  | OS-EXT-STS: Vm State | OS-EXT-AZ: Availability Zone | OS-EXT-SRV-ATTR: Hypervisor Hostname |
+--------------------------------------+------------+--------+--+----------------------+------------------------------+--------------------------------------+
| f7bf58f3-5292-4eb1-80d8-627fe747a6f8 | admin_vm01 | ACTIVE |  | active               | nova                         | node-28                              |
+--------------------------------------+------------+--------+--+----------------------+------------------------------+--------------------------------------+

Instance

$ ssh cirros@<INSTANCE> lsblk
NAME   MAJ:MIN RM    SIZE RO TYPE MOUNTPOINT
vda    253:0    0      1G  0 disk
`-vda1 253:1    0 1011.9M  0 part /
vdb    253:16   0      1G  0 disk
`-vdb1 253:17   0   1023M  0 part /mnt/vol01

$ ssh cirros@<INSTANCE> df -h /mnt/vol01/
Filesystem                Size      Used Available Use% Mounted on
/dev/vdb1              1006.9M     17.3M    938.5M   2% /mnt/vol01

Environment

Openstack : Juno
Ceph : Firefly

Reference

Nova Migration (non live migration) to Different Hypervisor
Nova Migration (non live migration) to Different Hypervisor between AZ
Nova Migration (true live migration) to Different Hypervisor
Nova Migration (true live migration) to Different Hypervisor between AZ
Nova Migration (Manually) between AZ
Nova Migration (Manually) between Region
Nova Migration (true live migration) to Different Hypervisor - Volume-backend live migration (instance)
Nova Migration (true live migration) to Different Hypervisor - Volume-backend live migration (instance) between AZ