We’re excited to announce that our Coriolis cloud migration project introduced earlier this year just reached a new important milestone. With the addition of new features specifically aimed at business continuity and Disaster Recovery as a Service (DRaaS), Coriolis now offers both one-off migrations as well as constantly up to date replicas of virtual workloads between different clouds.
Business continuity in cloud migrations and DRaaS
Coriolis v2 introduces the concept of migration replicas. A migration replica is a complete copy of one or more virtual machines on a separate cloud environment. For example, VMware virtual machines can be replicated on OpenStack, AWS or Azure. The replica is obtained by copying (replicating) incrementally the virtual machines data from the source environment to the target, without interfering with any running workload. A migration replica can then be finalized by automatically applying the required changes to adapt it to the target environment (migration phase).
Here are the objectives that we set for Coriolis V2:
No agent is needed inside the virtual machine itself: the user’s workload is a “black box” which means that Coriolis doesn’t require any guest access or credentials.
Guest OS independent: supports Windows, Linux, etc
Virtual machines can be either powered on or off when replicating data, the replication must not impact the business continuity.
Replica data transfers must be incremental to allow synchronization between source and destination with the minimum amount of network IO.
In case of running VMs, the replicated content must retain an application consistent state when available (e.g. VMware snapshot quiescing, VSS app consistency, etc).
All operations must be resilient as transient failures are to be expected in the cloud world.
Source and target hypervisor technology can differ: e.g. ESXi on a source vSphere, and KVM or Hyper-V on a target OpenStack.
In case of unavailability of the source environment (e.g. an unrecoverable failure), the virtual machines must be able to start on the target, retaining data and configurations, allowing a fast recovery.
Scalability: any number of replicas can be performed at a given time, limited only by the underlying resource availability, storage or network IO and QoS rules.
Testing: replicated VMs can be started anytime on the target environment to perform integration testing of guest operating systems and applications, without impacting the replica process or the source environment.
Run anywhere: all Coriolis components can be deployed as virtual, physical or containerized workloads / appliances anywhere, including directly on the source or target cloud.
Coriolis implements all the above requirements by leveraging the available cloud and virtualization technology (e.g. Changed Block Tracking in VMware) and cloud API abstractions (e.g. Cinder Storage Snapshot API in OpenStack) with a scalable and fully automated microservice based architecture.
How can Coriolis migration replicas be created and managed?
Coriolis offers a rich REST API, that can be consumed with our CLI tools, web GUI (coming soon) or any 3rd party tools.
Coriolis identity and authentication are managed via Keystone, which allows a large number of backends including common options like OpenLDAP, Active Directory, Azure Federation, etc.
Source or target cloud credentials can be managed via Barbican, another project originating within the OpenStack ecosystem, for the added security benefits.
Create a new replica
In this example two VMware VMs named “WebServer” and “DBServer” are going to be replicated to OpenStack (on any hypervisor, e.g.: KVM, Hyper-V or ESXi). The following command will create a configuration for the consistent replication of our “WebServer” and “DBServer” instances:
coriolis replica create --origin-provider vmware_vsphere --destination-provider openstack --origin-connection-secret $VMWARE_SECRET_REF --destination-connection-secret $OPENSTACK_SECRET_REF --instance DBServer --instance WebServer --target-environment "$TARGET_ENV"
Credentials for the source or target clouds can be securely stored in Barbican and safely referenced to Coriolis via their href:
barbican secret store -n "vsphere_connection_info" -t "text/plain" -p '{"host": "vsphere.local", "port": 443, "username": "user@vsphere.local", "password": "MyPassword", "allow_untrusted": true}'barbican secret store -n "openstack_connection_info" -t "text/plain" -p '{"identity_api_version": 3, "username": "demo", "password": "MyPassword", "project_name": "demo", "user_domain_name": "default", "project_domain_name": "default", "auth_url": "http://openstack.local:35357/v3", "allow_untrusted": true}'Note: if OpenStack credentials are omitted, the same Keystone token used by Coriolis is employed.
Lastly, the “target environment” parameter contains information about how to map virtual resources (such as networks, flavors etc) between source and target environments:
TARGET_ENV='{"network_map": {"VM Network Local": "public", "VM Network": "private"}, "flavor_name": "m1.small"}'Replica configurations can be easily managed with the following commands:
replica list replica show replica create replica delete
Executing the replica
A replica can be executed at any time, e.g. on a weekly, daily or hourly basis, or even continuously:
coriolis replica execute $REPLICA_ID
The replica execution is fully asynchronous and divided in multiple steps, each of which may be monitored with:
coriolis replica execution show $REPLICA_ID $EXECUTION_ID
During the first execution, a full data replication is performed while subsequent runs involve only the changes that occurred since the previous successful replica execution.
Optionally, you can also automatically shutdown the source VM before the replication starts by adding the “–shutdown-instances” command line option.
The replica process may involve the creation of temporary resources (such as virtual machines and volumes) both on the source and target infrastructures similar to what may be seen in the following replica execution output snippet:
id: b60ea340-da5e-40c7-9d2b-087c86fc952a task_type: DEPLOY_REPLICA_TARGET_RESOURCES instance: DBServer status: COMPLETED depends_on: b0cbd181-dba9-4908-89f5-1519f15fe3eb progress_updates: 2016-08-29T08:29:31.000000 Creating migration worker instance keypair 2016-08-29T08:29:32.000000 Creating migration worker instance Neutron port 2016-08-29T08:29:34.000000 Adding migration worker instance floating IP 2016-08-29T08:29:45.000000 Adding migration worker instance security group 2016-08-29T08:29:47.000000 Waiting for connectivity on host: 10.89.13.211:22 2016-08-29T08:30:06.000000 Attaching volume to worker instance id: bc04e461-ca2d-482e-90c4-70378522dfe7 task_type: REPLICATE_DISKS instance: DBServer status: COMPLETED depends_on: 68674a95-7bb8-40ad-8cbc-d9598237ea91, b60ea340-da5e-40c7-9d2b-087c86fc952a exception_details: progress_updates: 2016-08-29T08:30:26.000000 Creating snapshot 2016-08-29T08:30:32.000000 Performing incremental CBT replica for disk: [afsan1] Windows 2012 R2/Disk1.vmdk. Disk size: 14,294,967,296. Changed blocks size: 1,983,040 2016-08-29T08:30:40.000000 Disk [afsan1] Windows 2012 R2/Disk1.vmdk replica progress: 100% 2016-08-29T08:30:48.000000 Removing snapshot
The replication process will perform an app consistent snapshot on the guest OS when available (e.g. VMware quiescing). This ensures that the replicated data is application consistent in case of transactional activities running in the guest OS (e.g. Oracle, MS SQL Server, Exchange Server, etc).
How to start a replicated VM on the target cloud
Here’s where Coriolis truly excels: getting virtual machines to run across a wide range of hypervisors and clouds. Replicating the content of a VM is just a part of the process, getting the machine to actually run on the target cloud also requires actions particular to the underlying target hypervisor and cloud deployment tools. For example, some guest operating systems will require specific synthetic drivers to run in the best possible way on each of the individual hypervisors such KVM, ESXi or Hyper-V, while OpenStack needs cloud-init / cloudbase-init configured for proper guest initialization, Azure the WALinuxAgent, etc. All the required steps are fully automated and transparent for the user.
Once the replication is completed, the source environment becomes redundant thanks to the replica made available on the destination cloud. Thus, in the event of an actual disaster (the source VM became deleted, corrupted, etc), we may quickly bring the system back up on our target cloud with minimal loss to continuity.
The following command will snapshot the migrated disks / volumes and start the migration process asynchronously:
coriolis migration deploy replica $REPLICA_ID
Here’s some sample output:
task_type: CREATE_REPLICA_DISK_SNAPSHOTS
instance: WebServer
status: COMPLETED
depends_on:
exception_details:
progress_updates:
2016-08-28T21:06:30.000000 Creating replica disk snapshots
id: be587123-f01e-4ddc-a9e2-1474d7b6ca4c
task_type: DEPLOY_REPLICA_INSTANCE
instance: WebServer
status: COMPLETED
depends_on: f3bc535b-70b9-43b2-a6bd-0ec6fb9f21c6
exception_details:
progress_updates:
2016-08-28T21:06:39.000000 Creating migration worker instance Neutron port
2016-08-28T21:06:39.000000 Creating migration worker instance keypair
2016-08-28T21:06:41.000000 Adding migration worker instance floating IP
2016-08-28T21:06:53.000000 Adding migration worker instance security group
2016-08-28T21:06:55.000000 Waiting for connectivity on host: 10.89.13.226:22
2016-08-28T21:07:08.000000 Attaching volume to worker instance
2016-08-28T21:07:15.000000 Attaching volume to worker instance
2016-08-28T21:07:21.000000 Preparing instance for target platform
2016-08-28T21:07:21.000000 Connecting to SSH host: 10.89.13.226:22
2016-08-28T21:07:24.000000 Discovering and mounting OS partitions
2016-08-28T21:08:01.000000 OS being migrated: ('CentOS Linux', '7.2.1511')
2016-08-28T21:08:04.000000 Removing packages: ['open-vm-tools', 'hyperv-daemons']
2016-08-28T21:08:07.000000 Adding packages: ['dracut-config-generic', 'cloud-init', 'cloud-utils', 'parted', 'git', 'cloud-utils-growpart']
2016-08-28T21:08:40.000000 Generating initrd for kernel: 3.10.0-327.el7.x86_64
2016-08-28T21:09:43.000000 Dismounting OS partitions
2016-08-28T21:09:45.000000 Removing worker instance resources
2016-08-28T21:09:56.000000 Renaming volumes
2016-08-28T21:09:57.000000 Ensuring volumes are bootable
2016-08-28T21:09:57.000000 Creating Neutron ports for migrated instance
2016-08-28T21:09:59.000000 Creating migrated instance
2016-08-28T21:10:20.000000 Deleting Glance images
id: d4ebd9b7-76a9-46b5-b7c1-fbfed7aa5b99
task_type: DELETE_REPLICA_DISK_SNAPSHOTS
instance: WebServer
status: COMPLETED
depends_on: be587123-f01e-4ddc-a9e2-1474d7b6ca4c
exception_details:
progress_updates:
2016-08-28T21:10:22.000000 Removing replica disk snapshots
Congratulations, your workload has been successfully migrated!
What about testing the replicated workloads?
One important requirement consists in periodically testing the replicated workloads on the destination cloud. This can be easily done with Coriolis, even while the replicated resources on the source environment continue running unaffected. This also opens the way for numerous testing scenarios (e.g. integration / functional / white box / black box) to be performed on the replicated workloads at anytime, reassuring that the workloads will operate as expected and lead to a smooth recovery in the event of an emergency.
What are the supported guest operating systems?
One of the advantages of Coriolis is that the replica is guest OS agnostic, so differences between guest operating systems come only into play when performing the migration. Coriolis supports migrations for all Windows versions (Windows Server 2008, 2008 R2, 2012, 2012 R2, 2016 including Nano Server, Windows 7, 8, 8.1, 10), Ubuntu 12.04 and above, RHEL / CentOS 6.x and 7.x, SUSE SLE 11 and 12, openSUSE, Debian and Fedora. Support for other Linux distributions can also be added on demand.
How can you deploy Coriolis?
Just contact us for pricing and additional info! We are also offering a demo appliance that can be easily deployed on your environment for evaluating Coriolis!
The post Coriolis v2 – Adding DRaaS to Cloud Migrations appeared first on Cloudbase Solutions.