Migrate for Compute Engine

Overview
1. Migrate for Compute Engine decouples VMs from their storage and introduces capabilities that ease the move to Google Cloud
2. Easy deployment: Install Migrate for Compute Engine virtual appliances in just a few steps, without installing agents on servers
3. Secure by design: Data transfers between Migrate for Compute Engine components use TLS and AES-128 encryption. Data at rest is de-duplicated, compressed, and encrypted with AES-256
4. Boot over WAN: Migrate for Compute Engine performs a native boot in the cloud from VMs in a few minutes, regardless of image size
5. While the image boots, Migrate for Compute Engine adapts it for the target environment
6. No changes to the application, original image, storage, drivers, or networking are necessary
7. Intelligent streaming: Migrate for Compute Engine prioritizes the data necessary for an application to run and moves that data to the cloud first
8. Migrate for Compute Engine streams data to the cloud when needed
9. Multi-tier caching and optimization: Migrate for Compute Engine includes a multi-tier, read-write cache in the cloud
10. Migrate for Compute Engine cache stores data needed by the application
11. De-duplication, pre-fetching, asynchronous write-back, and network optimizations accelerate migration, reducing migration bandwidth by up to 75% in production migrations
12. Resiliency: Migrate for Compute Engine Cloud Extensions use an active-passive configuration across two availability zones
13. Data is written in both zones and then asynchronously transferred back on premises to reduce the risk of data loss
14. Writes can persist solely in the cloud for development and testing
15. Recovery point objective (RPO) is the maximum acceptable length of time during which data might be lost due to an incident
16. Migrate for Compute Engine's architecture ensures a 30-second RPO for sync to Google Cloud Storage in the rare case of a dual zone failure and a 1-hour RPO for sync on-premises
17. Supports multiple operating systems
Architecture
1. Architecture
  1. Migrate for Compute Engine provides a path to migrate virtual machines (VMs) running on VMware vSphere to Compute Engine
  2. Migrate for Compute Engine can also migrate physical servers and Amazon EC2 or Azure VMs to Compute Engine
  3. Migrate for Compute Engine Manager on Google Cloud manages all components and orchestrates migrations
  4. Migrate for Compute Engine Manager on Google Cloud serves the Migrate for Compute Engine user interface
  5. Cloud Extensions handle storage migrations and serve data to migrated workloads during migration
  6. A Cloud Extension is a pair of Cloud Edge nodes
  7. The Migrate for Compute Engine Exporter creates Google Cloud Persistent Disks when detaching disks
2. On Premises
  1. When performing on-premises to cloud migrations, the Migrate for Compute Engine On-Premises Backend virtual appliance on VMware establishes a secure data path with the Cloud Extension nodes
  2. Starts and stops VMs using VMware APIs
  3. Performs storage operations against virtual machine disks (VMDKs) using the VMware Storage API
  4. The Migrate for Compute Engine On-Premises Backend virtual appliance serves data from VMware to the cloud extension
  5. The Migrate for Compute Engine vCenter Plugin connects vCenter vSphere to the Migrate for Compute Engine Manager
  6. Encrypted migration data is sent from Migrate for Compute Engine Backend to Cloud Extensions
  7. A control plane is configured between Cloud Extensions and Migrate for Compute Engine Manager
  8. Data is synchronised between Cloud Extension nodes
  9. Migrate for Compute Engine Manager checks if SSH or RDP console on the migrated VM is available
  10. HTTPS access to Migrate for Compute Engine Manager is enabled for web user interface access
  11. iSCSI is used for data migration and syslog
3. AWS
  1. For migrations from AWS to Google Cloud, the Migrate for Compute Engine Manager launches Importer instances on AWS as required to migrate AWS EC2 source workloads and their EBS volumes
  2. Migrate for Compute Engine Importer instances run only when data is being migrated
  3. Migrate for Compute Engine Importer serves data from AWS Elastic Block Store volumes to Cloud Extensions
4. Azure
  1. For migrations from Azure to Google Cloud, the Migrate for Compute Engine Manager launches Importer instances on Azure as needed to migrate Azure source workloads and their disks
  2. Migrate for Compute Engine Importer instances run only when data is being migrated
  3. Migrate for Compute Engine Importer serves data from Azure disks to Cloud Extensions
5. GCP
  1. Migrate for Compute Engine Manager on Google Cloud orchestrates migration operations and serves the web user interface
  2. Migrate for Compute Engine Manager connects with the Migrate for Compute Engine On-Premises Backend virtual appliance and accesses Google Cloud API endpoints as well as Monitoring and Logs services
  3. Once the Migrate for Compute Engine Manager is launched, connect it to the Migrate for Compute Engine Backend, and create Cloud Extensions to manage storage migration
  4. Cloud Extension nodes (Cloud Edge nodes) run in pairs in separate Google Cloud zones
  5. Migrate for Compute Engine Manager and Cloud Extension require inbound access from the corporate datacenter to Google Cloud
  6. Inbound iSCSI access from on-premises VMs migrated to Google Cloud into the Cloud Extension nodes is necessary to migrate storage
  7. Subnets where Cloud Extension nodes are deployed must allow outbound access to certain services, such as Cloud Storage and Cloud Monitoring
Workflow
1. Modeling and planning migration waves
  1. Before beginning a migration, create a prioritized inventory of the machines to migrate.
  2. Understand application's dependencies
  3. Use Migrate for Compute Engine's Wave interface to create migration waves and plan the migrations
  4. Batch dependencies within the same wave, for example if an application depends on multiple VMs, migrate all of those VMs in the same wave
  5. If an application requires a database and a web server, the database can be migrated and started before the web server
2. Pre-migration testing (optional)
  1. To test migration before running it, clone an on-premises workload and run it in Google Cloud without modifying the original host
  2. Cloning an on-premises workload and running it in Google Cloud enables it to be tested without disrupting production
  3. On premises workload clones can be fully operational within minutes
3. Cutover to cloud
  1. Run the VM in Compute Engine in streaming mode
  2. Compute Engine VMs can be run in streaming mode per VM or using Waves for mass migration sprints
  3. The Migration Wizard or Waves can be used to move VMs to Compute Engine
  4. The Migration Wizard or Waves can be used to migrate application storage to Google Cloud
4. Detach
  1. After determining that the VMs in the cloud can be relied upon, schedule downtime to cut over application to Google Cloud
  2. To cut over application to Google Cloud, detach the VM and test the application to verify that it functions accurately
Running vs migrating VMs
1. Migrate for Compute Engine distinguishes between running a VM in Google Cloud and migrating the VM to Google Cloud
2. Both modes dramatically reduce the time to start up a VM on Google Cloud by making the VM available at the destination before all of the data is migrated there
3. When running an on-premises VM in Google Cloud, storage is streamed, and storage blocks are only cached on Google Cloud when needed
4. Storage can be optionally migrated as a next step to complete the migration to Compute Engine
5. Migrating an on-premises VM to Google Cloud involves running the VM in Google Cloud while actively migrating its data in the background to Google Cloud
6. When data is fully cached on Google Cloud, the VM is detached
7. Detaching a VM moves its storage from the Migrate for Compute Engine cache and creates native disks on Compute Engine
8. When detaching, the VM needs to be shut down briefly to switch from the streaming to native disk volumes and to allow for complete synchronization
9. Storage migration can be initiated at any time
Requirements
1. Requirements*
  1. Bandwidth between source environment and Compute Engine nodes should be larger than the total number of VMs migrated concurrently, multiplied by 0.5 Mbit/sec per VM
  2. Migrate for Compute Engine 4.8 recommends a minimum of 50 Mbit/sec for production use of up to 100 VMs running in the cloud
  3. VMs need to run a supported operating system version
  4. Network access outlines ports and connectivity requirements
  5. Migrate for Compute Engine components require a specific set of Google Cloud Service Accounts and Roles
2. On Premises
  1. Migrate for Compute Engine supports migrations from VMware vCenter and ESXi
  2. Each Migrate for Compute Engine release is compatible with specific VMware versions
  3. A VMware administrator needs to Deploy the Migrate for Compute Engine On-Premises Backend virtual appliance OVA
  4. Create and assign vCenter roles for Migrate for Compute Engine to manage migrations
  5. Migrate for Compute Engine Backend runs on a VM in a VMware environment
3. AWS
  1. Set up a VPN or Interconnect between AWS and Google Cloud
  2. Set up an IAM account and assign privileges to it
4. Azure
  1. Set up a VPN or Interconnect between Azure and Google Cloud
  2. Set up the necessary permission in Azure
Extensions
1. Overview
  1. A Cloud Extension is a conduit for VM storage between two hosting environments
  2. A Cloud Extension uses a dual-node active/passive configuration for high availability
  3. Each Cloud Extension node serves its own workloads while providing backup for the other
  4. A Cloud Extension is a pair of Cloud Edge nodes
2. Configuration
  1. Each Cloud Extension supports up to 20 or up to 50 concurrent VMs, depending on the Cloud Extension size
  2. More than one Cloud Extension can be used to simultaneously to handle a larger migration
  3. The total number of Cloud Extensions necessary for a migration is a function of the total number of VMs divided by the capacity of the Cloud Extension
  4. Cloud Extensions are created and configured either using the Migrate for Compute Engine vCenter or the Migrate for Compute Engine Manager
3. Impared
  1. A Cloud Extension is impaired when it has not fully failed, but is experiencing trouble with one of its nodes or network connectivity
  2. A Cloud Extension with an Impaired status functions differently depending on whether only one or both of its Cloud Edge nodes have failed
  3. Cloud Extension can be Impaired if the Migrate for Compute Engine Backend or the Migrate for Compute Engine Manager fail
  4. Cloud Extensions can be impaired due to incomplete deployment or initial failed health checks
  5. Incomplete deployment or initial failed health checks are most likely due to improperly configured networks or permissions
  6. Cloud Extensions can be repaired after fixing the underlying causes of a failure
  7. Repairing the cloud extension attempts to recreate the missing components, run relevant health checks, or both
  8. If a Cloud Extensions repair operation succeeds, the Cloud Extension is Active
  9. Stopping and starting the Cloud Extension might fix VM health issues after affected VMs restart on a healthy host
4. HA Model
  1. Cloud Extensions provide high availability using an active-passive model
  2. Workloads use a primary Cloud Edge node, and iSCSI multipath to connect to the secondary node
  3. If a Cloud Extensions primary node fails, the workloads fails over to the secondary node
  4. When a primary Cloud Extensions node recovers, the workload fails back to it
  5. If only one of the Cloud Edge nodes fails, most operations remain available, but write throughput decreases
  6. If both Cloud Edge nodes fail, VMs relying on that Cloud Extension will no longer have access to storage and will fail
  7. If both Cloud Edge nodes fail, the Cloud Extension can no longer run new VMs in the cloud
  8. If both Cloud Edge nodes fail, running VMs can be moved back to their source environment
  9. If both Cloud Edge nodes fail while a VM is fully cached, the Prepare to Detach operation can be used to migrate back to the source environment
  10. If both Cloud Edge nodes fail, VMs can be stopped
5. Repairing
  1. If the Cloud Extension creation task fails, loses connectivity, or edge node instances become unavailable, the Cloud Extension is considered Impaired
  2. Once the Cloud Extension is repaired, the status is set to Active
  3. Repairing the Cloud Extension attempts to recreate the missing components and/or re-run relevant health checks
  4. After fixing the underlying causes, the status can be can updated to Repaired using either the Migrate for Compute Engine Manager or vCenter
6. Data Loss
  1. When one Cloud Edge node fails, the Cloud Extension enters a fail-safe mode
  2. To avoid data loss, all data is written to the Cloud Extension's object store on Cloud Storage, which reduces the Cloud Extension performance
  3. Google does not recommend running or migrating VMs using an impaired Cloud Extension
Backend
1. On-Premises Backend virtual appliance connects to VM disks in on-premises data center and streams or migrates data to Google Cloud using Cloud Extensions
2. Backend is distributed as an Open Virtualization Format (OVF) package
3. Backend configuration is based on the number of VMs to be migrated concurrently
4. A PowerShell script is also available for manually adding a service role to the vCenter Server for Migrate for Compute Engine
5. Once the Backend has successfully connected and registered with the Manager on Google Cloud, the Migrate for Compute Engine VMware vCenter Web Client Plugin can be registered and deployed
6. Migrate for Compute Engine VMware vCenter Web Client Plugin enables Migrate for Compute Engine management operations and monitoring in the vCenter UI
7. Each Migrate for Compute Engine can communicate with only one Migrate for Compute Engine Manager per vCenter server
Adapting VMs
1. Customization
  1. Some custom configurations or adaptations are handled automatically, while others are enabled through the use of custom configuration script
  2. Custom configurations or adaptations allow a VM, its operating system (OS), and applications to run in the clou
  3. VM adaptations can be customised by installing drivers to boot VM
  4. Scripts can be executed during migration to enable adaptations
  5. Scripts can be developed by customers or provided as a support package
  6. Scripts may run in machine states (for Windows VMs) or phases (for Linux VMs)
  7. VM changes performed automatically
  8. Linux VMs can be prepared to boot automatically in the cloud using an installed package.
  9. Changes automatically applied to VMs are active only when the VM is running in the cloud
  10. Automatically installed packages can remain installed after the VM has been migrated
  11. If an automatically installed package is uninstalled, all changes are reverted and the VM may not boot in the cloud
2. Changes made to Linux and Windows
  1. Enable booting on cloud
  2. Enable the serial console
3. Cloud-specific changes for storage channel
  1. Hardware-specific adaptations for cloud migration
4. Modifications for run-in-cloud on Windows VMs
  1. When a run-in-cloud operation is started on a Windows VM, the VM is shut down and a snapshot taken
  2. Networking and storage drivers are modified after the VM is shutdown for the run-in-cloud operation to boot the VM in the cloud
5. Modifications for detaching a Windows VM
  1. During a detach of a Windows VM, a Google Cloud agent is installed
  2. The Google Cloud agent installation requires either an external IP address or Private Access to be enabled during run-in-cloud
6. Modifications for run-in-cloud on Linux VMs
  1. When a VM with VMware tools installed is migrated, the VM gracefully shut down and a snapshot of the VM is taken
  2. When a VM with VMware tools installed is shutdown, networking and storage drivers are modified to allow the VM to boot in the cloud
Rightsizing
1. Recommendations
  1. Before migrating a virtual machine to Compute Engine, determine the target instance types and sizes to use
  2. Migrate for Compute Engine has a rightsizing feature that determines the target instance types and sizes to use
  3. Migrate for Compute Engine rightsizing feature includes built-in usage monitoring and recommends instance types optimized for cost and performance
  4. Performance-based recommendations recommends Compute Engine instances based on the CPU and RAM currently allocated to the on-premises VM. This recommendation is the default
  5. Cost-based recommendations propose Compute Engine instances based on the current CPU and RAM configuration of the on-premises VM and the average usage of the VM during a given period
  6. To use Cost-based rightsizing recommendations, activate rightsizing monitoring with vSphere for the group of VMs and allow time for Migrate for Compute Engine to analyze usage
  7. Recommendations are accessible from the vCenter Web Client user interface, PowerShell module, REST API, and Migrate for Compute Engine Waves for mass migration planning
  8. Suggested options also include expected relative monthly costs estimates not intended for billing forecasts
  9. Cost estimates use on-demand compute prices with sustained-use discounts only, and does not include disk and network cost or consider reduced-cost options
  10. Migrate for Compute Engine Manager must have access to the internet to access the online price list to provide cost-optimized recommendations
2. vSphere
  1. When VMs are monitored, the Migrate for Compute Engine On-Premises Backend virtual appliance starts collecting and analyzing utilization statistics from VMware vSphere
  2. Migrate for Compute Engine On-Premises Backend classifies the activity it observes into patterns based on estimated memory and CPU needs
  3. For better recommendations, Migrate for Compute Engine recommends monitoring the migrated workloads for at least seven consecutive days
  4. Migrate for Compute Engine warns users when the monitoring period is insufficient for an adequate recommendation
  5. Migrate for Compute Engine offers a cost-optimized recommendation based on the data available
3. AWS
  1. Migrate for Compute Engine offers rightsizing recommendations for VMs hosted on AWS
  2. Unlike rightsizing recommendations for vSphere, recommendations for AWS instances are not based on monitoring of instance usage
  3. Rightsizing recommendations for AWS are based upon the CPU and memory configuration of the source instance
  4. Special instance types such as GPU or burstable instances need to be specified manually in the runbook
IAM
1. Migrate for Compute Engine uses service accounts to grant access permissions
2. Deploying Migrate for Compute Engine Manager creates the Manager Service Account attached to the Manager instance
3. The Manager Service Account allows the Manager to orchestrate migrations, deploy Cloud Extensions and create instances for migrated VMs
4. The Cloud Extension Service Account is attached to the Cloud Extensions nodes
5. The Cloud Extension Service Account allows Cloud Extensions nodes access to storage resources
6. Migrate for Compute Engine-specific roles enable permissions on Compute Engine and Cloud Storage
Networking
1. Network resources
  1. Virtual Private Cloud (VPC) on Google Cloud
  2. Firewall rules on Google Cloud, on-premises, AWS, or Azure
  3. VPNs or other network interconnections with routing and forwarding rules between Google Cloud and on-premises, AWS or Azure environment
  4. Google Cloud Network Tags that allow traffic to pass between instances
2. Network tags
  1. Google Cloud uses network tags to identify which network firewall rules apply to specified VM instances
  2. Components with the same network tags can communicate with each other
  3. Migrate for Compute Engine assigns network tags to facilitate workload migration
Storage
1. Write back mode (default)
  1. In write back mode, Migrate for Compute Engine writes all disk changes made to the VM on Google Cloud back on-premises
  2. Changes are buffered and synchronized
  3. When a VM is migrated with write back mode, Migrate for Compute Engine writes any data remaining in the buffer before restarting the VM
  4. When Migrate for Compute Engine completes a move back, it deletes all interim snapshots and virtual disks
  5. For migrations from Amazon EC2 and Azure, writing changes back to the source is not possible
2. Write isolation mode
  1. In write isolation mode, Migrate for Compute Engine does not write storage data on-premises while the VM is running on Google Cloud
  2. If a VM in write isolation mode is moved back to the source environment, changes are not copied back
  3. VM's disk is re-created from the initial snapshot when a VM in write isolation mode is moved back to the source environment
Lifecycle
1. Full migration
  1. Moves VMs in one step from source to target
  2. Performs the run-in-cloud process
  3. Waits for the VMs to be in the Cache on Demand state, when storage is streamed to the cloud.=
  4. Migrates the VM data to Google Cloud
  5. After storage is fully copied to Google Cloud, prepares the VM to detach
  6. When VM detach process has completed, the VM state changes to Ready to Detach
  7. Can run automatically or manually during full migration
    1. Run-in-cloud
    2. Storage migration
    3. Prepare to detach
    4. Run-in-cloud
  8. Moves the source VMs from on-premises data center to Google Cloud
  9. Does not completely move VM storage to the cloud
  10. Storage is migrated with storage migration
2. Run-in-cloud operation
  1. Shuts down the source VMs
  2. Attaches to the VM's volumes
  3. Starts the VM on Google Cloud, streaming storage as needed
3. Cleanup
  1. After VMs are detached and required validation completed, the detach cleanup can be started
  2. Each VM is then marked as unmanaged by Migrate for Compute Engine
4. Move back
  1. Moves the instances in Google Cloud back to their source, either on-premises or AWS
  2. Move-back operation
    1. Stops the VMs
    2. Moves storage back to source
    3. Deletes the Google Cloud instances
5. Test clone
  1. A test clone creates clones of selected VMs to test them in Compute Engine
  2. The test clone behaves like the live systems and leverages data from the source VM
  3. Test clones do not modify any live data because data from the test environment is not written back on-premises.
  4. Upon creating a test clone, Migrate for Compute Engine
    1. Attaches to the VM volumes
    2. Starts each instance in Google Cloud
    3. Streams storage from the VM to Google Cloud.
  5. Test clones are available only for migrations from vSphere to Compute Engine
  6. Test clones are not available for migrations from other cloud-based source platforms
6. Delete clone
  1. Deleting the test clone removes it from Google Cloud
  2. Deleting a test clone has no impact whatsoever on live system or data
  3. Any changes made to data in the test clone are not replicated back to the live system
7. Offline migration
  1. Migrates workloads with operating systems or file systems that Migrate for Compute Engine's streaming technology does not support, but that the cloud environment does support
  2. During the offline migration process, Migrate for Compute Engine
    1. Migrates storage
    2. Starts the new VM only after migration is completed
    3. Detaches the VM
Quotas
1. Compute instances are needed to support migrated VMs, Manager, Cloud Extensions, and worker nodes
2. When deploying Cloud Extensions, ensure SSD Persistent Disk quota is set high enough to support the number of Cloud Extensions
3. The migration process creates Compute Engine VMs, and can create IP addresses for access over the public Internet
4. During migration, Migrate for Compute Engine uses Cloud Storage to hold data that is used by applications on Compute Engine VMs
5. Migrate for Compute Engine uses Virtual Private Cloud (VPC) for secure access to GCP during migration
6. Ensure quotas for networks, subnets, IP addresses, and more will support the migration
Pricing
1. Google Cloud pricing calculator can be used to create an estimate of monthly GCP charges, including VM pricing
2. GCP charges for resources such as Compute Engine instances, storage, and networking which are necessary to perform migrations
3. Technical support is available with a paid GCP Cloud Support subscription
4. Migrate for Compute Engine uses Google Compute Engine instances
5. Users are billed for Compute Engine instances according to Compute Engine's pricing, until the instances are deleted
6. Compute Engine resources are billed on a per-second basis with a one-minute minimum usage cost
7. Prior to migration, Migrate for Compute Engine provides estimated costs for VMs using Compute Engine published pricing and sustained use discounts
8. Additional compute discounts may apply
9. During migration, customers incur costs for Compute VMs to support Migrate for Compute Engine Manager, Cloud Extensions, worker nodes, VMs Storage, Cloud Storage and Network traffic
10. GCP does not charge for ingress network traffic, such as migrating storage from another environment to GCP
11. If enabled in the Manager, customers incur costs for Metrics and Monitoring
12. After migration, workloads running on GCP incur costs according to regular Google Cloud pricing
13. Google Cloud Platform pricing calculator can be used to create an estimate of monthly GCP charges, including VM pricing
14. Migrations from AWS incur costs on AWS for EC2 instances launched by Migrate for Compute Engine on AWS, EBS volumes and usage, and network traffic