I don’t know how anyone can afford these migrations especially for production on prem workloads without building literally duplicate sets of hardware clusters then manually migrate workloads.
We usually reuse the VMware hardware and (most importantly) file storage. Some additional hardware is required temporarily so you can build out initial Openshift nodes. The VMware nodes are decommissioned and converted to OSV nodes as the conversion goes along. With some kinds of file storage (cough NetApp) the conversion is zero copy, the VM literally stays where it is. With others we will copy to new NFS storage areas which will be provisioned on the same physical hardware.
The 40k servers are probably made up of multiple redundant vSphere clusters with failover. You simply take one of those redundant clusters and migrate one half of it over. Then the other half. Then duplicate that process. As you build more compute in the new stack, you can decomission more and more of the old stack and convert it. The transition would progress like a cascade, with larger and larger groups of clusters being migrated at once until you're left with the one-off, ad-hoc, weirdo clusters at the end that need to be manually migrated (usually with great effort).
The actual hardware servers are clustered together into pools of resources. The pools are where the VMs live. The bigger the new pool becomes, the faster you can empty the old one. So the migration starts very slowly, ramps up quickly, and then tapers off.
> You simply take one of those redundant clusters and migrate one half of it over.
For that half you are migrating, you are essentially operating without redundancy. If these are serious production workloads, the tradeoff is not as simple as you make it seem.
Ha I have done migrations recently from vSphere to vSphere using vMotion and it was easy.
But it still took duplicate set of HW and I couldn’t imagine doing it without a lot of IaC and automation in place (plus physical space, power and cooling)