ARM Macs and Virtualization: It's going to be great

The transition from Intel to Apple Silicon Macs is a huge deal, but some are lamenting this shift as a loss to developers, especially those who rely on virtualization such as Docker. The concern is that under x86 emulation instead of hypervisor, the performance hit could be significant.

As the case may be, my take is the issue wouldn’t be that serious, while the upside for developers is huge, especially for those who primarily develop for Apple’s ecosystem. And if my hunch is correct, that ecosystem will expand to server-side soon enough. Let me explain.

Just to be clear, I’m a huge fan of Docker, and I’m a ML practitioner working primarily on server-side within Linux, so I’m very familiar with the importance of the development machines being consistent as server-side. Funnily enough, I’d say this is the biggest rationale for Apple going ARM for Macs, which is to make the development environment exactly the same as their deployment targets, namely iOS, iPadOS, Apple Watch, Apple TV, and soon, MacOS.

That is to say, the current situation is that Xcode development is done on x86, emulated in x86 simulators, and re-built for ARM for device deployment. This has been the state since the first iPhone and Apple has done a superb job maintaining the seamlessness of this process ever since (as far as I know), but more edge cases are beginning to crop up.

Edge Case or Tentpole?

From my narrow view of ML, the one I experience is the lack of emulation of Apple’s Neural Engine (ANE) for accelerating model inferencing. Once a model is compiled to Core ML, the emulator would execute the model on CPU and possibly GPU, but its behavior is different on device when offloaded to ANE. It’s not only a performance difference, but also floating point differences and possibly quantization nuances.

What this means is to truly test a ML model’s behavior on Apple devices, one has to deploy to actual devices, likely via a cable to enable debugging. As you guessed it, this also means that you’d have to have multiple physical devices if one is to test against different models. Multiply this with different iOS versions to test against, it quickly gets out of hand. If Apple wants to (and likely plans to) add more custom hardware, the situation is untenable for both Apple and its developers.

Virtualized Apples?

With ARM (and other custom silicon) on the development machines, we already know iOS apps will run natively so there’s no need for emulators. I wouldn’t be surprised if these apps are running within individual virtualized instances, to as closely as possible create the same behavior as actual devices. Apple could go as far as virtualize the hardware resources and capabilities so performance would be emulated faithfully also, greatly simplifying the development, testing, and debugging process for developers.

For ANE, this switch will be huge. People have had to deal with “numerical differences” when running ML models on CPU compared to ANE, a process that is slow to debug via tethered devices. With the Mac having the same hardware as the target devices, there’s consistency and no hidden surprises. Whenever the development cycle is shortened and opaque differences removed, it’s a good thing.

What about the downside of Docker becoming 2 to 5x slower without hypervisor? While that is indeed a downside, I’d argue that for local Docker instances, they are better used for functional testing and not part of the core development cycle. Coming from the Python world, I have had no problems developing software in native OS and only use local Docker for validation prior to deployment. The other main use case for Docker of bringing up server clusters locally would be slower, but again, shouldn’t be core part of the development cycle. There’s always the option for remote Docker or even better, deploy to sandbox and integration environments for testing.

ARMed Cloud?

That brings me to the last point about the ARM transition for Macs, which is the aspect of developing for server-side, not an area of focus for Apple. However, if Docker and virtualization needs are primarily for server-side development, and to be consistent of being consistent, the problem of x86 emulation can be dealt with if Apple offers ARM-based cloud servers. Having this uniformity would truly enable developers to write code on their ARM Macs, test across devices locally, and deploy to iOS devices and server-side, all from one code base, without translation, emulation, or hidden gotchas. Pretty sweet landscape for Apple developers if you ask me.

Closing

Ever since I bought my Titanium MacBook Pro many years ago, I have been a fan of Apple’s combination of software and hardware, but more importantly, its long term approach to their product developments. This ARM Mac transition has been long in the making, and I, for one, am excited again to buy an ARM MacBook and the future it’d usher in.