Win32 on macOS

https://www.winehq.org/pipermail/wine-devel/2019-December/156602.html

by yarapavan • 7 years ago

486 130 7 years ago

Okay that’s just impressive. I was wondering how they would adapt for 32bit support on Catalina.

kccqzy• 7 years ago

Speaking of code segments, the big thing that Catalina provides that enables this all to work is the ability to create 32-bit code segments in a 64-bit process. For that, they enabled the use of i386_set_ldt() in 64-bit processes. The big caveat though, is that this functionality is restricted by System Integrity Protection (SIP). For now, your best bet to get this working for yourself is to disable SIP. (CrossOver doesn't require that, but the mechanism by which we accomplish that is in flux internally to Apple. When it settles down, I'll update this thread.)

This is quite curious. Apple inconvenienced so many users by "removing" 32-bit support on Catalina, and then they added support for local descriptor tables? I'm not aware of any use of LDTs on modern systems (except perhaps Chrome for additional sandboxing).

toyg• 7 years ago

Maybe they plan on doing a big show and dance when they “settle” the issues around SIP: “macOS now allows you to run classic apps and games!!!!”

blinkingled2• 7 years ago

Apple has some weird history with doing crazy things with 32b/64b stuff just because they can - back in the Intel 64 early days they ran a 64-bit kernel that loaded 32-bit device drivers. I don't know if anyone has still figured out the how and why of it!

brigade• 7 years ago

They didn’t load 32-bit drivers in a 64-bit kernel; rather they ran a 32-bit kernel that supported 64-bit userspace processes.

throwaway8848402• 7 years ago

This is quite curious. Apple inconvenienced so many users by "removing" 32-bit support on Catalina, and then they added support for local descriptor tables? I'm not aware of any use of LDTs on modern systems (except perhaps Chrome for additional sandboxing).

For the specific purpose of enabling CrossOver's 32-on-64 support and things like it.

It's not too hard for the kernel to support user code merely entering the processor's 32-bit mode. CrossOver's processes don't even use 32-bit system calls or 32-bit Mach-O binaries, so support for those could hypothetically be removed, except that most of it is needed for watchOS anyway. But the most significant cost of i386 support was in userland, not the kernel: building separate x86_64 and i386 copies of every system library, and supporting the legacy i386 ABI, including an older version of the Objective-C runtime ABI. That's all gone now.

asveikau• 7 years ago

I am not privy to internal discussions, but it seems to me if there is really lots of ongoing maintenance needed at every release, and not simply exaggerating the need for occasional patches and bug fixes, then it is worth some consideration of re-architecting the way the compatibility mode works.

eg. Maybe draw the boundary of where you thunk at a different place. Maybe do more translation in user mode as this wine project did. Or focus on binary compatibility with old, frozen-in-time frameworks instead of rebuilding them with every OS build. Obviously the specifics will vary with actual details of breakage. But from outside, it seems like these type of questions weren't pursued to their maximum extent, resulting in pain for the end user.

joshumax• 7 years ago

I was only around at Codeweavers for a part of this huge undertaking, and only contributed a small part of the 32-on-64 MacOS patchset, but I must say this was one of the most impressive feats I've seen coordinated using mainly nothing but email and IRC for communication.

Everything from thunking layers to calling conventions to inline assembly in the Wine codebase had to be redesigned for this release of Wine; an absolutely massive proposition. Ideas such as dynamic binary translation and running Win32 executables in the MacOS hypervisor framework were considered, and some ideas I tried while hacking Wine gave me insight into other projects I hack on such as Qemu and LLVM.

In short; props to everyone who worked on this! It was a long time in the process!

saagarjha7• 7 years ago

nothing but email and IRC

But? Those are both extremely capable communication mechanisms.

r1nkgrl• 7 years ago

Exactly. When people are motivated to work together, even poor communication tools are adequate. But these are excellent communication tools.

hoistbypetard3• 7 years ago

I read "nothing but email and IRC" to mean "without in-person meetings" as opposed to "without more capable communications tools".

bonzini2• 7 years ago

running Win32 executables in the MacOS hypervisor framework

Interesting, why was this discarded? Performance reasons? Or the API not being satisfactory?

Spooky232• 7 years ago

How long will it exist?

Wowfunhappy4• 7 years ago

Presumably performance. At that point just use Virtualbox/VMWare.

heelix• 7 years ago

The Codeweaver folks are really great. I worked with one of their engineer's brother, which is how I can to discover this local (for me) company. The Linux version of WINE really shines too. They track, fix, and give back to the open source project while taking care of all the setup, etc. Ironically, a Win95 Garmin GPS simulator (airplane instrument) worked lovely on their kit... while it would not work on the 'run as' on Windows 7.

I've got their OSX version as well. There was a day where I needed Visio at work, and that would let me launch a Windows bottle. Funny enough, it also ran quite a few steam games... back before there were solid OSX ports for what I'd play.

19963• 7 years ago

I find it marvelous you can reliably use software 24 years later. And we talk about binaries, not sources. And we talk about software still with business uses.

The original storage media may be corrupted but the software lives!

My personal belief is Wine will stand next and maybe above the GNU project, the Linux kernel, the QEMU emulator and the GIT version management as the best things ever made by free software.

yitchelle2• 7 years ago

> ...coordinated using mainly nothing but email and IRC for communication.

I wonder what other projects (on a scale larger than your typical side hustle projects) are coordinated via IM chat or email?

On the surface, the Linux kernel coordination seems to be coordinated like this, but I am not 100% sure.

throwaway2048• 7 years ago

Basically every open source project that isnt "Group of devs at the same company" works like this.

Open source projects are likely the largest user of mailing lists.

ownagefool• 7 years ago

I'd argue it works for devs at the same company quite well, but having asynchronous communication is counter to your average business persons desire to lock you in a room for an hour and make you make a decision.

dewey3• 7 years ago

Every open source project that’s reasonable enough not to hide all their communications in proprietary silos like Slack and Discord.

jayflux• 7 years ago

Ok I’ll bite..

Seeing that both slack and discord save logs from the beginning (which irc doesn’t), isn’t communication less hidden if anything? It’s far easier for someone new to play catch up.

There’s a reason large open source projects moved to these (React, Rust, etc).

jononor• 7 years ago

Jayflux, most open source IRC channels are logged on webservers. Those logs are usually also searchable on Google, and you don't need to sign in to view them. Unlike Slack etc

mattip• 7 years ago

PyPy works primarily on #pypy, with yearly sprints and additional coordination on the PyPy-dev mailing list

arboghast• 7 years ago

When I was at Canonical back in 2007-2008, we were mostly communicating using email and IRC. I had to adapt but ended up finding it very productive.

st3fan2• 7 years ago

Many components of Firefox are.

cjbprime• 7 years ago

The kernel doesn't actually use IRC for coordination. There's a channel but technical discussion is rare.

dang• 7 years ago

We detached this subthread from https://news.ycombinator.com/item?id=21788357.

steve19• 7 years ago

Does apple have a good reason for killing 32bit support? I play a number of games that will now never run without a VM.

musicale• 7 years ago

The obvious development reason is to simplify and unify the code base, tools, and build process across all of their platforms. 32-bit bugs can also be ignored/deleted for future versions.

The obvious hardware reason on iOS is that Apple can remove support for ARM32.

The obvious business reason is to encourage subscriptions to Apple Arcade.

jaflo• 7 years ago

How is Arcade related to 32-bit?

oefrha2• 7 years ago

Probably a step towards the eventual arm64 transition.

chrisseaton• 7 years ago

Simplifying the software and hardware. Seems a good thing to clear out the cruft? Use a VM if you need to emulate ancient systems.

mhd2• 7 years ago

Killing it in the way they did? Not a dire technical one. In the past they managed to keep compatibility while switching operating systems ("Classic") and CPU architectures (68k -> PPC, PPC -> Intel). And that while being a much smaller company.

This is business, nothing else. Just like all the increased "security" restrictions. Corralling everyone towards the App Store. I think they've got too much legacy to ever totally cut off "regular" applications, but apart from that they're doing everything they can to move everything in the direction of iOS (or iPadOS now).

Not sure how long the days of the last commercial Unix will last.

soraminazuki2• 7 years ago

I'd guess a translation layer would require much more effort to develop and maintain than just keeping 32 bit support. It wouldn't make much sense in this case because dropping support is the sole objective, not some side effect caused by architectural/OS changes.

pvg• 7 years ago

Killing it in the way they did? Not a dire technical one. In the past they managed to keep compatibility

Killing it in what way? They kept compatibility for years and eventually ended it, just like the transitions you mention.

This is business, nothing else

Both the security changes and dropping 32 bits are technically reasonable things to do. You might not like them and that's perhaps not entirely unreasonable either but the notion that they were done for business, nothing else makes little sense. What business do you have in mind? Apple makes money selling iPhones.

throwaway8848402• 7 years ago

In the past they managed to keep compatibility while switching operating systems ("Classic") and CPU architectures (68k -> PPC, PPC -> Intel).

The Classic Environment (OS 9 -> OS X) was dropped after 6 years.

Rosetta (PPC -> i386) was dropped after 5 years.

i386 -> x86_64 is now being dropped after 12 years.

Kwpolska• 7 years ago

The Classic environment was retired in 2007 with the release of 10.5 Leopard, and was never available for x86 Macs. With 10.7 Lion (2011), they dropped support for 32-bit x86 Macs, and running PowerPC software via emulation (Rosetta). Eight years of support for 32-bit apps is quite long by these standards.

scarface749• 7 years ago

Apple has supposedly been “corralling every one” to the Mac App Store for a decade. Isn’t it more likely that the conspiracy theories are wrong?

goatinaboat• 7 years ago

Remember on 68k they also had to orchestrate the transition from 24-bit to 32-bit addressing.

ben509• 7 years ago

This is business, nothing else.

That's entirely the case. Apple has to weigh the cost of maintaining compatibility against the value it brings.

And it's hard to see how if the developer can't make a business case for updating their software that Apple can make a business case for maintaining compatibility.

That's the business reality of Apple's primarily consumer market: consumers are generally willing to let old software go and buy new stuff.

Microsoft is simply in a different market: they have many large enterprise customers who are willing to pay top dollar for extensive backwards compatibility.

4ad2• 7 years ago

My speculation is that it's for the upcoming arm64 transition (yet another speculation). It's easier to make one emulator, than two.

em500• 7 years ago

https://pilky.me/apples-technology-transitions/

Section "Why Drop 32 Bit?" (last quarter of the article).

zmzrr2• 7 years ago

"when you open a 32 bit app, 32 bit system libraries remain loaded forever!" okay how about you fix that instead?

tempodox2• 7 years ago

This is a great feat and it creates a somewhat ironic situation: When I want to run an app on Catalina that happens to be 32-bit only, I'm practically forced to use the Windows version. I can't see how that was a smart move on Apple's side.

andai• 7 years ago

Could some of the wizardry described here be used to get native 32-bit software working again?

tempodox2• 7 years ago

As I understand it, this would only be possible if you recompiled the software with the modified Clang they use for the Windows apps (and the underlying tech is in a state of flux). I'm not sure that would be more expedient than rebuilding your app as 64-bit.

saagarjha7• 7 years ago

The system frameworks you’d depend on don’t have a 32-bit slice anymore, unfortunately. You’d need to wrap every function in a think to make things work.

blablabla123• 7 years ago

Not sure how long this would work but the trick is to copy the 32 bit shared libraries over: https://apple.stackexchange.com/a/374038

It's apparently straightforward to install - an old - macOS in a VM. This can then be used to obtain the dylibs

dewey3• 7 years ago

The writing that 32bit apps are going away was on the wall for a long time. If you still rely on that (maybe for some enterprise application) then you are probably not going to upgrade to the newest version right away anyway.

It’s like removing the floppy drive, there will be a bunch of people complaining (probably the 1%) who still use it but in the end everyone is better of without it and it will be replaced by something better.

Almost nobody of the general users would know a difference between 32 / 64 bit as they just use new apps or use web apps. It’s a very small group that’s affected by this change.

oefrha2• 7 years ago

The writing has been on the wall for a long time but some big name software companies are slow to adapt, and even if they adapted you'll probably have to pay good money for a new license.

Take Mathematica for example. The first 64-bit version is 12.0, released April 2019. So a year-old permanent v11 license is useless on Catalina now. It would have cost me a couple hundred bucks to upgrade my v9 permanent license if I didn't happen to have an institutional license.

inferiorhuman3• 7 years ago

Find me a perpetual license for a version of Adobe Lightroom that has a 64-bit installer and I'll pay for it. But I'm not going to upgrade to a leased tool. Luckily the perpetual Lightroom licenses work on Windows too, so when Mavericks loses support I'll probably migrate at least the photography workflows to wine (on not-macOS though).

scarface749• 7 years ago

Why would Apple cafe if you run Windows on the Mac?

IggleSniggle2• 7 years ago

It weakens your incentive to buy a Mac the next time you purchase a computer if you find yourself forced into a different OS anyway.

Razengan• 7 years ago

It also relies on a new feature of macOS Catalina to allow the creation of 32-bit code segments in a 64-bit process.

How was this never brought up during all the "Catalina killed 32-bit" pitchforkings? Seems like a very important point, that has enabled the ability to emulate 32-bit apps.

saagarjha7• 7 years ago

It’s gated by an undocumented entitlement.

exikyut2• 7 years ago

I don't know how to mentally parse this sentence. (Not snark; stumped.)

4ad2• 7 years ago

Since wine is using it, it seems it's not gated enough.

soraminazuki2• 7 years ago

AFAICT, this won't let people run existing 32 bit apps out of the box. Instead, it enables the creation of 32 bit stub libraries that can call their 64 bit counterparts in behalf of the application. Although it's possible in theory to run existing 32 bit apps by creating stub libraries for 32 bit APIs, it's likely to require so much effort that I doubt this is going to happen.

nly• 7 years ago

It also relies on a new feature of macOS Catalina to allow the creation of 32-bit code segments in a 64-bit process

Is this roughly equivalent to MAP_32BIT on Linux?

http://man7.org/linux/man-pages/man2/mmap.2.html

burfog• 7 years ago

No, it is like modify_ldt on Linux.

http://man7.org/linux/man-pages/man2/modify_ldt.2.html

notaplumber• 7 years ago

Why OpenBSD/amd64 will not, and never has supported 32-bit binary compatibility on 64-bit.

https://marc.info/?l=openbsd-misc&m=148926149318522

Win32 on macOS

Related Stories