Mac Source Ports

 

I’ve started a new project, and it’s live and has been for a few months now, so I don’t know what took me so long to put something about it on this site but in any event here it is.

The site is called Mac Source Ports and, like you might imagine, it features source ports built for the Mac. The goal is to provide easy access to signed, notarized builds of source ports for the Mac, with instructions on how to use them. The apps provided on the site are designed to run on both Intel and Apple Silicon Macs at the very least. Some of them I’m responsible for building, packaging and updating, others are done by third parties and I’m just pointing to them.

The concept behind the site is “Play old games on new Macs” and, currently, I’m organizing things by game. So if someone comes to the site and says “I want to play DOOM 3 on my new 2021 MacBook Pro”, we’ve got you.

Here’s how this all came about:

So like you’re probably aware, a few years back it became my hobby to port id Tech engines to iOS and tvOS. It was a lot of fun, got me some very mild Internet fame and notoriety, and was just generally cool all around. I get about an email a week or so from someone who has stumbled across the work, either complimenting my work or asking for some tips to get past whatever hurdle they’re having with Xcode to get them going, or whatnot. My GitHub profile has followers and my repositories get starred by people literally all over the world. These games truly have global appeal.

And in all my interactions with the public, I don’t think I’ve had a negative one yet. I know the Internet at large can be a depressing place sometimes and maybe I’m just old and naive to think interacting with strangers is a good idea, but so far I haven’t had any issues.

One thing, though, is the number of people who write to me and ask me to send them the compiled apps and it’s disappointing to have to tell them that due to how the iPhone requires signed code, I can’t send them something that will work on their phone. I have a semi-canned response for people who ask me to send them a build, and on more than one occasion I’ve run into someone who tries to add the GitHub repo into whatever the App Store equivalent is when you have a jailbroken device since that uses the term “repo” as well.

I have had one or two instances where the person I’m talking with over a string of emails who is not a developer actually goes so far as to break out Xcode, which they’ve never used before, and actually gets through the process. This is sort of like walking someone through surgery over the phone, so it’s amazing when it works.

All of this is to say: it’s neat that I’ve made these things run, but the fact that I can’t put them on the App Store and can only distribute them in source code form limits the ability for anyone to actually run the ports.

Some of the ports I’ve done I did “from scratch” (as in, no one had this running on the device before – Quake II and DOOM 3 fall into this category) but several of them were based on ports that someone else had done or started. The id-developed ones like Wolfenstein 3-D and DOOM are known to be made from community source ports. The guy who did the original take on Quake (the first one) seems to have taken the original Quake source code – not a source port project – and worked with it directly.

At some point in working with the Beben 3 port that was the basis of the Quake III: Arena port, I realized it was derived from ioquake3. The code it used from ioquake3 was a few years out of date by then, so one of the things I did at one point was updated the common parts of the code (i.e., the parts not associated with the GL ES rendering) to the latest and greatest.

While I was in there I took the initiative to spruce up the Xcode project. I found it hard to believe at the time that the Xcode project hadn’t been touched since 2013, which this had to be 2018 or 2019 so we’re talking 5-6 years. A number of things didn’t work or build. One entire library had been removed and swapped with something else. The thing had been neglected. And I was so new to messing with these sorts of things that it took me a while to realize why: no one was using it.

The first class way to build ioquake3, and a lot of these projects, is the Makefile. I’ve railed about Makefiles here before but they work, and ioquake3’s Makefile is probably the best I’ve run across in so far as, out of the box, it builds just fine provided you have the libraries installed (basically just SDL2, and that’s just for Linux builds, it includes premade libraries for Windows and Mac).

In the course of fixing that up, plus a few Mac-specific issues that had cropped up, I became a member of the ioquake3 project.

It’s no secret that the Mac is not seen as a serious gaming option for the most part. The vast majority of PC games that have any sort of graphics horsepower are Windows-only. Games like Factorio are excellent but they don’t really require the kinds of 3D accelerator cards that cost hundreds of dollars if you can even find them.

There’s various reasons for this: Macs aren’t as customizable as PCs (really, outside of what you order from Apple, they’re not customizable at all). Mac gaming has never really been a force in the retail world, though that’s hard to judge really since there’s not really such a thing as a Mac-exclusive game (outside of some Apple Arcade entries). And the sheer number of times I’ve seen Apple’s deprecation of OpenGL interpreted as removing or breaking it is frustrating, though really it’s that developers don’t want to support Metal if they don’t have to same way they were nonplussed about Direct3D (another vendor/platform-specific 3D API).

But another big reason is: a lot of people don’t have Macs. And the more you head down the professionality spectrum, the more true this gets. Like, DOOM Eternal isn’t on Mac but I’m willing to bet that id Software can get as many Macs as they need if they wanted to. But I’ve seen multiple indie games that have a Windows and Linux port and no Mac port and the reason is simple: the (one) person who works on the project doesn’t have a Mac. Everyone has a Windows PC, getting Linux running in a VM or a spare or additional machine is trivial, but unless you specifically spend money to buy a Mac you’re out of luck. And the number of Mac gaming purchases doesn’t really inspire the need to invest in one either a lot of the time.

And this is especially true in the source port community. There are many projects that worked on the Mac at one point in time but “the person who handled that left” and no one’s taken up the reins.

And obviously I’m a Mac convert, I’m writing this on a Mac, I’m a big Apple fan but it bites me in the ass too. Apple announced in 2020 that they were switching processor architectures, again. Meaning, even if you have a Mac, or even a bunch of them, unless you bought one of these new Apple Silicon Macs, you, too were going to be in the situation where you can’t contribute to a situation due to a lack of hardware.

The Mac, then the Macintosh (and to some extent it’s still Macintosh it’s just that Apple’s pretty much exclusively called them “Macs” for decades now), launched in 1984 on the Motorola 68k processor line. In 1994, they switched to the PowerPC architecture. So that’s the second architecture now. In 2005, they switched to Intel processors, first 32-bit processors (x86 or i386) and then in 2008, 64-bit processors (x86_64). Many people consider the two Intel processors to be in the same line so Intel is now the third architecture. In 2020 they announced they were changing the Mac again to their own processors, with an ARM64 architecture, under the brand name Apple Silicon. Apple’s been making ARM64 processors for years, they run iPhones and iPads and Apple TVs. The first Mac chip, the M1, is essentially a souped up version of the A-line of processors they’ve had in mobile and embedded devices for years now. So, that’s now the fourth architecture for the Mac. And short of a bizarre circumstance, it’ll probably be the last one – Apple’s tendency is to reduce the number of dependencies as much as possible and bringing the CPU in-house is as reduced as you can get on that.

I found someone who had made modifications to ioquake3 to get it to run natively on Apple Silicon. Some of the changes made looked very similar to the kinds of changes necessary to get Quake III: Arena running on iOS. Mostly in the area of routing around Intel-specific code (notably, one section was still using MMX instructions of all things, which was a proprietary Intel thing). It seemed to me that this individual was disinterested in trying to get this merged into the main project so I took the changes (and gave him proper credit in the commit notes) and set about the business of making this be part of the main project proper.

Switching gears back again to Apple’s processor architecture changes, there’s been four architectures (provided you consider the Motorolas and Intels each collectively) so there’s been three transitions. I have no idea how the first transition worked, or even if there was one. Back when Apple started making computers the idea of making a computer that could not run the code of its predecessor was actually pretty common.

When they switched from PowerPC to Intel there were two main things they did. First, they shipped the version of Mac OS X for Intel Macs with a piece of software called Rosetta. As you might imagine, it translates PowerPC instructions to Intel instructions. Basically it allowed Intel Macs to run PowerPC apps. This way you could bring your apps with you while the developers made the transition. It was introduced in Mac OS X 10.4 (Tiger) and removed in Mac OS X 10.7 (Lion), which sounds quick but this was before annual releases, there were six years between these versions.

The other thing was something called Universal apps. Basically these were apps that contained both Intel and PowerPC code in them, so they could run natively in either location. Unlike Rosetta this concept couldn’t be removed or retired, it was just more of a natural trailing off seeing as how the number of PowerPC Macs in use declined over time.

I know I’m all over the place here but this ties into another reason why Mac game development is less popular amongst smaller indie devs and source port projects: developing for Mac is just weird sometimes.

When you install Quake III: Arena on Windows there ultimately winds up being this folder that has “quake3.exe” and a folder called “baseq3” that has the data files. The executable runs and knows to look for the “baseq3” folder and read the data. Simple.

Mac users are expecting something called “Quake III Arena.app” or something similar. The trick is, the .app file is not the executable. It’s an “app bundle” and strictly speaking, it’s not a file, though the Mac operating system sort of treats it like one. In the app bundle are a series of specifically named subfolders, and in those folders are the executable file and the data files (the “baseq3” folder, in this case), along with a data file that explains to the operating system things like what the executable is called and what the icon file is, etc.

Maybe Linux has something similar but it means you have to both make the thing build and run on the Mac but you’ve also got to do this whole packaging thing with the app bundle. Like many things it’s not that big a deal if you get the concept behind it but it’s One More Thing for people and organizations who might already be swamped with work.

Included in ioquake3 were a series of shell scripts that, at first, I thought were potentially redundant to each other but I realized they were all necessary – one of them could just run the Makefile with the necessary switches to properly build on the Mac. If you just fire off the Makefile it builds for whatever you’re running on but you need to hand it some parameters if you want to build for a different architecture. One of the scripts then handled the making of the app bundle. And then one of the scripts handled making a Universal app bundle for Intel and PowerPC. Each of these called the others.

I’ve said Apple has a very “fuck the past” attitude about things and it’s true that having a modern Mac with the latest operating system and tools can make it impossible to build for older machines. A few years back, Xcode dropped the ability to build 32-bit Intel executables, and years before that it dropped the ability to build PowerPC executables. Seeing as how 32-bit Intel Macs hadn’t been built since 2008 and PowerPC Macs hadn’t been built since 2005, it’s not an unreasonable cutoff timeframe, but it does mean that if you’re wanting to build for older machines you’re out of luck short of firing up an emulator or old hardware.

My first Mac was in 2009, a Mac mini that shipped with Snow Leopard (10.6). Over the years it qualified for a few Mac operating system upgrades but eventually it got to the point where it ran horribly, a combination of being loaded down with software and the natural inclination of newer operating systems to slow down older hardware, and really I hadn’t used it in years.

Well, just for the heck of it I dusted it off and basically nuked it and used install media to reset it to the 10.6 it shipped with. My goal was to get the Universal app version of ioquake3 building again but with the latest code. This was a journey – it’s quite the rabbit hole to go down. For one, the version of OpenSSL that ships with these things is too old to be able to reasonably access most of the Internet. You can upgrade OpenSSL with Homebrew but… you can’t install Homebrew because your OpenSSL is too old and the curl command to install it in the first place fails. I also did this in close proximity to taking another old Mac I have, an Intel 32-bit MacBook, which dual boots 10.4 and 10.6 and a lot of this runs together but I eventually found a web browser I could download that would run on the older hardware and also had its own version of OpenSSL recent enough to actually be able to download the Homebrew install script and then from there you can upgrade the version of OpenSSL on the machine, install git, etc.

Anyway long story short at some point I had it up and building the Universal app version of ioquake3 and I was able to get it simultaneously running on that 2009 Mac mini, my 2014 MacBook Pro, and the 2007 32-bit only MacBook at the same time in a very Frankenstein arrangement in my office.

We’re a happy family

So with that accomplished, now it was time to tackle Universal 2.

Universal 2 apps are Apple’s terms for the new versions of Universal apps that contain x86_64 and ARM64 code. The goal is the same – to give a way to provide a single app during the transition. The version of macOS that ships on Apple Silicon machine has Rosetta 2 as well which serves basically the same purpose as Rosetta on the original Intel machines – it runs Intel 64-bit code.

So after incorporating those changes to run on Apple Silicon (in such a way that it compiles differently based on architecture) and making a clone of the Universal script (the “ub” script) and making it build a Universal 2 app bundle (“ub2”), I had a process that would build a Universal 2 app of ioquake3.

The problem, however: I had no Apple Silicon Macs to test it on. That thing I’ve been lamenting this whole page? It happened to me. And for various reasons I wasn’t in a position to buy one.

We found some folks online and gave them the app to test out on Apple Silicon machines. The result? It crashed, horribly and cryptically. The kinds of messages that you might be able to unpack with your own hardware but I didn’t have it. It was basically debugging, tossing it over a wall, and finding out that the thing you tried still didn’t work.

Then the leader of the ioquake3 project suggested I crowdfund an Apple Silicon Mac. The reach of the ioquake3 project’s social media could probably make it happen. It felt weird but I set up a ko-fi page, essentially set the target to the cheapest possible option (an M1 Mac mini, baseline model) and a few donations came in from time to time. It wasn’t the quickest process but it was coming along. Then I did something that pushed it over the edge.

About 3,000 words ago (seriously, WordPress counts this shit) I mentioned signed and notarized builds. Some folks may know what that means, some might not. OK, strap in:

Go far back enough and computers were the wild west. You found a program, you downloaded it (or installed it from disks/discs) and you could run it. Simple. Easy. And also, prone to problems – like viruses and spam and malware. So at some point the concept of code signing came into play. Code signing is a little bit complicated but not overwhelmingly so and it essentially means: you use a certificate to “sign” the code and then when someone else downloads it they know it was from you and it hasn’t been modified or altered. Oh, and if you go rogue and start being an asshole and put malware in your own apps, then your certificate can be blacklisted and forbidden to run. And in closed off operating systems like iOS you can make it to where unsigned code can’t run at all.

Anyway up until one point code signing on operating systems like Windows and macOS was a “nice to have” but not really required or anything. To this day you download something, you try and run it, and Windows says “wait are you sure? We can’t find a signature” and you say “whatever just run it”.

Then in OS X 10.8 (Mountain Lion), Apple introduced a concept called Gatekeeper. Gatekeeper had three modes: you could let it run anything (so, essentially turning it off and having things run like before), you could let it run anything that had been code signed or was downloaded from the Mac App Store (which, in 2012 when this came out, wasn’t very popular – it’s a little bit better today), or you could let it only run apps from the Mac App Store (which is closer to what iOS does). The second mode was what was on by default. It wouldn’t outright stop you from running anything but it would force you to go into Security in the Preferences pane and authorize it. Essentially providing some friction before an unsigned app can run.

This should prompt an automatic *sigh* in pretty much everyone who sees it

The good news is signing isn’t that hard. In the case of Gatekeeper you need to sign it with an Apple Developer certificate which you can get by having an Apple Developer account, even a free one (as opposed to the paid ones which cost $99 and let you publish to the App Stores). But, most indie games or source port projects don’t do it.

At some point Gatekeeper removed the first option (the one that essentially shuts it off). People have speculated for years eventually they’d cut off the second one too and turn macOS into iOS but there’s a multitude of reasons I never see that happening, not the least of which is Apple repeatedly says it’s not going to happen.

But if you did sign the code then instead of the roadblock above, you’d get this one-time thing if you downloaded it off the Internet

Sym what now?

Easy enough, right?

The problem though, as you might imagine, is that signing code is nice and all but all it is is a way of saying “this person signed this code”, but there’s no guarantee the code is safe.

So starting in macOS 10.15 (Catalina), Apple started enforcing this thing called notarization. Notarization essentially means your app has been submitted to Apple, checked for malware, and been given the OK in the form of a “receipt” you can include in the app bundle.

This is different than submitting something to the App Store. Besides the fact that it doesn’t involve publishing to the App Store, it doesn’t involve humans at all. The App Store requires a literal human being to look over the app, which could take days. The notarization process involves submitting the code to a server that essentially runs a virus scan on the code.

So for all apps compiled after a certain date (that way old apps that will never be updated again are grandfathered in) they need to be notarized or else, even for signed code, you’ll get the “can’t be opened” dialog.

OK, so what’s the big deal? Well besides the fact that this is another part of the already kinda complicated process, notarization requires a paid developer account. For a certain level of project complexity, the combination of the two is a bridge too far.

But I have a paid developer account (it’s how I published Disasteroids 3D). So one weekend I dug in and decided to figure out how to sign and notarize via a script.

I was successful. At the end, we now had a Universal 2 binary for ioquake3 that was signed and notarized. You could now run it with no bullshit in macOS. We published it on the ioquake3 site and made posts and tweets about it and everything.

Well this proved to be quite popular because it caused a surge in crowdfunding donations to the point where I could buy the M1 Mac mini. And I was able to verify: it worked.

It turns out the thing was bitching about code signing all along. The Apple Silicon version of macOS is a bit more strict than the Intel builds when it comes to that. Especially with regards to embedded frameworks.

One of the things that makes Mac development tricky is that the library situation can get hairy. If your game needs SDL, it needs to be able to find it somewhere. One way is it for to be installed via whatever mechanism your operating system uses for centralizing libraries. If you go to the SDL site, they have a disk image you can open and it’s got the SDL2.Framework file/folder and a text file showing how to install it, but for some number of end users this is going to be too much hassle. You can also install it with a package manager like Homebrew but again, for some number of folks this is going to be a bridge too far. Once you’re talking about command lines, like what Homebrew requires you’ve lost a good chunk of your audience.

You can instead embed the libraries and frameworks inside the app bundle itself. You can then tell the executable to look for them there. It can be a bit on the tedious side but it’s do-able. More recently though I discovered a utility called dylibbundler which essentially automates the process – it looks for what the executable is looking for, it copies them where you want them to be, an then modifies the executable to know to look for them relative to the executable path.

At this point, since the Mac gaming community saw fit to help me buy a Mac mini, it only seemed fair not to stop here.

Quake III: Arena was covered, naturally more Quake games seemed natural. The hottest source port of the original Quake appears to be vkQuake, authored by a former id Software employee that swaps out the OpenGL renderer for one written in Vulkan. Vulkan is, on some level the successor to OpenGL, though it has problems of its own, but macOS does not officially support it, favoring Metal, which is Apple’s proprietary API. However, an open source project called MoltenVK works so well that vkQuake can run on macOS with the instructions provided. However, the result is an executable and not an App Bundle, so I wrote a script to make a Universal 2 build of that and sign and notarize that as well.

So all that was left of the Quakes was Quake II. I did similar build script magic with the Yamagi Quake II project.

At this point, or possibly after doing DOOM 3: BFG with the RBDoom3BFG project, I started to like the idea of making a central point to distribute these things.

I can’t really put these on the Mac App Store. For example with Quake III: Arena, I’m obviously not id Software so I can’t put it on there myself. Conceivably, ioquake3 could be put on the Mac App Store, with the caveat being you have to provide your own data files, but since that’s a process that involves having a human being having to review it I’m not sure if that would work making them track down data files themselves. And I probably couldn’t call it “ioquake3” since that has “quake3” in it, so the discoverability would be diminished. And that’s assuming the source port project would be ok with all of this to begin with.

There are other distribution methods, like the aforementioned Homebrew, but Homebrew is a command line utility aimed more towards developers and power users. They do have some source ports available but they’re still susceptible to the same signing and notarizing issues that most source ports have since Homebrew’s authors don’t maintain these things, individuals do.

And none of these cover the main discoverability hurdle: most people have no idea what a source port even is, they just know the name of a game they want to play. You might figure out from ioquake3’s name that it plays Quake III: Arena but you might not know to look for that name. You might not guess that a source port named Raze can play Duke Nukem 3D, or that a source port named DevilutionX can play Diablo.

So that was the angle I was going for. I wanted some place where normal people can go with their brand new M1-based Mac (or slightly older Intel Mac) and say “I want to play Quake II” and find it and be pointed to a source port they can download, run, and be able to run without having to go into their System Preferences and authorizing it manually.

And either because I thought this was the best way to do it, or because I’m now considered an old man who doesn’t know how things work any more, I decided the best way to handle this was to make a new website. I decided to name it Mac Source Ports.

Here lately I find myself fighting with WordPress more than I want to, plus at my full time job I’m using Angular to make a web project, so I decided to use Angular to make a custom site as well. It’s possible that this might outgrow being a custom site but for now it works.

It’s an interesting change in gears. Making games run on iOS/tvOS is fun and the feedback I get is satisfying, but it’s a deal where you’re always fighting against the concept that this is not supposed to work. These games were written for computers that have OpenGL and mice and keyboards, I’m trying to get them to run on something without any of those. And I can’t easily share my work with anyone who isn’t inclined to use development tools. But these source ports often have macOS support out of the box, either through the original authors or from community maintainers, they just don’t always have someone willing to do the work of getting them packaged up and signed and notarized. So instead of fighting the project I’m just using them as designed and I’m able to share the results with others.

And sometimes the projects do the work themselves. I as all geared up to make GZDoom a signed and notarized Universal 2 build, but the GZDoom project already does that so it’s not necessary to do it myself. But since I still want my site to help people who search by game name, I added DOOM to the site and linked to GZDoom’s latest release. I developed a badge to denote when it’s a MacSourcePorts build versus a third party build.

The original idea I had behind the site was simple – make a bunch of entries for games and then link to the source ports. Quake is vkQuake, Quake II is yquake2, Quake III: Arena is ioquake3. Easy. But of course it gets a little trickier from there.

Originally I just had an entry for DOOM and linked to GZDoom. But GZDoom can run other games as well like DOOM IIFinal DOOM, HexenHereticStrife, etc. Maybe people could guess it can run DOOM II, but they might not know about the others. So, I added multiple entries for the various games. They all point to the same source port, but they’re individual entries. I don’t know how many people will ever say “man I want to play Hexen” and come to the site but if they do, we’ve got that info covered. I figure there will be a “many to many” relationship since more than one source port can play a game (in addition to GZDoom there’s the predecessor ZDoom which is discontinued but still runs on PowerPC Macs) and a source port can sometimes play more than one game. Right now I have everything listed by games, in the future I plan to make a listing for the source ports so the info can be seen from the other direction.

And the site is also designed to show Universal 1 builds as well. We don’t always have Universal 1 ports and the goal is to have Universal 2 at the very least but where there is one I figure I’ll provide it. I may at some point have to offer non-Universal apps for the occasions where the Apple Silicon ports don’t work but I’m resistant to the idea.

I made the logo be a line drawing from the Noun Project of a 1984 Mac and a very stereotypical joystick

The logo is “wrong” in a couple of ways. First in that few of these source ports are the kind anyone is going to play with a joystick but second in that the Mac is an old Mac (the oldest actually) and the tagline of the site is “Play old games on new Macs”.

The reason I went with it was that I tried logos that used modern things like a Mac mini or a MacBook Pro or an iMac and I couldn’t find one that didn’t seem like it could also be a line drawing of an average laptop or computer monitor. They didn’t make it immediately obvious that it was a Mac. Part of the reason was that the drawings I found for use did not use the Apple logo (i.e., on top of the Mac mini or the chin of the iMac screen) for copyright reasons. Same reason I didn’t want to use them. But the 1984 Mac is iconic, it communicates the point immediately. It’s the equivalent of the floppy disk icon still being used for “save” years after the floppy disk went the way of the dodo – it’s not an icon so much as a idiom.

I’ve done all the things that I figure someone needs to do when they launch a venture like this – I registered domain names, a GMail address, a Twitter account, an Instagram account, started a Discord, etc. I think I signed up for Facebook but Facebook sucks so I don’t really care. I’ve done a smidge of “advertising” in the form of posting on message boards and Reddit, and occasionally someone stumbles across it, but I haven’t made a bigger push since I want to get more games on there. Traffic has been brisk, though probably still in the range of rounding errors for more significant websites traffic. Not entirely unlike this site.

This might sound dumb but I figure Mac gaming isn’t taken that seriously by a lot of people. A number of things would need to happen in order for that to change. People can attack the problem from a few different angles. This site is my angle.

And in any event, it’s neat to be able to offer up my work as something you can actually download and use. We’ll see where it goes next. Check it out and let me know what you think.

Categories: