Mac Source Ports Progress Report: April 28, 2022

I’ve added a build of j2mv to the site, it’s a multiplayer client for Jedi Knight II. The OpenJK project had a single player client for JK2 but not one for multiplayer. Daggolin was able to figure out the trick to getting OpenJK working on arm64, he was able to apply similar logic to j2mv, which I incorporated.

I’ve updated DevilutionX, the source port of Diablo, to the latest code. They had a formal release of 1.4.0 and this includes that, plus some stuff off the main branch. I’m not sure what my long term strategy should be yet, I think at some point I’ll want to make builds based off labeled or tagged commits but I’m not sure the best way to do that. As in, for DevilutionX they have a commit they tagged as 1.4.0 and then there’s like 60+ commits after that. My build includes the 60+ commits after that which, so long as they didn’t break anything, is probably fine. Maybe at some point I’ll have a “stable release” and “latest code” thing happening.

My focus at the moment is to try and get as many games/ports as possible going, with an eye on automation down the line. At some point I need to get releases packaged into .dmg files instead of just zip files of the app bundles. Not sure if this would be less likely to trigger the “this file is not commonly downloaded” thing Chrome does when (I assume) the zip file contains executable code for the first X number of downloads.

Someone pointed out that the dhewm3 bundle for DOOM 3 did not feature the code to run the Resurrection of Evil expansion pack. I rectified that and did a new upload.

Iliyas Jorio let me know that the fourth and final Pangea Software game he maintains, Mighty Mike, is now signed and notarized so I added it to the site as well. I wasn’t sure if I should have listed it as Mighty Mike (Power Pete) or not – it was released in 1995 as a commercial CD-ROM title from Interplay/MacPlay under the Power Pete name, in 2001 Pangea regained the rights to the game but not the name so they re-released it as Mighty Mike via shareware. Maybe I have my timeframes mixed up but I’m kinda surprised to hear there was still shareware happening in 2001.

And then on something of a lark, despite still trying to get ES3 happening, I decided to do The Ur-Quan Masters, aka Star Control 2. I’ve never been into this game but it’s one of those titles where it’s frequently on the lists of best games ever made, and subsequent attempts at sequels or remakes haven’t really been a hit, sort of a RoboCop effect.

I’m always fascinated by backstories like this – the developer is Toys for Bob which, unlike a lot of developers from the 90’s is still technically around, though for a while there they were stuck in licensed game hell. In any event, Star Control II was ported to the 3DO where they used the then-novel CD-ROM technology to add speech and different music to the game.

Apparently the source to the original DOS version is completely lost, but the 3DO port’s code was still available so they made it open source. Then at some point they made the content for the game be freeware under a creative commons license. The one thing they didn’t own and couldn’t release was the Star Control name, which is in some legal limbo clusterfuck that continues to this day, so the project is The Ur-Quan Masters, which was the subtitle of the original game.

Since the game uses very little if any 3D graphics (there’s some level of OpenGL in there but it’s primarily a 2D game) I figured it would come over to Apple Silicon fairly easily. And for the most part it did, with a couple of hitches.

Most of the source ports I work with use either Make or CMake to handle builds, with probably 2/3 or more of those being CMake. I’m still figuring out the best practices there but I’m coming around to liking CMake projects the most because they’re much closer to uniformity with what works.

There’s some outliers – a few projects use nonstandard stuff, like Ninja. The DXX-Rebirth ports used SConstruct which I’ve never even accidentally run into anywhere. The Ur-Quan Masters, however, uses its own homegrown build system. It’s not that bad, it is to some extent a scripted wrapper around Make, and they have another script that builds an App bundle, not entirely successfully but nothing I can’t work around

The problem is it has a menu system. You have to select a few things by specifying numbers, and even that’s no big deal except it requires intervention and I can’t automate it. Star Control 2 came out in 1992 and The Ur-Quan Masters started in 2002, so it’s a 20-year-old source project for a 30-year-old game. Maybe they’d do it differently today but a number of today’s practices either didn’t exist or weren’t mature in 2002.

Anyway without a ton of hassle I had it doing a Universal 2 build and even included the data files so it can be the full game (according to the FAQ on the website, this is permissible). But the problem was – the sound was fucked up on Apple Silicon. The graphics were fine, the sound had an issue.

There’s two or three libraries included in source code form that appear to be sound-related. My first thought was perhaps one of these has an updated version that works with Apple Silicon so I could just copy over newer code, but the UQM devs actually left a note to future developers that these were custom versions of these libraries, so that was out – no telling if a new version of the code would work at all.

Then on a Hail Mary hunch I figured – ok, so if it’s working on Intel Macs and screwed up on Apple Silicon Macs, could there be some sort of code in these audio libraries that’s being configured for Intel. So I did a search for “x86_64” and found two places doing an #ifdef thing. I added arm64 to the list of defined variables and it worked. Essentially it was two places that had to define a typedef for 64-bit machines and this code predates arm64 (or at least Apple Silicon). The website for the port says the 0.8 version released late 2021 was the first release in almost a decade.

In that vein, I added a new “Full Game” badge to source ports on the site to indicate releases that include the game data or at least don’t require you to download or configure anything out. Looks like right now with the Pangea titles, the Marathon titles, Star Control II and a few others, there’s a dozen full game releases on the site.

Mac Source Ports Status Report: April 16, 2022

So just to post something, here’s what I’m in the midst of with regards to Mac Source Ports.

I have the ioquake3-based game World of Padman building but it crashes when it connects to most servers. It appears the authors provide their own builds for Mac, but they’re the result of an automated process. Like a number of source port projects they don’t have someone involved with Mac experience. They’re working on a new version soon so I may hold off and try again when that ships.

I caught the attention of a Jedi Knight fan site named JKHub and someone from the site asked me to take a look at jk2mv which looks to be a port of the multiplayer for Jedi Knight II. As I’ve done the other Jedi Knight open source games and also ioquake3 it seems straightforward and I have an Intel build working but there’s a snag on M1 I’m trying to work out.

I’ve got a list I’m maintaining of the source ports I want to do, and I’m doing them in phases/passes. First pass, anything that builds into an app bundle (so, an “.app”) and that I can get going on Apple Silicon fairly quickly gets done. Second pass, anything that builds and runs on Apple Silicon and is cognizant of the Mac but isn’t doing its own app bundling, third pass, anything I can get to build that runs but clearly has no Mac-specific considerations. I’ve deviated from this a little bit (System Shock/Shockolate and Rise of the Triad/rottexpr would really be third pass ports) but I’m trying to prioritize. The holdups on those deviations were they didn’t have any code in them to look for data outside of some place like a subfolder of the executable location. It’s not hard to put code in there to look under Application Support, but it’s more work.

Back when I first announced the site on Reddit someone asked if I had a Discord. I didn’t but with a few taps on my phone, I did. It was absolutely an afterthought, but it’s turned out to be a surprisingly active thing, we have over eighty members now and some of the more active ones have gone and found more games to port or link to, so shout out to my Discord community.

One thing I was surprised to see had the potential to build out of the box for the Mac was the Serious Engine. Back in 2016, Croteam released the source to Serious Engine 1, which powered the first two games in the Serious Sam series, Serious Sam: The First Encounter and Serious Sam: The Second Encounter. Those games have been remastered/remade multiple times now but the original files work with this release. I haven’t completely worked through what’s going on yet but I was able to get it to build, however the exact arrangement of data files for the two games is a little weird. By default it wants to build the second game, but you can also have it build the first game, but then the first game won’t run without some files it appears are only part of the second game? It’s weird. But it’s got potential.

The big fish I’m working on right now and one I’d love to get going on the site is The Elder Scrolls III: Morrowind, courtesy of the OpenMW project. Morrowind was never open sourced but these guys have created an engine that can run it anyway. Lots of projects like this get started, very few make it to a playable state, OpenMW is an exception. I know people have had it running on Apple Silicon before and I have it building but I can’t get it to run, and out of the box the Intel version through Rosetta has issues (though, when you mess with stuff like this enough sometimes it’s not clear if it’s because there’s an out of the box problem or if you’ve messed with something and some preferences file hidden somewhere on your system is gumming up the works). Anyway I’m going to keep picking at it.

At some point I’m going to need to explore the possibility of letting either Intel-only games on the site or games that are maintained but not signed and notarized. Obviously the goal would be to get everything up to the state of being Universal 2 but at some point you have to think – what is the more important goal, having everything up to this rigid standard or providing access to source ports for Mac owners even if they’re not perfect?

For example Warzone 2100. This is a game that was released in 1999, open sourced in 2004, and then its data released as public domain in 2008. It has a fantastic open source project that has a slick, professional website with downloads for macOS, Linux and Windows, and it’s even a Universal 2 app. Should be a no-brainer to add them to the site as a Third Party Build.

So what’s the holdup? They’re not signed and notarized. Their app is Ad Hoc signed but it’s not signed by an Apple Developer account or notarized. These guys are so close to being perfect that instead of trying to do my own build I reached out to them. Naturally it’s not like they weren’t aware of the notarization thing, but the holdup seems to be: there does not appear to be a way, currently, for an unincorporated entity to get a paid Apple Developer account. You’d either have to get it in someone’s name or incorporate as a legal entity. Someone’s name means someone would have to get the account and no one on their project can or is willing to do this (and I’d volunteer but then I’d have to share my Apple credentials) and incorporating as a legal entity is a costly proposition for a non-profit project.

Indeed, all the signed/notarized builds on MSP that I make are signed by me, Tom Kidd. But if you didn’t know about me you’d check the signature and see it’s just “some rando” who signed them. I looked at the other third party builds I’m pointed to and yeah, all of them are just “some rando” as well. In some cases it makes sense (Iliyas Jorio personally maintains the Pangea builds so his name on them makes sense) but it’s sort of a flaw in the concept since if you’re firing up, say, dhewm3 it’s just signed by some random person.

The one exception I found was OpenTTD, they actually have a signed app whose signer is named “The OpenTTD Team” but it’s because they incorporated, in Scotland apparently, as OpenTTD Distribution Ltd. So, a legal entity.

As a side note, I was initially surprised that an Ad Hoc signature meant anything. So, there’s unsigned code where it’s not signed at all, and then there’s code signed with a certificate, like an Apple Developer certificate, and in the middle there’s Ad Hoc signed where you’ve run the codesign utility on it. I think Xcode just does this for you automatically, or it can. A big change with Apple Silicon was that on Apple Silicon you can’t run unsigned code at all. It has to be at least Ad Hoc signed (and even as I type that I’m not sure I have that right).

Thing is anyone can Ad Hoc sign something. There’s no proof of who did it or why you should trust it. So why is it a requirement now for Apple Silicon? My best guess is that, in addition to the fact that signed code (with a certificate) can tell you who signed it, it also tells you it hasn’t been tampered with since being signed. An Ad Hoc signature doesn’t leave an audit trail but you’re at least reasonably certain that it hasn’t been tampered with since whoever signed it did so. At least that’s my best guess.

My current thought process is: I want to get more stuff on the site, then I can consider letting in exceptions. I may need to reconfigure how the little cards work too, make it more clear that something is signed or just Ad Hoc signed, or is Intel-only, or whatever.

Granted if it takes as long to get Morrowind going as it did Ion Fury I may bend that rule a little quicker, we’ll see.

Mac Source Ports

On the one in a million chance that there’s someone out there who still reads my blog, or is still subscribed to the RSS feed despite me not posting in *checks notes* years, and does not look at the front page, here’s a post to point my latest venture: Mac Source Ports.

The short version is I’m using it to post signed and notarized app bundle builds of video game source ports for the Mac to run on Intel and Apple Silicon processors.

The long version is here.

So that’s the first reason I’m making this post.

The second reason is: I’m actually going to use the categories on this site to categorize thing. And by categories I mean category: one category for Mac Source Ports posts. I’m going to link to it from the main Mac Source Page.

I figure a blog to say what I’m working on and where I’m at with things would be relevant.

Why not just put that on I guess I could, but for right now I’ve got that thing hosted somewhere it would be nontrivial to say “and run a PHP site in this subdirectory, oh and can you even do databases?”

Right now is entirely custom code. I don’t know how long term feasible that is or how long before I’ll need to suck it up and move to a proper CMS, but for now it works and I can do anything I want with it, although it’s a touch on the tedious side.

Anyway, I’ve tried using this as a platform to put my thoughts out there and I’ve kinda fallen off of that, then I tried using this as a platform for my professional works and, well that kinda fell off too. So either this latest attempt at using my blog will really, no really, prove to be useful, or I’ll fall off of it, too. We’ll see.