And an update on the base.iPack file.
First things first, the work has now been added to the same repo as before, so here’s the link for all of the above, but it’s the same one as before.
DOOM, DOOM II and Final DOOM for iOS and tvOS
I chose to add targets to the existing repo rather than make new ones. Now there’s six targets, two each for DOOM, DOOM II, and Final DOOM. It would probably be possible to make one app that could launch all three games on the given platform but I like separating them out this way.
But before that, let’s talk about that base.iPack file.
The deal with that file was, if you’ll recall, that at the end of the process of getting DOOM working, I realized one of the files necessary to get it to run wasn’t included in the source code, and it wasn’t a file that you could retrieve from an installation on a PC because it was unique to the iPhone version.
They did include an additional project, “doomtool”, to create it but you’re still in the dark on what is required to make it – like sound files, graphics, etc. I went ahead and posted the updated project and the detailed the process of fixing it in the blog post but it was a bit of a cloud over the thing since anyone who wanted to play the game had to source a copy of this file from somewhere, and it’s not available in the brightest spots on the Internet.
After posting the repos and articles to Reddit, I started hearing some more feedback, including from some of the people whose work I incorporated as part of the process. It’s all been very positive and everyone’s been thrilled to see their work be used elsewhere.
In particular, JadingTsunami, whose work was very helpful in getting DOOM working, got in touch. He saw the article and offered some assistance on the base.iPack thing.
To recap, base.iPack is a memory-mapped file containing images and sounds whose initial purpose was to make DOOM perform well on the hardware of the day, namely the original iPhone and the iPhone 3G (those two phones effectively had the same hardware, processing-wise, the 3G having advances in other areas like cellular speeds and GPS). The graphics in the file seem to be unique to the iOS port (the on-screen buttons, directional pad, etc.) but the sound effects were pulled from the WAD file.
The best way to make DOOM compile right out of the box would be to make a “clean room” base.iPack file – one that replaced the copyrighted graphics with new ones, and that provided its own sound effects, probably pulled from the WAD file directly. As provided, doomtool didn’t do this. I figured I would need to make new graphics, come up with some sort of code – existing or new – to pull the sound effects from the WAD file and then make them work for the base.iPack file, and the whole thing seemed daunting.
Then JadingTsunami got in touch. He verified to me that the performance need for base.iPack to exist was long gone – there was no reason for it to provide memory-mapped sound effects. He then shared with me some code he wrote to traverse the iPack file. Essentially this utility would extract the sound effects in PCM format and then a separate script would turn them into WAV files to work with. However, in the long run he wound up not needing the output of it at all because he modified the code to just look for them in the WAD file directly and ignore what’s in the iPack file. And since I had grafted in his work, I vicariously got this code, too. It turns out the base.iPack file that came with the game was missing DOOM II sound effects and had some odd modifications to a couple of the originals so it’s better this way.
So then it was on to the graphics. The utility he provided me with didn’t extract the images but since it was derived from the code that traversed the iPack file in the engine itself, I figured I could expand on it to extract them. The code to extract the PCM files (which came out with the .wav extension since the index in the file uses the input filenames regardless of what they wind up as) was basically extracting chunks of the file based off of a calculation of the size using length, number of samples, channels, etc. I figured I could do something similar to extract the TGA files.
Yeah, no. Long story short at various points in the process I had the glob of data that I’m guessing the engine could use to render the graphics on the screen, but writing it out as a TGA file didn’t work. Or at least, it would produce TGA files that no operating system or graphics package had any clue what to do with. On scrutinizing the code in doomtool, I realized that while it was taking the WAV files, turning them into the PCM data, and then adding that to the iPack file, the stuff it was doing to the input TGA files was Herculean in nature. It was doing all kinds of manipulation, including making them square even if they were uploaded as rectangular because apparently iPhone OS 2.0 had issues otherwise, and doing all sorts of things with the color and pixel data. To extract those images I would have to basically perform all the steps backwards, and I’m not even sure if that would even be possible (like if it did some one-way conversions of the sort to where a conversion in the opposite direction correctly would be impossible).
So I wound up doing what JadingTsunami suggested in the first place – just have the utility output the dimensions of the images and come up with replacements from scratch. I came up with a list of the graphics and then made appropriately sized placeholder replacements (a white background, a black “X” stretching the corners of whatever size it was, and the filename across it in red so I’d have a clue in-game which one I was looking at).
I created replacement graphics from FontAwesome icons and The Noun Project, which have free resources for use with attribution, which isn’t a problem for me. Some of the filenames were intuitive (I bet the fire.tga file is the Fire button, etc.) some were not intuitive, and not even all of the files were referenced from the code. I think perhaps all of the files had been referenced at various points, but the older version of the game where it did less with Cocoa Touch and more with in-engine rendering probably used some graphics from the iPack file that proved unnecessary later.
So then armed with the from-scratch iPack I loaded it up… and it crashed as soon as I fired a gun. So, as it turns out, while the code is getting the sound effects from the WAD file, it’s still actually loading them up from the iPack file, it’s just not using them. But it still needs them to be there. I could probably have gone in and taken out the code that still looks for the wav file and changed it to not need to do that anymore, but I came up with a “pencil in outer space” style solution instead: I just created a one-second silent WAV file, made however many copies I needed to and then renamed it to be the filenames the engine is expecting. Once I built one using that everything worked – and the correct sound effects were loaded from the WAD file.
The only thing left was the font file. This was an interesting little journey.
One of the files in the iPack file is called “arialimagelal.tga” which appears to be an image of just text characters. I say appears because like I said I can’t extract it, but one of the files in the project is clearly an array of structures whose properties map out to coordinates within that image. When a message is drawn on the screen like “You got the shotgun!” or “You found a secret!”, the code uses these structures to determine how to draw the text on the screen.
Well, without the original image file I was either going to have to recreate it or create a new file with a different font, and also figure out what the coordinates are.
I was going to write up the whole, long process of how I figured out how to do it but really it’s not that interesting. What is interesting is the aha moment towards the end and the tool I used.
I’m a regular participant in the Chatty community at Shacknews.com. One of the regularly contributing members is a guy named Thom Wetzel, who I’ve met at QuakeCon a couple of times. He’s contributed a range of community projects, including an external voting apparatus onto the Chatty itself, sort of a meta third party Reddit voting system.
Anyway, back in the year 2000 he came out with a program called Bitmap Font Builder. Basically it constructs a TGA file of text characters, the sort of thing you can then map coordinates onto. He wrote it in Visual Basic 6 and it’s been used by dozens of games since then. He’s received credit in game manuals, game development books tell their readers to go out and download it, and despite not having been touched since 2006, he still gets updates from folks who have used it.
So what better time to conscript it back into service than to fill in a gap in my DOOM iOS project? I downloaded and installed it and it still installs and runs just fine in Windows 10 x64, complete with an install wizard that would be at home in Windows 3.1.
So I started messing with it, at first using the optional grid that the app can provide. The first efforts were unusable gibberish but I started to get to where it looked like some sort of ransom note
I used a public domain version of Chicago, the font used by the original Macintosh (because I thought it would be hilarious) and after some trial and error and some Excel wizardry with the INI file of font widths the program can also give you, I had something workable.
And so the end result, after some scaling, works for me. I played around with making the text red like in the original DOS game but it seemed too hard to read.
So then it was on to the other part of this addition to the project – the sequel and the sequel-ish
For a while there id Software followed a pattern – shareware game followed by retail sequel. I don’t completely follow what the releases of the Commander Keen games were, but Wolfenstein 3-D was followed by Spear of Destiny. Quake was followed by Quake II (and later Quake III: Arena, which broke the pattern, also Quake II had broken the pattern of both games using the same engine), and in the middle there was DOOM followed by DOOM II.
The shareware games were episodic, which was seen as necessary in order for the model to work – Wolfenstein 3-D had six episodes, the first was shareware, free to distribute. DOOM had three episodes, the first was shareware (it sounds like Wolfenstein 3-D was the better value, but there were much fewer things a level could do so they were much more involved in DOOM).
The retail games were not episodic because they didn’t need to be. So whereas the original DOOM game was three episodes of nine levels each, DOOM II is just 32 levels in one go, no episodes.
When the source code for DOOM was released, what they basically released was the source code that would run both DOOM and DOOM II, there was not a separate release for DOOM II‘s source code. I don’t completely recall how it worked but I believe the way they distributed the original games, the DOOM executable couldn’t run DOOM II levels and vice versa, but once the source came out it became a “one executable to rule them all” sort of thing.
So consequently when DOOM Classic launched on the iPhone, the source port that Carmack used – PrBoom – had the ability to run DOOM II levels the whole time, it’s just that id never released anything that could. With Wolfenstein 3-D on iPhone, they later released the ability to buy the Spear of Destiny sequel’s levels as an in-app purchase, as well as give it the ability to run custom levels, but they never did the same for DOOM. The fork that JadingTsunami made had replaced some screen in the app, like the credits menu I think, with a screen that would load up different WAD files, so clearly I wasn’t the first to want to run DOOM II or later in the app.
I settled on the approach of making additional targets within the Xcode project, one to run DOOM II on iOS and another to run DOOM II on tvOS. Since I wasn’t trying to switch platforms could duplicate the existing targets and just rename them. I modified them for the naming scheme – so the “DOOMtv” target became the “DOOM-tvOS” target, and named the new ones “DOOM2-iOS” and “DOOM2-tvOS”. I decided to use one more preprocessor macro to determine which game was which, both because I needed code to sniff out which WAD file to load up, but also because when you launch DOOM II, you need to skip the episode selection screen because DOOM II doesn’t have them.
The original port didn’t use image asset catalogs, mostly because they didn’t exist back then, so I had adhered to that as much as possible but for this update I did switch some things over to use asset catalogs, mainly because I could then use a trick of sorts – different asset catalogs for the different targets with the same image names. So when the DOOM port runs, it uses the “logo” image which is the DOOM logo, and when the DOOM II target runs, it uses the “logo” image which is the DOOM II logo. I did have to wind up using multiple different launch screen storyboards because for whatever reason I couldn’t get this trick to work with them.
In that vein, as much as I did the work to make the base.iPack files be “clean room” files free of copyrighted material, I did cheat a little and use the actual logos for the games in the icons, opening screen and splash screens. Seeing as how I used them in the original ports and it didn’t cause any issues, and the ports still require you to supply your own copies of the WAD files, hopefully no one will mind. Plus they’re just too awesome to not use, but I did try and use them sparingly.
The issue with moving to asset catalogs is now it’s no longer a simple matter of replacing the placeholder images with the originals, so I decided to make the placeholder images be a little better. Since the menus in the original game used a “cursor” in the form of a brown skull with blinking red eyes, I decided to make the placeholder buttons be skulls with eyes that glow red once you’ve highlighted something. I also figured out the best way to use that Chicago font everywhere in the game – I had replaced the file itself with the font back when I published the ports originally but because I didn’t change the name of the font within the font file in the code (the name of the font within the code is a separate thing from the font filename itself) it didn’t take before. I factored it out in the code to all come from the app delegate so if you do want to switch it back to the original font – or anything else – all you have to do is change it in one place.
And so for one last addition, I added a third pair of targets for Final DOOM. In some ways, Final DOOM is the most fascinating of the three to me, mainly because I had completely ignored it back in the day, yet it’s important enough to be available as its own separate release on Steam and GOG, and apparently a version of the game was even released for the PlayStation.
Back when DOOM came out it predated the ubiquity of the Internet and web browsers so some enterprising individuals would take tons of levels and put them on CD-ROMs and sell them in software stores. This was bad for a few reasons, besides using trademarks and copyrights for DOOM without rights or permission, the levels themselves had authors and copyrights and none of them were being honored either – these authors were not being compensated for their levels being on a commercial product.
To some extent, id was caught between having an open attitude about modification of their games – to the extent that DOOM explicitly didn’t lock any of it out – and not having the legal resources necessary to go after everyone. So at some point during the lifecycle of DOOM II they did the “if you can’t beat ’em, join ’em” thing and released a product called Master Levels for DOOM II. It had basically two parts – a selection of curated levels chosen from some of the best the community had to offer, and a shit ton of other levels on the disc of varying degrees of quality. It was actually sort of tedious because when you launched it, it would launch a DOS-based frontend wherein you picked which levels you wanted to be in the game. DOOM and DOOM II levels had to specify which level they were (or, which level they were replacing) so if the level you chose was designed to be MAP02, then when you would fire up the game, MAP01 would be the normal first level and then MAP02 would be the custom level. If you did it right you could replace all the levels in the game with custom levels and blow through them (incidentally the change from DOOM levels following the ExMx convention to DOOM II following the MAPxx convention was a big reason why you had to pick one or the other when making your level).
So there was that. There was also a retail release of the original DOOM under the title The Ultimate DOOM, which was just the original game again with a new fourth episode, “Thy Flesh Consumed”. This is the version you’re purchasing today from Steam and GOG. It’s also why when you own both it and the 2016 DOOM you don’t have two entries named “DOOM“ in your Steam library because the 1993 DOOM is listed as The Ultimate DOOM (and as far as I can tell the only two games on Steam with literally the exact same name are 2006’s Prey and 2017’s Prey and if you install them both at the same time dogs and cats start living together)
There also appears to have been separate releases of both The Ultimate DOOM and DOOM II for Windows 95, which presumably used DirectX or something and I remember at the time being a big deal that they could run in 640×480 instead of 320×240.
Point being, there were a lot of DOOM releases once id got a publisher. And if you had purchased DOOM shareware on floppy disk, you didn’t get the fourth episode of The Ultimate DOOM unless you bought that, too. And when Windows 95 versions of these games came out, you didn’t get them unless you bought the games again (the boxes on Mobygames even have “$10 Rebate Upgrade Offer!” printed on them in the corner). I can’t say I blame id or their publisher seeing as how piracy of the originals was so rampant, but this means that when yet another retail release came out, and less than a month before the launch of Quake, a lot of people like myself just ignored it, so I find it probably the most interesting of the three since it’s effectively new to me.
Final DOOM consists of two WAD files, each of which had 32 levels, and each was designed to basically be a complete replacement for the levels in DOOM II. Final DOOM was a standalone title, it didn’t need DOOM II, but it basically shipped with DOOM II‘s executable and when you launched it you chose which of the 32-level WAD files you wanted. The levels weren’t made by id Software themselves, they were made by map groups in the community. The WAD named “TNT: Evilution” was from a group called TeamTNT and the WAD named “The Plutionia Experiment” is credited to the Casali Brothers, reportedly a couple of guys from TeamTNT.
In any event, adding a second set of targets for this was actually pretty easy. I just re-used the “episode” structure of the original game to choose which WAD file to load, and PrBoom actually lets you load in WAD files more or less on the fly so I didn’t have the issue of the engine being committed to one set of data and not being able to switch like I did with Quake.
So all that was left was making the branding and icons. I chose one color for each of the three games and just stuck with it. I did have to do some trickery to make the “II” of the DOOM II logo be split out for the purposes of the animated tvOS icon but it worked out pretty well actually. I did wind up removing the circular thing from the tvOS logo since I wanted to minimize what I was using from copyrighted resources but I think the results wind up being pretty slick.
Also, I redid the top shelf banners, and for the Final DOOM one I used this rather badass and unsettling art they used for the advertisements for it back in the day.
So anyway, that’s what I’ve added. It’s a minor update but it lets you build the games without needing anything other than the WAD files, and now you can play all three major titles.
If anyone wants to contact me I can be reached at firstname.lastname@example.org.