Posts on felsqualle.com

Saving the Masters of the Elements From Getting Lost to Time: Epilogue.

Sun, 19 Oct 2025 22:30:00 +0200

Every journey comes to an end, and this is the end of a very personal journey that started back in May 2025 . A lot has happened since then: I uncovered what breaks the game, how to fix it , bought all available versions of the game I could find (including some quite expensive shipping from Japan to Germany), did some last-minute fixes , and discovered some secrets .

Thanks to the help of my incredibly talented friends from the ScummVM project, and thanks to everyone following me on this adventure over the last couple of months, I finally proudly present: The patch files required to play “Masters of the Elements” on Windows 2000 and higher, up to Windows 11.

Enjoy!

Note: All downloads listed below only contain the changes necessary to fix the compatibility issues. An origial copy is required to play the game. Without the original game, these files are worthless.

Edition	Patch	Data directory
Masters of the Elements	melements-win-en_patch.zip	MVM
Masters of Elements: Unmei no Shihaisha	melements-win-jp_patch.zip	(none)
Le Maître des Éléments	melements-win-fr_patch.zip	MdE
Meister Zufall und die Herrscher der Elemente	melements-win-de_patch.zip	MVM
I Signori degli Elementi	melements-win-it_patch.zip	MVM
Meesters van Macht	melements-win-nl_patch.zip	MvM

Installation

Download the patch set that matches your version.
Copy the contents of the data directory from your disc to your hard drive.
Copy the contents of the Xtras directory from the patch files in your new game directory.
Copy the file scripts.cst to data/global/, and delete the scripts.cxt in the same directory.
Copy the file intro.dir to data/intro/, and delete the file intro.dxr in the same directory.
Start the game with the executable (MVM.EXE or MDE.EXE).
Have fun!

README & FAQ

I can’t find the data directory on my disc

On most of the releases, the data directory is hidden by default. Enable the “Show hidden items” option in Explorer to reveal it.

When copying the data directory to my hard drive, my PC locks up on the folders ZDAT, ZYDAT, or ZZDAT

Some releases of the English and German versions of the game use a copy protection called “copy-X”. This copy protection uses a “blank” area on the disc that contains absolutely no information. In the data directory, you’ll find a (hidden) folder called ZDAT, ZYDAT, or ZZDAT containing about 100 MB of data. And here’s the trick: The data is not there, but the files are “mapped” to that blank area, causing your CD reader to hang. To check if your disc is affected, visually inspect the data side of the disc: If there’s a visible ring (about 1-2mm thick) at the outer area of the disc, then this disc is protected.

To mitigate this, open the data directory and copy everything except the dummy folders: The game never checks these files.

The game runs in a tiny window with a very large black border. What can I do?

When you start the game, you’ll notice that the game is rendered really, really small, especially when using a larger screen resolution. The game is designed around a screen resolution of 640x480 pixels, with a color depth of 256 colors. Since the game cannot change its resolution, it relies on helper applications that do the resolution switching. Unfortunately, these helper applications won’t work anymore. The perfect solution for this problem is a small utility called Res-o-Matic by bCheck. This tool allows you to select an application and specify the resolution it should run in. Then, it will create a shortcut file that switches your display to the chosen resolution, launches the application, and returns to the original resolution after the program exits. Really convenient! I used this tool for the entirety of this series, so I can’t recommend it enough. Thank you, bCheck!

At startup, I get an error message regarding temporary file access with error code -34

This issue mainly occurs on the French and Italian releases, and on my version of Meesters van Macht (which is a re-release primarily marketed in Belgium). The game is notoriously susceptible to breaking whenever anything else on your system is trying to access the game data files the moment the game itself requests access to them. Unfortunately, the affected versions bundle one data file required by the game (CHECK.DXR) in the main executable.

The problem: When trying to start the game the first couple of times, Windows Defender will reliably check the main executable, thus blocking read access to the embedded CHECK.DXR file. For the French and Dutch versions, you can safely ignore this error message, since by the time the intro screen from the publishers finishes playing, access to CHECK.DXR is released, and the game continues just fine.

The Italian version, however, has no additional intro screen, so it tries to directly jump into the (blocked) CHECK.DXR file. One thing CHECK.DXR does is to determine if you are running the game on Windows or Macintosh, and if this check fails, you’ll get some very bad side effects, like broken audio and missing speech output. If you see this error message on the Italian version, the game will break, so you have to close the game and start over.

To mitigate this, I recommend excluding the game directory of the affected versions from your antivirus and any backup/cloud synchronisation software. What also worked pretty reliably for me was to wait a couple of minutes after copying over the game files to let things settle down. YMMV.

The Japanese version shows a text box with garbled text

That’s normal and will happen if your machine is set to any other locale than Japanese. You can safely ignore this error.

The Dutch version complains that my system is not set to 256 colors

You can safely ignore this warning and continue to play the game.

In the Italian version, the colors at the beginning of the Room of Gravity are messed up

Yes, and right now, I don’t understand why this happens. This is one of those bugs that should either happen on all versions or not at all. I’m currently investigating.

Will the game still work on Windows 95 and 98?

Yes, though I can’t guarantee that there are no color issues on some configurations. I recommend to stick to the unpatched version.

What’s next?

Well, for this project, this is it. I will look into the remaining issues related to parallel file access and the graphics issues in the Italian version, but right now, it is time for me to take a break and focus on something else. I still have some hardware from 1999 sitting around here that’s waiting to see a datacenter from the inside again for way too long, but that’s a story for another time.

…to be continued… one day.

But there is one more thing.

…to be continued. For real.

Saving the Masters of the Elements From Getting Lost to Time: Part 5

Sat, 04 Oct 2025 10:40:00 +0200

While working with the game files of Masters of the Elements , I found something exciting hidden in one of the script files: Multiple, undocumented cheat and debug codes.

Well, they were undocumented up until now, so without further ado, but a hefty spoiler warning, let’s see how we can bend the laws of time, train bats, or get infinitely lucky.

I broke down the debug and cheat codes into three sections: Common codes, room codes, and debug codes. The common codes are primarily used for navigating between the different rooms. The room codes, however, will trigger specific actions that are present in this room only. The debug codes are interesting, but not really useful except for development purposes.

To use a code, all you have to do is type it at the correct location (or everywhere if no location is specified). There is no visible input field or cheat mode you have to enter before typing the first code. Capitalization does not matter; for documentation purposes, I left them as they are written in the game files.

Common cheat codes

Code	Effect	Needs master control
willy	Enable master control code.	no
allPages	Immediately collect all 5 pages that are hidden in the rooms.	yes
allStories	Immediately unlock all story pages in the entire book.	yes
noStories	Remove all story pages from the book.	yes
intro	Go back to the intro screen	yes
goGrav	Enter the Room of Gravity	yes
goWarm	Enter the Room of Warmth	yes
goTime	Enter the Room of Time	yes
goElec	Enter the Room of Electricity	yes
goLight	Enter the Room of Light	yes
goChance	Enter the Garden of Chance	yes

Room codes

Code	Effect	Location	Needs master control
IJsfontein	Enable/disable two player mode in the juggling game. The blue hand can be controlled using the Ctrl and Alt keys.	Room of Gravity	no
Bombilla	Enable/disable aim assist when trying to catch the flies.	Room of Light	no
Merlyn	Re-shuffle the locations of all windows in the tower.	Room of Light	no
twoHands	Enable two player mode in the juggling game. The blue hand can be controlled using the Ctrl and Alt keys.	Room of Gravity	yes
oneHand	Disable two player mode in the juggling game.	Room of Gravity	yes
Spiral	Immediately finish the electric loop puzzle. You may have to change locations within the Room of Electricity in order to see an effect.	Room of Electricity	yes
Poort Gate	Open the small gate at the end of the electric loop puzzle. Both cheat codes work exactly the same.	Room of Electricity	yes
Train	The train holding the battery will immediately appear.	Room of Electricity	yes
Co	Summon your companion, even if you haven’t solved the juggling game yet.	Room of Light	yes
Pan	Immediately catch the bat and start your journey to the top.	Room of Light	yes
BigBen	Toggle between night and day mode. You will have to wait up to one minute until you see an effect.	Room of Light	yes
Mirror	Automatically orient all mirrors to their correct location.	Room of Light	yes
boom	The railway gate will open at the next drop.	Room of Time	yes
fuel	The train to the Room of Electricity will depart soon, all aboard!	Room of Time	yes
plant	The plant will grow by one step. But be careful, it will die very quickly too!	Room of Time	yes
fire	Immediately ignite the stove.	Room of Warmth	yes
boat	Skip all mini-games.	Garden of Chance	yes
lucky	Use the cheat code, press and hold the Ctrl key at the very last choice you have to make in the game, and your luck will increase to infinity.	Garden of Chance	yes

Debug codes

Code	Effect	Needs master control
showmem	Show memory information.	yes
removemem	Hide memory information.	yes
serial	Show internal serial number. Purpose unknown.	yes
machineslow	When starting the game, it will calculate your current system speed and make some adjustments to it dynamically based on the results. This mainly affects drawing and animation speed, so slower machines are not overloaded. The higher the “slowness” value, the slower the game runs. Not relevant on modern machines, based on testing, the “cut-off” speed is somewhere above a Pentium 133 MHz where no slowdowns are introduced.	yes
setms1	Restore the playback speed, simulating a system that is powerful enough to run the game at full speed	yes
setms2	Set slowness factor to 2	yes
setms3	Set slowness factor to 3	yes
setms4	Set slowness factor to 4 (almost unplayable)	yes
setms5	Set slowness factor to 5 (unplayable)	yes
unloadcast	Appears to trigger a function that unloads cast members.Function call seems to be broken, no visible effect. Do not use.	yes
directsound	Show DirectSound information.	yes
SoundOff	Disable audio output.	yes
SoundOn	Enable audio output.	yes

…continue reading Part 6 of this series…

86Box Now Supports the SafeDisc Copy Protection for CUE/BIN Image Files

Wed, 03 Sep 2025 23:30:00 +0200

While recently going through the latest commit messages for the incredible PC emulator 86Box in preparation for their release of version 5.0 , I stumbled across commit 8944c92 with the following commit message: CD-ROM: Parity and CRC checking support, System Shock 2 now works with a directly mounted .CUE image as well.

My initial thought? “Wow, they just added support for the SafeDisc copy protection, this is awesome! I can play The Sims and Age of Empires 2 now!”.

Well, the games themselves were already compatible with 86Box, but it was the SafeDisc copy protection that rendered the games unusable.

Let me explain.

Wait, what is SafeDisc?

Whenever I’m writing about “SafeDisc protected discs”, I am strictly referring to CD-ROMs. While there are also SafeDisc variants that support DVD, they only use the ATIP and virtual drive checks. Therefore, the code changes discussed here will have no effect.

SafeDisc is a copy protection technology that was popular in the late 90s to the early 2000s. Coincidentally, two of my favorite titles, The Sims (including all their expansion packs and later “Deluxe” and “Complete Collection” editions) and Age of Empires 2, are protected with SafeDisc as well. Each time the protected application or game is started, SafeDisc performs various checks to ensure that the original disc is inserted into the user’s computer. If SafeDisc detects the original disc and passes all authentication steps, it decrypts the program data and launches the application. If authentication fails, the program won’t start, and you’ll receive an error message, typically asking you to insert the correct CD-ROM into your drive.

The term copy protection is very accurate: SafeDisc was effective at preventing copies written to blank CD-Rs, but not very effective at preventing dumping.

Depending on the version of SafeDisc, the tool incorporates various technologies to prevent copies:

Intentionally bad sectors that cause read errors
So-called “weak” sectors that exploit flaws in the EFM encoding
ATIP detection being able to recognize CD-Rs
Detection of virtual drives

The detection of virtual drives was quickly bypassed by drive emulation software, allowing it to “hide” the virtual drive from the rest of the system, making SafeDisc unable to detect it.

The ATIP detection tries to read the ATIP information of the disc. The ATIP is an area that sits “before” the data area of a (re)writable disc. It contains information about the characteristics of the disc, such as the type, manufacturer, and additional data that helps CD writers calibrate their writing parameters. If SafeDisc detects that you are running the application from a CD-R, it will refuse to work. To bypass the ATIP check, you must either use drive emulation software or a CD-ROM drive that can only read, but not write, CDs.

The “weak” sectors are an interesting phenomenon. When reading the disc, the weak sectors will read just fine like any other sector. However, when you try to write the image to a CD-R again, your CD writer will likely either refuse to burn the image during the process or the copy will not work. Before the data is sent to the laser that actually burns the data onto the disc, your CD writer will have to go through a step called “EFM encoding ”.

The data within the weak sectors is laid out in a way that tries to confuse the EFM encoding by creating special patterns. To this day, making copies of the weak sectors is not easily possible with modern drives. During the authentication process, SafeDisc will try to read the weak sectors and refuse to work if they can’t be read correctly. Please note that the weak sectors only affect writing new copies, but not reading the original disc.

If you are interested in all the details regarding the weak sectors and how EFM works, there’s a great article about it available online .

Intentionally breaking data

To recap, we are - in theory - already clearing 3 out of the (up to) 4 checks SafeDisc implements:

The virtual drive detection is not effective since we can emulate multiple drive models, and due to the full-system emulation, all applications will see a valid drive.
The ATIP check will always pass since we are creating the image from a valid, original disc, so there’s no ATIP information to pass to SafeDisc.
The weak sectors will always read fine since we are not working with a burned disc that’s potentially affected by the bad EFM encoding.

The most important part of SafeDisc, starting with the very first version, is the introduction of intentionally bad sectors. Reading CDs is hard. You are reading data with a laser from a thin metal layer with tiny grooves in it, through a thin sheet of clear plastic. That’s not reliable! To mitigate reading errors through smudges or scratched discs, CDs contain multiple levels of error detection and (to a degree) correction.

For demonstration purposes, I’ll work with a CUE/BIN dump I created from my original German Age of Empires 2 disc, which has a matching entry in the redump.org database .

When creating a dump file from a disc protected with SafeDisc, you’ll notice some read errors that will always show up, usually in the first 10000 sectors:

[LBA:    841] C2 error (bits:  312)                          
[LBA:    844] C2 error (bits:  312)                          
[LBA:    866] C2 error (bits:  312)                          
[LBA:    869] C2 error (bits:  312)                          
[LBA:    872] C2 error (bits:  312)                          
[LBA:    899] C2 error (bits:  312)                          
[LBA:    902] C2 error (bits:  312)                          
...                         
[LBA:   9961] C2 error (bits:  312)                             
[LBA:   9991] C2 error (bits:  312)                             
[LBA:  10006] C2 error (bits:  312)                             
[LBA:  10009] C2 error (bits:  312)                             
[LBA:  10012] C2 error (bits:  312)                             
[LBA:  10039] C2 error (bits:  312)                             
[LBA:  10042] C2 error (bits:  312)                             
media errors: 
 SCSI: 0
 C2: 812

C2 errors are reading errors that can be corrected to a certain degree by the drive’s error correction capabilities. If the damage is too severe, the drive will start to interpolate the missing data with a “best guess”. For CD audio, this is not a big deal: You either don’t hear the side effects of the interpolation or you’ll get a crackling or popping noise for a fraction of a second. But for a CD-ROM containing data, that’s not acceptable; even if we flip one single bit, the data will be invalid.

For this reason, CD-ROMs contain a second, higher level of error correction and detection: EDC/ECC.

A sector on a standard Mode 1 CD-ROM contains 2352 bytes of data (excluding subchannel information which are not relevant here), and in CUE/BIN images, we capture all 2352 bytes. 2048 bytes are dedicated to the “usable” data itself, and the 288 bytes “above” contain the error correction information. From these 288 bytes, the lower 4 bytes are reserved for error detection using a CRC checksum. Then, there’s a “gap” of 8 reserved bytes, followed by 276 bytes of error correction information using parity data.

When a drive reads a sector, it will check the 2048 data bytes against the 4 bytes of the checksum. If the checksum matches, it knows that the sector is good and tells the operating system that it could read the sector without problems. Otherwise, if the data and checksum don’t match, the drive knows that it read incorrect data and tries to rebuild the missing data using the 276 bytes of parity data. Assuming the read error is not too severe, the parity data contains enough information to reconstruct the original data. In this case, the drive will also report a good read to the operating system.

However, if there is too much original data missing, the error correction will fail because there’s not enough parity information to reconstruct the missing bits. Then, the drive will return a reading error to the operating system.

SafeDisc intentionally places damaged sectors on the disc. It doesn’t contain any game data; it is only there for internal use by SafeDisc itself. During the authentication process, SafeDisc instructs the drive to (randomly) read some of the intentionally bad sectors. And unlike the weak sectors, the sectors need to be unreadable. If the drive can read the deliberately damaged sectors, the authentication will fail, and SafeDisc reports the disc as missing or a copy.

Using bad sectors serves two purposes: Many drives will try really hard to correct the bad sectors, sometimes needing hours for a simple copy. Earlier software wasn’t designed with this protection in mind, so it would often simply fail if it encountered a reading error. Why would you want to burn bad data in the first place? And speaking of burning, many earlier disc copying software couldn’t burn bad data. The software or the drive would either refuse to work, or silently rebuild the checksum fields to “match” the user data it could read, fixing the reading errors that should not be fixed.

Sometimes, fixing errors is not good

So, if SafeDisc can read the damaged sectors, it knows it is a copy. And until the commit mentioned at the beginning of this article, 86Box couldn’t emulate read errors. Even though the image is dumped correctly and contains the “bad” sectors, filled with dummy data to ensure a broken checksum and invalid parity information, 86Box’s drive emulation would happily report a good read. And yes, this breaks SafeDisc.

To demonstrate the emulation flaw, I loaded the image I created from my original disc into 86Box and ran a surface scan with an old version of IsoBuster.

With the old drive emulation code, the disc reads without errors, even though it should have 812 errors:

When I run the surface scan with the new drive emulation code, the 812 errors are reported correctly as reading errors:

And while the game previously wouldn’t pass the copy protection, asking me to insert the correct CD-ROM into the drive…

… it now allows me to go back to medieval times!

About version compatibility

Age of Empires 2 is using the very old SafeDisc 1.30.010. SafeDisc 1.x only uses the intentional reading errors to detect copies. Especially the ATIP checks and the virtual drive detection are a novelty to SafeDisc 3.x and 4.x. The newest disc using SafeDisc I own is my copy of The Sims: Complete Collection , which uses SafeDisc 4.50.000. Right now, I couldn’t get the game to work on 86Box 5.0, using the stable build with the old dynarec. When it tries to perform the copy protection, the application segfaults. This is not related to the game, but also happens when starting The Sims Creator that’s bundled with the game and also does the SafeDisc check. I also tried with build 7706 using the new dynarec and ran into the same issue.

So while earlier versions of SafeDisc now seem to work just fine, you might run into problems with “newer” games. Time to dust off my stash of discs to get some more samples…

Thanks, dear 86Box team - keep up your great work, you are awesome!

Saving the Masters of the Elements From Getting Lost to Time: Part 4

Sun, 31 Aug 2025 10:00:00 +0200

In the previous part of this series , we finally fixed the game itself by removing the broken library and replacing it with a more modern alternative.

However, we are not done yet - there are still some minor issues left to fix. Let’s go!

Starting the game is a whole puzzle by itself. Depending on the release, the intro screen, which also serves as the main menu, is broken starting with Windows 2000. This issue significantly hinders the game’s usability, as you’ll either get almost all UI elements (except the one that actually starts the game) or none (unless you know the exact location and hover the mouse over them).

When rendered correctly, the intro screen is supposed to look like this:

The rendering issue is not too bad in the German release. The only UI element that is missing is the “pull switch” that starts the game. Luckily, you can just use the Hilfe button, since that one will start the game as well.

For comparison, here’s a screenshot of a properly rendered intro screen of the English release:

As you can see in the screenshot below, the English (as well as the French and Dutch) releases are affected more severely. The UI elements are entirely missing, unless you hover the mouse cursor at the exact location. This makes it basically impossible to start the game unless you played it before and remember the exact locations of the UI elements.

Like many Macromedia Director games, Masters of the Elements relies on a screen resolution of 640x480 and a color depth of 256 colors. While 256 colors were common back in the day, modern Windows releases and graphics drivers basically only support this color mode through emulation and by wrapping the necessary functions. Furthermore, Macromedia Director supports multiple color palettes and even provides options to swap them mid-movie, allowing more colors than the initial color palette provides.

Setting the screen resolution and proper color depth is accomplished by a small launcher executable that all versions of Masters of the Elements provide. This application sets the screen resolution and color depth, then launches the Macromedia Director projector executable, ultimately starting the game. And yes, this launcher executable is also broken on modern Windows. The non-native 256 colors combined with the palette assignment in Macromedia Director is what breaks the intro screen.

I was able to track down the broken palette assignment in the INTRO\INTRO.DIR file. The relevant code is the on startIntro function in Movie Script 31:

on startIntro
  set navigationZones to []
  set navigationZones to [rect(0, 0, 640, 480)]
  set navigationPoints to [[300, 170]]
  set navigationDirection to [1]
  set navigationScripts to ["zoomInOnBook"]
  puppetPalette("black palette", 25)
  updateStage()
  set the loc of sprite 1 to point(2000, 2000)
  setSprite(4, iassets, 21)
  setSprite(2, iassets, 11)
  showButtons()
  ...

Removing the line

puppetPalette("black palette", 25)

which transitions to the palette called black palette with a speed setting of 25 (where 1 is the slowest and 60 the fastest possible setting) fixes the rendering issues without any notable side effects. Since using this palette won’t cause issues when running the game on Windows 95/98/ME using the native 256 color space, my conclusion is that a) the palette transition breaks the 256 color mapping into the 32-bit color space of current Windows and b) the color mapping is working well enough to avoid any negative side effects.

The odd one out is the Italian release. Right now, I don’t fully understand what is wrong with this version. The UI elements in the intro screen render correctly; however, both the clouds at the beginning and the transition to the Room of Gravity are rendered using the incorrect color palette. Luckily for us, this is the only issue I could find, so this is more likely related to the transition itself and not directly to the color palette.

When running the game natively without any modifications on Windows 95/98/ME, the transition to the Room of Gravity looks like this:

Comparing the screenshots from above with the broken rendering reveals that the solid blue background of the “first half” of the room transition is replaced with solid black:

Interestingly, this version includes the line in the INTRO.DIR movie that breaks all the other versions, so they likely used an entirely different palette or changed something I have yet to find when building the movie files themselves.

Minor changes across the different language releases are a recurring theme for this game. Many strings are hard-coded into the movie files, so they vary slightly across all versions. Furthermore, there’s a movie file called CHECK.DIR that’s embedded into the projector file for the French, Italian, and Dutch release I own, while the other releases have this movie as a seperate file. At this point, I’m convinced that I don’t own the truly original Dutch release, but rather the later re-release primarily marketed in Belgium. If someone has the original Dutch release from 1997 for sale, please get in touch with me.

When you start the game, you’ll notice that the game is rendered really, really small, especially when using a larger screen resolution. As I briefly mentioned, the game originally switched to 640x480 by using the (now broken) helper application. Initially, I tried to use the compatibility option provided by Windows, but the program never notifies Windows when it closes, so you then have to manually revert back from 640x480 to your native screen resolution. This gets annoying really, really fast.

The perfect solution for this problem is a small utility called Res-o-Matic by bCheck. This tool lets you pick an application and the resolution it should run in. Then, it will create a shortcut file that switches your display to the selected resolution, launches the application, and returns to the original resolution after the program exits. Really convenient! I used this tool for the entirety of this series, so I can’t recommend it enough. Thank you, bCheck!

To summarize, the game is now fully functional, for the first time in about 20 yearss. We have successfully fixed all the issues that we could. Mission accomplished?

Well… I think I just found some subtle changes in the Lingo code across the different releases, and they are not related to the localization. This can only mean one thing: There might be multiple versions of the game engine.

…continue reading Part 5 of this series…

Saving the Masters of the Elements From Getting Lost to Time: Part 3

Sun, 13 Jul 2025 15:00:00 +0200

I briefly talked about Xtras and XObjects in the first part of this series . To recap, Xtras and XObjects are specially crafted libraries that extend Macromedia Director’s functionality by exporting additional functions that can then be used in the Lingo scripts powering the individual application. This allows a deeper integration with the operating system, since the Xtras and XObjects can provide a bridge between Macromedia Director and the operating system’s API.

Masters of the Elements uses such a third-party XObject for controlling the mouse cursor as we discovered in the previous articles .

Now that we know how it breaks the game and what it actually does , it is time to look for a proper replacement.

Finding the Xtra

Thankfully, the Macromedia Director community was very active back in the day, featuring countless developers who created both commercial and non-commercial Xtras. When looking for an Xtra that covers a specific function, the place to go is the Director Mile High Table O’Products provided by Dean’s Director Tutorials . This website is an incredible resource for anything related to Macromedia Director, so I highly recommend checking it out if you are interested in learning more!

In the Mouse and Keyboard section of this list, I spotted two possible candidates for a replacement: SetMouseLoc Xtra by Laurent Cozic, and SetMouseXtra by Stephan Eichhorn from Scirius Development.

According to the Director Mile High Table O’Products list, SetMouseLoc Xtra is only compatible with Macromedia Director 8.5 and higher. Since we are working with a game made with Director 6.0.2, choosing this Xtra is not a viable option. SetMouseXtra, however, claims to be compatible with Macromedia Director 5 to 11. Since SetMouseXtra is distributed as freeware without any special licensing requirements, this seems to be an excellent choice.

Looking at the interface description of SetMouseXtra reveals that this Xtra is not only a viable option, but a complete match. It exports one function, and expects the same parameters as PUTCURS.dll does: The x and y coordinates.

xtra SetMouseXtra               -- Version 1.0
-- --------------------------------------------------------------------
-- this Xtra allows to set the mouse position through Lingo
-- --------------------------------------------------------------------
1997 by Stephan Eichhorn, Scirius Multimedia
-- e-mail:  <redacted>
-- WWW:     http://www.scirius.com
* SetMouse integer x,integer y     -- set the cursor to position x,y
-- all x,y are in screen coordinates,
-- not relative to the stage!

The Integration

After downloading the .zip archive with the Xtra files, I extracted the file SMXTRA.X32 from the Windows directory inside the archive to the XTRAS directory of my copy of the game.

Contrary to the old XObjects, the newer Xtras introduced with Macromedia Director 5 support autoloading. As soon as the Xtra is placed in the application’s XTRAS directory, all functions will be available immediately after startup without having to use OpenXLib. Both the old XObject and our new Xtra are so similar that we can use it as a simple drop-in replacement by modifying the moveTheCursor function, where we replace the old myMouse(mSet...) call with SetMouse().

Old function call

on moveTheCursor X, y
...
  if platform = #windows then
    set offSetX to the stageLeft + (4 * (the runMode = "Author"))
    set offSetY to the stageTop + (42 * (the runMode = "Author"))
    myMouse(mSet, integer(X + offSetX), integer(y + offSetY))
  else
...
end

New function call

on moveTheCursor X, y
...
  if platform = #windows then
    set offSetX to the stageLeft + (4 * (the runMode = "Author"))
    set offSetY to the stageTop + (42 * (the runMode = "Author"))
    SetMouse(integer(X + offSetX), integer(y + offSetY))
  else
...
end

This is how the final script looks after replacing the “broken” calls to the old XObject with the new ones from our replacement Xtra:

on definePlatform
  global platform
  if the machineType = 256 then
    set platform to #windows
  else
    set platform to #macintosh
  end if
end

on initCursorObject
  global myMouse, platform, objectPath
  if platform = #windows then
  else
    openXLib(objectPath & "MoveMous.XOB")
    set myMouse to MoveMouse(mnew)
  end if
end

on exitCursorObject
  global myMouse, platform, objectPath
  if platform = #windows then
  else
    closeXLib(objectPath & "MoveMous.XOB")
  end if
end

on moveTheCursor X, y
  global myMouse, platform, offSetX, offSetY
  if platform = #windows then
    set offSetX to the stageLeft + (4 * (the runMode = "Author"))
    set offSetY to the stageTop + (42 * (the runMode = "Author"))
    SetMouse(integer(X + offSetX), integer(y + offSetY))
  else
    myMouse(mSetMouseLoc, integer(X), integer(y))
  end if
  set a to the mouseH
end

What about Macintosh?

The modified script works on both Windows and Macintosh, since I left the parts specific to Macintosh intact. However, it is very unlikely that this modified file will ever affect a Macintosh version. Masters of the Elements is distributed as a so-called “Hybrid disc”, which means that the disc contains two distinct file systems. The ISO9660 file system, which you probably know from .iso files, contains all the files that are responsible for the Windows version. The Macintosh files, however, are stored in a separate HFS file system on the same disc.

On a Windows system, the HFS partition won’t even be visible without using third-party tools. Hence, the chance of unvoluntarily overwriting the “wrong” file is pretty much zero, but for the sake of completeness, I didn’t unnecessarily remove the Macintosh integration.

There and Back Again

There should have been a video here but your browser does not seem to support it.

Finally, after more than 20 years, the game is fully playable again. I replayed the first scene in the Room of Gravity and successfully solved the Room of Warmth. Yes, I had to cut the video a bit; I’m not that great at baking pancakes, and I couldn’t trigger the shortcut through the egg at the juggling puzzle.

Fun fact: If you are running the game on a slower system, it gives you an advantage by slowing down the animation speed. “Slower” in this context means something along the lines of a Pentium 133. To determine the system speed, the game runs a couple of checks at the beginning of the game, where it tries to draw sprites and transitions as fast as possible. If you are below a certain threshold, then the code path that slows down the entire game triggers.

I wonder if I can use this to create a cheat that makes the game easier. Oh, and there’s one thing left to do.

…continue reading Part 4 of this series…

BTS#3: Removing Unwanted Services and Modules for a Headless Raspberry Pi Zero 2 W

Wed, 09 Jul 2025 14:00:00 +0200

For my next small side project, I’m running a Raspberry Pi Zero 2 W as a proxy server for my website. In fact, right now, fabulous.system is served from this exact Pi!

For this task, the limited CPU capabilities are not that big of a deal, but the limited RAM is. Sometimes, 512 MB is really not enough memory. Clearly I can’t simply add more RAM to the Pi, so I decided to disable as many hardware features as possible.

I don’t need WiFi, sound or accelerated video; all I need is USB for my Ethernet adapters.

Modifying the startup configuration and kernel modules can lead to a non-bootable system or unwanted behavior. Be careful if you don’t have physical access to the device!

I started by removing all services I don’t need, since I have no use for WiFi, Bluetooth, sound handling, or local console setup. Since I am not going to use the GPIO pins and the Raspberry Pi Zero 2W has no built-in EEPROM, I can deactivate these services as well.

sudo systemctl disable --now avahi-daemon Bluetooth console-setup rpi-eeprom-update triggerhappy ModemManager wpa_supplicant alsa-restore rpc-statd-notify hciuart Bluetooth

Next, on to the hardware side of things!

In the startup configuration located at /boot/firmware/config.txt, remove the line

dtoverlay=vc4-kms-v3d

and add

dtparam=i2c_arm=off
dtparam=i2s=off
dtparam=spi=off
dtparam=audio=off
dtoverlay=disable-wifi
dtoverlay=disable-bt

display_auto_detect=0
camera_auto_detect=0
gpu_mem=16

Reboot the system to apply the changes. This modification disables (most) of the audio and higher-level video handling as well as some lower-end components like i²c and SPI. Setting gpu_mem to 16 restricts the GPU core to 16 MB of VRAM. Unfortunately, we can’t go any lower; 16 MB is the bare minimum. On this model of the Raspberry Pi series, the GPU (also known as VideoCore) does much more than just video output; it is also responsible for certain aspects of memory management and loading the initial firmware.

Finally, I blacklisted all kernel modules that are not essential for my use-case. To do this, modify the file /etc/modprobe.d/raspi-blacklist.conf and add the following entries:

blacklist bcm2835_codec
blacklist bcm2835_isp
blacklist Bluetooth
blacklist bnep
blacklist btbcm
blacklist drm
blacklist rfkill
blacklist snd
blacklist snd_bcm2835
blacklist snd_compress
blacklist snd_pcm
blacklist snd_pcm_dmaengine
blacklist snd_soc_core
blacklist snd_soc_hdmi_codec

This eliminates Bluetooth, DRM, and sound support. While it is tempting to disable the vc4 module as well: Don’t. I tried, and it somewhat worked, but led to issues with the USB Ethernet adapters causing hanging kernel threads when they were under heavy load. As far as I can tell, the remaining modules that are loaded at boot time are absolutely required.

With these modifications in place, the fully functional system now requires 83 Megabytes of RAM! This leaves me with plenty of space for further additions, such as an rsync server, a caching DNS proxy, and some caching for nginx.

Saving the Masters of the Elements From Getting Lost to Time: Part 2

Sat, 05 Jul 2025 18:00:00 +0200

In Macromedia Director, everything is related to cinematography and theaters. A Director project consists of individual movies, timed and coordinated by the score. The canvas where you place all the objects is the stage; objects and scripts compose the cast, and each object and script is a cast member.

Then, there’s the Projector, the executable file that plays all movies in the defined order. And just like a real movie projector shines light through the film, we have to reverse this process: Splitting the beam of light to reveal the real source.

Introducing ProjectorRays

As briefly mentioned in my previous article , more or less complex Director movies contain scripts written in a small BASIC (and later JavaScript)-like language, called Lingo. The (human-readable) Lingo code is then compiled into a (machine-readable) bytecode. To protect their software and intellectual property from reverse-engineering efforts, Macromedia Director offers a feature to “protect” a movie. The protection consists of stripping down the plain text Lingo sources, leaving only the compiled parts intact, and on top of that, adding a checksum algorithm that prevents any tampering with the Director movie internals.

The protected movies get special file extensions, .DXR or .CXT, and are impossible to be opened with the Macromedia Director Authoring system.

Thankfully, there’s a project called ProjectorRays , which solves this problem. ProjectorRays takes the protected .DXR and .CXT files, and decompiles them to their editable .DIR and .CST counterparts. Except for the original Dutch release of the game, Masters of the Elements uses this kind of protection as well.

After downloading the latest version from the release page , we can start feeding the encrypted files to ProjectorRays itself.

For Windows, ProjectorRays ships as a single executable file, so we can run all .DXR and .CXT files in one go without having to install any additional dependencies.

Starting from the directory containing our working copy of the game files, we have to open a PowerShell terminal and pass all .DXR and .CXT files to ProjectorRays. While we’re at it, I highly recommend deleting the .DXR and .CXT files after the decompilation step—I noticed that the Director software gets really confused if both variants are present.

1Get-ChildItem -Recurse -Filter '*.dxr' | ForEach-Object { X:\Path\To\projectorrays.exe decompile $_.FullName }
2Get-ChildItem -Recurse -Filter '*.cxt' | ForEach-Object { X:\Path\To\projectorrays.exe decompile $_.FullName }
3Get-ChildItem -Recurse -Filter '*.dxr' | ForEach-Object { rm $_.FullName }
4Get-ChildItem -Recurse -Filter '*.cxt' | ForEach-Object { rm $_.FullName }

The debugging begins

After decompiling the Director files and deleting the original, protected files, we are ready to open the starting movie in Macromedia Director. The German variant I’m currently working with was built using Macromedia Director 6.0.2. To avoid any compatibility issues, I’m using this exact version of the authoring software.

And indeed, this is the correct file.

As soon as I hit the “Play” button in Director, the movie starts to play back, and eventually confronts me with the same error message. Well, not exactly the same, as this time, we get the option to start debugging!

There should have been a video here but your browser does not seem to support it.

Clicking the “Script” button takes us right to the point where everything starts to fail, and unsurprisingly, the problematic script is called Cursor Script.

Lingo? Lingo!

As I mentioned in my previous article, Director is using a scripting language called Lingo. When I first looked at it, it thought it was very readable, even without any prior experience with it.

Let’s break it down, shall we?

on definePlatform
  global platform
  if the machineType = 256 then
    set platform to #windows
  else
    set platform to #macintosh
  end if
end

The script begins by checking whether we are running on Windows or Macintosh. This check is important because we can’t load a Windows Xtra file or XObject on a Macintosh and vice versa.

on initCursorObject
  global myMouse, platform, objectPath
  if objectp(myMouse) then
    myMouse(mdispose)
  end if
  if platform = #windows then
    openXLib(objectPath & "Putcurs.dll")
    set myMouse to Putcurs(mnew)
  else
    openXLib(objectPath & "MoveMous.XOB")
    set myMouse to MoveMouse(mnew)
  end if
end

on exitCursorObject
  global myMouse, platform, objectPath
  if objectp(myMouse) then
    myMouse(mdispose)
  end if
  if platform = #windows then
    closeXLib(objectPath & "Putcurs.dll")
  else
    closeXLib(objectPath & "MoveMous.XOB")
  end if
end

Every time we want to use the myMouse object, we have to load (the broken) Putcurs.dll file on Windows or MoveMous.XOB, which is responsible for handling the cursor-related functions on the Macintosh. Upon initialization of myMouse, we load the XObject with openXLib, and when we don’t need the object anymore, we unload the XObject by using closeXLib.

on moveTheCursor X, y
  global myMouse, platform, offSetX, offSetY
  if not objectp(myMouse) then
    return 
  end if
  if platform = #windows then
    set offSetX to the stageLeft + (4 * (the runMode = "Author"))
    set offSetY to the stageTop + (42 * (the runMode = "Author"))
    myMouse(mSet, integer(X + offSetX), integer(y + offSetY))
  else
    myMouse(mSetMouseLoc, integer(X), integer(y))
  end if
  set a to the mouseH
end

Calling moveTheCursor uses the function mSet exported by Putcurs.dll. The values X and y refer to screen coordinates and are then passed to mSet. We don’t need to worry about mSetMouseLoc though, because this is only used by the Macintosh version we are not working on.

Bare minimum

We already know that Putcurs.dll is broken beyond repair, so if we want to do any further work about it, we first need to check what it does inside the game itself.

Looking at the Lingo code, we know that it controls the position of the mouse cursor, but not how the game uses it. And surely, the game will use it somehow, because otherwise, there would be no reason to include it in the first place.

To solve this mystery, I modified the Lingo script and removed all parts that are not absolutely necessary. All that’s left over are some stubs for the functions themselves, so Lingo won’t trip over when trying to call them.

on definePlatform
  global platform
  if the machineType = 256 then
    set platform to #windows
  else
    set platform to #macintosh
  end if
end

on initCursorObject
end

on exitCursorObject
end

on moveTheCursor X, y
end

I saved the modified file global/scripts.cst, and started the game, this time using the original game executable.

The moment of truth, the moment we all waited for.

There should have been a video here but your browser does not seem to support it.

As expected, the Tivola intro runs, but this time, I get past the error message, and the next movie (called INTRO.DXR/.DIR) is loaded. I kept the dialog with the mole in the video to check if playing back the audio samples works correctly, because the game uses some legacy DirectX components that are thankfully still included in Windows 11.

The conversation with the mole works just fine, and so does the transition to the Room of Gravity. And here is where the first quirks are visible.

So close, yet so far

When I try to turn the ring to the left (instead of the right), the cursor also jumps to the left. In the original game, the mouse cursor stays at the same position. This effect is also visible when I try to manipulate the handle controlling the crane in the first room.

Purely cosmetic at this point, so it’s not a deal breaker.

But a couple of seconds later, in the Room of Warmth, the game breaks. Here, you are supposed to light a match by striking it along the priming surface of the matchbox. You control the mouse cursor in two dimensions, vertically and horizontally. Horizontally, it works without problems, but vertically, it is impossible; the mouse cursor will always slowly creep up or down. The same problem also happens when trying to control the pan; there is no way to control vertical movement. Not included in the video is the second room with an unsolvable puzzle: The Room of Electricity, where you won’t be able to solve the maze due to a lack of control over the mouse cursor.

To summarize, every time the game tries to perform a drag-and-drop movement, it will break. With a working Putcurs.dll implementation, the game continuously resets the cursor position to the “real” starting location when it encounters said drag-and-drop operations. And without it, it cannot be completed.

Somehow, we need to find a way to replicate the features provided by Putcurs.dll; there’s no way around it. How, you might ask? Well, as I mentioned, the Macromedia Director community was very active back in the day…

…continue reading Part 3 of this series…

Saving the Masters of the Elements From Getting Lost to Time: Part 1

Wed, 14 May 2025 18:00:00 +0100

Masters of the Elements (or Meister Zufall und die Herrscher der Elemente in Germany) is an educational adventure video game developed by the Dutch studio IJsfontein Interactive Media , initially released in late 1997 for Macintosh and Microsoft Windows.

I don’t remember when I got the game; it was most likely Christmas 1999 or 2000. What I do remember, though, is that I spent many hours wandering around the game and trying to solve the puzzles.

And I remember the day the game would stop working forever: The day I updated my PC to Windows 2000.

The game will not work on anything newer than Windows 95, 98, or ME. No matter which compatibility mode you try, as soon as you try to launch the game on Windows 2000, it will fail with a very non-descriptive error message.

When you click on “Stop”, the game immediately quits. When you try “Continue,” the game sits there forever and eventually crashes.

For many years, I have not found a solution to this problem. Yes, there was a patch for running the game on Windows XP, but it was so hard to find that I got it as late as the year 2023. Furthermore, it only works with the version released in Belgium and introduces issues in all other releases.

Hello, Director!

Luckily for me, the game was created with Macromedia Director , a multimedia authoring suite developed between the mid-90s and mid-2000s. If you played any educational title during this period, there’s a high chance it used Director as well—thousands of applications and games are known to use it. In case you have never heard of Director, I’m sure you have heard about Shockwave—yes, that’s the web variant of Director applications.

The ScummVM development team is currently working on support for Director applications, so I know some very knowledgeable people who are incredibly familiar with it.

Macromedia Director started as a tool for creating multimedia presentations and interactive applications, and it became popular thanks to its relative ease of use. Since it features a complete scripting language, Lingo, it can also be used for advanced games.

It also includes the concept of XObjects and Xtras, which are third-party add-ons that enhance Director’s functionality even more. XObjects were the initial variant of these add-ons, and evolved to Xtras starting with Director 5.0. Thanks to backward compatibility, Director 6.0 supports XObjects as well.

The investigation begins

Let’s look at the error message: “Script error: Handler not defined, #Putcurs”.

The error message tells us that a movie or Lingo script tries to call a function called Putcurs, but fails to find it. Since Putcurs is not a command built into Director’s language set itself, it is very likely that one of the XObjects or Xtras is causing the problem.

Conveniently, Director applications usually have a directory called Xtras which include both the XObjects and the Xtras, and Masters of the Elements is no exception:

...
-r-xr-xr-x 1 fabulous None 273K Jan 29  1997 PMATIC.X32
-r-xr-xr-x 1 fabulous None 4.5K Aug 31  1993 PUTCURS.DLL
-r-xr-xr-x 1 fabulous None 112K May 11  1995 SOUNDIMP.X32
...

There it is, a suspiciously small DLL file. Looking at the string table of the DLL reveals that it is indeed an XObject, and a very old one as well.

--Put Cursor XObject. 31 ago 95. Mauricio Piacentini Tabuleiro da Baiana Multimedia
putCurs
I      mNew                --Creates a new instance of the XObject
X      mDispose            --Disposes of XObject instance
III    mSet, Xpos, Ypos    --Set the cursor to a new position

Why does it work on Windows 95/98/ME, but not on Windows 2000, XP, or anything else?

Windows backward compatiblity is not always amazing

A quick check with file reveals that it is a 16-bit DLL, written with Windows 3.1 in mind. Remember, the game was released in 1997 using Director 6.0, so this is an odd choice.

PUTCURS.DLL: MS-DOS executable, NE version 5 for MS Windows 3.10 (DLL)

Director 6.0 is a 32-bit application, and now we know it needs to include the 16-bit DLL file to access the functions exported by said DLL. And that’s the problem: This is impossible on the Windows NT family.

Windows generally can’t mix 32-bit and 16-bit code in the same process. Instead, it relies on a method called “thunking”. There are two types of thunks: Generic thunks and flat thunks .

Generic thunks support loading a 32-bit DLL from within a 16-bit application, and they are supported in Windows 9x and the NT series, including Windows 2000. Generic thunks are a relic from transitioning the 16-bit DOS-based world to modern Win32: Developers could update their libraries first, and then adapt their applications accordingly.

The other type of thunk is called a “flat thunk”, and it works in both directions: A 32-bit application can call 16-bit code, and a 16-bit application can call 32-bit code. And there is one catch: Flat thunks only work on Windows 9x.

That’s why it breaks, and that’s why it will never work again. Unless…

…continue reading Part 2 of this series…

The Day Anubis Saved Our Websites From a DDoS Attack

Thu, 01 May 2025 14:00:00 +0100

One part of my work for the ScummVM project is helping to keep the server infrastructure up and running, including our primary server, which hosts our website, wiki, forums, and some internal applications.

About three weeks ago, I started receiving monitoring notifications indicating an increased load on the MariaDB server. This in itself is nothing too unusual. It usually means nothing but a sudden influx of new visitors, and in most cases, it is just a link being shared somewhere or a single IP trying to annoy us.

The notifications popped up and disappeared as quickly as they appeared. I started to look into the log files of our web server, and I didn’t notice anything too unusual, maybe a bit more background noise. This went on for a couple of days without seriously impacting our server or accessibility–it was a tad slower than usual.

And then the website went down.

We use a stack consisting of Apache2, PHP-FPM, and MariaDB to host the web applications. The server logs revealed that everything was saturated. Apache2 refused to accept new connections, the PHP-FPM pools were completely filled, and MariaDB also had no connections left.

Now, it was time to find out what was going on. Hoping that it was just one single IP trying to annoy us, I opened the access log of the day and was greeted by this:

127.0.0.1 - - [24/Apr/2025:23:42:29 +0000] "GET /index.php?days=30&from=20250417123108&hidemyself=1&limit=500&target=Lure_of_the_Temptress&title=Special%3ARecentChangesLinked HTTP/1.1" 200 6366 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/534.2 (KHTML, like Gecko) Chrome/16.0.843.0 Safari/534.2"
127.0.0.1 - - [24/Apr/2025:23:42:29 +0000] "GET /index.php?from=20250417205327&hidemyself=0&limit=100&target=California_Pacific_Computer_Company&title=Special%3ARecentChangesLinked HTTP/1.1" 200 6363 "-" "Mozilla/5.0 (compatible; MSIE 6.0; Windows NT 4.0; Trident/3.1)"
127.0.0.1 - - [24/Apr/2025:23:42:29 +0000] "GET /index.php?days=30&from=20250410022141&hidebots=0&hideliu=1&hideminor=1&target=The_Big_Red_Adventure&title=Special%3ARecentChangesLinked HTTP/1.1" 200 6368 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_5_3; rv:1.9.4.20) Gecko/8520-08-18 14:24:31.076782 Firefox/3.8"
127.0.0.1 - - [24/Apr/2025:23:42:29 +0000] "GET /index.php?days=1&from=20250424060651&fromFormatted=06%3A06%2C+24+April+2025&hideminor=1&limit=100&target=RAMA&title=Special%3ARecentChangesLinked HTTP/1.1" 200 6368 "-" "Mozilla/5.0 (X11; Linux i686; rv:1.9.7.20) Gecko/4195-09-07 16:38:05.879333 Firefox/3.8"
127.0.0.1 - - [24/Apr/2025:23:42:29 +0000] "GET /index.php?days=30&from=20250424183156&fromFormatted=18%3A31%2C+24+April+2025&hideminor=1&limit=250&target=AGOS%2FVersions&title=Special%3ARecentChangesLinked HTTP/1.1" 200 6367 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/534.0 (KHTML, like Gecko) Chrome/39.0.887.0 Safari/534.0"
127.0.0.1 - - [24/Apr/2025:23:42:29 +0000] "GET /index.php?days=30&from=20250411043805&hidebots=0&target=OpenTasks%2FEngine%2FImprove_WME&title=Special%3ARecentChangesLinked HTTP/1.1" 200 6367 "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_8; rv:1.9.3.20) Gecko/9958-03-18 16:15:48.117981 Firefox/14.0"
127.0.0.1 - - [24/Apr/2025:23:42:29 +0000] "GET /index.php?days=30&from=20250411042538&hidebots=0&hidemyself=1&limit=250&target=Compiling_ScummVM%2FPlayStation_Portable&title=Special%3ARecentChangesLinked HTTP/1.1" 200 6363 "-" "Opera/9.13.(X11; Linux i686; ce-RU) Presto/2.9.173 Version/11.00"
127.0.0.1 - - [24/Apr/2025:23:42:29 +0000] "GET /api.php?action=feedrecentchanges&days=14&feedformat=atom&from=20250405110953&hidebots=1&hidemyself=1&limit=50&target=Summer_of_Code%2FGSoC2010&urlversion=1 HTTP/1.1" 200 6364 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/531.2 (KHTML, like Gecko) Chrome/24.0.862.0 Safari/531.2"
127.0.0.1 - - [24/Apr/2025:23:42:29 +0000] "GET /index.php?days=30&from=20250421165249&fromFormatted=16%3A52%2C+21+April+2025&limit=100&target=Template%3AMain_Contact&title=Special%3ARecentChangesLinked HTTP/1.1" 200 6366 "-" "Opera/9.61.(X11; Linux x86_64; st-ZA) Presto/2.9.160 Version/12.00"
127.0.0.1 - - [24/Apr/2025:23:42:29 +0000] "GET /index.php?returnto=Special%3ARecentChangesLinked&returntoquery=from%3D20250418162237%26fromFormatted%3D16%253A22%252C%2B18%2BApril%2B2025%26hidemyself%3D1%26target%3DAGIWiki%252FAl_Pond_-_On_Holiday&title=Special%3AUserLogin HTTP/1.1" 200 6365 "-" "Mozilla/5.0 (compatible; MSIE 7.0; Windows 98; Win 9x 4.90; Trident/3.1)"
127.0.0.1 - - [24/Apr/2025:23:42:29 +0000] "GET /index.php?days=30&from=20250417091241&hidebots=1&limit=250&target=Summer_of_Code%2FApplication%2F2007&title=Special%3ARecentChangesLinked HTTP/1.1" 200 6366 "-" "Mozilla/5.0 (iPod; U; CPU iPhone OS 4_1 like Mac OS X; nr-ZA) AppleWebKit/535.26.3 (KHTML, like Gecko) Version/3.0.5 Mobile/8B114 Safari/6535.26.3"
127.0.0.1 - - [24/Apr/2025:23:42:29 +0000] "GET /api.php?action=webapp-manifest HTTP/2.0" 200 2102 "https://wiki.scummvm.org/index.php?title=Hopkins_FBI" "Mozilla/5.0 (Linux; Android 10; K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Mobile Safari/537.36"
127.0.0.1 - - [24/Apr/2025:23:42:29 +0000] "GET /index.php?days=30&from=20250417023112&hidebots=0&hideminor=1&hidemyself=1&limit=250&target=AGIWiki%2FSpecial_flags&title=Special%3ARecentChangesLinked HTTP/1.1" 200 6367 "-" "Mozilla/5.0 (compatible; MSIE 7.0; Windows 98; Trident/3.1)"
127.0.0.1 - - [24/Apr/2025:23:42:29 +0000] "GET /index.php?days=30&from=20250416060403&hideanons=1&limit=100&target=Summer_of_Code%2FApplication%2F2007&title=Special%3ARecentChangesLinked HTTP/1.1" 200 6367 "-" "Mozilla/5.0 (Linux; Android 4.3) AppleWebKit/536.0 (KHTML, like Gecko) Chrome/51.0.880.0 Safari/536.0"
127.0.0.1 - - [24/Apr/2025:23:42:30 +0000] "GET /index.php?days=1&hidebots=0&hideminor=1&hidemyself=0&limit=250&mobileaction=toggle_view_mobile&target=HOWTO-Tips_And_Tricks&title=Special%3ARecentChangesLinked HTTP/1.1" 200 6366 "-" "Mozilla/5.0 (Android 4.4.3; Mobile; rv:58.0) Gecko/58.0 Firefox/58.0"
127.0.0.1 - - [24/Apr/2025:23:42:30 +0000] "GET /index.php?days=30&from=20250415120719&limit=250&target=Time_Zone&title=Special%3ARecentChangesLinked HTTP/1.1" 200 6366 "-" "Mozilla/5.0 (iPad; CPU iPad OS 1_1_5 like Mac OS X) AppleWebKit/532.1 (KHTML, like Gecko) FxiOS/12.3t5461.0 Mobile/69A052 Safari/532.1"
127.0.0.1 - - [24/Apr/2025:23:42:30 +0000] "GET /index.php?title=SCI/Testing&direction=next&oldid=14195 HTTP/1.1" 200 6364 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5; rv:123.0esr) Gecko/20100101 Firefox/123.0esr"
127.0.0.1 - - [24/Apr/2025:23:42:30 +0000] "GET /index.php?days=14&from=20250417034946&hideliu=1&hideminor=1&target=Nippon_Safes_Inc.&title=Special%3ARecentChangesLinked HTTP/1.1" 200 6364 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1; rv:1.9.6.20) Gecko/9899-07-01 03:29:48.393829 Firefox/3.8"
127.0.0.1 - - [24/Apr/2025:23:42:30 +0000] "GET /index.php?returnto=Special%3ARecentChangesLinked&returntoquery=days%3D30%26from%3D20250410005945%26hidebots%3D1%26hideminor%3D1%26hidemyself%3D1%26target%3DUser%253ASpookypeanut&title=Special%3AUserLogin HTTP/1.1" 200 6367 "-" "Mozilla/5.0 (Windows; U; Windows 95) AppleWebKit/533.2.2 (KHTML, like Gecko) Version/4.1 Safari/533.2.2"
127.0.0.1 - - [24/Apr/2025:23:42:30 +0000] "GET /index.php?days=30&from=20250410094930&hidebots=1&hideminor=1&hidemyself=1&limit=100&target=Loom&title=Special%3ARecentChangesLinked HTTP/1.1" 200 6364 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 13_7 like Mac OS X) AppleWebKit/533.1 (KHTML, like Gecko) FxiOS/9.0k8480.0 Mobile/92A641 Safari/533.1"
127.0.0.1 - - [24/Apr/2025:23:42:30 +0000] "GET /index.php?days=30&from=20250425184120&fromFormatted=18%3A41%2C+25+April+2025&hideminor=1&hidemyself=1&target=Indiana_Jones_and_the_Fate_of_Atlantis&title=Special%3ARecentChangesLinked HTTP/1.1" 200 6365 "-" "Mozilla/5.0 (iPod; U; CPU iPhone OS 4_1 like Mac OS X; pl-PL) AppleWebKit/535.5.2 (KHTML, like Gecko) Version/4.0.5 Mobile/8B116 Safari/6535.5.2"
127.0.0.1 - - [24/Apr/2025:23:42:31 +0000] "GET /index.php?diff=39241&oldid=29636&mobileaction=toggle_view_desktop HTTP/2.0" 200 2104 "https://wiki.scummvm.org/index.php?diff=39241&oldid=29636&mobileaction=toggle_view_desktop" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.3"
127.0.0.1 - - [24/Apr/2025:23:42:31 +0000] "GET /index.php?days=30&from=20250407050329&hideliu=1&hideminor=1&hidemyself=1&target=Summer_of_Code%2FGSoC_Ideas_2020&title=Special%3ARecentChangesLinked HTTP/1.1" 200 6367 "-" "Mozilla/5.0 (Android 2.2; Mobile; rv:51.0) Gecko/51.0 Firefox/51.0"

For privacy reasons, I replaced the real IPs with 127.0.0.1, but trust me, there were many IPs–around 35.000, to be precise–from residential networks all over the world. At this scale, it makes no sense to even consider blocking individual IPs, subnets, or entire networks. Due to the open nature of the project, geo-blocking isn’t an option either.

The main problem is time. The URLs accessed in the attack are the most expensive ones the wiki offers since they heavily depend on the database and are highly dynamic, requiring some processing time in PHP. This is the worst-case scenario since it throws the server into a death spiral.

First, the database starts to lag or even refuse new connections. This, combined with the steadily increasing server load, leads to slower PHP execution. Eventually, all resources in the PHP-FPM pools are used up, and since Apache2 doesn’t get a reply from PHP-FPM in time, it waits until it runs out of free connections. At this point, the website dies. Restarting the stack immediately solves the problem for a couple of minutes at best until the server starves again.

To bring the website back up, I cranked up the configuration of our stack to insane values, risking that the server would eventually run out of memory.

I needed a proper solution, something that takes the load away from the web application stack.

Hi, Anubis!

Anubis is a program that checks incoming connections, processes them, and only forwards “good” connections to the web application. To do so, Anubis sits between the server or proxy responsible for accepting HTTP/HTTPS and the server that provides the application.

Designed to protect websites from AI scraper bots, Anubis primarily focuses on parameters like the user agent sent with the request and looks for oddities in the connection. “Known good” and harmless clients are always accepted, and “Known bad” clients are always denied. In case the defaults are not working for your application, Anubis allows extensive configuration with customizable bot policy definitions .

And then, there’s the in-between, the part where the real magic happens. Many bots disguise themselves as standard browsers to circumvent filtering based on the user agent. So, if something claims to be a browser, it should behave like one, right? To verify this, Anubis presents a proof-of-work challenge that the browser needs to solve. If the challenge passes, it forwards the incoming request to the web application protected by Anubis; otherwise, the request is denied.

Solving the challenge–which is valid for one week once passed–takes a couple of seconds on the client side, occupying CPU time. Checking if the browser solved the very fast on the server side, taking up virtually no resources.

As a regular user, all you’ll notice is a loading screen when accessing the website. As an attacker with stupid bots, you’ll never get through. As an attacker with clever bots, you’ll end up exhausting your own resources. As an AI company trying to scrape the website, you’ll quickly notice that CPU time can be expensive if used on a large scale.

Long story short, deploying Anubis immediately solved our issues. In fact, you can see the exact time in our monitoring.

I didn’t get a single notification afterward. The server load has never been lower. The attack itself is still ongoing at the time of writing this article. To me, Anubis is not only a blocker for AI scrapers. Anubis is a DDoS protection.

Credits: Anubis is created by Techaro . The Anubis mascot is created by CELPHASE

Yes, You Can Still Run Internet Explorer in Windows 11

Thu, 23 Jan 2025 20:00:00 +0100

In June 2022, Microsoft announced that Internet Explorer would be phased out. In February 2023, they deactivated it with a Windows 10 updated and encouraged users to switch to their Edge browser.

If you ever tried to start Internet Explorer through iexplore.exe after this date or on a fresh Windows 11 installation, Windows opens the Edge browser instead. If you want to use a web application that only supports Internet Explorer, Microsoft recommends using the compatibility mode in the Edge browser.

But what if I tell you that to this date, Internet Explorer is deeply embedded in Windows 11 and can be used without even having the Edge browser installed?

Booting a new Windows 11 installation that has been modified by tiny11 to remove all components of the Edge browser reveals the first problem: You can’t even access Edge to get to the compatibility mode. Instead, all you get is an error message that Internet Explorer 11 is no longer supported, and you should visit the Windows App Store and get a new browser instead.

So—what do we have to do to get the world’s once-most famous browser back?

Create a new script (for example called iexplore.ps1) with the following two lines of PowerShell code:

1$iexplore = New-Object -ComObject "InternetExplorer.Application"
2$iexplore.Visible = $true

That’s it!

After setting the PowerShell ExecutionPolicy to Unrestricted, you can launch the new script.

At this point, you might wonder how this is possible.

The magic word is ComObject, which refers to the Component Object Model . Introduced with Windows 3.1 back in 1992, it is by no means a new technology.

Luckily, COM also covers Internet-enabled components using ActiveX, which relies on Internet Explorer to run. By creating a new instance of the InternetExplorer.Application object, we can still launch the complete program without any compatibility modes enabled. As I said—to this day, Internet Explorer is still considered an OS component in Windows.

How long will it last?

Microsoft announced that the Internet Explorer compatibility of Microsoft Edge will be present through at least 2029 . And even after this date, there’s a high chance that using the COM as a neat “loader” will still work.

Windows truly has exceptional backward compatibility.

BTS#2: Fix Font Rendering for Hugo's Code Blocks in iOS

Mon, 20 Jan 2025 14:00:00 +0100

For my previous article , I added Syntax highlighting to my Hugo installation.

Reviewing the page on my iPhone, I noticed something weird—the font rendering for the code blocks was entirely off.

Time to fix it!

Hugo’s support for syntax highlighting provides an easy way to make your code snippets more readable. To add them to your page, you add the highlighting shortcode and describe the language in which your code is written.

{{< highlight bash>}}
...
{{< / highlight >}}

Unfortunately, the rendering engine on iOS is not fully compatible with the CSS generated by the syntax highlighting. This leads to broken font sizes, resulting in code that is very hard to read.

Thankfully, there is an easy fix.

Applying the following code snippet to your stylesheet mitigates the broken rendering of the code blocks:

1 /* Fixes iOS font sizing anomaly */
2 code {
3    text-size-adjust: 100%;
4    -ms-text-size-adjust: 100%;
5    -moz-text-size-adjust: 100%;
6    -webkit-text-size-adjust: 100%;
7}

Besides adding the workaround, no additional changes to your Markdown files are required.

BTS#1: Submitting Entire Websites to web.archive.org Through Sitemap Parsing

Sun, 19 Jan 2025 22:00:00 +0100

This is the first episode of a new series: BTS, short for “Behind the Scenes”. In this series, I’m going through some scripts and techniques I use to build and maintain felsqualle.com.

Over the weekend, I wrote a script that parses my entire website and submits all URLs to the Internet Archive’s Wayback Machine , including all outgoing links.

Update 2025-01-21: I found out that the Wayback Machine has a limit of 200 submissions per day and IP address. In case your site is larger, you might want to limit your submissions to the latest changes of your website.

Why do I care about the Archive

I have to admit that I was not always a massive fan of the Archive’s approach of keeping websites for all eternity—in the end, I want to have the opportunity to remove content if I no longer identify with it, right?

Well, I can. A few years ago, I spotted a website in the Archive hosted on a domain I previously owned and let expire. Naturally, my content was there, then a gap of a couple of years, and suddenly, the new owner started to publish political stuff on the domain—content I can not identify with.

I contacted their support, and after providing evidence and the exact period where I owned the domain, they removed any references to the previously published content. We solved the situation within a couple of days, and working with their support staff was really great.

Then felsqualle.com happened. Given the more obscure nature of the content presented here, finding reliable sources is increasingly challenging. There are multiple articles on this website that wouldn’t exist without the help of the Wayback Machine. One day, people will think about your website the same way.

A brief introduction to sitemaps

In short, sitemaps represent your website’s table of contents. Following a standardized format , their primary purpose is to help web crawlers like search engines to find your content. In their most basic form, sitemaps only list all URLs on your website. Additionally, they can include information about a URL’s “weight” (or importance) and the date of the URL’s last modification.

For example, this is the beginning of this website’s sitemap:

 1<?xml version="1.0" encoding="utf-8" standalone="yes"?>
 2<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
 3  xmlns:xhtml="http://www.w3.org/1999/xhtml">
 4  <url>
 5    <loc>https://felsqualle.com/categories/</loc>
 6    <lastmod>2024-06-22T22:00:00+02:00</lastmod>
 7  </url>
 8  <url>
 9    <loc>https://felsqualle.com/posts/2024/06/new-installation-media-for-ms-dos-4/</loc>
10    <lastmod>2024-06-22T22:00:00+02:00</lastmod>
11  </url>
12  <url>
13    <loc>https://felsqualle.com/</loc>
14    <lastmod>2024-06-22T22:00:00+02:00</lastmod>
15  </url>
16...

The simple structure of the sitemap XML makes parsing relatively easy.

In fact, it is so simple that I managed to use a single Bash script with curl as the only dependency.

The script

 1#!/bin/bash
 2
 3# Auto-announce all pages and related links to archive.org by
 4# crawling the sitemap.xml rendered by Hugo, fetched locally
 5SUBMISSION_URL="https://web.archive.org/save/"
 6SOURCE_URL="https://felsqualle.com"
 7
 8try_count=0
 9
10announce_url() {
11	curl -s -o /dev/null -w "%{http_code}" "${SUBMISSION_URL}""${url_to_announce}"
12}
13
14archive_url() {
15	# Increment trial counter on each run since we don't know about the result here
16	((try_count++))
17
18	if [[ $(announce_url "${url_to_announce}") == '302' ]]; then
19		echo "Successfully archived \"${url_to_announce}\""
20		try_count=0
21	else
22		echo "Retrying to archive \"${url_to_announce}\""
23		
24		# Give the archive.org API some time to cool down...
25		sleep 30
26
27		# Retry announcing the same URL again until we reach the failure threshold
28		if [ "$try_count" -lt 5 ] && [ "$try_count" -ne 0 ]; then
29			archive_url "${url_to_announce}"
30		else
31			echo "Giving up on url \"${url_to_announce}\""
32			try_count=0
33			return 1
34		fi
35	fi
36}
37
38extract_external_urls() {
39	curl -s -f -L "${url_to_announce}" | grep -Eo '"(http|https)://[a-zA-Z0-9#~.*,/!?=+&_%:-]*"' | grep -v "${SOURCE_URL}" | sed 's/\"//g' | sort -u
40}
41
42# Starting the main loop, iterating through the
43# entire content of sitemap.xml.
44grep loc public/sitemap.xml | grep -v legal | grep -v privacy | sed 's/    <loc>//g' | sed 's/<\/loc>//g' | while IFS= read -r url_to_announce; do
45
46	# Start with building a list of all external URLs present on the current page
47	external_url_list=$(extract_external_urls "${url_to_announce}")
48
49	# Send archive request for current (internal) URL
50	archive_url "${url_to_announce}"
51
52	# Iterate through any external URLs we found and send archive request
53	for url_to_announce in ${external_url_list}; do
54		archive_url "${url_to_announce}"
55	done
56done

The script — demystified

Let’s break it down, shall we?

First, we need to parse the sitemap.xml file (located at public/sitemap.xml) and extract all URLs we can find in our main loop. Then, we call extract_external_urls, and archive_url for both the internal and the outgoing/external links.

 1grep loc public/sitemap.xml | grep -v legal | grep -v privacy | sed 's/    <loc>//g' | sed 's/<\/loc>//g' | while IFS= read -r url_to_announce; do
 2	# Start with building a list of all external URLs present on the current page
 3	external_url_list=$(extract_external_urls "${url_to_announce}")
 4
 5	# Send archive request for current (internal) URL
 6	archive_url "${url_to_announce}"
 7
 8	# Iterate through any external URLs we found and send archive request
 9	for url_to_announce in ${external_url_list}; do
10		archive_url "${url_to_announce}"
11	done
12done

For extract_external_urls, we fetch the HTML code of all URLs from the sitemap with curl again. grep -v "${SOURCE_URL}" ensures that we exclude all internal links. This grep command can be further expanded to exclude external links that are present on every page or in a navigation area.

1extract_external_urls() {
2	curl -s -f -L "${url_to_announce}" | grep -Eo '"(http|https)://[a-zA-Z0-9#~.*,/!?=+&_%:-]*"' | grep -v "${SOURCE_URL}" | sed 's/\"//g' | sort -u
3}

We can now track down all pages from the sitemap, and also have a list of all external URLs included on every page. To submit the URL to the Wayback Machine, we only have to send a single request to their endpoint.

1announce_url() {
2	curl -s -o /dev/null -w "%{http_code}" "${SUBMISSION_URL}""${url_to_announce}"
3}

For the front page, this expands to the following expression:

1announce_url() {
2	curl -s -o /dev/null -w "%{http_code}" https://web.archive.org/save/https://felsqualle.com/
3}

Finally, we need some (now annotated) logic to glue everything together.

 1archive_url() {
 2	# Initially, we start with a try_count of 0.
 3	# Since web.archive.org is often overloaded, we have to expect
 4	# multiple retries until we get a successful submission.
 5	# We have to avoid deadlocks, so we need to limit the amount
 6	# of retries somehow.
 7	((try_count++))
 8
 9	if [[ $(announce_url "${url_to_announce}") == '302' ]]; then
10		# For checking the status of our submission, we rely on the
11		# 302 redirection web.archive.org performs after it successfully
12		# saved a website.
13		#
14		# In case we receive any other HTTP status code ...
15		echo "Successfully archived \"${url_to_announce}\""
16		try_count=0
17	else
18		# ... we need to do some work.
19		echo "Retrying to archive \"${url_to_announce}\""
20		
21		# Give the archive.org API some time to cool down.
22		# Sometimes, multiple subsequent calls lead to rate limiting
23		# that we can mitigate by slowing down our requests.
24		sleep 30
25
26		# Retry announcing the same URL again until we reach the failure threshold.
27		# We want to try until we reach our retry threshold of 5 attepts.
28		# At the same time, we have to ensure that our current counter is
29		# _NOT_ 0. In theory, this should never happen, but _if_ it does,
30		# we are stuck in an endless loop.
31		if [ "$try_count" -lt 5 ] && [ "$try_count" -ne 0 ]; then
32			archive_url "${url_to_announce}"
33		else
34			# When reaching our threshold, we simply give up and continue
35			# with the next URL.
36			# We reset our counter and exit with an errorcode (which we don't check yet)
37			echo "Giving up on url \"${url_to_announce}\""
38			try_count=0
39			return 1
40		fi
41	fi
42}

With this script, I was able to update the copy of my website that is currently stored on archive.org. And thanks to some extensive testing, I discovered that archive.org has a limit of 5 submissions per URL and day — oops.

Creating New Installation Media for MS-DOS 4.0

Sat, 22 Jun 2024 22:00:00 +0200

On April 25, 2024, Microsoft released the source code of MS-/PC-DOS 4.00 to the public, and the community managed to provide MS-DOS 4.01 as well.

While the original release was available as a set of six 5,25" floppy disks (and optionally three 3,5" disks for version 4.01), this open-source release does not have official installation media.

Recreating the original media would theoretically be possible, but I want to focus on the 720k images for this project. Having to swap around six floppies multiple times isn’t practical. Furthermore, we can build images for 1.44 MB floppy disks for increased compatibility: The SELECT utility used for the installation process doesn’t like it if you install it on floppies with a different format than the source disks are written in.

The open-source release is missing a few program files and the entire DOSSHELL environment, so that’s something we also have to deal with.

The missing LINK

Let’s start with the missing files. To check if the correct floppy is present in the drive, the SELECT utility used to install MS-DOS 4.0 doesn’t check the disk label but directly checks if the files it is supposed to copy are available on the disk. If a file is missing, it simply tells you you have the wrong floppy inserted.

While this isn’t an issue for DOSSHELL at the moment—it is optional and only installed upon request—this is a massive problem for two other missing files.

In total, the following core files are missing from the open-source release:

HIMEM.SYS
GWBASIC.EXE
LINK.EXE

HIMEM.SYS is the easiest one. It is not referenced in the SELECT code at all, which means it can’t be one of the “marker files” indicating a specific floppy disk. We can skip it altogether, and nothing will break.

The other two files, however, are mentioned in SEL_FILE.INC, which would cause an issue if the files were absent. To fix this, I created two simple executables with Turbo Pascal 5.5:

program Gwbasic;
begin
  writeln('This release of MS-DOS 4.0 is based on the code released');
  writeln('to the general public in April 2024.');
  writeln('');
  writeln('GWBASIC.EXE is not included in this release.');
end.

Since SELECT doesn’t do any checksum comparison, it won’t complain about these stubs, and the installation will happily continue.

Removing DOSSHELL

DOSSHELL, however, is an entirely different problem. While it is optional and not selected by default, the user can enable it during installation. This will inevitably break the installation since we don’t have any DOSSHELL components in the open-source release.

We need to disable DOSSHELL in two places:

The file responsible for handling both screens is SELECT3.ASM.

Conveniently, we get a hint on how to disable the first screen:

     82 ;  The CHOOSE SHELL SCREEN is always presented.
     83 ;  This screen allows the user to decide whether or not the DOS
     84 ;  shell will be installed.
     85 ;  Valid keys are ENTER, ESC, F1, F3 and numeric 1 and 2.
     86 ;----
     87 ; Note:  This screen (and, hence, all shell support) can be eradicated
     88 ;       by defining the symbol NOSHELL.
     89 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
     90 choose_shell_screen:                                            ;AN000;
     91 IFNDEF NOSHELL
     92         INIT_PQUEUE             PAN_choose_shell                ;AN000; initialize queue
     93         PREPARE_PANEL           PAN_HBAR                        ;AN000; prepare horizontal bar
     94         PREPARE_CHILDREN                                        ;AN000; prepare child panels

NOSHELL EQU 1 defines the required symbol, and as expected, the first screen is skipped.

     82 ;  The CHOOSE SHELL SCREEN is always presented.
     83 ;  This screen allows the user to decide whether or not the DOS
     84 ;  shell will be installed.
     85 ;  Valid keys are ENTER, ESC, F1, F3 and numeric 1 and 2.
     86 ;----
     87 ; Note:  This screen (and, hence, all shell support) can be eradicated
     88 ;       by defining the symbol NOSHELL.
     89 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
     90 choose_shell_screen:                                            ;AN000;
>>>  91 NOSHELL EQU 1
     92 IFNDEF NOSHELL
     93         INIT_PQUEUE             PAN_choose_shell                ;AN000; initialize queue
     94         PREPARE_PANEL           PAN_HBAR                        ;AN000; prepare horizontal bar
     95         PREPARE_CHILDREN                                        ;AN000; prepare child panels

Unfortunately, the “Review Selection” screen doesn’t honor this symbol. This screen can override all previously set variables regarding the features that should be enabled. Furthermore, the entire menu system is scattered across multiple files, and changing one entry means we have to rewrite all other menu options.

Forcing the return value to E_SHELL_NO, which would remove DOSSHELL integration, doesn’t work either.

I got it somewhat working once, but this caused the menu system to break, and I managed to “enable” XMA support on an emulated machine that definitely doesn’t support it. At this point, I realized that modifying the menu system was beyond my non-existent assembler skills.

The “Review Selection” screen is controlled by the REVIEW_DISK_SCREEN and REVIEW_DISKETTE_SCREEN functions in SELECT3.ASM. Since I couldn’t disable the menu option, I dusted off my crowbar:

    221            RET_SUPPORT          SUPPORT_STATUS,E_GRAPHICS_C,F_GRAPHICS;AN000; GRAPHICS support
    222            RET_SUPPORT          SUPPORT_STATUS,E_SHARE_C,F_SHARE;AN000;      SHARE support
    223            RET_SUPPORT          SUPPORT_STATUS,E_SHELL_C,F_SHELL;AN000;      SHELL support
    224            RET_SUPPORT          SUPPORT_STATUS,E_VDISK_C,F_VDISK;AN000;      VDISK support
>>> 225            INIT_VAR             F_SHELL,E_SHELL_NO
    226            PUSH_HEADING         REVIEW_DISK_SCREEN              ;AN000;    save screen address on SELECT STACK
    227            GOTO                 DOS_PARAMETERS_SCREEN           ;AN000;    goto the next screen (DOS_PARAMETERS)

    277            RET_SUPPORT          SUPPORT_STATUS,E_SHELL_B,F_SHELL;AN000;      SHELL support
    278            RET_SUPPORT          SUPPORT_STATUS,E_VDISK_B,F_VDISK;AN000;      VDISK support
>>> 279            INIT_VAR             F_SHELL,E_SHELL_NO
    280            PUSH_HEADING         REVIEW_DISKETTE_SCREEN          ;AN000; save screen address onto SELECT STACK
    281            GOTO                 DOS_PARAMETERS_SCREEN           ;AN000; goto the next screen (DOS_PARAMETERS)
    282         .ELSE                                                   ;AN000;

Using INIT_VAR here always forces F_SHELL to E_SHELL_NO, discarding the user’s selection which was previously stored with RET_SUPPORT. Thus, it effectively prevents DOSSHELL from ever being installed.

Yes, this is incredibly ugly, but also very effective.

Creating the disk images

With the fixed SELECT executable on hand, it was time to create the floppy images. Luckily, it fits on two 720kB floppies, so disk swapping is not too bad. Please note that it would also fit on a single 1,44MB floppy, but this will heavily confuse SELECT and break support for installing to a set of floppies.

For 3.5" floppy disks, the disk size doesn’t matter, so both 720kB and 1.44MB disks are a viable option, but you really should create both a dedicated INSTALL and OPERATING floppy. Unfortunately, you can’t split the images on two 1.2MB 5.25" disks since SELECT will expect all six floppies in this case.

Creating the floppy images was as simple as copying all required files using the official release as a template and making the first disk bootable.

Installing MS-DOS 4.0 on a hard drive worked without any issues. While the installation to a set of floppies should work, I had mixed results even when using the images from the original release, so your mileage may vary.

Downloads

Have fun!

If I Ever Get a Dog, I'll Name It Rover: A Brief Introduction to Microsoft Bob

Sun, 16 Jun 2024 20:00:00 +0200

March 1995. Although PCs had been around for many years at this point, less than 40% of households in the US owned one. Back then, people mostly associated PCs with being a machine used at work, not a thing you’d even need at home. They were complicated and… unfriendly.

Just a few months before the launch of the all-new Windows 95, Microsoft Bob was an attempt to create a user-friendly interface that runs on top of Windows to increase the popularity of PCs at home and show how valuable a PC might be for purposes other than business.

This interface provides a virtual home inside your computer, divided into multiple rooms resembling your house’s rooms. It’s designed to be intuitive and user-friendly, so you don’t need to know what a word processor is or how the applications are named — for example, just click on the pen and paper on your virtual desk and start writing a letter.

Introduction

When starting Microsoft Bob for the first time, you are greeted by Rover Retriever, the main mascot who will guide you through the program.

While the concept of having a virtual assistant as an integrated online help was new at the time, it would later gain popularity in the form of Clippit, the virtual assistant integrated into Microsoft Office .

After telling Rover a bit about yourself, setting a password for your account, and selecting a secret room only you have access to, you will enter your virtual home for the first time.

Then, Rover will take you on a tour of your new home so you know exactly where to find the different applications Microsoft Bob offers.

But wait, there are more objects on the screen, so what do they do? At first, I was worried I’d miss an application hiding behind one of the other objects.

However, the applications Rover mentioned are really the only ones included in Microsoft Bob. Everything else is cosmetic.

If you are unsure where your application objects are located, simply press the “F1” key to reveal all clickable hotspots.

Clicking on the books on the shelf, for example, reveals that they are purely decorative and don’t add any functionality. As you can see on the screenshot, you can replace the decorative objects, but more on that later.

The applications

Time to get some tasks done! Let’s have a look at the various applications bundled with Microsoft Bob.

The Letter Writer

At first glance, the Letter Writer feels like a stripped-down version of Microsoft Word. When launching the application by clicking the “Pen & Paper” object, Rover will guide you through the steps for, well, writing a letter.

The application itself looks pretty straightforward and allows for many simple customizations. You can pick from various border styles, and premade cliparts for every occasion. It even includes the famous WordArt!

Letter Writer includes numerous templates. The pre-filled address book includes politicians, airlines, travel information agencies, departments for environmental concerns, newspapers, magazines, insurance companies, and others.

Yes, it is pretty versatile and a helpful start if you never used a word processor before. However, if you have some experience with Microsoft Word, it feels slow and tedious to use in comparison.

Will the next click allow me to start typing? Ah, no, another step in the assistant. No, I don’t want to pick an address from my address book because I don’t have any addresses yet. Yes, I’d like to have a nice picture in my letter. Sure, why not? Makes things a bit more personal. Thank you!

When starting the application, you can’t skip the assistant altogether. You have to click three times to leave it and being able to work.

Rover either didn’t mention it, or I missed it when skipping through the assistant: There’s no dedicated way to save the letter (file). Just exit Letter Writer, and it will be automatically saved for later use.

The Calendar

Clicking on the calendar on your virtual wall opens the Bob Calendar application.

Since the calendar offers fewer options than the Letter Writer, Rover is also much less intrusive. You can access all options right at the beginning and don’t have to click through a lengthy guide.

Add events, set reminders, and create your To-Do list — there’s not much to say about this application.

The Checkbook

The Checkbook is the leading financial application of Microsoft Bob. First, you are introduced to another assistant: Lexi.

Considering that Checkbook is a fairly large application, having a tour makes sense and matches the philosophy of Microsoft Bob: “Explain everything”.

The application is divided into multiple sections:

The Account Book
The Bill Basket
The Report Folder

In the account book, you add your various bank accounts for payments and deposits. You’ll record all your transactions there, so you see exactly where your money goes.

Next is the bill basket. It contains your upcoming bills and deposits. Thanks to the integration in the Bob Calendar, you’ll never miss a payment again. A payment or deposit will be automatically recorded in the account book when it is due.

The third and final section is the report folder. As the name implies, you can create all sorts of reports here to check your financial health.

After showcasing the different areas, Lexi continues with the initial account setup.

Ironically, one of the first options shows that Microsoft didn’t have a clear target audience for Microsoft Bob: You can import an existing Microsoft Money file. If you already have a financial application that offers more options than Checkbook and is way more sophisticated, why do you even want this… downgrade?

The tutorial itself consists of several steps. You add your accounts, your first round of upcoming bills, and your paycheck. Additionally, you can import these transactions to your personal or household calendar, the latter being shared across all users with an account in your Microsoft Bob installation.

Bob Checkbook also provided an online service: Pay On-Line. Using this (now defunct) service, you could send payments to individuals and businesses in the US - right from your computer and in the blink of an eye.

Once you manage to get past the tutorial, Lexi will not be as intrusive as Rover in Letter Writer. On subsequent starts, you can get straight to the various areas of the program without having to look at how to get rid of too much helpful advice.

The Financial Guide

Bob Financial Guide is best described as a weird mix of a tutorial on life and creating oddly specific lists.

You start with an overview covering important aspects of life: Finance, buying a house, some personal milestones, your retirement.

Here, you can either modify the premade lists or generate your own.

Sure, some things in life require quite a lot of planning, but I don’t know if Microsoft Bob is the right tool. It works, but it feels clumsy, and I had a hard time finding the lists I created — did I put them in “Buying a Home” or “My Home”? I am trying to remember.

The Household Manager

Essentially, the Household Manager is the same as the Financial Guide, with different topics. And if I said “oddly specific” for the lists in the Financial Guide, then I have to say that we are entering the “absurdly specific” territory for the Household Manager.^

It suffers from the same flaw as the Financial Guide: It is easy to add information, but finding it again is hard. I appreciate the included cat manual, though.

E-Mail

Bob E-Mail was an E-Mail service provided by MCI Mail in collaboration with Microsoft. After signing up for their service, you get a nice @bob.com address to send and receive emails. Thanks to the integration into Letter Writer and the Address Book, you could send messages directly through these applications.

Unfortunately, the service is long gone, so there’s nothing I can show except for the startup screen.

The Address Book

The Address Book integrates with Letter Writer and Bob E-Mail. You can add, modify, and find addresses and use them as recipient addresses when writing a letter or sending it electronically.

It is simple and effective - exactly what you’d expect from an address book.

GeoSafari

GeoSafari is the only educational title and thus more geared towards children. This time, Hank will replace Rover, who will guide you through the program.

The main objective of this application is to solve geographical puzzles: find capitals and countries, name monuments, have fun with flags, and visit the solar system.

Feeling at home

Microsoft Bob provides an advanced customization system. You start in your living room, but there are more rooms to explore. Functionally, the rooms do exactly the same: They contain objects that either start programs or are purely decorative. Objects can be added, replaced, or rearranged.

The only room that works slightly differently is the personal room you selected when you launched the application for the first time. It will be marked as “hidden”, so the other users won’t be able to see it. In addition to the premade rooms, you can create your very own rooms. There’s a surprisingly extensive catalog of objects to choose from, providing a high level of customization.

However, transitioning between the different rooms takes quite a lot of time, even on the Pentium with 75 MHz backed by 16 MB of RAM I’m emulating for this review.

Conclusion and legacy

Microsoft Bob is one of the most spectacular product failures the company ever experienced. Reviews and sales were so bad that they discontinued it in early 1996, giving it a lifespan of just under a year.

Microsoft Bob had high system requirements and even felt slow on PCs faster than standard hardware at the time. Combined with a retail price of $99, which adjusted for inflation equals around $200 in 2024, the limited set of included applications wasn’t helping sales either. Initially, Microsoft planned a whole software ecosystem around Bob, but in the end, only Microsoft Great Greetings, a software for creating greeting cards at home, was released specifically for Microsoft Bob.

Looking at the visuals, the number of different objects, and the written content (remember, there’s a tutorial on how to handle your cat!), you can clearly see that the development team and designers put a lot of effort into Microsoft Bob.

Do I think it is a bad product? No, but I think it is pointless. It looks nice but provides little additional value. The bundled applications are stripped down, redundant, and sometimes even hard to use. Sure, you can add your Windows programs to Bob and assign them to objects, but this will make things way slower than just starting them through the Program Manager. All in all, it feels like Microsoft Office: Preschool Edition or a weird crossover between Windows and The Sims.

My recommendation for 1995 computer users: Get a good video tutorial for Windows 3.1 instead and learn how to use the real thing.

Let's Build a fabulous.community

Fri, 07 Jun 2024 20:00:00 +0200

Today, I’m excited to finally announce a new side project I’ve been working on for quite a while.

To be more precise, this will be our new side project that we’ll be working on in the future.

Please welcome fabulous.community , a brand new message board for everything related to retro-computing, and the contents of this website. Yes, my initial idea was to find a way to make comments on the articles published here easier than sending me an e-mail.

Since just adding a comment section would be way too easy (and boring), I thought about… well, more engagement.

So, here it is.

While many communities use instant messaging and chat services, niche topics like the cursed computing stuff well beyond any mainstream stuff benefit from a slower-paced message board. Oh, and it provides some essential features like proper search functionality and a way to access and archive the content published here in the future!

I created a comment section in the sub-forum dedicated to this website for my most important articles and already linked the “comment sections” to the website.

For the next couple of weeks, the Building a fabulous.community forum will be the most important place for the entire project since this is where I’d like to collect everything that matters to you.

Registrations are open, so please take a seat!

About Improving MS-DOS Screenshots With Calculated Distortion

Thu, 30 May 2024 17:00:00 +0200

Introduction

Have you ever noticed that a pixel-perfect screenshot from a virtual machine running MS-DOS in the default VGA text mode looks slightly off? The characters are too wide, and the whole image seems stretched.

At the very least, it doesn’t evoke the same nostalgic feeling as the old CRT days, does it?

But did you know that what you’re seeing is a perfect representation of the actual image, while your monitor from 1995 was deceiving you? Let’s have a closer look.

Using a VGA or VGA-compatible display adapter, MS-DOS draws an image with 80 characters per line and 25 lines of text in total. In this mode, each character is 9 pixels wide and 16 pixels tall.

Knowing the size per character and number of characters per screen, we can calculate the exact image size: (80*9)x(25*16)=720x400.

So, even though we know that 720x400 is the correct image size, why does it look so wrong?

Compared to the usual screen resolutions from the time, 720x400 is an odd number. When considering monitors from the mid-90s, we are familiar with resolutions of 640x480, 800x600, and 1024x768, which all have one thing in common: An aspect ratio of 4:3.

4:3? Sounds familiar!

Yes, that was the aspect ratio of monitors from the past. Since you wanted to use the full screen size, you adjusted the settings on your monitor until you got a nice image. Contrary to modern displays, CRTs didn’t have a fixed pixel size - in theory, you could draw a “pixel” that filled an entire line if you told the magnets bending the electron beam to do so.

Yep, that’s it. The pixel-perfect screenshots are correct; your settings were wrong 30 years ago. Sort of.

With this in mind, we can easily recreate an image resembling the original monitors’ visuals. We just have to stretch the image to an aspect ratio of 4:3.

The best resolution for the images rendered in 720x400 is 720x540, which perfectly hits the 4:3 ratio. Since we want to introduce as few artifacts as possible, we only stretch the image in one dimension.

And suddenly, the images look perfectly how we remember them:

Behind the scenes

To illustrate what happens with the image, I prepared a little demo.

Let’s draw an image at 720x400 with alternating black and white lines, each line being exactly 1 pixel tall. The picture looks incredibly sharp - it is pixel-perfect.

Now, let’s stretch the image to 720x540 because we want to fix it.

I assume you don’t want to count all the individual lines, but if you did, you’d notice that the number stayed the same. We made the pixels a bit larger, adapting them to our desired screen size. Remember, we have an infinite number of possible sizes for the individual pixels.

So, is it always 720x400?

Unfortunatly not. VGA, the VESA standard, and custom modes used by some manufacturers allow for a wide variety of resolutions and character sizes .

For example, the Windows 98 boot screen, technically using MS-DOS at this boot stage, is rendered at 640x400:

In this case, our target resolution is 640x480:

To make matters even worse, other graphic modes, like CGA and EGA, again support a multitude of resolutions. The general rule is to stretch your image to the closest size that matches the magic aspect ratio of 4:3.

Check the width of the rendered image and adjust the height accordingly. You’ll soon get used to the most common modes.

One line of ImageMagick

For image manipulation, my tool of choice is ImageMagick whenever possible. You can easily apply the aspect-ratio correction using the convert command. All you need to know is the exact size of your source image and the target resolution.

To convert the picture of the Windows 98 boot screen, I used the following command:

convert win98boot_640x400.png -resize 640x480\! win98boot_640x480.png

That’s it!

3D Pinball for Windows: The Space Cadet's Mission Continues

Mon, 27 May 2024 21:00:00 +0200

Do you know what computer games and the Voyager space probes have in common? Clever engineering can extend their missions way beyond their intended lifetime.

3D Pinball for Windows – Space Cadet, the iconic game bundled with the Microsoft Plus! packs and included in Windows NT4 up to Windows XP, is preserved for decades to come thanks to a reverse engineering project by Andrey Muzychenko.

Andrey’s work involved decompiling the executable that shipped with Windows XP and creating a modern reimplementation using SDL2 . This means you can now enjoy both the bundled version and the ‘Full Tilt! Pinball’ version on Windows, Linux and macOS.

Back in the day, I learned about the game when I got it with my Windows XP installation. I remember spending way too much time with it, even during IT classes when the teacher wasn’t paying attention.

The required game assets are not included in the reimplementation for legal reasons. The easiest way is to copy the original files from a Windows XP installation CD. The files are stored in the I386 directory.

Copy the following files to your hard drive:

pinball.da_
pinball.ex_
pinball.mi_
pinball2.mi_
font.da_
table.bm_
sound*.wa_

The files are compressed using a DEFLATE algorithm. You can decompress them with 7-Zip or by using the EXPAND command, which is included in basically all Windows versions up to Windows 11.

After downloading the latest release for your platform, copy the files in the release archive to the directory where you extracted the original game files.

That’s it! Start the game and have fun!

A Minor Update: The Source Code for MS-DOS 4.01 Is Here

Sat, 25 May 2024 17:30:00 +0200

A couple of weeks ago, Microsoft released the source code of MS-/PC-DOS 4.00 to the public.

Due to various bugs, DOS 4.00 was a relatively short-lived release, and it was replaced by DOS 4.01 just a couple of months later.

Howard M. Harte (hharte), who already fixed various flaws in the official source code release of MS-DOS 4.00 , managed to figure out the differences between DOS 4.00 and 4.01 — we now have access to the improved version as well!

I imported Howard’s patch into my fork of the MS-DOS source code and prepared a new source code bundle on a set of floppy disk images.

Downloads

You can either use the files in the ZIP archive on DOSBox or try the floppy images containing a self-extracting RAR archive on real hardware or in an emulator.

Please note that if you want to use the floppy images, you must copy the contents of all four disks to a new directory and run V401_SRC.EXE from there, which will extract all files in the current working directory. Please note that the self-extracting RAR archive requires an 80386 or higher and 8 MB of RAM.

The compiled files in MS-DOS 4.01 (compiled from source, bootable, 1x 1.44MB) match the April 1989 MS-DOS 4.01 release, available at archive.org .

You can flip two switches in INC/VERSION.INC, which allows the IBM PC-DOS variant to be built instead of the “original” Microsoft variant. The generated binaries are available in the image PC-DOS 4.01 (compiled from source, bootable, 1x 1.44MB). However, due to compilation errors, the PC-DOS 4.01 build lacks the XMA2EMS.SYS driver.

I don’t know if this driver was meant to be included in the IBM variant or if it is an MS-DOS exclusive, so you should consider this release to be incomplete as of now.

The Broken Source Code for MS-DOS 4.00 Has Been Restored

Wed, 01 May 2024 18:00:00 +0200

On April 25, 2024, Microsoft released the source code of MS-/PC-DOS 4.00 to the public.

While this release is exciting and essential to computing history, Microsoft’s release process was not entirely flawless: An accidental conversion from codepage 437 to UTF-8 destroyed parts of the source code and caused build failures.

In less than a week, on April 30, 2024, the community provided all necessary patches to restore the source code to its original state, matching the 1988 release of MS-DOS 4.00.

To get a working source tree for myself, I imported the patches by Howard M. Harte (hharte) and E. C. Masloch (ecm) to my MS-DOS fork on GitHub .

Additionally, I converted the source files from LF to CRLF because it is very likely that all source files were originally stored with CRLF line endings. A fix to the SETENV.BAT batch file was also applied to fix broken include paths. I want to make absolutely clear that all changes that brought the code to a working, complete state are not mine and I was not involved in figuring out the correct code fixes.

Downloads

You can either use the files in the ZIP archive on DOSBox or try the floppy images containing a self-extracting RAR archive on real hardware or in an emulator.

Please note that if you want to use the floppy images, you must copy the contents of all four disks to a new directory and run V400_SRC.EXE from there, which will extract all files in the current working directory. Please note that the self-extracting RAR archive requires an 80386 or higher and 8 MB of RAM.

The compiled files in MS-DOS 4.00 (compiled from source, bootable, 1x 1.44MB) match the October 1988 MS-DOS 4.00 release, available at archive.org .

You can flip two switches in INC/VERSION.INC, which allows building the IBM PC-DOS variant instead of the “original” Microsoft variant. The generated binaries are available in the image PC-DOS 4.00 (compiled from source, bootable, 1x 1.44MB).

However, this build comes with an interesting oddity. The boot loader and SYS.COM expect IBMBIO.COM and IBMDOS.COM instead of IO.SYS and MSDOS.SYS, and there are multiple checksum mismatches, so we get a different build.

But here’s the catch: Booting the system and running VER shows the Microsoft copyright notice instead of the IBM one — there’s no matching entry in MESSAGES/USA-MS.MSG.

The Making of the MS-DOS Command Reference Bot

Sun, 21 Jan 2024 12:00:00 +0100

A brief introduction

I launched my Daily MS-DOS command bot a few months ago.

The initial idea was straightforward: Pick a random command or program that ships with MS-DOS 6.22 and post a brief description.

With a bit of help from some websites that archived the MS-DOS reference manual, I finished the initial revision of the JSON-based data file including all the comments with a short description in a matter of hours. I wrote a simple Bash script to pick a random command, fetch the description from the JSON file, and publish a post through the Mastodon API.

Easy. But shortly after that, a question arose: “What about screenshots?”

At first, I thought about going through every command in my data file, spinning up a VM, executing the command, taking a screenshot, and linking to it in the data file itself. While this approach is possible, it is tedious. There’s no real challenge in repeatedly typing in commands and trying to keep track of the screenshots I already took. Moreover, any change in the command list would require spinning up a VM, executing the command, taking a screenshot, and modifying the data file to include the modified screenshot.

But what if I could do all of this dynamically by taking screenshots at the moment my MS-DOS VM is executing the command I requested?

Setting up the MS-DOS 6.22 virtual machine

The first step was to create a virtual machine with QEMU to run the commands from the data file. For this scenario, QEMU works exceptionally well thanks to the built-in monitor interface that will allow interacting with the VM from the host system.

I started by creating an image file that will be the virtual hard drive for the MS-DOS installation.

qemu-img create msdos622.img 512M

A basic installation of MS-DOS 6.22 consists of three floppy images. The first floppy is bootable and will start the installation process. For convenience, I named the floppy images Disk1.img, Disk2.img, and Disk3.img in the same order the setup program will request them.

I went with a very basic 486-based VM with a whopping RAM size of 8 MB - this should be enough for getting all the included tools up and running.

qemu-system-i386 -drive format=raw,file=Disk1.img,index=0,if=floppy -drive format=raw,file=msdos622.img,index=0,if=ide -m 8 -cpu 486

After starting QEMU, the VM booted from the floppy image Disk1.img and initiated the installer.

Guided by the installer, I partitioned the virtual hard drive and let the installer start copying the contents from the floppy image to the virtual hard drive. Once the installer is done copying the files from the first floppy, it will request the user to insert the second floppy into the disk drive.

To replace Disk1.img with Disk2.img (and Disk3.img afterwards) in the VM, I switched to the QEMU monitor by pressing Ctrl+Alt+2 inside the QEMU window. The QEMU monitor gives us complete control over the VM, so replacing the floppy image is possible with the following command:

change floppy0 Disk2.img

With the new floppy image in place, I switched back to the VM with Ctrl+Alt+1 and let the setup continue. Inserting the third floppy image works in the exact same way.

Once the installer completes processing all three floppies, it asks the user to eject the floppy from the drive, ensuring that the machine will boot from the fresh installation instead of the floppy drive.

Using the QEMU monitor again, ejecting the floppy image is as easy as

eject floppy0

One reboot later, we have the new installation up and running. Success!

Connecting to the monitor

After the initial setup, it is time to hook up the QEMU monitor to the host system.

Thankfully, QEMU provides a way to connect to the QEMU monitor via the network so that I can send arbitrary commands to the VM. Additionally, I switched the virtual console to text-only mode because the bot and thus QEMU is running on a server without a graphical interface.

qemu-system-i386 -hda msdos622.img -m 8 -cpu 486 -monitor tcp:localhost:9999,server,nowait -nographic &

Starting the VM this way will expose the QEMU monitor to port 9999 of the host machine. With the monitor exposed, the host machine can send the commands to port 9999 using tools like nc.

QEMU’s sendkey command will only accept one character at a time, which is not ideal. One way to mitigate this problem would be splitting the DOS commands into single characters. The much more convenient way is using a script called sendkeys.awk by zambonin .

This script accepts full strings and split them into single characters, including some conversion magic:

$ echo "ver" | awk -f sendkeys.awk | timeout 1 nc localhost 9999
QEMU 8.2.0 monitor - type 'help' for more information
(qemu) sendkey v
(qemu) vsendkey e
(qemu) esendkey r
(qemu) rsendkey ret
(qemu)
...
MS-DOS Version 6.22


C:\>

With the DOS VM listening to the input, it is time to take care of the output!

Creating screenshots with the screendump command

QEMU provides the screendump command through the monitor, which does exactly what I needed to create the screenshots. The command creates a bitmap file in the PPM format, which resembles a simple bitmap file.

Because this format is pretty non-standard nowadays and incompatible with most web applications, the best way to convert the files is with the convert utility provided by ImageMagick .

convert ms-dos-vm-screendump.ppm ms-dos-vm-screendump.webp

The JSON data file

As mentioned earlier, I am using a JSON file to store the commands along with their descriptions. Each entry in the data file follows the same pattern:

  {
    "command": "SYS",
    "description": "Copies MS-DOS system files to a disk's master boot record.",
    "verbose_description": "The SYS command is used to copy MS-DOS system files to a disk's master boot record (MBR), making it bootable. It's often used to create bootable system disks for MS-DOS installations.",
    "show_help": "yes"
  },

Each message the bot posts contains the command, the description and a more verbose description in a second paragraph. If show_help is set to yes, the bot passes the /? argument to command.

I use this option for commands that provide no meaningful screen output when invoked without additional parameters like files or disk drives.

The script

With the VM prepared and the JSON file in place, there is only one step left: Writing a simple script that picks a command, launches the VM, grabs the screenshot, and uploads it through the Mastodon API.

Because the content of the JSON file is dynamic and can change quite often, I want to do as little hard coding as possible. First, I check the length of the JSON file to see how many commands the script can pick from.

After picking a random command, the individual elements for the JSON entry are concatenated into the final string QUOTE, which contains the complete post message.

# Calculate number of entries in json datafile
MAXNUMBER=$(jq '.[].command' dosreference.json | wc -l)
MAXNUMBER="$((MAXNUMBER-1))"
RANDOMNUMBER=$(shuf -i 0-$MAXNUMBER -n 1)

# Fetch random entry from json datafile
COMMAND=$(jq -r --argjson randomnumber "${RANDOMNUMBER}" '.[$randomnumber].command' dosreference.json)
COMMAND_DESCRIPTION="$(jq -r --argjson randomnumber "${RANDOMNUMBER}" '.[$randomnumber].description' dosreference.json) \n\n"
COMMAND_VERBOSE_DESCRIPTION="$(jq -r --argjson randomnumber "${RANDOMNUMBER}" '.[$randomnumber].verbose_description' dosreference.json)"
COMMAND_SHOW_HELP="$(jq -r --argjson randomnumber "${RANDOMNUMBER}" '.[$randomnumber].show_help' dosreference.json)"
QUOTE="status=$(echo "${COMMAND}": "${COMMAND_DESCRIPTION}""${COMMAND_VERBOSE_DESCRIPTION}")"

if [ "$COMMAND_SHOW_HELP" = "yes" ]; then
    COMMAND="${COMMAND} /?"
fi

Then, I start the VM and pass the command crafted from the JSON output to it.

After taking the screenshot, the VM is killed through the monitor, ready to relaunched the next time the cron job responsible for the script is triggered.

qemu-system-i386 -hda msdos622.img -m 8 -cpu 486 -monitor tcp:localhost:9999,server,nowait -nographic &
sleep 30

# Execute command in VM
echo "cls"        | awk -f qemu-sendkeys.awk | timeout 1 nc localhost 9999
echo "${COMMAND}" | awk -f qemu-sendkeys.awk | timeout 5 nc localhost 9999

# Create screenshot of VM
echo "screendump ms-dos-vm-screendump.ppm" | timeout 1 nc localhost 9999

# Kill VM
echo "quit" | timeout 1 nc localhost 9999
convert ms-dos-vm-screendump.ppm ms-dos-vm-screendump.webp

The timing here is completely hard-coded because I cannot determine if the VM has fully launched. Since the server hosting the bot also provides some other services, I went with a safe timeout of 30 seconds - this should be well enough to get a bare MS-DOS VM ready. The timeouts for the calls to nc are required, though - without it, nc would wait ’til eternity for me to hit the “Enter” key.

The final step: Upload the screenshot to Mastodon, wait for it to process, and attach the post message.

# Upload image to Mastodon
MEDIA_ID=$(curl -s -X POST -H "Content-Type: multipart/form-data" https://manitu.social/api/v2/media -H 'Authorization: Bearer XXXXXXXXXXXXXXXX' --form file="@ms-dos-vm-screendump.webp" | jq -r '.id')

# Wait for the image to process...
sleep 10

# Go!
curl -g https://manitu.social/api/v1/statuses -H "Authorization: Bearer XXXXXXXXXXXXXXXX" -F "$QUOTE" -F "media_ids[]=${MEDIA_ID}"

And we are done!

Connecting MS-DOS 6.22 to the internet --- and the fediverse

Sat, 05 Aug 2023 22:00:00 +0200

Microsoft released the last standalone version of MS-DOS as MS-DOS 6.22 in 1994.

Even though this operating system is decades old and a true relic of the past, people still develop new software for it. Thanks to a couple of modern tools, we can connect MS-DOS to the internet. Since just having a connection to the outside world itself is pretty boring, we connect MS-DOS 6.22 to something fairly modern and amazing: the fediverse.

Let’s have a look.

A little bit of history

Today, getting connected to the internet is very simple: Just enable your WiFi or connect an ethernet cable and you are good to go. In many cases, everything just works without having to configure anything.

Modern systems include all the drivers and tools you need for an internet connection right out of the box.

Back then, things were not that easy.

For connecting a machine running MS-DOS to the internet, you needed the matching packet driver for your network card and a set of tools for providing the TCP/IP stack. Yes, TCP/IP is not the only way to do networking, and since other protocols were a lot more popular in the world of small to medium-sized businesses, it was a common practice to ship the OS without any networking capabilities so you could install whatever software your network required.

Setting up the connection

Thanks to emulation, hardware compatibility isn’t a problem anymore, so we can just emulate a network card compatible with the widespread NE2000 standard which 86Box conveniently supports.

As I mentioned before, we need both the networking driver and the TCP/IP stack. For the driver, we are going to use the NE2000/NE2100 driver available at http://www.georgpotthast.de/sioux/packet.htm . Support for TCP/IP will be provided by the mTCP toolset .

If you are mainly interested in connecting MS-DOS to the fediverse, you only need the packet driver, since DOStodon, the software we will be using, includes a full TCP/IP stack. However, if you want to use your brand new Internet connection for other software, I recommend that you install mTCP as well.

In 86Box, we can use SLiRP for connection handling, so we don’t need to worry about it on your host machine at all. SLiRP’s virtual router is able to provide dynamic networking configuration with DHCP, making it the perfect match for mTCP’s DHCP client.

To add the virtual network card to the VM, open the VM settings and use the following configuration:

Mode: SLiRP
Adapter: [ISA16] Novell NE2000
Address: 0x300
IRQ: IRQ 3 (or any other IRQ if IRQ 3 is already used by other virtual cards)
BIOS address: Disabled

The hardware is done, now to the software side of things!

Part 1: The internet.

To add some life to the virtual network card, we extract the .ZIP archives of the NE2000/NE2100 driver and the mTCP package. The software is pretty small and will fit on a single 1.44MB floppy disk. For this experiment, I created a floppy image with WinImage and copied all the files to it.

Do not skip the SAMPLES directory of the mTCP package - we’ll need it later!

Installing the networking stack is pretty easy. After booting the VM, we mount the floppy image containing the driver and mTCP to the VM and copy everything to the C:\NET directory.

After copying the files, it is time to check if the networking driver works. To do so, we try to load the NE2000 driver manually:

C:\NET\NE2000.COM 0x60 3 0x300

Great, the driver found the network card!

Now, we can try to get an IP address. First, we have to copy the file SAMPLES\SAMPLE.CFG to C:\NET\MTCP.CFG. Since we are using only the DHCP client and none of the other services (like an FTP or HTTP server) from the mTCP package, the default configuration works just fine.

Prior to using mTCP, we have to tell it where the configuration file is located by using the SET MTCPCFG=C:\NET\MTCP.CFG command. Finally, we can try to get an IP address.

As you can see, the DHCP server provided by the virtual router assigned an IP address to the VM. The DHCP server doesn’t seem to support requesting a hostname, but for our purposes, this won’t matter at all.

Time to test if outbound connections work: C:\NET\PING.EXE felsqualle.com.

Now that we confirmed that the setup we want to run is working properly, it is time to auto-load it in the AUTOEXEC.BAT file with the following modifications:

SET PATH=C:\DOS;C:\NET
SET MTCPCFG=C:\NET\MTCP.CFG
LH C:\NET\NE2000.COM 0x60 3 0x300
C:\NET\DHCP.EXE

These modifications will ensure that the networking stack will be loaded each time the system starts up.

Part 2: The fediverse.

Now that we have a working internet connection, it is time to connect MS-DOS to the fediverse.

Let me introduce you to DOStodon , a Mastodon client for MS-DOS, created by SuperIlu . DOStodon is powered by DOjS , a JavaScript implementation for MS-DOS which is also created by SuperIlu.

Installing DOStodon is very simple. All we have to do is to get a copy of the git repository , which includes the necessary program files already compiled.

I transferred the extracted files to an ISO image and mounted the image in the virtual machine. Then, I copied all files to a new directory.

When starting DOStodon for the first time, we have to pass the login credentials as arguments. Since DOStodon stores the login information, we only have to do this on the first start.

C:\PROGRAMS\DOSTODON\DOSTODON.BAT example.com youraddress@example.com yourPassword123`

After a couple of seconds, DOStodon starts up and shows us the Home timeline. As you’ll see in the next couple of screenshots, this is not just a simple tech demo. DOStodon is a complete Mastodon client with all the timelines, hashtag search, and a compositor for new posts and replies.

And because a picture is worth a thousand words, I’ll close this article with some screenshots.

Enjoy!

Smashing the limits: Installing Windows XP in DOSBox-X

Wed, 26 Jul 2023 00:00:00 +0200

In my previous article , I described how I managed to install Windows 2000 in DOSBox-X.

Even though this experiment was successful, I was not really happy with the results. While I got Windows 2000 working, I didn’t want to stop there. The final goal for the project was to get Windows XP running instead. However, after multiple attempts I gave up, thinking that Windows XP was impossible to use.

Well - I was wrong. But let’s start at the beginning.

At the time of writing this article, running Windows XP isn’t officially supported by the DOSBox-X project. Don’t blame the developers for any issues - this is very experimental.

Previously…

Since I wasn’t able to do a clean install of Windows 2000 due to driver and compatibility issues as described in my previous article, I started with the upgrade route right away. Upgrading from Windows 98 to Windows XP is officially supported, so I was pretty confident this would work for me as well.

I created a copy of the hard drive image after each step, so I was able to start the upgrade procedure with a clean installation of Windows 98 (Second Edition).

After mounting the Windows XP ISO image, I was greeted by the Windows XP Setup.

Because of the issues I had with the clean installation of Windows 2000, I used the “Upgrade” installation type instead because that’s what worked for Windows 2000 as well.

After typing in the product key, the Setup was about to start the file copy process. In fact, it even started copying the installation files but failed at the very first file.

“The ISO must be bad!” I said to myself. I obtained this ISO file many years ago, so some silent file corruption was not entirely impossible. To avoid any potential issues caused by bad images, I always create checksum files immediately after obtaining the image. So, I compared the ISO image with the checksums I had - and to my surprise, I got a perfect match. The ISO image itself is absolutely fine since I have used it multiple times before.

To further investigate, I even went ahead and reinstalled Windows 98 from scratch to rule out any possible installation issues, but to my dismay, I encountered the same error message again.

Great. Getting another ISO image or starting with a fresh installation would have been way too easy, right?!

Back to square one.

Since the upgrade didn’t work as planned, I decided to proceed with a clean installation instead. Unfortunately, I encountered difficulties booting DOSBox-X from the ISO image. As a workaround, I launched the Setup from within another fresh Windows 98 installation just like I did for Windows 2000.

After my previous experience with the Windows XP Setup, I mentally prepared myself for yet another potential failure. However, to my surprise, this time the Setup didn’t just throw an error message at me; instead, it smoothly proceeded to copy over the temporary installation files. Crisis averted!

One reboot later, I was greeted by the beloved and very familiar blue hue of the Windows XP Setup.

And now - the moment of truth. After confirming that we want to overwrite the existing Windows installation, Windows XP Setup started the next step of the installation by copying over the temporary installation files to the Windows directory.

I can’t say that I was surprised when the project stumbled across yet another issue. After copying over some files atrociously slow, it failed. No matter how often I retried, it always failed. I couldn’t track down the issue to one particular file though because on subsequent runs (yes, I tried!), it failed at a different file.

Based on the experience I had with the Windows 2000 installer I think this is related to either the IDE driver or the emulation of the IDE controller and the devices are simply dropping out of the system.

The next obvious way to solve this issue was to reboot the system and try to run Setup again. Unfortunately, this wasn’t an option either because after a reboot it simply failed to copy the very first file - turns out that Windows Setup already started to overwrite vital parts of the previous installation.

At this point, I gave up. Running Windows XP in DOSBox-X clearly wasn’t possible. Heck, I couldn’t even get the installation to work!

New Technology from 1999

“It is the NT conversion, stupid!”, I yelled at myself a couple of days later.

While Windows 98 is using MS-DOS to bootstrap the system and uses parts of it in its kernel, Windows XP uses the Windows NT platform instead. Just like Windows 2000!

What if swapping out the kernel requires some sort of low-level access I don’t have in DOSBox-X?

So I installed Windows 98 again, updated it to Windows 2000, fixed the missing taskbar, verified the system was working, mounted the Windows XP ISO, and started the installer again.

Because I was afraid that a clean installation would fail again, I went with the upgrade route once more. Not really knowing what to expect, I clicked on the magic button and waited.

And yes - this time, the upgrade seemed to work! No error messages this time!

One reboot later, I noticed something very promising.

Instead of starting in the text-based installer again, I got a colorful graphic installer, indicating that Windows Setup was taking a different route this time. And amazingly, the installation ran just fine without any issues.

With this part of the installation complete, there was one last reboot to survive.

Windows XP was bootable and the installation itself was complete!

Soon thereafter, things started to go downhill though. After the first boot, Windows XP is supposed to start the so-called “Out of Box Experience” (OOBE) in order to finalize the installation by setting up some options and the user account.

The OOBE started, but what an experience it was! This wasn’t an experience. This was pure madness!

The beautiful setup music was cluttered with tons of distortion sounding like incredible compression artifacts caused by a very unhappy WMA decoder.

In fact, the performance was so poor that the system completely locked up and I had no other choice than rebooting the system.

Fortunately, Windows XP is able to migrate the user accounts from Windows 2000 because I had no option to create a new account - I guess I avoided yet another dead end.

And yet another reboot later - there it was. In all its glory.

I did it. Windows XP is running successfully in DOSBox-X.

Obviously, I had to retry the games I tried on Windows 2000. The Sims in the various versions I tried before still failed, but this was expected. Roller Coaster Tycoon improved by a lot. I couldn’t spot any sound issues this time and the game was perfectly playable.

And last but not least: Yes, the iconic 3D Pinball: Space Cadet worked flawlessly as well.

Closing words

This is what I like about emulation and playing around with those systems so much. Even though some tasks seem to be impossible at first, there are always new routes to explore, helping you to achieve things many people (including yourself!) say are impossible.

I love it.

Breaking the limits: Installing Windows 2000 in DOSBox-X

Mon, 03 Jul 2023 22:00:00 +0200

DOSBox was one of the first emulators I used when I learned about emulation back in the early 2000s. Originally developed with its own DOS implementation for running games and legacy applications in mind, newer forks like DOSBox-X and DOSBox Staging started to support “real” system emulation as well, providing similar functionality as PCem or 86Box .

While originally only targeting MS-DOS emulation, DOSBox-X provides official support for Windows 95 and Windows 98.

“Well,” I said to myself, “if they already support Windows 9x, I could try running Windows 2000 on it! It shouldn’t be that hard, right?!”

At the time of writing this article, running Windows 2000 isn’t officially supported by the DOSBox-X project. Don’t blame the developers for any issues - this is very experimental.

But Windows 2000 isn’t DOS!

Yes, that’s correct. While Windows 95, 98, and ME (also referred to as the “Windows 9x series”) are using MS-DOS as part of their kernel, Windows 2000 runs on the Windows NT platform. Clearly, I won’t be able to start it using the DOS runtime environment DOSBox-X provides, but using their full system emulation should be possible.

As the base configuration for DOSBox-X, I used the configuration template for Windows 98 provided in the DOSBox-X wiki . Since the recommended system specifications for Windows 2000 include 128 MB of RAM, I modified the configuration file accordingly.

I started by creating a virtual hard drive image with a size of 8GB using the IMGMAKE command and mounted both the virtual hard drive as well as the Windows 2000 ISO file.

Since I wasn’t able to boot the Windows 2000 ISO image, I used the boot floppy images shipped on the Windows 2000 disc to start the installer.

A promising start

I was pretty surprised that the floppy-based installer started without any issues. Expecting a smooth journey, I cycled through the three floppy images.

After loading the files from the third floppy, the Windows 2000 installer checked the system hardware, tried to find the virtual hard drive - and failed.

Great, what an amazing start! I tried to reboot the virtual system a couple of times, but without success. It looks like the IDE driver in Windows 2000 doesn’t support the virtual IDE interface created by DOSBox-X. Sure, you can load additional drivers when starting the Windows 2000 setup by pressing the F6 button, but without a matching driver, this won’t work either.

Plan B: Choosing the update route!

Luckily, I might be able to work around this issue by attempting to upgrade a Windows 98 installation to Windows 2000 instead.

To do so, I mounted the Windows 98 (Second Edition) ISO file in DOSBox-X. To make sure that the hard drive image is clean, I used the FORMAT utility shipped on the Windows 98 CD-ROM to initialize the virtual hard drive.

While the retail versions of Windows 98 are not bootable by default, the CD-ROM for the OEM release can boot straight from the CD using the El Torito format. Since the release I had was indeed a OEM version, I didn’t have to work with a boot floppy this time. As recommended by the DOSBox-X documentation, I set the emulation core to normal instead of dynamic_x86 to avoid crashes during the installation.

The installation itself was pretty easy and completed without any issues. I opted for a “minimal” installation since my plan was to replace the Windows 98 installation with Windows 2000 after the first boot.

Part one: Done!

Part 2: The Windows 2000 upgrade

Now it’s time for the exciting part of the project: attempting to upgrade the new Windows 98 installation to Windows 2000.

I loaded the Windows 2000 ISO image and launched the installer. After selecting the upgrade option, the setup asked if I wanted to convert the hard drive to the NTFS filesystem. Since I wasn’t able to boot from NTFS during my earlier attempts, I skipped this step and went with FAT32 instead.

I assume that the hard drive emulation is only able to boot from FAT, but this is just an educated guess and not being able to boot from NTFS might just be the result of a configuration error on my end.

Now that the first part of the setup is complete, I should be able to boot into both the Windows 98 installation and the Windows 2000 installation.

The moment of truth…

I restarted the virtual machine and selected the Windows 2000 setup from the boot menu. After loading the initial setup files from the hard drive, the Windows 2000 installer checked the system hardware, tried to find the virtual hard drive - and successfully detected it!

Immediately, the installer proceeded to the next stage of the installation by copying all the required systems files. Usually, these files are copied from the CD-ROM, but since I’m performing an upgrade instead of a clean installation, the files are copied from the hard drive itself.

A few minutes later, the system attempted to boot into the newly installed system. I honestly didn’t expect it to work, but to my surprise, it booted successfully on the first try.

What a relief!

Being able to boot into the next stage of the Windows 2000 installation indicated that it might just work.

And it did! The system started the final part of the installation, performing the hardware initialization and completing the final setup.

Mission complete?

After completing the final steps of the installation, I had to reboot the system one more time. At this point, I was feeling very confident that I would have a system that just works. I pressed the magic “reboot” button, patiently waited for the boot process to complete, logged into the system, and…

Oops.

The system started, but for some reason that I have yet to discover, explorer.exe constantly crashes. I attempted to restart it multiple times through the Task Manager and even rebooted the system, but without success.

As a result, I don’t have a functioning taskbar, and I’m unable to browse the system using the file explorer.

Luckily, I was able to run cmd.exe through Task Manager, so accessing the system and launching additional programs was still possible.

Even though the Explorer was completely broken, Internet Explorer on the other hand worked just fine. Since Internet Explorer allows local file browsing, I had a very nice replacement on hand.

But well, I should probably fix it.

Update Rollup 1 to the rescue!

Since I had no idea what caused the crashes, my first troubleshooting step was to install all available updates.

The Windows 2000 ISO I used for the installation already included the latest Service Pack for Windows 2000. However, at the end of the Windows 2000 lifecycle, Microsoft released a package called “Update Rollup 1” which bundled all the updates they had released for Windows 2000.

The Update Rollup 1 package (Windows2000-KB891861-v2-x86-ENU.exe) is still available online through various archival sites. I created a new ISO image with the update file, mounted it in DOSBox-X, and started the installation.

After what felt like an eternity and a reboot later, I noticed a slight difference compared to the previous reboot.

No more Explorer crashes and a working taskbar!

What about gaming?

With a working base system, I wanted to see what I can do with the system. First, I tried to run 3D Pinball Space Cadet.

The game works perfectly fine, with smooth graphics and flawless sound.

However, the next games I tried provided a much worse experience. Rollercoaster Tycoon starts just fine with smooth graphics, but the sound is completely unusable with heavy stuttering. I tried to work around it by tuning the DirectX acceleration settings, but I had no luck.

Next, I tried to install a copy of The Sims. The original installer for the base game repeatedly crashed DOSBox-X with an error message indicating some unimplemented features. The installer for The Sims: Deluxe Edition crashed Windows 2000 with a bluescreen. The only version I was able to successfully install was The Sims: Complete Collection.

Notice how I wrote install instead of run here? As soon as I tried to start the game, it simply crashed with an error message. At this point, I wasn’t too surprised - I guess that the CD-ROM emulation in DOSBox-X is simply not able to handle the SafeDisc copy protection of the game.

Closing words

From a usability point of view, this project doesn’t make any sense at all. For running games from the late 90s to the early 2000s, Windows 98 provides much better compatibility. You can either run Windows 98 in DOSBox-X or - if you want to be able to emulate a specific system - use a full system emulator like 86Box instead. Since 86Box emulates real, existing hardware, driver issues are much less likely as well.

But from a more technical perspective, this project was both interesting and impressive. Trying to see what emulation can achieve is always fascinating - and sometimes, you get results that seem to be impossible at first glance.

Encounter wild Pokémon in your terminal!

Wed, 21 Jun 2023 21:00:00 +0200

A few days ago, I posted an image that showcased an ASCII art representation of a Raichu from the Alola region. This image appeared after logging into my server using SSH. Surprisingly, it sparked curiosity, and I received several inquiries about the magic behind it.

Let’s delve into the details.

If you haven’t heard about pkmn.li , you should definitely have a look. pkmn.li is a website that shows random ASCII art of Pokémon each time you open the website. glow, the author of the project, provides a lot of interesting details about it on their website .

Since the output provided by the website contains all the necessary ANSI sequences for coloring, and assuming that your terminal supports true color (which it probably does), you aren’t limited to a web browser.

Instead, you can fetch the website with curl and view the sprites right in your terminal.

All I had to do was add the command curl https://pkmn.li to the .bash_profile file of my shell account. Since the contents of the .bash_profile file (or .bashrc on some systems) are executed each time a shell session is opened, you’ll get a random Pokémon encounter each time you open your terminal or launch an SSH session.

However, this only works for Bash, so if you are using a different shell on your system, you’ll have to find the matching “autostart” file.

Gotta catch ’em all!

Meta is a danger to the fediverse and we have to be prepared

Mon, 19 Jun 2023 21:00:00 +0200

Back in March 2023, we heard about Meta’s plans to join the fediverse for the first time after they confirmed that they are working on a Twitter-like alternative using the ActivityPub protocol. Codename: Project 92.

Great! Now we can connect with even more people using our familiar apps! But there’s a catch.

In the last couple of days, new rumors appeared.

According to various sources, some fediverse admins had secret meetings with Meta under NDAs.

Talking about an open network like the fediverse with a huge company behind locked doors is an incredibly bad idea. I think the fact that they don’t want to publically discuss what Meta’s plans for the fediverse are proves that they most certainly won’t care about openness or federation.

Sure, they will connect to the existing network. But what if their platform gains enough traction so that a substantial amount of users switch over to their instance? At this point, they only have to lock down their network again. Or maybe they propose changes to ActivityPub that implement a recommendation algorithm. Or an ad platform.

Meta doesn’t care about the users. They care about their company and increasing value for their sharesholders. Always remember that the users are nothing but a source of revenue for them.

If we want the fediverse to survive as it is right now, we have to step in right at the beginning. We shouldn’t federate with them. We should once and for all prove that social media platforms run by big corporations are nothing but a failed experiment and a relic from the past.

Be vigilant.

Export your own followers from Mastodon using the Mastodon API

Fri, 16 Jun 2023 21:00:00 +0200

With decentralization and federation in mind, Mastodon allows exporting a list of all the accounts you follow directly from within the GUI.

This function generates a CSV file you can later import to another server. This function is especially helpful if you want to create a backup of your data or if you don’t want to do a full migration (and lose your first account), but open up a second account instead.

Unfortunately, Mastodon doesn’t provide an option to export a list of your own followers. Well, at least not from within the GUI. But hey, we have an API to play with!

This is a proof of concept. While the following script works for me, there is absolutely no sanity checking. It worked for my pretty small accounts, but I can’t guarantee that you won’t hit rate limits on larger accounts or crowded servers.

Please do not use this script to scrape accounts that are not yours!

If you want to run this script against an instance that is not entirely controlled by you, I recommend asking the owner/operators first if they agree with mass-querying their API and importing large amounts of data.

I came up with the idea when I started my soft migration to a secondary account. In order to grow my network, I decided that I want to follow back as many accounts as possible.

Importing the accounts I follow was no problem at all since the Mastodon web interface provides an option to export your own follows. The generated CSV file can be imported on a second account, bringing back all your follows in the blink of an eye!

But what about your followers? As you might notice, the CSV option is missing in the web interface. Well, usually you don’t need to export your followers. While you can control which accounts you want to follow, your followers are an entirely different story.

Regardless of account migrations or recovery after data loss, you can’t bring back your followers unless they decide to follow your account again. This might be the reason why a GUI option to export your followers is missing - most people will never use it.

However, the necessary data is accessible using the Mastodon API. All we need is bash, curl, jq, and sed.

The Mastodon API provides an endpoint called /api/v1/accounts/$userid/followers that returns the followers of any account. Fortunately, we don’t have to deal with authentication since the endpoint is entirely public.

But there’s a catch: Calling the API only returns 40 to 80 results, depending on the configuration of the instance and the query options. While this looks like an artificial limitation at first glance, it prevents the API to return too large replies - just assume your account has 10.000 followers.

To fetch all accounts, we have to iterate over multiple virtual pages of the API result until we run out of pages to query. After the last page is fetched and we get no reference to a “next” page, we know that the operation is finished.

The script itself is pretty straightforward:

 1#!/bin/bash
 2# Fetch own followers via the Mastodon API
 3# Tested up to Mastodon v4.3.3
 4
 5# Enter your user details
 6username="yourMastodonUsername"
 7instance="https://example.com"
 8
 9# Preparations
10echo "Get account ID from the given username..."
11userid="$(curl --silent "${instance}/api/v1/accounts/lookup?acct=${username}" | jq -r .id)"
12
13echo "Found account ID: $userid"
14initial_url="${instance}/api/v1/accounts/${userid}/followers"
15
16# Fetch initial JSON output
17echo "Checking URL ${initial_url}..."
18json_output=$(curl --silent "${initial_url}")
19nextstep=$(curl --head --silent "${initial_url}" | grep -i "next" | cut -d ";" -f1 | sed -n 's/.*<\(.*\)>.*/\1/p')
20
21# Main loop: Fetch all remaining accounts
22while [ "${nextstep}" ]; do
23  echo "Checking URL ${nextstep}..."
24  json_output+=$(curl --silent "${nextstep}")
25  nextstep=$(curl --head --silent "${nextstep}"  | grep -i "next" | cut -d ";" -f1 | sed -n 's/.*<\(.*\)>.*/\1/p')
26done
27
28# Save output to CSV file
29echo "Writing CSV file..."
30timestamp=$(date +%s)
31echo "Account address,Show boosts,Notify on new posts"       >  ${username}_follower_accounts_${timestamp}.csv
32
33echo "${json_output}" | jq -r '.[].acct | .+ ",true,false,"' >> ${username}_follower_accounts_${timestamp}.csv
34echo "done!"

First, we have to get our account ID. We can easily perform a lookup using the /api/v1/accounts/lookup?acct= endpoint. Next, we grab the first batch of JSON by querying /api/v1/accounts/$userid/followers. This will not only return the first round of accounts but also provide the “next” URL in the HTTP response. Now, we simply extract the next URL, get the JSON reply from the API and check if we have a new “next” URL. We repeat this process until there is no “next” URL present in the HTTP response which indicates that we are on the very last page.

Finally, we have the entire API response in the json_output variable.

The last step consists of writing a proper CSV header and querying the JSON output we got earlier using jq. Finally, we write the prepared output into the CSV file.

The CSV we created has the same format as the following_accounts.csv created by the web interface when using the export of your follows. Thus, we can use this CSV and import it to the “Following list” using the web interface and follow all of our followers at once!

Virtual archaeology: Recreating the very first PC I ever used

Mon, 12 Dec 2022 08:00:00 +0100

There are some things in life you will remember forever. In case you are just like me, then the very first PC you ever used is most likely one of these things.

I still remember the specifications of the family computer 6-year-old me used in 1999, even though its mainboard died around 15 years ago.

It was a Packard Bell PC, powered by an Intel Pentium II @ 233MHz, 32 MB of RAM, a 20x CD-ROM-Drive (no DVD!), and a Matrox Mystique GPU.

I knew that the system was running Windows 95 with a custom OEM software bundle. For years, I thought that rebuilding the system with the original software was impossible. The recovery CD was long gone and there was no copy to be found on any archival site.

By pure luck, I recently found a copy of the original recovery CD for sale online - I simply had to get it. It turned out that trying to recreate the system on a virtual machine was quite an adventure involving a deep dive into the inner workings of the Packard Bell recovery process.

The system in detail

Even though I still know most of the specifications, the exact model of the system was unknown, so I couldn’t easily try to find a recovery CD based on the model number or name. According to the specs I had, the most likely candidate is a Packard Bell Platinum 3000 from late 1997. Looking at the picture of the case, I was sure that this was the model I had. Luckily, this case model (codenamed ‘Milano’) was pretty rare and this model was the only one I could find with this exact case and system specifications.

The specifications don’t match exactly though, e.g. I remember that the hard disk was a bit smaller and closer to the 2 GB range. Furthermore, the included software listed on that page doesn’t match, but this might be caused by regional differences.

Finding some detailed information about the exact hardware used for this model turned out to be more difficult than I initially thought. After some research, I think the most accurate match I could find is a system with the codename Pulsar 24 based on the ‘Hermes’ platform.

Since Windows 95 is horribly broken in any modern virtualization environment, I used 86Box instead. Choosing 86Box also allowed me to pick the hardware I wanted to emulate, helping to recreate the system as closely as possible. 86Box doesn’t support the exact hardware though, so I had to improvise a bit.

Type	Original hardware	Emulated hardware	Explanation
Base machine	PB Pulsar 24	Micronics Spitfire	Original model unsupported, same chipset (Intel 440LX), same mainboard?
CPU	Pentium II (Klamath) @ 233 MHz	Pentium II (Klamath) @ 166 Mhz	Host system limitations
RAM	32 MB	32 MB	./.
Hard Drive	~2 GB	~8 GB	Allowing for future experiments and upgrades
GPU	Matrox Mystique 200	S3 Vision964 (ELSA Winner 2000 Pro/X)	Original model unsupported
Sound card	Aztech Sound 16C	Sound Blaster 32 PnP	Original model unsupported
Network	USR X2 Winmodem	Realtek RTL 8029AS	Original model unsupported, no networking with Host
CD-ROM	NEC CD-ROM drive, 20x	HITACHI CDR-8130, 20x	Original model unsupported
Floppy drive	3.5in, 1.44 MB	3.5in, 1.44 MB	./.

This configuration was still very close to the original machine, thanks to the Micronics Spitfire I could emulate, which is likely the same mainboard as the one used for the Packard Bell Platinum 3000. It even shares the same BIOS revision.

The other components are pretty sane defaults, offering good driver support in Windows 95.

Getting started

Luckily, I remembered parts of the software collection that was bundled with the system I used back in the day. The online listing for the CD I bought only referred to the software collection, so I wouldn’t have been able to locate it just by the system’s model name.

Unfortunately, the part number for the Master CD listed for the Hermes platform doesn’t exactly match the copy I have, but it is still a very close match. While Hermes is supposed to have a Master CD with the part number 6709881101, the copy I have has the part number 6709880503.

The last two digits are not important because they are indicating the language used on the CD. While 01 marks an English release, 03 is the language code for Germany. All in all, I wasn’t too worried about the part numbers because the most likely explaination is a regional difference again.

A much bigger problem was the missing accompanying boot floppy that initializes the system restore process, and the fact I couldn’t find a copy online. I tried to start with a different period-correct Packard Bell boot floppy from archive.org, but all I got was an installer that immediately crashed.

Thinking it was a bad boot floppy that caused the issue, I started the system with a generic Windows 95 boot floppy and launched the setup file on the CD manually. For some reason though, the installer started and immediately told me the restore process was completed. Unsurprisingly, it was not. The hard drive was as empty as before.

This meant that I had to find another way to start the installation, bypassing the original setup file.

A first look at the Master CD

To be able to understand how the recovery process works, I started by examining the file layout on the recovery CD.

[ROOT DIRECTORY]
│   CRCLOG.DAT
│
├───ARJ
│       ARJ.EXE
│       ARJ.PIF
│       WINDOWS.ARJ
│
├───CABS
│       ACROREAD.CAB
│       ADIBOU.CAB
│       AIMS.CAB
│       AUTOEXP.CAB
│       CDEXTRA.CAB
...
│       WORD8_5.CAB
│       WORD8_6.CAB
│       WORKS_1.CAB
│       WORKS_2.CAB
│       WWHACKE.CAB
│
├───TSRC
│   ├───3DMOUSE
│   │       3DMOUSE.INF
│   │
│   ├───ATI
│   │       NOFILE.TXT
│   │
│   ├───AZTECH
│   │       AZT472.CAB
│   │       AZT480D.CAB
...
│   │
│   ├───COMMAND
│   │       ANSI.SYS
│   │       ATTRIB.EXE
│   │       CHKDSK.EXE
│   │       CHOICE.COM
│   │       COUNTRY.SYS
│   │       DEBUG.EXE
...
│
└───WIN95
        ARJ.EXE
        ARJ.PIF
        BIOSLOCK.EXE
        BIOSLOCK.PIF
        KILL.COM
        KILL.PIF
        MASTCOUN.INI
        OEMSETUP.EXE
        PBHOUSE.PCX
        PBRESTOR.EXE
        PCXDUMP.EXE
        PSCRIPT.EXE
        PSCRIPT.PIF
        ROMAN.FON
        SVGA256.BGI
        TREECRC.EXE

The file layout doesn’t look too complicated. ARJ contains the main archive with the base installation, CABS contains all the additional software. By looking at the names of the .cab files, I was able to verify that the entire preinstalled software library as I remembered was there. Yay!

In the TSRC directory, we find some additional drivers and what looks like a very basic DOS installation. The WIN95 directory holds the installer and restoration executables I didn’t manage to get up and running previously.

Looking into the WINDOWS.ARJ archive was very promising. Considering the filenames and folders present in the archive, it looked like a complete factory image of the hard drive. In theory, all I had to do is uncompress it to the hard drive and start working from there.

Moments later, I was incredibly disappointed when I noticed a very small, but very important detail in WinRAR’s file listing.

Do you see the * at the end of each file? This means that the files are encrypted and you can’t uncompress them without the proper password. And unfortunately, I don’t have the password.

Back to square one.

Great. With all the files encrypted, there is no way to simply extract the archive as I initially wanted to do. I had to take a closer look at the executables for the restore process itself.

A quick examination of the non-matching-but-close boot floppy revealed that the file for starting the system restore process is \WIN95\OEMSETUP.EXE. Not sure where to begin, I quickly ran grep "WINDOWS.ARJ" OEMSETUP.EXE - and there was a match in the binary file. This was very promising because it meant that there was at least a reference to the magical archive file in the setup program.

The next step was to try to find some more valuable information in the executable by using the strings command provided by my MinGW64 environment. While looking through the program output, I found something amazing:

%c:\win95\arj.exe x -y -i -gP127A783D %c:\arj\windows.arj c:\
md c:\cabs
%c:\win95\arj.exe x -y -i -gP127A783D %c:\cabs\inf.arj c:\cabs
%c:\content.txt
[switches]
%s %c:\cabs\*.* C:\cabs /S

There it is! The archive password and all the instructions I needed to get the base installation up and running.

Extracting the base archive

With the instructions I found in the OEMSETUP file, it was time to start the manual reinstallation process. I booted the system using my generic Windows 95 floppy and prepared the hard drive using FDISK and FORMAT. Since the archive contains the files MSDOS.SYS, IO.SYS, and COMMAND.COM, I made the hard drive bootable by running FDISK /MBR immediately after partitioning to get a bootable system.

Time for the moment of truth.

I started the extraction process by running D:\WIN95\ARJ.EXE x -y -gP127A783D D:\ARJ\WINDOWS.ARJ C:\, not exactly knowing what to expect. After all, it wasn’t an exact match for the Master Boot Floppy, so all I could do was hope for some sort of password-sharing.

Needless to say, I was very happy after seeing the following lines scrolling by:

Extracting the WIN95\WINDOWS.ARJ archive finished without further issues, so I continued by extracting CABS\INF.ARJ the same way. And luckily, this archive extracted just fine as well.

With both archives extracted, I was ready to copy over the remaining bits and pieces I previously figured out by examining the OEMSETUP executable. To document the process, I created a batch file that performed all the steps I knew the original installer would do.

@echo off
echo Prepare drive...
FDISK
FDISK /MBR
FORMAT C: /Q /U /V:PACKARDBELL

echo Extract encrypted archives...
D:\WIN95\ARJ.EXE x -y -gP127A783D D:\ARJ\WINDOWS.ARJ C:\
D:\WIN95\ARJ.EXE x -y -gP127A783D D:\CABS\INF.ARJ C:\CABS

echo Copy remaining cab files
D:\TSRC\COMMAND\XCOPY.EXE D:\CABS\*.* C:\CABS /S

echo Extraction done, please reboot!

After copying over the remaining .cab files from the CD, I was ready to remove the boot floppy and reset the system, hoping for a flawless recovery process.

For the fraction of a second, things looked pretty promising as I was greeted with the “Starting Windows 95…” banner.

However, instead of getting a successful boot I was confronted with an image telling me that the hard drive was locked and the boot process stopped. I tried resetting the system multiple times, but I couldn’t get the system to boot.

Not to be sold seperately

Back in the day, Packard Bell invented a couple of methods to ensure their recovery disks would work on the original system only. After some research, I learned that this led to the introduction of a hidden sector that is placed on the hard drive during manufacturing that holds some critical information.

The setup process checks if this sector is present which would indicate a genuine Packard Bell system. Moreover, this allowed Packard Bell to use the same Master CD for multiple system variants, e.g. by shipping various software bundles depending on the exact system specifications.

I realized that this also means easily swapping the hard drive wasn’t possible back then because of the missing hidden sector. In theory, a copy of the hidden sector could be placed on the Master Boot Floppy, but since I don’t have a valid copy, I can’t confirm that theory.

On the other hand, the service technicians had to have a way to rebuild the hidden sector in case the original software was lost and the hard drive had to be replaced.

Introducing HSCENTER

The tool that was used for managing this hidden sector is called HSCENTER. I was pretty surprised that this tool was included in the base image of the restore CD, so I didn’t have to find a third-party source.

I booted the system from my Windows 95 boot diskette again. HSCENTER is hidden in the C:\DRIVERS\ directory after the initial uncompressing stage is completed. Using HSCENTER /? revealed quite a lot of options:

Without a proper manual for the application, I had to figure out how to use the tool. /CREATEHS wasn’t an option because I didn’t have the SAUDIT.TXT file (I guess that this holds the sector information in case you have a proper Master Boot Floppy). Instead, I had to build the hidden sector by myself.

Since none of the options seemed like an obvious solution, I simply launched HSCENTER without any additional options.

My first choice was to select the HS MANAGER option, so I could check if the tool was supposed to do what I needed.

Reading and modifying an existing hidden sector seems to be possible from within this menu. But currently, there is no sector to modify and I have to create it.

The next obvious choice was to try the ENTER HS PARAMETERS FOR BURNIN PROCESS option. After selecting the manufacturer (this was around the time when Packard Bell was sold to Zenith Data Systems), I seemed to have hit a dead end again.

The main issue is that I don’t have the necessary information to proceed with the setup process. This means that the validity checks for the entered fields will fail, potentially causing a broken hidden sector and triggering the disk lock.

A FIELD trip to the service technicians

Additionally, the setup process uses the part number and format switch to determine the exact machine configuration, so even in the best-case scenario, I would likely end up with a bare-bones system without any additional software or drivers.

The command line switches /BURNIN and /MASTERCD both sent me to the same screen where I had to enter the missing manufacturing details.

/FIELD looked like a way better option. According to the description, this was meant to be used by the technical support department - e.g. after replacing a failed hard drive.

I then realized that the various options were primarily targeted toward the different service departments. /MASTERCD refers to the Master CD development process itself, /BURNIN is meant for a technician that assembles and prepares the system, while /FIELD is referring to a field-service technician.

This was exactly what I was looking for. With the /FIELD option, HSCENTER guided me through a simple interface where I could pick some of the components and software collections, presumably based on the different model variants sold at the time.

After completing the process by using the SAVE AND QUIT option, I started HSCENTER again to verify if the hidden sector had been written properly.

Finally! I was confident that I now had a properly written hidden sector. But there was only one way to find out.

I rebooted the system again without the boot floppy attached, noticed the “Starting Windows 95…” banner again, and prepared for yet another failure. Much to my surprise, the boot process finally continued, starting the usual Windows 95 setup procedure.

Flawless victory and a pretty usual setup procedure

The remaining installation was pretty simple. After the initial hardware detection and the first Windows 95 bootstrap setup, the machine started into an additional unattended installer that performed all further installation steps.

After each reboot, you can see parts of the build environment that is running in the background before it launches the ‘Please Wait’ overlay. The text message in the editor refers to this environment as ‘Audit mode’, allowing for additional tests.

Unfortunately, I couldn’t find any additional information about this environment, so I don’t know if this was only meant for the installation steps or if there was an additional service environment the service technicians could use.

After about 30 minutes, the installation was complete and I could enter my username and password as well as the OEM product key.

Closing words

And there it is. I had to reboot the system one more time before I was greeted by this ugly yet still somewhat familiar desktop. Even though I hadn’t seen the icons for more than 15 years, I immediately recognized the “Packard Bell Navigator” and the “CyberTrio” icons.

Surprisingly, the system overview dialog claimed that the machine I was emulating is powered by a Pentium Pro CPU, even though I was running a Pentium II emulation. I’m not sure if this is a flaw caused by the emulation itself (it is properly detected in the BIOS though) or if the version of Windows 95 that shipped with the system was a bit too old in order to be able to properly detect the CPU.

Obviously, I didn’t do this whole restoration process just for a minimal Windows 95 installation. I was looking for the software that was originally bundled with the system.

While skipping through the various categories of the Start menu, there was only one thing left to say: “This is it.”

Was it worth it?

Well, technically: Totally not. For retro-gaming purposes, it would have been way easier to just install Windows 98 (SE) without having to worry about this rather complicated process.

But personally: Absolutely.

This was the machine I used to learn how to use a computer. The machine I used to play my very first games, the machine I used to learn for school, and the machine I later used for my first Linux experiments.

I’d say this machine was pretty important for me.

I miss it.

Using qemu-guest-agent as interface between VMs and Proxmox host systems

Fri, 09 Dec 2022 18:00:00 +0100

Since virtual machines created with KVM/QEMU are not simple containers but quite isolated from the host’s environment, exchanging data and information between the host and the VMs can be a bit tricky. Therefore, QEMU offers a companion service called qemu-guest-agent for Linux guests. qemu-guest-agent acts as an interface between the VMs and the host system.

Some features like passing ACPI information for a clean guest shutdown are pretty well-known. However, did you know that you can even send commands to your VMs directly from your Proxmox host system?

Proxmox Virtual Environment uses KVM/QEMU as virtualization technology. Since calling the qemu-guest-agent interface is not very intuitive by itself, Proxmox provides the qm guest command which acts like a bridge between the host system and the VMs.

Using the qm guest command requires qemu-guest-agent to be installed inside your Linux guests. Since the guest agent is pretty essential in case you want to be able to do properly shut down the VMs without performing virtual powercuts or being able to create consistent backups, you want to install it on all of your Linux guests anyways.

In order to use the following commands, you need to get the VMID of your virtual machines. You can get them either from the Proxmox dashboard or by running the qm list command on your host system.

Reset user passwords (including the root user)

I basically learned about qm by accident. Usually, I tend to lock all local user accounts in order to make (virtual) console access impossible. Especially on VMs, this might not even be exactly helpful regarding security, but this is some sort of personal preference. Due to a configuration error on my end, I locked myself out of a VM running on my Proxmox host, so I was not able to SSH into it anymore.

The obvious solution would be to shut down the VM, boot a rescue environment or live distribution, mount the disk and fix the configuration there. I knew the VM was performing some maintenance tasks in the background which do not like sudden interruptions, so I needed some sort of backdoor.

This is where qm guest passwd comes in handy. As the command suggests, it can be used to set user passwords in your VM, even if there’s no direct access to it anymore.

As an example, the following command resets the password for the user root on VMID 100:

qm guest passwd 100 root

After invoking this command, qm will interactively tell you how to set a new user password for the account in question. If the user was previously locked with passwd -l, the user will be automatically unlocked.

Run arbitrary commands in your VM

As you might have noticed, we have full access to the guest operating system using the QEMU Guest Agent (or QGA for short). Therefore, we can pass any command to the guest. Side note: This is a friendly reminder that you really want to care about the security of your host system.

The command we are looking for is qm guest exec (again using VMID 100 as an example):

qm guest exec 100 -- ping google.com -c 3

qm exec will pass the command to the VM, execute it and wait for the command to finish or break with an exit code. Afterwards, it will return the exit code as well as the contents of stdout and stderr in JSON format. Please be aware that all commands are executed with root privileges.

Long-running tasks can be sent to the background using the --synchronous=0 parameter. You can check the state of the process by using the qm guest exec-status $yourVMID $PID command.

Get additional VM information and perform low-level tasks

Next, let’s take a look at qm guest cmd. Now, we are not executing commands within the guest operating system itself. Instead, with cmd we call various QGA commands on a lower level. Those commands are more focused on maintenance and less on interaction with the guest operating system.

In order to use this feature, use the following command:

qm guest cmd $yourVMID $Command

I compiled a list of valid commands below. Just like the qm guest exec command, the output will be JSON formatted.

Command	Description
fsfreeze-freeze	Send the fsfreeze-freeze command to the guest, locks the file system on the guest machine
fsfreeze-status	Query fsfreeze-status, returns ‘frozen’ or ‘thawed’
fsfreeze-thaw	Thawing a frozen filesystem returns control to the guest operating system
fstrim	Issue the TRIM command to the VM’s block device(s). The VM has to be configured with the discard option or running with SSD emulation enabeld and the guest’s filesystem needs to be TRIM aware. Note: Since running a full fstrim takes quite a while, it’s not uncommon that the command seemingly fails with ‘qmp command ‘guest-fstrim’ failed – got timeout’. Check the guest’s message log for ‘guest-fstrim called’.
get-fsinfo	Returns all mounted filesystems
get-host-name	Returns the hostname of the guest system
get-memory-block-info	Displays memory information
get-memory-blocks	Displays memory information
get-osinfo	Displays various OS information like the distribution, kernel version and architecture
get-time	Get the system time of the guest OS
get-timezone	Get the timezone set in the guest OS
get-users	Get all users currently logged into the guest OS
get-vcpus	Get information about the assigned virtual CPUs
info	Return a list of available commands that can be issued through QGA
network-get-interfaces	Return a list of configured network interfaces along with traffic statistics
ping	Check if the VM is up and qemu-guest-agent is running. If this command times out, the VM is either offline or qemu-guest-agent is not working properly. Note: This is not a ‘traditional’ ping over a network connection. Instead, the host is trying to communicate with the VM over QGA.
shutdown	Send the ACPI shutdown command to the VM
suspend-disk	Send the suspend-to-disk command to the VM
suspend-hybrid	Send the suspend-hybrid (faster suspend-to-disk?) to the VM
suspend-ram	Send the suspend-to-ram command to the VM

Conclusion

The QEMU Guest Agent provides an extensive interface for communication between the host system and the VMs.

You can either send any arbitray command to the guest systems or use QGA to get some details about the VMs themselves. It is also possible to go a bit more low-level, so you can shutdown your VMs or let the host system take control over the file system by essentially freezing it.

This is especially helpful if you want to use Proxmox’ built-in snapshots. However, if you use the backup and snapshot functions from the Proxmox GUI, freezing and thawing is handled automatically.

Currently, only a subset of the QGA protocol is abstracted by the qm command. Since calling the other commands supported by QGA is a little bit more complicated, I’ll cover this in a seperate article.

Restoring support for 16-bit applications in modern Windows versions

Fri, 02 Dec 2022 21:00:00 +0100

Windows has some pretty amazing backward compatibility. In many cases, you can run ancient 32-bit Win32 applications just fine on your current system.

However, there’s one issue: If you ever tried to run a 16-bit application from the Windows 3.x days, any 64-bit Windows version (starting from Windows XP) will refuse to run the application with an error message indicating that you should ask the vendor for a compatible version.

On the other hand, the modern 32-bit versions of Windows run these applications just fine.

Thanks to two amazing open-source projects, you can bring back 16-bit compatibility to the 64-bit Windows era.

Why did Microsoft drop support for 16-bit applications?

Back in the old days, things were rather simple. Up to (and including) Windows ME, all consumer versions of Windows ran on top of MS-DOS, so support for 16-bit applications was basically built-in (Windows 98 for example even includes the Program Manager PROGMAN.EXE from Windows 3.x). For the NT series, Microsoft had to pick a different route because it didn’t have any MS-DOS kernel to rely on. This led to the invention of NTVDM, the NT Virtual DOS Machine.

NTVDM was first introduced with the release of Windows NT 3.1 back in 1993. The NTVDM subsystem provides a runtime environment replicating a system running on MS-DOS 5.0 including all required BIOS calls, the Windows 3.1 kernel, and the Win16 API itself. Since NTVDM doesn’t provide real hardware emulation, it still relies on the CPU being able to run the original 16-bit code natively.

Microsoft removed the NTVDM in all 64-bit Windows releases, so running DOS- and Win16-based applications is not possible anymore.

This removal was not a deliberate choice made by the Windows developers. Instead, the lack of 16-bit support is caused by the design of the x86_64 architecture itself.

CPUs based on the x86_64 architecture can run in two modes: Long mode and Legacy mode.

In Long mode, the CPU executes 64-bit code natively, while 32-bit applications run in a special compatibility mode. Long mode doesn’t provide support for the VM86 CPU code required by NTVDM in order to enable 16-bit segments for addressing, so it had to be removed in all x86_64 versions of Windows.

When the CPU runs in Legacy mode, it provides full support for both 32-bit and 16-bit applications.

Unfortunately, the only way to switch between both operating modes is a hard reset of the CPU itself, making a seamless integration of both operating modes impossible without restarting the whole system. Because booting a 64-bit Windows system will always force the CPU to run in Long mode, there is no way to get back to a mode that allows native execution of 16-bit applications at runtime.

In fact, even switching to the 32-bit compatibility mode within Long mode itself (and not the legacy mode) requires an additional abstraction layer. Every time you launch a 32-bit application on your 64-bit Windows installation, the operating system invokes the WoW64 subsystem that acts as a compatibility layer providing the interface required to run the 32-bit application on a 64-bit system.

Like fine WINE

Considering the limitations the x86_64 architecture imposes, the only way to get 16-bit code running is by introducing an emulation layer that is able to replicate a CPU with the proper instruction set. Additionally, we need some software that provides a runtime environment for the Win16 applications.

Combining an 80386 emulator and a Win16 runtime environment - that’s exactly what otya128 accomplished with their otvdm project . otvdm uses the 80386 emulator developed by the MAME project and combines it with the well-known Windows runtime environment WINE.

Installing otvdm is incredibly easy and provides a seamless integration in your current Windows installation.

First, you have to download the current version from GitHub’s release page .

I recommend extracting the contents of the .zip archive you just downloaded to C:\otvdm since this directory is available for all users on the system and is also not protected by UAC. The latter is important because in order to save DLLs in the virtual WINDOWS folder provided by otvdm, write access to the otvdm directory is required.

All you have to do to get otvdm added to your system is to click on the install shortcut. The installer tool adds numerous keys to your Windows registry, so administrative access is required. After a couple of seconds, the installer window closes and the installation is complete.

That’s it. Now, each time you want to launch a Win16 application, your system detects it and passes the program to otvdm which in turn is able to run it. In theory, otvdm also provides native DOS emulation as well, but since it is rather limited, a solution like 86Box or PCem might be a better choice for MS-DOS applications.

You don’t have to start otvdm manually after booting up your system or in case you want to launch a Win16 application. Thanks to the installed registry keys, it will automatically kick in as soon as it detects a Win16 application.

Updating otvdm is pretty easy too: Just download the new release, replace all files (but do not delete the contents of the C and WINDOWS directories) and run the installer again.

Pretty incredible, if you ask me. And a great example why the open-source community is so important for software preservation. But that’s a topic for another day.

Welcome!

Sun, 20 Nov 2022 15:00:00 +0100

“A new project? Again?!”

Yes, I know. My personal projects are as persistent as volatile memory. But hey - not this time!

Welcome to felsqualle.com, where we take a deep-dive into the wonderful world of mostly ancient (but awesome!) technology.

For those who don’t know me: Hi, I’m felsqualle. I’m a systems administrator at the German webhosting company manitu by day and a general-purpose tinkerer and project co-lead for ScummVM at night.

I got my first PC when I was 6 years old, which means that I’m getting pretty close to hitting 25 years of PC usage. During my long-lasting journey, I used each and every consumer version of Windows (yes, I tried Windows 1.0!) and most versions of MS-DOS. My very first GNU/Linux experience was back in 2003 when I got my hands on a copy of Knoppix , which was one of the most famous distributions of live GNU/Linux systems back then while still being maintained today.

At some point, I got interested in retro-computing - and for some reason, this passion never faded away. And this is what I want to share with you as well.

So, what should you expect? Well, in this blog, I’ll not only share some of my sysadmins tricks as I did previously, but I’ll also tell you about devices and software that nobody really talks about anymore.

You’ll find some familiar stuff and pieces of software where I believe that I have one of the very last copies in existence.

Sounds pretty exciting, doesn’t it?

Immersive slowness or why I added artificial loading times for Myst to ScummVM

Mon, 19 Jul 2021 09:00:00 +0100

Ever since I discovered the Myst series back in 2005, I’m in love with it. To me, the Myst series feels like an immersive trip to another world - it is truly something different compared to your average point-and-click adventure game. Needless to say that especially the first entries in the series - the original Myst and its successor Riven - are truly remarkable games.

In my opinion, the immersion these games provide is partially created due to technical limitations.

The original Myst was released in 1993 on this incredible new format called ‘CD-ROM’, allowing for a whopping 650 Megabytes of storage.

At that time, CD-ROM drives were slow - like double-speed slow. This means access times of 80 to 200ms as well as a blazing transfer speed of 300KB/s under ideal conditions. Since even the largest hard drives had a capacity of around 1000 Megabytes, installing the game wasn’t an option. Furthermore, RAM constraints made any caching of the datafiles impossible.

This is where immersion kicks in

Myst uses some pretty clever compression techniques and an optimized file layout on the disc. Since each time you move through the game, your computer needs to load the next image. On the next move, it needs to load yet another image. The game basically forces you to slowly walk through the worlds of Myst.

I’m not sure if this was even part of the original design concept. However, it really feels that your are not supposed to rush through the game. Everything tells you to slowly explore while making sure not to miss any hints you need to solve the puzzles.

While many adventure games are limiting playback speed on their own (e.g. due to character movement), this does not apply to Myst. When you play the game on original hardware, then the loading times seem perfectly fine since you expect them.

As soon as you eliminate the loading times at all, e.g. by playing the games with ScummVM, then you might notice something is off. Now, you are not forced to slowly explore, but you can simply run through the game. Even though I’m obviously not forced to take advantage of the instant loading, the feeling that I can do this is affecting the experience in a negative way for me.

The solution

Looking for a way to improve the experience, I recently implemented loading time simulation to ScummVM . After activating this feature, it adds a delay between the different screen transitions in order to simulate the loading times of a real CD-ROM drive.

In order to show you the difference, I prepared a small video. The first half of the video shows the previous without any loading times behaviour. Obviously, I’m really exaggerating here. It should be pretty obvious that this is clearly not how the game should be played. I enable the new loading time simulation at around the 50 second mark.

There should have been a video here but your browser does not seem to support it.

While the implementation was pretty straight-forward, it was not exactly easy to get the timing right. The main issue is that I don’t have really period correct hardware at the moment. The oldest system I own is a Pentium III with 500 MHz, featuring a 16x Philips CD-RW drive and a 48x “generic” drive. Fortunately, this generic drive is compatible with Jörg Fiebelkorn’s CD-Bremse . This tool allows you to throttle your CD-ROM drives by enforcing lower reading speeds. Unfortunately, I wasn’t able to go down to double-speed since the drive wouldn’t go slower than 4x.

This meant I couldn’t rely on measurements alone. Instead, I read through many, many datasheets of various CD-ROM drives. I also tried to emulate slower drives with 86Box since they allow setting transfer speeds as well. Even though I didn’t look at the code, I have the feeling that 86Box only emulates transfer speeds, but not access time.

Limitations and what’s coming up next

All in all, the current delays are pretty arbitrary and might be subject to change in future versions. I tried to be as accurate as possible, but I simply can’t guarantee I got them absolutely right. It is also noteworthy that applied delays are the same on the original Myst and Myst: Masterpiece Edition. Due to the lack of samples, I wasn’t able to verify this, but I’d expect Myst: Masterpiece Edition to have longer loading times on the same hardware due to increased file size compared to the original edition.

One option would be to make the delay a configurable option, allowing for a wider variety of ‘drives’. For now, the new option is disabled by default. Eventuelly, we’ll probably consider making this the new default. However, this will most likely not happen until the next release is out since I want to gather feedback from our users first.

Closing words

Currently, the loading time simulation is available for Myst and Myst: Masterpiece Edition. The same concept could be applied to Riven as well. There are two things to consider:

Timing: Internally, Riven uses some counters and timers to keep track of - well - time-based events. I need to dig deeper into the codebase to make sure that adding artificial delays between scene transitions won’t mess with the timing. This might even involve multiple playthroughs with varying conditions.
The installation itself: At least the version distributed on five CDs I own provides three options during the installation: Minimal, standard and full installation. Depending on which type you choose, loading times will obviously vary as well.

If you want to give this new feature a try, you need to download a current daily build from the ScummVM website and grab your copy of Myst or Myst: Masterpiece Edition. In you don’t own the game yet: Myst: Masterpiece Edition is available at GOG.com as digital download* .

After adding the game to ScummVM, you find the new feature in the in-game settings by hitting the ‘F5’ key.

What do you think? Any feedback is very welcome, especially when you have the chance to play the game on original hardware.

*: This link is a special affiliate link. If you visit GOG.com via this link, the ScummVM project gets a commission for any purchases you make. For you, the price is exactly the same while you are still supporting the project. Thank you ❤️!

Using Snap packages in Gentoo

Mon, 09 Mar 2020 14:36:48 +0100

One of the main advantages of snap packages is the possibility to use them not only on one Linux distribution like ’traditional’ packages, but on a wide variety of distributions without having to modify or rebuild them. Many distributions provide the necessary snapd daemon in their repositories.

It is entirely possible to use snap packages with Gentoo too. Even building new snap packages with snapcraft and multipass or LXD will be possible afterwards.

Prerequisites

First and foremost: systemd is a mandatory requirement since snapd is not compatible with OpenRC. Since snapcraft requires elevated privileges, sudo should be installed and properly configured too.

In order to aquire the necessary ressources and permissions on the host system, snapd is using the AppArmor framework. In case you are using your own kernel, it might be necessary to manually enable AppArmor support in its configuration. The binary packages provided by the Gentoo project have AppArmor support already enabled.

I won’t cover the migration of a Gentoo based system to systemd at this point. Since this topic is so extensive, I’ll publish a separate article at a later point in time.

AppArmor and systemd

We need to compile systemd with the USE flags policykit and apparmor enabled, libseccomp with static-libs. Therefore we’ll add the following entries to the /etc/portage/package.use file:

sys-apps/systemd policykit apparmor
sys-libs/libseccomp static-libs

The snapd ebuild is masked by default, so we unmask it in the /etc/portage/package.accept_keywords file:

app-containers/snapd ~amd64

In the next step, we (re-) build systemd and AppArmor.

emerge sys-apps/systemd
emerge sys-apps/apparmor

In order to make AppArmor available right after the system is booted up, it’s necessary to modify the bootloader configuration. We need to add the following line to /etc/default/grub or modify it accordingly:

GRUB_CMDLINE_LINUX_DEFAULT="apparmor=1 security=apparmor"

Afterwards, the GRUB configuration gets rewritten with the command

grub-mkconfig -o /boot/grub/grub.cfg

Reboot the system to apply the chenges. In case you are using a different bootloader, please check the proper documentation.

Installing snapd itself

Finally, we install the snapd package and enable the necessary systemd units:

emerge --ask app-containers/snapd
systemctl enable --now snapd
systemctl enable --now snapd.socket
systemctl enable --now snapd.apparmor

Additional permissions for snapcraft

In case we want to allow unprivileged user accounts to create new snaps with snapcraft, we need to add them to the groups adm or lxd. The adm group is required if you want to use multipass for providing the build VM, lxd in case you want to use LXD instead.

gpasswd --add username adm
gpasswd --add username lxd

That’s it - now you should be able to run all snap packages you want. You can verify that your installation is working by checking out the hello-world snap provided by Canonical.

apache2: Redirect multiple domains to the same website

Sun, 12 Jan 2020 00:00:00 +0000

Over the last couple of years, I aquired a nice collection of various domains. Since I only keep them for historical purposes, I decided to redirect all domains to this blog domain. Usually, you can achieve such a redirection by simply pointing all your domains to the same virtual host within your web server configuration in case you are using WordPress. However, this won’t work when using some caching plugins. Some .htaccess magic will help though.

The following .htaccess snippet redirects every domain that’s not called www.targetdomain.com to the - well - desired target domain:

RewriteEngine on
RewriteCond %{HTTP_HOST} !^www.targetdomain.com
RewriteRule (.*) https://www.targetdomain.com/$1 [R=permanent,QSA,L]

The redirection also works for all queries - https://olddomain.com/foo will always redirect to https://www.targetdomain.com/foo.

apache2: mod_autoindex and UTF-8

Fri, 10 Jan 2020 15:00:00 +0100

For my ScummVM nightly server , I’m using the apache2 module mod_autoindex. I noticed that – even though I enabled UTF-8 support in the apache configuration – one vHost was still served with ISO-8859 encoding.

The problem is that the autoindex module simply ignores the global charset settings. Instead, it insists on using ISO-8859 for the vHost without further configuration.

In order to enable UTF-8 for sites generated by mod_autoindex, the parameter Charset=UTF-8 must be added to the IndexOptions of the specific vHost configuration.