As anyone that is knowledgable in the field of security is probably aware, fuzz testing is an essential part of the toolkit. After all, it would be virtually impossible to test as many inputs as something like AFL can - with a proper fuzzing setup and a fast executable (or a slower executable with some patches to skip things such as disk writes and checksums, in some cases), you can hit thousands of executions per second. Well, recently, I’ve been getting a bit more serious about fuzzing, and since I primarily have a desire to fuzz Windows applications, I decided I’d talk about my experience using CERT’s Basic Fuzzing Framework and share a few tips.

First off, I’d like to say that I am aware that AFL has a port to Windows called WinAFL. I’ve used it in the past, and it is just as every bit as excellent of a fuzzer on Windows as it is anywhere else. However, it can be tricky to setup, especially if you’re fuzzing complex black box targets like I mainly do (I really love fuzzing games as a hobby).

I’ve recently been relying a lot more on CERT’s BFF. Sure, it’s not as fast as AFL. It’s also not as smart (AFL’s heuristics are seriously impressive) however I do think that it is the quickest way to get into fuzzing a black box target. You can set up a new campaign in about 5 minutes and you’re off to the races, and that was the idea CERT had in mind when they were creating it. They intended for it to be a quick, intelligent enough fuzzer that anyone can get into fuzzing with, and it serves that purpose well.

One of the problems I’ve ran into with fuzzing game engines is that, a lot of the time, you can’t directly launch into a specific map or game session right from the command prompt. However, I have come up with a solution, and that solution is basically to set the fuzz target to a batch file that sets up the game files properly so that fuzzing can still occur. Generally this involves copying the current fuzzing iteration to the game directory, and launching the game. The one limitation to this is that you’re limited to a corpus of only the files that the game will need to load immediately. Some games have a built in feature to load up a specific map or run commands on startup (Source Engine games are great for this) but most games don’t. For example, one of my prior fuzz targets was Halo Combat Evolved: there are no command line parameters to load up Halo CE into a specific map, so I simply fuzzed the ui.map file. It is loaded instantly upon starting the executable, so I was still able to fuzz the .map format.

Here is the batch file that I’ve got going for Halo Combat Evolved:

copy "%1" "C:\Program Files (x86)\Microsoft Games\Halo\maps\ui.map

cd "C:\Program Files (x86)\Microsoft Games\Halo"

start /wait halo -window -safemode -vidmode 320,240,60

To break it down a bit, first we copy the seed into the proper location with the proper name so it will overwrite the original UI.map with the current fuzz iteration. Next, we’ll switch directories over to the Halo Custom Edition directory. Then, we’ll start Halo with the /wait flag (which allows BFF to catch crashes that occur, otherwise the batch file will just close after launch and BFF will think the fuzzing iteration is finished) and we also set the video mode to a low resolution and run it in a window. That’s not strictly necessary, I just do that to speedup the game’s launch for faster fuzzing. The batch file runs every fuzzing iteration, so a new, freshly mutated ui.map gets copied into the right place every single time.

This can also be applied as a generic processing step for the seed. I’ve used a similar batch file in the past to put the seed into a zip file first before copying it over as the application in question would only take ZIPs and I didn’t want to fuzz the ZIP container parsing code. You could probably do something like fix checksums if you want (however, I’d strongly recommend just patching out checksum code if you can).

Anyway, that’s been my experience with BFF. While there are definitely better fuzzing options out there, BFF can be setup to run very quickly with any executable in existence, and I think it’s definitely a valuable tool to have around. I’ve found several good bugs with it and I hope to keep finding more in the future.