Portability games

I found out why the sky wasn't working: it turns out that q2bsp (the new name for qbsp3) gets the flags that tell it if a face is part of the skybox by reading the headers of the texture files. It wasn't able to load the textures, so it couldn't work out that the sky-textured bits were actually sky.

One thing led to another, and I ended up rewriting not only the PAK loader, but just about all the code that messes around with filenames and directories.

A PAK file is basically a linear collapse of a directory tree into a binary object (it's entirely uncompressed) and I believe that the reason for doing it was to avoid the overhead of opening, reading and closing a lot of files. Anyway, in order to find a file such as a texture, the tools need to look in up to four different places, including scanning through PAK files containing thousands of files. I decided that it would be fairly straightforward to get rid of the necessity of scanning for files more than once: the tools now scan through all the places textures could be as they start up, and store the locations of the files in a hashtable, along with the file pointer of the PAK file they're in. So not only do I only read the header of each PAK file once, I only do a full scan once, and after that its just an O(1) lookup to find a file.

Anyway, the upshot of this all is that all the tools now compile and run without errors on Windows/Cygwin and on 64-bit Linux, and even create playable maps! Get the latest release now, and tell me about any problems you find.

Except q2rad, that is, because the default settings of q2rad are rubbish -- they leave the map totally unlit. Oh, for the ability to merge my code with ArghRad...

And progress continues apace

I think I've finally cracked the maths behind the Brush Primitives format, which is a good start. I've written it up on the buildutils wiki. Now I just need to work out the maths behind the other two formats that I need to support (Classic Quake and QuArK ETP) and all will be well.

I took the time to move the tools over to using GNU getopt to parse command line options -- something I've been meaning to do for a while, because not only is it easier to maintain, but it needs a lot less code: you can see what each of the command-line options do at the code level much more easily now.

I also renamed them: the package is now buildutils-q2bsp and the four tools are q2bsp, q2vis, q2rad and q2bspinfo.

I have, however, just noticed a somewhat worrying problem. When I load a map I've compiled in Quake II, the sky doesn't work; I just see a flat texture. However, when I load one the Quake II game proper (for example), the skybox works just fine. I'll investigate further, but not now because it's the middle of the night...

Quake II tools update

Well, it's a while since I posted on the subject of compiling maps for Quake II, but I'm pleased to announce that things have started moving again. I was contacted yesterday by whitelynx of the Precursors project, asking what was going on.

Anyway, one thing lead to another, and now there's a Trac site for the project: wiki, subversion repository and bugtracker. I'm very grateful to servegame.org for the hosting (AutoPackage is hosted on the same server, I was suprised to learn). However, the package will continue to be distributed from here.

whitelynx has decided that it would be a good idea to bring lots of different tools "under one roof", so the package is now called "buildutils", with the Quake II compilers called "q2bsp". The new name for the Quake II package is of course inspired by the Quake II build tool, "q3bsp" (which we're trying to get source code for as well). In fact, whitelynx wants to merge in tools for as many different games as possible; I think that's a bit ambitious, so I'm going to watch with interest.

I've made a few small changes in the last few hours; the tools now get linked against libm (which helps), and I've fixed all outstanding compile-time warnings on 64-bit Linux (yay). Lots of stuff to do still, though! I still need to implement Brush Primitives support, for instance...

Fun & maths

I haven't updated for a while. Two main reasons: firstly, my PC has been broken for weeks with a fried motherboard (I should be receiving my new one soon, hopefully), and secondly, I've been at university, somewhat excitingly.

However, I have managed to find some time to look at QBSP3 and try to improve my understanding of how it works. For example, I now know how to compile the texture transformation provided by the Brush Primitives MAP format into something the Quake II engine can understand.

I'm gradually working through the QBSP3 source code working out why it does what it does and how it does it. The big problem is that there is no documentation. There are very few comments in the code (and what few comments there are are cryptic enough to be entirely opaque to me) and although I've been able to find out what it should generate at the end, I can't seem to find a good explanation of how it works anywhere. So I'm going to try my best to document the maths involved at some point.

Hopefully when I get my PC back I'll be able to get some code out there.

As far as university is concerned, it's been quite intense. The workload has been heavy (of course), and I can't believe how much I've learned! In between struggling through examples papers, I've made it onto the university small-bore rifle team (thanks to shooting 196 ex 200 against Edinburgh at the beginning of the month), as well as joining the Catholic chaplaincy's Schola Cantorum (the choir specializing in Gregorian chant and Renaissance music).

Logging framework

Well, I've now written the bones of my framework for doing log or console output at various levels of detail. Unfortunately, I now need to go through all the source code and work out what deserves to be output at each different level of verbosity: Error, Warning, Normal, Verbose, Debug. As the approach in the source code at the moment seems to be, 'What I feel like doing in this file,' I've got my work cut out. I've done errors and (I think) most warnings, but... yeah. Lots of printfs.

Ongoing development

I've been busy!

• I've started working on a manual, in Texinfo format (so it can easily be converted to a number of different formats). Problem is, most of the command-line options for the various tools are totally undocumented in what small amount of documentation I've got from Armin/GDW. So I suppose I'm just going to have to gradually fill in the gaps as time goes on.
• I've started tracking down another bug with using origin brushes with brushes with rotated/scaled textures. I've got a few ideas, and I'm hard on its tail.

In other news, I've been working on the QuArK bug reports, and trying to clarify where exactly the problems are.

Somehow, it's working

Somehow, my QBSP3 is now correctly aligning textures on entities with 'origin' brushes. This is wierd. I have no idea how it's doing it.

It only happens for QuArK ETP map files, so that would imply that it's something to do with Armin's code that I added yesterday. But it's still wierd, since Armin's code doesn't do it correctly!

Testcase as it appears in QuArK:

Testcase as it appears ingame (using my QBSP3 + my QVIS3 + ArghRad 3.00 beta):

Unfortunately, I've got a fairly serious regression: something I've done has broken my QRAD3. It just doesn't work at all, which is pretty worrying, and I'm not sure how to find out where the breakage is...

Update: It's not my fault! GDW's QRAD3 doesn't work either! Oh, wait... that means I don't have any clues as to how to get it working.

Let's just hope I can get my hands on the ArghRad source code.

Success!

I've successfully added support for QuArK's enhanced texture positioning MAP format, so that means that my version of QBSP3 matches Armin's in terms of functionality, while incorporating most of GDW's enhancements. Yay!

Lots more stuff to do, though. In particular, I want to remove the way-to-frequent appearance of printf in the code to do debugging/error reporting and centralise it into some utility functions (probably in cmdlib.c -- this is the way Armin's QBSP3 does it, by the way). And that's on top of fixing the origin-brush-displacing-textures problem.

Get the Windows build from my downloads directory.

Update: Uploaded new Win32 build with stripped binaries (i.e. download is now a tenth of the size).

Fun and games with QBSP3 and textures

Among other things, the QBSP3 tool is responsible for aligning textures on the faces of polyhedrons, taking its information from the .map file output by whichever editor you're using.

In my opinion, the QBSP3 tool should ensure that the way the texturing appears ingame is identical to the way it appears in the map editor. How else can the mapper know how to get the textures looking the way he wants them to?

So, I thought I'd compare the performance of the various versions of QBSP3 I've got available to me. First off, the testcase, as it appears in QuArK. The two floating boxes are func_wall entities, and the left-hand one has an brush with the 'origin' texture on it to set the entity origin somewhere away from the default. Note that both boxes are identicaly textured in the editor, and the textures are square with the face edges:

This is how it appears using the TXQBSP3-derived QBSP3 modified by Armin Rigo and distributed as the recommended QBSP3 for QuArK. Note that right hand box is textured correctly, but the left hand one has had the textures displaced by the 'origin' brush.

Next, the output of Geoffrey DeWan's QBSP (my build). The textures are distorted, and it fails the 'origin' displacement test.

Finally, the output of the same tool with the -origfix command-line option. This makes an attempt at fixing the 'origin' displacement test, but fails.

So my current aims are to:

1. Fix texture distortion in my QBSP3 build
2. To fix displacement of textures when an 'origin' brush is used.

QuArK, MoinMoin, and Quake II build tools

A bit of a saga to relate.

Since I'm going to be leaving my job, I decided to get back involved with QuArK development. I thought that, since this was a long weekend, I'd be able to get some stuff done.

The first order of business was to get shell access to the QuArK SourceForge project again, because cdunde had removed them while I was inactive (I'm now an admin again -- yay). The reason this was urgent was that SourceForge.net changed the way that project web servers worked, and this had terminally broken the QuArK wiki.

Well, thanks to a very helpful page on the MoinMoin wiki, I manage to get the thing working again, and it's actually working better now than it was before. There are two big problems though: firstly, I couldn't get access to the old database to copy it across, so I had to reset the wiki, which sucks; secondly, the data directory is now world-writeable, which means that anyone can just rm -rf the whol wiki if they want to. So I suppose I'll just have to get round to actually making backups...

Having done that, I moved on to doing some housekeeping tasks: restoring the LatestVersion page (and all of the older version pages), creating page templates, etc, etc. I also wrote an article on HowToInstallQuark.

This morning, I came to start writing some more howtos, and decided to start with an explanation of how to set up QuArK for Quake II editing. While doing so, I realized something suprising, and slightly worrying: the tools recommended for use with QuArK haven't been updated in years, and there are tools with enhanced features that don't work properly with QuArK. So I decided to do something about it.

I got the source code to Geoffrey DeWan's tools, which are currently accepted as the best available, and decided to add the QuArK support functions to them. The only problem was, no Microsoft Visual C compiler. So I duly got out my automake and autoconf manuals, and got busy writing the necessary scripts.

The end result? I've got my own distribution of the Quake II map compilers, available from my website. Madness.

Now to try and add Armin's enhancements for QuArK into the mix. Should be lots of fun!