Standard Trek.exe IDA Database

[anjel]

the same here, worked like a charm.

[QuasarDonkey]

That's great. I'm hoping it will be useful for future modding. It may not be completely useful until the functions are given more descriptive names. I've already labeled a few functions for turn processing, etc.

But I'll have another release real soon. I've identified about 50 new functions and lots of variables. Right now I'm in the guts of the user interface code, trying to make sense of it.

[QuasarDonkey]

I've posted another update.

I've labeled a lot of unknown functions and variables, and I've entered function signatures for most library functions, which should make analysis easier.

I've also started using structure member offsets to make code more readable, e.g.

Code: Select all

mov eax, galInfo.Systems
mov ebx, [ebp+eax+4Dh]

becomes this:

Code: Select all

mov eax, galInfo.Systems
mov ebx, [ebp+SystemInfo_t.owner]

But I've only done this for a few functions so far, like Game_Borders_Calculate.

I've also identified all functions from the file system/memory.c. It turns out Trek uses a third-party library called SmartHeap for memory management (this library overrides standard C functions malloc, free, etc.; this is why IDA missed them). Trek has numerous memory allocator functions, based on a memory pooling scheme, so you have one function for allocating memory related to UI, one for AI, etc. I've identified the important functions from the SmartHeap library too. If people want more information on BotF memory management, just ask.

As a proof of concept, after I figured out the function names and function signatures for system/memory.c, it was trivial to reverse engineer the file back into C source code Unfortunately, since it requires the SmartHeap library which costs $2500+, the code won't compile I'm not sure whether I should start posting BotF source code since I'm really not sure of the legal status of reverse engineered source code. Before anyone gets excited, I'd estimated that file I reversed accounts for <0.5% of the total source code.

[Flocke]

I just had a look at your ida file, looks good, great work!

Well, decompilation doesn't seem to be that a well option to me, though there are some decompiler programs that might allow to decompile most of the botf code. As long there is asm code left and there are cross references, it's too much work and risky to readdress the code & data offsets. Not to talk about legality issues.

Instead simply choose a function and link in your own implementation via codecaves as I already suggested. That way you also can 'decompile' parts of the source if you want and still depend on functions like memory management ones that were compiled to trek.exe.
The cool thing I didn't realize for a long time either, is that dll files that were loaded by botf are in same process space and allow to directly call botf functions by their asm offsets. To call functions of your library you can use an array of function pointers that you can pass to botf by an initial call and only need to store the single pointer to the array somewhere in botf data section.
When calling functions vice versa, of course you've to regard calling conventions like __cdecl, but that's rather easy and you can use inline assembler if there's something special to match.
I already posted about codecaves here: viewtopic.php?f=4&t=598&start=15

cheers & looking forward to see your first break throughs in coding

[QuasarDonkey]

Thanks Flocke, you flatter me sir!

I do think decompilation is possible, with or without decompiler programs, but not yet. Like I said, once I figured out the function signatures for system/memory.c, it was easy to reverse the ASM back into C. As regards legal issues, I don't think Microprose will come back from the grave to sue anyone (but people should be aware that trek.exe contains at least one third party library, SmartHeap, which is still actively developed -- we should stay away from reversing that code).

Anyway, it took me about a day in total to reverse that one file (including information gathering, etc.), and that file accounts for about 0.3-0.5% overall code. Projecting that estimate, it should take me a year, working full-time, to decompile BotF . Although I do think that as certain critical subroutines and especially data structures are deciphered, it will become faster and easier to figure out what the code does overall. We'll get the source code back one day, at which time I have no doubt a lot of it will be rewritten.

It's interesting being able to hook in code and make calls to arbitrary subroutines. From what I noticed, __cdecl is not used much for the functions in trek.exe. It's mostly Watcom's funky register-based calling convention, using eax, edx, ecx, ebx, and then the stack for argument passing. But some functions use __stdcall. So I think your suggestion of using inline assembly is good, we could create wrapper functions with inline assembly that set the registers, set the return address, and jump to real subroutine's address in trek.exe.

[thunderchero]

Hi QuasarDonkey,

For all the great work you have done analyzing of the trek.exe, I have given you a new rank of "Code Analyst"

I hope you continue your great work on this project.

thunderchero

[Tethys]

I think it suits him nicely. Nice blue too

[QuasarDonkey]

Awesome! Thanks guys, I'm liking it.

I should also have another update for this soon.

[anjel]

congratulations mate !!!

[Tethys]

Its a faaaaaaaaaaaaaaaaaaake

[QuasarDonkey]

Its a faaaaaaaaaaaaaaaaaaake

It is REAL!

Compulsory viewing:
http://www.youtube.com/watch?v=6lHgbbM9pu4

[QuasarDonkey]

It's that time again. You can get the latest release from the usual place:
http://sourceforge.net/projects/botftrekidb/files/

I've done quite a bit of work for this release, especially after I learned that the 3D library from Falcon 4 is the same one used in BotF. (Some years ago, some crackers broke into Microprose.com and stole the Falcon 4 source code. If anyone is interested in seeing the relevant source code, grab yourself a copy, and look at Falcon4sc/flyer/3Dlib -- NOT the version under Falcon4sc/Graphics/3Dlib. Note I while I do not encourage illegal activities, I don't see any harm here, especially since no action has been taken against the developers of FreeFalcon).

I've labelled most of the equivalent functions in trek.exe based on the info I found there. Unfortunately, I've had a lot of trouble inputting the function signatures in IDA. If anyone is doing any interfacing work with the 3D engine, give me shout and I'll give you more info on the calling conventions (all that code is in C++, and they seem to pass the "this" pointer in eax, and use either registers or the stack for the other parameters; check the Falcon source code for function prototypes).

A few other things I've done include fixing more problems in IDA's analysis, entering function signatures for MSS32.DLL (Miles' Sound System -- the backend used for sound in BotF), labelled all SmartHeap functions, randomly labelled other functions for user interface, zip file handling, and more.

I hope someone finds it useful. I'll have another release soon, I'm not nearly finished yet. Actually I'm swimming in information

P.S. Special thanks to Flocke for pointing me in the direction of useful information on multiple occasions. Cheers!

[Flocke]

Nice work!
That brings some light to the botf 3D engine. I'm looking forward to if you can make some use of this complex old beast.

Also great you found function signature info on the Miles Sound System!

and thx for your help on all this stuff

[QuasarDonkey]

No problem Flocke. I found a header file for MSS here (I'm not certain if it's exactly the same in BotF, it seems to be, and it's from 1995):
http://svn2.assembla.com/svn/legion-2/t ... form/Mss.h

I'm looking forward to if you can make some use of this complex old beast.

I actually have a few patches in the works for various parts of BotF. I'll start releasing them soon enough.

[thunderchero]

I actually have a few patches in the works for various parts of BotF. I'll start releasing them soon enough.

sounds interesting I can't wait

thunderchero