Standard Trek.exe IDA Database

Standard Trek.exe IDA Database; support/discussion/questions

Moderator: thunderchero

User avatar
anjel
Past Administrator
Past Administrator
Posts: 666
Joined: Thu May 08, 2008 2:00 am
Location: Bs As - Argentina

Re: Standard Trek.exe IDA Database

Post by anjel » Tue Aug 16, 2011 8:38 pm

the same here, worked like a charm.
Live long and propser

User avatar
QuasarDonkey
Code Analyst
Code Analyst
Posts: 433
Joined: Tue Jul 26, 2011 8:29 pm
Location: Ireland

Re: Standard Trek.exe IDA Database

Post by QuasarDonkey » Tue Aug 16, 2011 9:23 pm

That's great. I'm hoping it will be useful for future modding. It may not be completely useful until the functions are given more descriptive names. I've already labeled a few functions for turn processing, etc.

But I'll have another release real soon. I've identified about 50 new functions and lots of variables. Right now I'm in the guts of the user interface code, trying to make sense of it.

User avatar
QuasarDonkey
Code Analyst
Code Analyst
Posts: 433
Joined: Tue Jul 26, 2011 8:29 pm
Location: Ireland

Re: Standard Trek.exe IDA Database

Post by QuasarDonkey » Thu Aug 18, 2011 10:29 am

I've posted another update.

I've labeled a lot of unknown functions and variables, and I've entered function signatures for most library functions, which should make analysis easier.

I've also started using structure member offsets to make code more readable, e.g.

Code: Select all

mov eax, galInfo.Systems
mov ebx, [ebp+eax+4Dh]
becomes this:

Code: Select all

mov eax, galInfo.Systems
mov ebx, [ebp+SystemInfo_t.owner]
But I've only done this for a few functions so far, like Game_Borders_Calculate.

I've also identified all functions from the file system/memory.c. It turns out Trek uses a third-party library called SmartHeap for memory management (this library overrides standard C functions malloc, free, etc.; this is why IDA missed them). Trek has numerous memory allocator functions, based on a memory pooling scheme, so you have one function for allocating memory related to UI, one for AI, etc. I've identified the important functions from the SmartHeap library too. If people want more information on BotF memory management, just ask.

As a proof of concept, after I figured out the function names and function signatures for system/memory.c, it was trivial to reverse engineer the file back into C source code :D Unfortunately, since it requires the SmartHeap library which costs $2500+, the code won't compile :x I'm not sure whether I should start posting BotF source code since I'm really not sure of the legal status of reverse engineered source code. Before anyone gets excited, I'd estimated that file I reversed accounts for <0.5% of the total source code.

User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 2546
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: Standard Trek.exe IDA Database

Post by Flocke » Fri Aug 19, 2011 5:39 am

I just had a look at your ida file, looks good, great work!

Well, decompilation doesn't seem to be that a well option to me, though there are some decompiler programs that might allow to decompile most of the botf code. As long there is asm code left and there are cross references, it's too much work and risky to readdress the code & data offsets. Not to talk about legality issues. :lol:

Instead simply choose a function and link in your own implementation via codecaves as I already suggested. That way you also can 'decompile' parts of the source if you want and still depend on functions like memory management ones that were compiled to trek.exe.
The cool thing I didn't realize for a long time either, is that dll files that were loaded by botf are in same process space and allow to directly call botf functions by their asm offsets. To call functions of your library you can use an array of function pointers that you can pass to botf by an initial call and only need to store the single pointer to the array somewhere in botf data section.
When calling functions vice versa, of course you've to regard calling conventions like __cdecl, but that's rather easy and you can use inline assembler if there's something special to match.
I already posted about codecaves here: viewtopic.php?f=4&t=598&start=15

cheers & looking forward to see your first break throughs in coding :D

User avatar
QuasarDonkey
Code Analyst
Code Analyst
Posts: 433
Joined: Tue Jul 26, 2011 8:29 pm
Location: Ireland

Re: Standard Trek.exe IDA Database

Post by QuasarDonkey » Fri Aug 19, 2011 2:34 pm

Thanks Flocke, you flatter me sir!

I do think decompilation is possible, with or without decompiler programs, but not yet. Like I said, once I figured out the function signatures for system/memory.c, it was easy to reverse the ASM back into C. As regards legal issues, I don't think Microprose will come back from the grave to sue anyone (but people should be aware that trek.exe contains at least one third party library, SmartHeap, which is still actively developed -- we should stay away from reversing that code).

Anyway, it took me about a day in total to reverse that one file (including information gathering, etc.), and that file accounts for about 0.3-0.5% overall code. Projecting that estimate, it should take me a year, working full-time, to decompile BotF 8O. Although I do think that as certain critical subroutines and especially data structures are deciphered, it will become faster and easier to figure out what the code does overall. We'll get the source code back one day, at which time I have no doubt a lot of it will be rewritten. :D

It's interesting being able to hook in code and make calls to arbitrary subroutines. From what I noticed, __cdecl is not used much for the functions in trek.exe. It's mostly Watcom's funky register-based calling convention, using eax, edx, ecx, ebx, and then the stack for argument passing. But some functions use __stdcall. So I think your suggestion of using inline assembly is good, we could create wrapper functions with inline assembly that set the registers, set the return address, and jump to real subroutine's address in trek.exe.

User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 6066
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.
Contact:

Re: Standard Trek.exe IDA Database

Post by thunderchero » Sun Aug 21, 2011 7:38 pm

Hi QuasarDonkey,

For all the great work you have done analyzing of the trek.exe, I have given you a new rank of "Code Analyst" :D

I hope you continue your great work on this project.

thunderchero

User avatar
Tethys
Past Administrator
Past Administrator
Posts: 2065
Joined: Fri Jul 18, 2008 2:00 am
Location: Your mom's bed ;)
Contact:

Re: Standard Trek.exe IDA Database

Post by Tethys » Sun Aug 21, 2011 7:42 pm

I think it suits him nicely. Nice blue too ;)
Not for the weak of heart...
Image
GALM <--- GALM/Galaxies Mod

User avatar
QuasarDonkey
Code Analyst
Code Analyst
Posts: 433
Joined: Tue Jul 26, 2011 8:29 pm
Location: Ireland

Re: Standard Trek.exe IDA Database

Post by QuasarDonkey » Sun Aug 21, 2011 9:25 pm

Awesome! Thanks guys, I'm liking it.

I should also have another update for this soon.

User avatar
anjel
Past Administrator
Past Administrator
Posts: 666
Joined: Thu May 08, 2008 2:00 am
Location: Bs As - Argentina

Re: Standard Trek.exe IDA Database

Post by anjel » Sun Aug 21, 2011 9:38 pm

congratulations mate !!! :)
Live long and propser

User avatar
Tethys
Past Administrator
Past Administrator
Posts: 2065
Joined: Fri Jul 18, 2008 2:00 am
Location: Your mom's bed ;)
Contact:

Re: Standard Trek.exe IDA Database

Post by Tethys » Sun Aug 21, 2011 11:34 pm

Its a faaaaaaaaaaaaaaaaaaake
Not for the weak of heart...
Image
GALM <--- GALM/Galaxies Mod

User avatar
QuasarDonkey
Code Analyst
Code Analyst
Posts: 433
Joined: Tue Jul 26, 2011 8:29 pm
Location: Ireland

Re: Standard Trek.exe IDA Database

Post by QuasarDonkey » Mon Aug 22, 2011 12:19 am

Tethys wrote:Its a faaaaaaaaaaaaaaaaaaake
It is REAL! :D

Compulsory viewing:
http://www.youtube.com/watch?v=6lHgbbM9pu4

User avatar
QuasarDonkey
Code Analyst
Code Analyst
Posts: 433
Joined: Tue Jul 26, 2011 8:29 pm
Location: Ireland

Re: Standard Trek.exe IDA Database

Post by QuasarDonkey » Sat Aug 27, 2011 7:12 am

It's that time again. You can get the latest release from the usual place:
http://sourceforge.net/projects/botftrekidb/files/

I've done quite a bit of work for this release, especially after I learned that the 3D library from Falcon 4 is the same one used in BotF. (Some years ago, some crackers broke into Microprose.com and stole the Falcon 4 source code. If anyone is interested in seeing the relevant source code, grab yourself a copy, and look at Falcon4sc/flyer/3Dlib -- NOT the version under Falcon4sc/Graphics/3Dlib. Note I while I do not encourage illegal activities, I don't see any harm here, especially since no action has been taken against the developers of FreeFalcon).

I've labelled most of the equivalent functions in trek.exe based on the info I found there. Unfortunately, I've had a lot of trouble inputting the function signatures in IDA. If anyone is doing any interfacing work with the 3D engine, give me shout and I'll give you more info on the calling conventions (all that code is in C++, and they seem to pass the "this" pointer in eax, and use either registers or the stack for the other parameters; check the Falcon source code for function prototypes).

A few other things I've done include fixing more problems in IDA's analysis, entering function signatures for MSS32.DLL (Miles' Sound System -- the backend used for sound in BotF), labelled all SmartHeap functions, randomly labelled other functions for user interface, zip file handling, and more.

I hope someone finds it useful. I'll have another release soon, I'm not nearly finished yet. Actually I'm swimming in information :D

P.S. Special thanks to Flocke for pointing me in the direction of useful information on multiple occasions. Cheers!

User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 2546
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: Standard Trek.exe IDA Database

Post by Flocke » Sat Aug 27, 2011 12:05 pm

Nice work!
That brings some light to the botf 3D engine. I'm looking forward to if you can make some use of this complex old beast. :)

Also great you found function signature info on the Miles Sound System!

and thx for your help on all this stuff ;)

User avatar
QuasarDonkey
Code Analyst
Code Analyst
Posts: 433
Joined: Tue Jul 26, 2011 8:29 pm
Location: Ireland

Re: Standard Trek.exe IDA Database

Post by QuasarDonkey » Sat Aug 27, 2011 12:15 pm

No problem Flocke. I found a header file for MSS here (I'm not certain if it's exactly the same in BotF, it seems to be, and it's from 1995):
http://svn2.assembla.com/svn/legion-2/t ... form/Mss.h
Flocke wrote:I'm looking forward to if you can make some use of this complex old beast. :)
I actually have a few patches in the works for various parts of BotF. I'll start releasing them soon enough.

User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 6066
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.
Contact:

Re: Standard Trek.exe IDA Database

Post by thunderchero » Sat Aug 27, 2011 1:04 pm

QuasarDonkey wrote:I actually have a few patches in the works for various parts of BotF. I'll start releasing them soon enough.

sounds interesting I can't wait

thunderchero

Post Reply

Return to “Standard Trek.exe IDA Database”

Who is online

Users browsing this forum: No registered users