Standard Trek.exe IDA Database

Standard Trek.exe IDA Database; support/discussion/questions

Moderator: thunderchero

User avatar
QuasarDonkey
Code Analyst
Code Analyst
Posts: 433
Joined: Tue Jul 26, 2011 8:29 pm
Location: Ireland

Standard Trek.exe IDA Database

Post by QuasarDonkey »

This is an attempt at creating a well-documented standard IDA database (IDB file) for Trek.exe, readable by humans.

Here you'll find the sixth release of the Trek.exe IDB:
I used IDA Pro Freeware 5.0 (http://www.hex-rays.com/idapro/idadownfreeware.htm), and the Trek.exe from the Main Multi installer, the one with the no-CD patch.

Code: Select all

Trek.exe Size: 1781248 bytes, MD5 sum: 483448abf30f84d19c7a5b59b0536935

What I've done:
  • Cleaned up IDA's auto-analysis, fixing almost all problems in the code analysis.
  • Cleaned up much of the data section.
  • Labelled most functions with prefixes indicating the source code files they belong to. Most functions have names like AI_AIEmpire_407020, indicating it originally came from AI/AIEmpire.c
  • Given some functions proper names like System_Pref_ReadPrefs, Game_Borders_Calculate, etc.
  • Identified most C library functions that IDA missed, like malloc, fopen, etc. These are marked with L in the functions list.
  • Identified many functions related to the 3D subsystem.
  • Entered function signatures for most library functions, the MPR library (courtesy of Flocke), MSS32.DLL (Miles' Sound System), the SmartHeap library, all functions from system/memory.c, gdllist.c, hash.c, gheap.c, and others. (These functions are marked with T in the functions list, T meaning Type information).
  • Entered numerous data structures, and labelled many global variables.
  • Flagged any functions called by library functions as library functions themselves (marked with L in the functions list). You can ignore these functions.
  • Flagged many functions as static (marked with S in the functions list). This means they are only called within their own module (i.e. they are private). Note I've only partially completed my labelling of static functions.
  • Functions coded in C++ have the letters CPP in their name; most of Trek was coded in C, but some C++ was used.
I'm still calling this a preview release because I'm still not finished my analysis. There's still a lot of useful info in Trek.exe that I haven't looked at yet. I plan on making many future releases as I gather more info.

Where to start
Open up the database with IDA, and click on the Functions tab. You'll see most functions have been categorized.
Some functions have also been properly labelled. You should be able to locate code of interest from there.
Functions marked with L (Library function) or S (Static function) are less interesting than other functions.
Last edited by QuasarDonkey on Tue May 22, 2012 4:22 pm, edited 25 times in total.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7849
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: trek.exe - My Initial Analysis

Post by thunderchero »

Interesting,

While I do not understand any of it I look forward to seeing your next post.

thunderchero
User avatar
QuasarDonkey
Code Analyst
Code Analyst
Posts: 433
Joined: Tue Jul 26, 2011 8:29 pm
Location: Ireland

Re: trek.exe - My Initial Analysis

Post by QuasarDonkey »

Basically what I'm saying is that we now know what roughly what each function in trek.exe relates to, whether it be AI, User Interace, Tactical Combat, Intel, Science, Economy, etc. So if you wanted to locate all code for minor race AI, I know where to look for it in trek.exe. Figuring out exactly how the code works is still hard, but this should make things a little easier.
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3197
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: trek.exe - My Initial Analysis

Post by Flocke »

that's nice work, I might have done this too if I had found the time, but I know SCT has a real good and advanced documented ida file.
Never have seen it and haven't been in need, but with all his work on the asm part, I'm sure it's worth an ask. ;)
If he didn't label all these functions already, a merge would be cool.
And I agree such labels are much easier to read. :)

To figure out how the code works the ida debugger is very helpful, though it's a little tricky to get it work with botf. On the one hand you have to get botf run in windowed mode, on the other it throws some exceptions especially on video playback and some have to pass while most have to be skipped or botf will crash, at least on my system.

One side note, you posted in the solutions forum, but analysis isn't meant to be part of it, see viewtopic.php?f=9&t=1309&sid=cf9f3143b5 ... 805d5768e9 instead the modding section should be used.
But many people misinterpret this forums name.
User avatar
DCER
Code Master
Code Master
Posts: 683
Joined: Sat Apr 26, 2008 2:00 am

Re: trek.exe - My Initial Analysis

Post by DCER »

Interesting read, good work. Having a well documented ida file available for download would be nice.
User avatar
QuasarDonkey
Code Analyst
Code Analyst
Posts: 433
Joined: Tue Jul 26, 2011 8:29 pm
Location: Ireland

Re: trek.exe - My Initial Analysis

Post by QuasarDonkey »

I wouldn't actually call my IDA file well documented. I've only given the functions prefixed names, and input a few data structures, named a few variables, etc. It still needs a lot of work. I might get on to SCT about it, see it he's willing to share his knowledge... It'd be a good chance for me to familiarize myself with the code. I'll have to try get the debugger working, that would be a massive help. Better than trying to just read the assembly :)
User avatar
Tethys
Past Administrator
Past Administrator
Posts: 2392
Joined: Fri Jul 18, 2008 2:00 am
Location: Your mom's bed ;)
Contact:

Re: trek.exe - My Initial Analysis

Post by Tethys »

Im interested to see if trek.exe could be extended, or recompiled to be larger, which Im sure would be the ultimate goal. BTW, Ive moved this topic to Modding Info for the time being.
Not for the weak of heart...
Galaxies MOD v0.4.0 <--- GALM/Galaxies Mod latest version
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3197
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: trek.exe - My Initial Analysis

Post by Flocke »

well, this isn't a trouble anymore tethys, only if you want to keep asm only but QuasarDonkey is a programmer and knows how to use dlls, right ;)
hint: read about codecaves

QuasarDonkey, for debugging turning off the videos in stbof.ini might help, and I think I only had to let first exception on video-playback pass and block the others, but I use windows not linux. As far I know on wine it's also no problem to run in windowed mode.
User avatar
Spocks-cuddly-tribble
Code Master
Code Master
Posts: 1884
Joined: Sun Apr 27, 2008 2:00 am

IDA database project

Post by Spocks-cuddly-tribble »

DCER wrote:Having a well documented ida file available for download would be nice.
Of course, I'll provide my IDA-database for this worthy project. But it will be less helpful than some people expect.

Also maybe the topic should be renamed in order to make the project clearly recognisable?

Flocke wrote:I know SCT has a real good and advanced documented ida file.
Unfortunately, my database doesn't look like I suggested in The "avoid nerd stuff" ASM Hints & Tips (formatting of most of my old postings is corrupted due to forum updates).

NOTE; I started editing this database without any asm-knowledge i.e. ~80% of my labels are wrong or just nonsense and I never had the time/motivation to correct this. Also the file was created by IDA-version 4.9 and based on a very old (AFAIK outdated muli-installer exe) which even was slightly edited (gal-density & stellar object reference values IIRC).

Nevertheless, here it is:
SCT version
sct_trek.idb
I don't know how many bugs is too many but that point is reached somewhere before however many in BotF is.
User avatar
QuasarDonkey
Code Analyst
Code Analyst
Posts: 433
Joined: Tue Jul 26, 2011 8:29 pm
Location: Ireland

Re: trek.exe - My Initial Analysis

Post by QuasarDonkey »

Thanks a lot SCT! I just took a look through it. It will be very helpful. It's much appreciated.
If anyone else has done work on Trek.exe, labeling functions, variables, etc., feel free to email everything on to me. I can quickly locate interesting information by dumping the file to ASM and scanning for differences with the vanilla disassembly.

What I'll do is merge the databases together with the new information I've discovered, and create an Ultimate Trek.exe IDA Database for the community to use. I'll be starting a new job soon (possibly next week or the week after), but I should still have time to work on BotF. My plan is to have an IDB ready for download in a few days.
P.S. Don't worry about the labels being wrong SCT, I'll try to verify them as I go along.

I'm familiar with many of IDA's features, and I'll be inputting all known variables as data structures and arrays which are easier to work with (I've already done much of this in my own IDB). I'll also be setting up proper function prototypes, so that the ASM will be self-documenting (IDA will comment the code for us). I'll post an article soon on calling conventions, but basically coders should be aware that most functions in Trek.exe follow the calling conventions of the Watcom C compiler (basically function arguments are stored in eax, edx, ebx, ecx in that order, and the return value is stored in eax; well there's a little more to it... so stay tuned).

The end result will be far easier to work with, and should ultimately make decompilation a realistic possibility :D
I guess we should rename this thread to something more appropriate, like "Standard Trek.exe IDA Database". Does anyone know how to rename a post?
User avatar
DCER
Code Master
Code Master
Posts: 683
Joined: Sat Apr 26, 2008 2:00 am

Re: Standard Trek.exe IDA Database

Post by DCER »

Thanks SCT!

Looking forward to the final database, QuasarDonkey.
User avatar
QuasarDonkey
Code Analyst
Code Analyst
Posts: 433
Joined: Tue Jul 26, 2011 8:29 pm
Location: Ireland

Re: Standard Trek.exe IDA Database

Post by QuasarDonkey »

Heys guys. I've updated the first post in the thread with the first release of the IDB file. I hope someone finds it useful.
User avatar
anjel
Past Administrator
Past Administrator
Posts: 666
Joined: Thu May 08, 2008 2:00 am
Location: Bs As - Argentina

Re: Standard Trek.exe IDA Database

Post by anjel »

downloading, and downloading IDA Pro 6.1 Evaluation version so i can see it well, is there another usefull tool for dummys like me in these area ???
Live long and propser
User avatar
QuasarDonkey
Code Analyst
Code Analyst
Posts: 433
Joined: Tue Jul 26, 2011 8:29 pm
Location: Ireland

Re: Standard Trek.exe IDA Database

Post by QuasarDonkey »

anjel wrote:downloading, and downloading IDA Pro 6.1 Evaluation version so i can see it well, is there another usefull tool for dummys like me in these area ???
The Evaluation version is limited in many ways (you can't save changes, etc), so I don't recommend it.

The Freeware version is excellent, and does everything you need (that's what I use). You can get it here:
http://www.hex-rays.com/idapro/idadownfreeware.htm

There is excellent help online for using IDA, and it has help files too. I know SCT posted lots of links for learning ASM if you need that (I think it's under Structured Modding).
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7849
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Standard Trek.exe IDA Database

Post by thunderchero »

while I don't understand most of this I did take a look, and will report it looks like it converted it to the 5.2 version without any problems.

thunderchero
Post Reply

Return to “Standard Trek.exe IDA Database”