Direct Access to Trek.exe's internal data

General Modding Information/Questions; support/discussion/questions

Moderator: thunderchero

Forum rules
:idea: Please search before starting new topic. :idea:
There is a good chance it has already been asked.
Post Reply
User avatar
QuasarDonkey
Code Analyst
Code Analyst
Posts: 433
Joined: Tue Jul 26, 2011 8:29 pm
Location: Ireland

Direct Access to Trek.exe's internal data

Post by QuasarDonkey »

This is an advanced article for programmers. I discovered this info while working on a new API for Trek.exe.

Direct Access to Trek.exe's Variable and Structures

If you are interfacing code with Trek.exe (e.g. using DLL injection), accessing Trek.exe's data can be a real pain-in-the-butt. However, the gcc compiler has a nice trick that let's us directly reference data from Trek.exe without using pointers. We do this by defining the symbols during the linking phase.

First declare the symbol in your C program with external linkage:

Code: Select all

extern unsigned char MuddCheats;
Now use the variable directly:

Code: Select all

MuddCheats = 1;
Now the Magic
--defsym creates a global symbol in the output file, containing the absolute address given by an expression.
When compiling, pass --defsym=SYMBOL=ADDRESS as an option to the linker (with -Wl, if calling gcc):

Code: Select all

gcc -Wl,--defsym=_MuddCheats=0x5B1A90 -mdll your.c -o your.dll
:arrow: Note the leading underscore in _MuddCheats. The gcc compiler in Windows prefixes symbols with an underscore.

If all is working correctly, you should not get any undefined reference errors.

On an Industrial Scale

The above approach works fine for defining a few symbols, but what about when we need a lot? Just use --just-symbols=FILE

Code: Select all

gcc -Wl,--just-symbols=symbols yoursource.c -o yourapp
where your symbols file contains the list of used symbols:

Code: Select all

_MuddCheats = 0x5B1A90;
_galaxy = 0x5a36b0;
_GCurrentGame = 0x5a28d8;
_GAliens = 0x5a2010;
Note each symbol definition must end in a semicolon.

That's it! Now you can directly access memory from Trek.exe from your code, without having to result to ugly pointer hacks.


Calling Trek.exe Functions from Your Code: A Tale of Two Compilers

As some of you coders already know, Trek.exe was compiled using the Watcom compiler. Watcom was very popular for compiling DOS video games back in the nineties, for it generated highly efficient code (at the time). Now the company that made Watcom scrapped the compiler. It's still available, in open source form, but it's quite outdated, and lacks many modern features that gcc has (like C99/C++11 support).

Anyway, Watcom has a unique register-based calling convention. Let me explain.

Here's a C function prototype:

Code: Select all

int find_root (int a, int b, int c, int d, int e);
And we call it as follows:

Code: Select all

x = find_root (1, 2, 3, 4, 5);
Most C compilers by default (using __cdecl calling convention) produce assembly code like this:

Code: Select all

push 5
push 4
push 3
push 2
push 1
call find_root
add esp, 0x10
The Watcom compiler uses registers (eax, edx, ebx, ecx) for passing the first four arguments, and the stack for more:

Code: Select all

push 5
mov ecx, 4
mov ebx, 3
mov edx, 2
mov eax, 1
call find_root
This makes interfacing with Trek.exe functions tricky when not using the Watcom compiler, since gcc doesn't support Watcom's calling convention. Or does it? :grin:

Some More Magic

gcc has some limited support for register-based calling conventions, as it just happens to pass the first two arguments in the same order as Watcom! How convenient for us. Let me reiterate: only the first two are the same as Watcom's (eax, edx). So you'll have to resort to inline assembly for functions with three or more parameters. Ugh!

You can flag a function to use the register-based calling convention using this function attribute in gcc: __attribute__((regparm(2))). Let's use Trek.exe's UI_Screens_SwitchScreen() function as an example:

Code: Select all

extern void UI_Screens_SwitchScreen (int screen, int data) __attribute__(regparm(2));
As with the data, we can define the symbol during linking -- add a symbols file entry:

Code: Select all

_UI_Screens_SwitchScreen = 0x4B9180;
Now feel free to switch screens from your injected DLL. This code switches to the science screen:

Code: Select all

UI_Screens_SwitchScreen (12, 0);
Assimilating Trek.exe Functions

Overriding code in Trek.exe is pretty easy -- except for memory protection of course. If you try to write new code into Trek.exe code section (really .text section), it will crash with an access violation. So you need to make the memory pages writable first. Here's some code to do that:

Code: Select all

// Safe replacement for memcpy that is aware of memory protection.
void SafeCopy (void *dest, const void *src, size_t n)
{
        // Make the memory page writable.
        DWORD OldProtect;
        VirtualProtect (dest, n, PAGE_READWRITE, &OldProtect);

        // Apply the patch:
        memcpy (dest, src, n);

        // Restore memory page protection.
        VirtualProtect (dest, n, OldProtect, NULL);
}
That's fine for loading pre-compiled patches at run-time. What about hijacking Trek.exe's functions and replacing them with your own functions? We can write some binary code into Trek.exe that does a far jump to the code in your DLL:

Code: Select all

// Replace code at the given address by far jumping to the new address.
// It will overwrite 10 bytes at the given address to do so.
void MonkeyPatch (unsigned int offset, void * targetAddr)
{
        /* Far jump to absolute target address.
         * This takes 10 bytes, two for the opcode, 4 bytes for the jump 
         * address (at trampoline+2), which points to (trampoline + 6),
         * which contains the absolute address of the target function.
         */
        unsigned char * trampoline = (unsigned char *) offset;

        // First make the memory page writable!
        DWORD OldProtection;
        VirtualProtect (trampoline, 10, PAGE_READWRITE, &OldProtection);

        // Write the jump instruction:
        trampoline[0] = 0xff;
        trampoline[1] = 0x25;
        *(void**)(trampoline+2) = trampoline + 6;
        *(void**)(trampoline+6) = targetAddr;

        // Restore memory page protection:
        VirtualProtect (trampoline, 10, OldProtection, NULL);
}
Use it like:

Code: Select all

MonkeyPatch( 0x458260, ProcessTurn );
Assimilating Functions with more than Two Parameters: Resistance Is Not Futile

Let's say we want to call the InitGraphics() function from our code:

Code: Select all

int InitGraphics (HWND hwnd, uint32_t resnum, const char *mode3d, uint32_t language, void *callback, uint32_t data);
Here's the inline assembly to do that:

Code: Select all

static int __wrap_InitGraphics (HWND hwnd, uint32_t resnum, const char *mode3d, uint32_t language, void *callback, uint32_t data)
{
        uint32_t target = 0x513636; // real address of InitGraphics
        int retval;
        asm (
                "push    %6;"
                "push    %5;"
                "movl    %4, %%ecx;"
                "movl    %3, %%ebx;"
                "movl    %2, %%edx;"
                "movl    %1, %%eax;"
                "call    *%7;"          // indirect absolute call
                "movl    %%eax, %0;"    // return value
                : "=r"(retval) // output operands
                : // input operands
                "m"(hwnd),
                "m"(resnum),
                "m"(mode3d),
                "m"(language),
                "m"(callback),
                "m"(data),
                "r"(target)
                : "%eax", "%ebx", "%ecx", "%edx" // clobbered registers
           );
        return retval;
}
Now you can call it from your code, no problem!

What about overriding such a function? You'd have to write a thunk (taking no parameters) using inline assembly to pull the arguments from the registers and stack, and then pass them to your own implementation. I leave it as an exercise to the reader :razz:
User avatar
Tethys
Past Administrator
Past Administrator
Posts: 2392
Joined: Fri Jul 18, 2008 2:00 am
Location: Your mom's bed ;)
Contact:

Re: Direct Access to Trek.exe's internal data

Post by Tethys »

So, will this ever be translated into english? :P

Jokes aside, Im not sure what I am looking at but it looks damn good. Great work! :D From what I read, it looks like we might be able to eventually make use of Sociology tech fields, and perhaps even some entirely new screen categories.
Not for the weak of heart...
Galaxies MOD v0.4.0 <--- GALM/Galaxies Mod latest version
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7849
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Direct Access to Trek.exe's internal data

Post by thunderchero »

Nice, I don't understand it but sounds very nice. :up:

So I have to ask what will use this approach first?

thunderchero
User avatar
QuasarDonkey
Code Analyst
Code Analyst
Posts: 433
Joined: Tue Jul 26, 2011 8:29 pm
Location: Ireland

Re: Direct Access to Trek.exe's internal data

Post by QuasarDonkey »

mmobley1 has started working on the MPR++ extentions. I've been helping him out by prototyping an API (Application Programming Interface) for Trek.exe. See I already have many of the game's data structures coded up for the BotF clone and other tools. The tricks I've outlined above just make the interfacing a lot nicer and easier to work with.

I'm not sure what to do with it yet.
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3197
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: Direct Access to Trek.exe's internal data

Post by Flocke »

great article! :up:
and amusing answers by the novice

an interesting option to add botf keyvariables by a symbols file
I'm not a fan of these oldish c extern declaration one must include everywhere
but I guess the code will become a bit cleaner anyway :)

as for the ugly asm wrapping, maybe some templates could be written so one doesn't have to type it all down over and over again.

edit: you're not sure what to do with it? isn't that obvious? come back to start coding on botf again! :D

edit2: btw, when you assimilate functions, resistance is futile
User avatar
QuasarDonkey
Code Analyst
Code Analyst
Posts: 433
Joined: Tue Jul 26, 2011 8:29 pm
Location: Ireland

Re: Direct Access to Trek.exe's internal data

Post by QuasarDonkey »

Flocke wrote:I'm not a fan of these oldish c extern declaration one must include everywhere
Well, that's how the BotF developers designed it. We have to do it too if we want easy access their API. That's also why I'm making my API in C -- it directly maps to BotF's internal structures.

There's no reason why we can't put a C++ abstraction layer on top of it.
Flocke wrote:as for the ugly asm wrapping, maybe some templates could be written so one doesn't have type it all down over and over again.
MACROS to the rescue! e.g. for calling a Watcom function with 3 parameters:

Code: Select all

#define WATCALL3(target, retvar, parm1, parm2, parm3) asm( \
                "movl %3, %%ebx; movl %2, %%edx; movl %1, %%eax; call *%4; movl %%eax, %0;" \
                : "=r"(retvar) : "m"(parm1), "m"(parm2), "m"(parm3), "r"(target) \
                : "%eax", "%edx", "%ebx" )
It's butt-ugly, but it would make writing thunks easier:

Code: Select all

int somefunc (int arg1, int arg2, int arg3) {
    uint32_t trek_offset = 0x123456;
    int retval;
    WATCALL3 (trek_offset, retval, arg1, arg2, arg3);
    return retval;
}
Disclaimer: I haven't actually tested that code, I just assume it will work. :cool:
User avatar
anjel
Past Administrator
Past Administrator
Posts: 666
Joined: Thu May 08, 2008 2:00 am
Location: Bs As - Argentina

Re: Direct Access to Trek.exe's internal data

Post by anjel »

i totally agree... with Tethys and TC... lol
for what you´ve written looks from bynars guys, but i see where it´s going and looks really exciting, keep going with hard work !!!

regards
User avatar
QuasarDonkey
Code Analyst
Code Analyst
Posts: 433
Joined: Tue Jul 26, 2011 8:29 pm
Location: Ireland

Re: Direct Access to Trek.exe's internal data

Post by QuasarDonkey »

anjel wrote:for what you´ve written looks from bynars guys
Classic.
User avatar
Max315
Cadet 3rd Year
Cadet 3rd Year
Posts: 14
Joined: Fri Jul 26, 2013 8:50 pm
Location: Canada

Re: Direct Access to Trek.exe's internal data

Post by Max315 »

You know, when I learned assembly in CompSci I honestly didn't think it would be all that useful. I stand corrected!

This is really good work QD, everytime I visit the forum it seems like you've made another codding breakthrough for BOTF!
Post Reply

Return to “General Modding Information/Questions”