Extension approaches for the trek.exe executable

General Modding Information/Questions; support/discussion/questions

Moderator: thunderchero

Forum rules
:idea: Please search before starting new topic. :idea:
There is a good chance it has already been asked.
Post Reply
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3196
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Extension approaches for the trek.exe executable

Post by Flocke »

When modding the trek.exe assembler code, the limited exe size has always struggled when people wanted to add new code. And while for the most time you can find enough empty code space or can free some space, this easily leads to either coding mess or lots of work and compromise. Further it soon becomes quite complex to program some simple routines compared to the mighty of modern C++. :roll:

So lets investigate the options for code extensions we have. Details on the coding I might provide with some separate posts.

C++ programming wise, you could:
  • load, attach or replace some library
    - to attach to the trek.exe process you could either abuse some windows debug routines or attach a dll from an external app, which however I never investigated, cause it might cause trouble with some virus detection and there is no reason to run another app
    - to replace some library, like I did with mpr++, you simply could pass through each of the calls to the copied original dll, which already is much handier
    - or you add in a new dynamic library call like documented in the structured modding index coding guides
  • load trek.exe as a non-relocatable dll and call it from your own executable
    - the relocation table here needs to be disabled if not properly fixing all the code references
    - to fix the relocation table you'd have to adjust or remove all entries that became invalid and add a new entry for each new absolute code reference like those of a far jump, which likely is hopeless for the many mods
    - to disable relocation you need to set the relocation table address and size of the PE (portable executable) header to 0
    - the library might not conflict with another absolute positioned library, which however is unlikely with the included and all the common windows libraries. Usually all libraries are build with a relocation table, so on conflict they can be moved and all absolute code references be patched automatically.
  • use monkey patching to dynamically replace some executable code while the app is running
    - to extend some routine a common approach is overwrite some code instructions with a far call or far jump to some loaded dll routine and then perform the original code instructions followed by your own, therefore the moved instructions must be absolute or you need to include all relative referenced code
    - further you have to unlock the write protection of the executable code with VirtualProtect https://docs.microsoft.com/en-us/window ... ualprotect and patch the code from within the trek.exe process or some loaded or attached library which is part of the process
  • patch the trek.exe code to read the call or jump target address from a writeable dynamic key variable, in case you wanted to skip the write protection unlock step
  • patch the executable section tables and add in some further static library calls
    - enlarging some middle section however breaks all code references to what follows, so you likely have to mess with the assembler adjustments described below and add in some new section to the end of the executable
  • or rebuild a perfectly aligned executable like I did with my unlimited asm space executable container side project, that was part of an attempt to build a full development framework for coding with BotF - a project that is still on my list
    - here you copy over the trek.exe code from 0x400 to file end of the original exe to the new exe location just like when you add some new asm section described below
    - in addition the new exe adds some own sections that allow you to compile normal c++ code and far jump to the original exe entry point once ready. It is all in your hands just like if you loaded trek.exe as a dll, just that it still is a full executable and you don't need to mess with making the dll non-relocatable and care to not conflict with the virtual executable address if you'd like to have your exe non-relocatable
    - for reference I attached the gcc ld linker script I used to build such an exe with aligned sections, see bottom of the post
Assembler wise, you could:
  • add new sections to the end of the executable
    - When you check on the PE-Header of executables https://en.wikipedia.org/wiki/Portable_ ... _fixed.svg you find that it includes a section table where each of the sections is listed with file offset, size, attributes and the virtual assembler address. This already tells it all what you need to do to add a new section.
    For full executable structure also check https://raw.githubusercontent.com/corka ... /pe101.png for a Hello World example.
    - As a tool for investigating the PE-Header I can recommend the CFF Explorer you can find over here: https://ntcore.com/?page_id=388 Ofc there also exist other tools with detailed information and in part also with editing capabilities. I can't tell what tool is best.
    - The problem with it is, that you need to enlarge the section table and therefore move all the onfollowing code. Luckily this does not break the assembler code, given each section has the virtual assembler address listed to where the sections are loaded in memory when you run the application. It is just the file contents you need to move.
    - Obviously this will break editors like UE and QD patcher that rely on hex file offsets and executable size.
    But there exist a simple workaround by copying all the code from the original sections over to the old executable and back and forth when ever you need to comply.

    - Update: As I figured trek.exe left enough empty space to add another 10 sections without having to move any code! So moving the code is a non-issue! More info over here: viewtopic.php?p=53846#p53846
    - Beside of this, by inspiration from Tethys viewtopic.php?p=53841#p53841 I realized that you could as well split first section and move some code to free more space if ever needed.
  • just enlarge the last section instead
    - But then you mess with the usage of the different section types and I guess the last section was no code section so you need to patch the access restrictions of the section characteristics. Not much of a deal actually if you aren't too worried on security, which is why data sections should contain no executable code.
  • patch some existing dll instead
    - Dlls have the same file structure, so can be easily patched in same manner. Further all loaded dlls share same virtual memory space with the exe, so you just need to know which offset and can do far calls and jumps to the dll code.
    - As a benefit you don't need to mess with the trek.exe file layout here, but you either need to make the dll non-relocatable, or you can only use indirect references and calls writing the actual load offset from the dll to some new trek.exe key variable once it is loaded.
  • create a plain empty dll that you load along with trek.exe and can adjust to your needs
    - Again make the dll non-relocatable or dynamically adjust the far call and far jump addresses into the dll for the actual virtual load address.
    - Adding a new dll for this however seems to be a bit improper, cause then you really rather program some C++ when you already build and injected the dll.
  • or make use of already existing exe space that has no use like the relocation table that you can easily deactivate and convert to an executable code section by changing the section characteristics
    - of course this still would be limited space but enough for some nice little projects
    - like already mentioned, the relocation table for sure is broken for most of the mods anyhow given not only new absolute references must be included, but also the old ones must be updated whenever there is a change
    - second if there is no collision with a non-relocatable dll, the exe is always loaded to its preferred virtual asm address and never moved, so it is discarded anyhow
    - if disabled in the section header you enforce that instead the dlls must be relocated, and other than for executables, for dlls it is nature that they are relocatable
This is a ton of options to hack into and extend the trek.exe code. Might be that I missed some option, but I think it is pretty comprehensive. :smile:

My personal choise is to add more sections by compiling a new exe with the attached gcc ld linker script. Details on this however is out of scope for this overview and I need to recap on my research and the coding framework I aimed to create.

:arrow: One key aspect you need to care when you start to extend the asm space for patches, is that others might easily conflict with their own patches. So you should always report what offset and size you have in use, specially the virtual assembler addresses.

I hope this finds some good use and am sorry for not having provided any further details as of now. When you have a serious request, post me a note and I'll see what I can help.

cheers
Attachments
ldscript.ld.txt
(14.41 KiB) Downloaded 145 times
Post Reply

Return to “General Modding Information/Questions”