How to extend trek.exe code sections

General Modding Information/Questions; support/discussion/questions

Moderator: thunderchero

Forum rules
:idea: Please search before starting new topic. :idea:
There is a good chance it has already been asked.
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3197
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

How to extend trek.exe code sections

Post by Flocke »

While in viewtopic.php?f=4&t=4038 I brought some light to the many options you could hook or patch the exe and inject your code, here I want to specifically address how to enlarge trek.exe itself.

When you check on the PE header https://en.wikipedia.org/wiki/Portable_ ... _fixed.svg of trek.exe, you find that there are 6 sections listed in the section table
trek_sections.jpg
trek_sections.jpg (136.69 KiB) Viewed 3474 times
Each of those section entries is 40 bytes large, so the section table can be enlarged by another 10 sections without having to move and patch any code:
free_sections.jpg
free_sections.jpg (149.11 KiB) Viewed 3474 times
All that needs to be done is enlarge the exe file, update the PE Header for the new sections and write some proper values for the new section locations, virtual address mapping and characteristics of whether they are read, write or execution protected.

But step by step:
  1. With your hex editor, enlarge your exe file size by whatever space the new sections should be.
  2. Next, starting at 0x268 hex file offset, append your sections to the section table according to the PE header documentation above, by entering:
    - up to 8 bytes for the new section name,
    - skip 4 bytes, this is the virtual memory size used but doesn't seem to be relevant or is auto-calculated from the raw file size of the section
    - enter the virtual memory address for the assembler addresses to where the code is loaded when the app is started,
    - enter 4 bytes for the raw file size of the section,
    - enter 4 more bytes for the raw file offset of the new section,
    - skip 16 bytes for unused relocations,
    - enter 4 bytes for the characteristic flags of the section
    For the characteristic flags, either copy over the characteristics from another section you know or refer https://docs.microsoft.com/en-us/window ... tion-flags
    The most important flags likely are:
    FlagValueDescription
    IMAGE_SCN_CNT_CODE0x00000020The section contains executable code.
    IMAGE_SCN_CNT_INITIALIZED_DATA0x00000040The section contains initialized data.
    IMAGE_SCN_CNT_UNINITIALIZED_ DATA0x00000080The section contains uninitialized data.
    IMAGE_SCN_MEM_EXECUTE0x20000000The section can be executed as code.
    IMAGE_SCN_MEM_READ0x40000000The section can be read.
    IMAGE_SCN_MEM_WRITE0x80000000The section can be written to.
    As you might spot, the trek.exe .reloc relocation table section is flagged with "IMAGE_SCN_MEM_DISCARDABLE 0x02000000 The section can be discarded as needed.".
  3. Then update the File header to increase the NumberOfSections
    file_header.jpg
    file_header.jpg (59.74 KiB) Viewed 3474 times
  4. Further update the 'Optional Header' SizeOfImage value to your last section virtual address + raw size, but increased to match the section alignment of 1000 in hex. Means the trek.exe .rsrc resource section is set to start at virtual address 0x29D000 with a size of 0xA00 so ends at 0x29DA00 but by the section alignment of 1000 the image size actually ends at 29E000.
    Image
  5. In addition in the 'Optional Header' there are some further size attributes named 'SizeOfCode', 'SizeOfInitializedData' and 'SizeOfUninitializedData'
    I dunno if these are important, but they cumulate the size of the different section types. So when you have two sections of executable code you cumulate the raw sizes of both of them.
This should be it, now you can add some code and data to your new sections and do some far calls or jumps or reference by absolute address. :wink:
Last edited by Flocke on Fri Apr 09, 2021 10:10 am, edited 3 times in total.
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3197
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: How to extend trek.exe coding sections

Post by Flocke »

reserved to upload and link some images
Attachments
image_size.jpg
image_size.jpg (91.89 KiB) Viewed 3472 times
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7851
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: How to extend trek.exe code sections

Post by thunderchero »

Flocke wrote: Fri Apr 09, 2021 9:53 am As you might spot, the trek.exe .reloc relocation table section is flagged with "IMAGE_SCN_MEM_DISCARDABLE 0x02000000 The section can be discarded as needed.".
could the .reloc relocation table section be changed to be used as code without increasing the size of the trek.exe?

Note some patches and even some mods have used this section already.

1. QD uses 0x0019f400 - 0x0019f717 for energy screen project
2. UCW I placed enlarged ship range table at 0x0019fbd0 - 0x001a031f (not used in latest version)
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3197
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: How to extend trek.exe code sections

Post by Flocke »

thunderchero wrote: Tue Apr 13, 2021 5:21 pm could the .reloc relocation table section be changed to be used as code without increasing the size of the trek.exe?
Sure, like already mentioned in last point of viewtopic.php?f=4&t=4038 you just need to clean all references in the section headers, see first image. In addition you should disable the whole relocation table in the section table, see https://en.wikipedia.org/wiki/Portable_ ... _fixed.svg bottom left.
Then change the section characteristics to allow for executable code. Oh and you should unflag that the section may be discarded.

You however should not share that section for both data and code. It probably works but if writable you make it a mutable application, with access to re-write it's own routines, and on error that can lead to unexpected, potentially harmful results. So it is convention to separate code from data sections for good reason. Only when you do scientific research on ai programming you might find actual reason to allow for mutable code, and even then there are better options than to change the section characteristics. I'd not be surprised when some virus detection alerts on this either.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7851
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: How to extend trek.exe code sections

Post by thunderchero »

I did make an attempt last night, but it failed.

What I attempted was to split the relocation area into 2 areas 1 for data for energy screen project and second for code. But not sure how to "disable the whole relocation table in the section table" I have not read the link you have provided yet.

here is what my failed edited trek.exe looks like in CFF Explorer.
reloc.jpg
reloc.jpg (247.54 KiB) Viewed 3422 times
a couple questions
do the sections need to be in order?
is there a location/pointer for icon section? when I moved the .rsrc section icon stopped displaying.
User avatar
Spocks-cuddly-tribble
Code Master
Code Master
Posts: 1884
Joined: Sun Apr 27, 2008 2:00 am

Re: How to extend trek.exe code sections

Post by Spocks-cuddly-tribble »

thunderchero wrote: Wed Apr 14, 2021 7:21 amWhat I attempted was to split the relocation area into 2 areas 1 for data for energy screen project and second for code.
The section type classifications are sometimes counterintuitive for non-expert programmers:

- IIRC the energy screen project data area is read-only (no write-access -> this would be indicated by displaying question marks in hex view of IDA-pro e.g. loaded saved game files)
- read-only executable code areas can also be used as read-only data areas with no issues i.e. no separation needed in this case (many of my old patches do this already)
- IIRC by default BotF does not use advanced features like dynamic (write-access) executable code areas


But better Flocke confirms and elaborates this, and maybe provides a hex patch example for changing the relocation table into read-only executable code (to be on the safe side for noobs).
I don't know how many bugs is too many but that point is reached somewhere before however many in BotF is.
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3197
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: How to extend trek.exe code sections

Post by Flocke »

thunderchero wrote: Wed Apr 14, 2021 7:21 am I did make an attempt last night, but it failed.
On first sight, you messed the section sizes. Your DGROUP2 is now larger than the .reloc section before, not to talk of the AUTO2 section. Sections may not overlap! The 4KB alignment I think is only relevant to the virtual address offset and looks fine to me. But add the raw size to the virtual address and it should not exceed next section address! The raw size is what is copied from the exe file. Of course it should match and not overlap with the raw addresses either.

Sharing a section for both code and read-only data technically is ok, but it is not common practice and I'd rather avoid it if not enforced to. You easily clutter the code and need to move the data when the code is changed. And when you reference that data from some other routine, you need to track and update all the references, while when in a separate data section you can write a much cleaner listing.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7851
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: How to extend trek.exe code sections

Post by thunderchero »

Here is my latest attempt,
test.jpg
test.jpg (238.49 KiB) Viewed 3389 times
Game will load, energy screen added and loads properly.

then I attempted to add my X Y code to new area. I added changes and adjusted call to new location and edited call to sprintf_

I expected this to work, but when entering into F2 screen game hangs and crashes.
User avatar
Spocks-cuddly-tribble
Code Master
Code Master
Posts: 1884
Joined: Sun Apr 27, 2008 2:00 am

Re: How to extend trek.exe code sections

Post by Spocks-cuddly-tribble »

thunderchero wrote: Wed Apr 14, 2021 11:09 amI added changes and adjusted call to new location
By "call" to new location you mean jump? Since "call" would break your code judging from your posted example, like other miscalculated adjustments. EDIT: I missread your code "call loc_4B6640" -> you mean now sub_4B6640, I guess.

Other than that, the conventions of the bitmask flags are still hard to assess for noobs; e.g. "contains executable code" vs "can be executed as code".
I don't know how many bugs is too many but that point is reached somewhere before however many in BotF is.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7851
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: How to extend trek.exe code sections

Post by thunderchero »

Spocks-cuddly-tribble wrote: Wed Apr 14, 2021 12:35 pm
thunderchero wrote: Wed Apr 14, 2021 11:09 amI added changes and adjusted call to new location
EDIT: I missread your code "call loc_4B6640" -> you mean now sub_4B6640, I guess.
correct, I moved loc_4B6640 to new code area (it should show as new loc_??????).

when I look at windows event viewer fail location is 3080 bytes past new code location in a 00 00 00 00 00 area. So I adjusted call 3080 bytes, then it was 4 bytes before new code, so I adjusted it again by 4 bytes and it was still 4 bytes before code. then I moved my code to that location and still failed at same point. :sad:

sad part is even if I get this call figured out my new code calls sprintf_ and I am sure that will not be easy to call correct code location.
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3197
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: How to extend trek.exe code sections

Post by Flocke »

thunderchero wrote: Wed Apr 14, 2021 11:09 amI expected this to work, but when entering into F2 screen game hangs and crashes.
By the section headers you posted I only can see you changed the characteristics flags on the now named .EXTEN section to:
IMAGE_SCN_CNT_CODE|IMAGE_SCN_MEM_EXECUTE|IMAGE_SCN_MEM_READ like the already available AUTO section, so that should be fine.

But what I agree with SCT is, that you indeed need to jump to the moved local routine. Calls only work for whole subroutines, that push and pop registers for parameters and return to and from the stack. If 'calling' a local routine, you likely corrupt the stack. But there also exist far jumps.
What you probably need is the EA JMPF: http://ref.x86asm.net/coder32.html But I havn't tried.


Edit: Better don't mess with jmp far but stay with relative near jmp and calculate the address. Far jumps are meant for a different purpose. It once was invented to address 20bit memory for 16bit CPUs by memory segmentation, but soon became obsolete and was replaced to index different memory segments defined by the os instead, mostly for separating kernel and user privileges.
Last edited by Flocke on Fri Apr 16, 2021 8:24 am, edited 3 times in total.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7851
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: How to extend trek.exe code sections

Post by thunderchero »

Flocke wrote: Wed Apr 14, 2021 1:56 pm Calls only work for whole subroutines.
then I do not understand, I thought my code for XY is a sub-routine. it is separate from any other code and returns to previous location. It was labeled as a "loc" because it was within another sub-routine since I had no other place to put it. while writing this I thought I might need to now add push's for registries that are needed for my code?

but still to me the call is not going to correct location/offset that it is set to go. This I do not understand why.
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3197
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: How to extend trek.exe code sections

Post by Flocke »

thunderchero wrote: Wed Apr 14, 2021 2:19 pmthen I do not understand, I thought my code for XY is a sub-routine. it is separate from any other code and returns to previous location. It was labeled as a "loc" because it was within another sub-routine since I had no other place to put it.
Well, I'd have to look into your code changes and read up on how subroutines differ actually, and what was the watcom calling conventions. But you should never mess local jump routines with global call routines. When you did not copy a full sub-routine, it likely is no sub-routine.

On what is wrong with the offset called I can't tell either without having checked. Might be that you simply calculated the location wrong, possibly due to section alignment. Might also be that indeed the correct location is called, but the code is mis-interpreted due to the call which leeds to false assumptions on what instructions come next.

Better try a far jump with some simple statements and jump back, or copy over some proper but simple sub-routine and see if that works. Or just try read some values first.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7851
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: How to extend trek.exe code sections

Post by thunderchero »

I did some editing to my XY patch to place code outside of that sub-routine that it was within to confirm it is a sub-routine.

here is image of my new sub for X Y displayed in ida
sub_4B6879.jpg
sub_4B6879.jpg (275.24 KiB) Viewed 3339 times
This makes me think my issue is due to section alignment? But no idea how to fix issue. Or if there is a different issue.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7851
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: How to extend trek.exe code sections

Post by thunderchero »

Well I have attempted many different ways to get it to work with no luck.

I did notice when looking at section info in ida it might be missing some setting?
sections.jpg
sections.jpg (196.88 KiB) Viewed 3327 times
section 5 does not list;
OS type : MS Windows
Application type: 32bit

Ideal
p686
pmmx
model flat

also unknown data is dd while unknown data in section 1 is marked db

this makes me think section 5 is still not setup properly to hold code.
Post Reply

Return to “General Modding Information/Questions”