Increasing the trek.exe Palette Limit project

[Flocke]

By the Falcon code, MPRFreeTexturePalette is called with the resource context (not texture_list_entry_24h_36dec, but a unique handle usually used for multi-window rendering), and the MPR palette id.
MPRFreeTexturePalette(MPRHandle_t hRC, MPRHandle_t ID);

And in Texture.cpp LoadTexturePalette you indeed find 2nd value is loaded with the chroma key:

Code: Select all

			GlobalPaletteFlag[i].paletteID = j;
			GlobalPaletteFlag[i].chromakey = palentry[0];
			GlobalPaletteFlag[i].flag = 1;

With:

Code: Select all

struct GLTexturePalette {
	GLint	flag;
	GLint	chromakey;
	GLint	paletteID;
};

[Flocke]

I've had another look at mpr565.dll and updated SCT's IDA file with all that is posted:

fl_mpr565_2025-06-21.zip

Would be nice if thunderchero you can upgrade the file.

From the locations already reported, there seem to be only a few relevant ones, that is:

palette allocation:

Code: Select all

1001860C                 cmp     esi, 80h      // 0x17A0E -> for malloc and data stack (by thunderchero)

palette initialization:

In sub_10018C70 all the 128 palette entries of 418 bytes each are getting zero initialized by rep stosd.
That command repeats to write the 4 byte eax value 8300h times, so 418h/4 = 106h * 80h entries = 8300h.

palette release:

loop_free_max_0x80_palettes:
100187E5 cmp esi, 20C00h ; 0x418 * 0x80 default

I still believe we could simply relocate the whole palette data structure, increase limits and be fine. It should be way less data values to change.
Nonetheless it's too much effort for me right now and I have way too much on my TODO list.

Edit: minor IDA typo update, plus:
For relocation I'd probably try to relocate whole start_main_data_palette_list.
That should skip updates on relative offset computions and therefore avoid relocation table mess.
Furthermore, there are only a few onfollowing data offset references that possibly need update.

[thunderchero]

Would be nice if thunderchero you can upgrade the file.

I presume you wanted ida i64 file with comments, done

fl_mpr565_2025-06-21_i64.zip

note; ida is looking for pdb file would this be part of source?

C:\botf\ECM\mpr565.dll: failed to load pdb info.

Do you want to browse for the pdb file on disk?

palette initialization:
In sub_10018C70 all the 128 palette entries of 418 bytes each are getting zero initialized by rep stosd.
That command repeats to write the 4 byte eax value 8300h times, so 418h/4 = 106h * 80h entries = 8300h.

00 83 00 00 -> 00 06 01 00 (double) = ctd on game start. further investigation will be needed lol

edit;
error given on start up

Thread 484 is named 'MSS Timer'
0: The instruction at 0x0 referenced memory at 0x0. The memory could not be executed -> 00000000 (exc.code c0000005, tid 14964)

[Flocke]

I presume you wanted ida i64 file with comments, done
fl_mpr565_2025-06-21_i64.zip

Yes, thanks, works great but I just realized I was still running IDA Free 8.2 but needed 9.0 or above.
Starting with version 9 they however require a subscription, which even for the free version seems to be one year time limited.
I'm not too happy about this new subscription model. You can't even change your temporary email address after subscription.
Wonder what SCT has to say about it. So long I possibly best keep using old IDA Pro Free 5.0.

note; ida is looking for pdb file would this be part of source?

.pdb files are debug files generated on compilation.
They are great for tracing bugs, but never have been released, because they are both large and give deep insights to the source code.

00 83 00 00 -> 00 06 01 00 (double) = ctd on game start. further investigation will be needed lol

This can only work, when mpr565.dll is extended with some new section or extending the .data section and the whole start_main_data_palette_list is relocated along with updating all the data pointers. Otherwise it overwrites other data areas in use.
Adding or extending data sections however is not too complicate. I showed how this can be achieved before.

edit:
BTW, start_main_data_palette_list at asm_100841E0 should be labelled the "resource_context". It is initialized by open_device and gets 424F0h bytes allocated per resource context. BOTF however only has one window and therefore I guess only uses one such resource context. Wonder how much unused space we have for optional resource contexts.

loc_10016219:                  cmp     ebp, 8

suggests that there are max 8 resource contexts supported, so much of empty space I guess

edit2:
Ok, forget about the relocation, there is a perfect match for 8 resource contexts ending at asm_10296960 = 100841E0h + 8 * 424F0h resource contexts.
Let's try limit the resource contexts to a single one but increase size by 8 and just move the palette table to not conflict with all those hard-coded 20C00 data pointers SCT warned about.

[thunderchero]

sound like best way would be to edit and compile source code if possible.

[thunderchero]

I can't find the comment to quote

But you can load the mpr565.dll in debug, this is how I am doing it



but you must be in debug to open modules, and warns location may change next debug load. so no way I know of to load comments

[Flocke]

I can't find the comment to quote

But you can load the mpr565.dll in debug, this is how I am doing it

You probably refer to the missing .pdb debug symbols:

note; ida is looking for pdb file would this be part of source?

.pdb files are debug files generated on compilation.
They are great for tracing bugs, but never have been released, because they are both large and give deep insights to the source code.

I might have explained better. .pdb debug symbols map actual source code locations by their function names and function parameter types and names etc. If we had them, we probably could make much more sense of the assembler code. IDA might extract function names or if we had the code, even allow to debug while watching the source code files. That is what these .pdb files have been made for. And of course like with every closed source application they were not released.

[thunderchero]

I might have explained better.

no you explained it good enough, I just thought I remembered you asking to have mpr565 loaded when trek.exe was running debug.

but I must have mis-understood

[Flocke]

no you explained it good enough, I just thought I remembered you asking to have mpr565 loaded when trek.exe was running debug.
but I must have mis-understood

I can remember I once asked for how to load them both in IDA, but that is long ago and I think you already showed how to do this.
Anyway, debugging DLLs never was a problem to me.

But back to the palettes issue, I have some file for you to try:

mpr565-xpal.zip

To my huge astonishment, it ran first try without me having to debug any crashes!
How could this not have crashed first try?

But in lack of time I didn't do much testing yet. I only checked that it loads and both the research screen and combat doesn't crash.
I neither did try adding more ships nor did I check memory usage and not even the screensaver.
But even if I missed some stuff, I call it a huge progress and great success!

I even attached IDA to confirm it actually loaded that library, because hell it usually takes many tries even for much simpler modding.
How could this damn thing not have crashed even the first try?

Enjoy and please report back!

Here is an updated mpr565 IDA database by the way with lots more code annotations:

fl_mpr565_2025-06-23.zip

[Spocks-cuddly-tribble]

I'm not too happy about this new subscription model (...) Wonder what SCT has to say about it.

Give Hex-Rays 60 days to reconsider the error of their ways.
Then leak something about strange centrifuge stuff in their server rooms and wait for (Mc)Donald T's doordash delivery.

sound like best way would be to edit and compile source code if possible.

You lost me there. We agreed there is no reasonable chance to provide the BotF source code. I understood from Flocke/QD that BotF uses heavy modified versions of Falcon/MPR, so even their original source codes might not help much?

[thunderchero]

But back to the palettes issue, I have some file for you to try:
mpr565-xpal.zip

To my huge astonishment, it ran first try without me having to debug any crashes!
How could this not have crashed first try?

so is this an edited or created dll? I am guessing created.

but tested with 255 palettes with no issues in tech, combat or screensaver either.
loaded multi races and started many games so far.
all billboards functioning properly. (torpedoes, damage, sun)
tested 100+ fed ships vs 36 heavy card ships, no issues (other card ships ran away)
funny thing was combat felt quicker.

but what really surprised me was new dll allowed 512 x 512 still. Falcon dll's have always limited to 256 x 256.

any chance of 1024 x 1024? or should I check this version (Edit; still limited to 512 x 512)

any limits I should know about?

You lost me there.

Flocke understood
edit; no Flocke went code crazy.

Been testing so much, and just happen to test 267 palettes and it worked. so I went back and read readme.txt

the palette limit now is increased from 128 to 1814!

Edit2;
You know this just gives me hope of fixing combat stack issue.

File: stack.c, Line: 69, stack->bottom != NULL

[Flocke]

any chance of 1024 x 1024? or should I check this version (Edit; still limited to 512 x 512)
...
You know this just gives me hope of fixing combat stack issue.

You know, asm coding is not in my wheel house!
Might have a look another day.

[thunderchero]

any chance of 1024 x 1024? or should I check this version (Edit; still limited to 512 x 512)
...
You know this just gives me hope of fixing combat stack issue.

You know, asm coding is not in my wheel house!
Might have a look another day.

as for 1024 x 1024 the more I thought about it, it would be counter productive.
larger textures would need a larger color palette to keep best colors for the texture.

[Flocke]

as for 1024 x 1024 the more I thought about it, it would be counter productive.
larger textures would need a larger color palette to keep best colors for the texture.

No, detail is more important than the color count. It would still be great when there were larger ones suported.


Regarding the texture image formats..

nope can`t be done, also tried useing .bmp and .tga instaed of gif but that failed as well

I converted all the gif's to bmp's changed texture.lst gif -> bmp
added bmp's and texture.lst and Game did not crash

loc_538D4E list GIF, BMP, APL and TGA as valid texture types to load from stbof.res.

If possible, wouldn't it be the best solution to just disable this limiting feature? I.e. allow any color to be rendered from any texture.

I would not care if it was limited to gif -> bmp

I wouldn't be surprised when by some other code location in trek.exe, the expected image format is very much hardcoded, and forcefully expects to load GIF palettes. But did anyone actually try to alter the MPR render calls?


Relevant to the rendering are the MPRNewTexture and the MPRLoadTexture calls:

0053904E         push    eax    -> pixel height
00539055         push    eax    -> pixel width
00539056         push    8      -> 8bit
00539058         push    0E0h   -> MPR_TI_PALETTE flag
0053905D         push    edi    -> the render context
0053905E         call    MPRNewTexture

005390C5         push    ebx    -> texture buffer
005390C9         push    ebp    -> chroma key
005390CD         push    ecx    -> texture ID
005390D1         push    eax    -> texture width bytes = width * 1 byte for 8bit textures
005390D8         push    eax    -> height
005390DF         push    eax    -> width
005390E0         push    0      -> mipmaps?
005390E2         push    8      -> 8bit
005390E4         push    edi    -> the render context
005390E5         call    MPRLoadTexture

for the palettes there is also:

00538FA9         push    100h   -> 256 color entries
00538FAE         push    20h    -> 32bit
00538FB0         push    esi    -> the render context
00538FB1         call    MPRNewTexturePalette

00538FC3         push    ebp    -> palette buffer
00538FC4         push    100h   -> 256 color entries
00538FC9         push    0      -> start index
00538FCB         push    20h    -> 32bit
00538FCD         push    80h    -> MPR_TI_PALETTE
00538FD2         push    edi    -> palette ID
00538FD3         push    esi    -> the render context
00538FD4         call    MPRLoadTexturePalette

refer mpr.h

Code: Select all

UInt __stdcall 
MPRNewTexture   (MPRHandle_t hRC, 
                 UInt16      info,
                 UInt16      bits,
                 UInt16      width, 
                 UInt16      height);
                 
UInt __stdcall 
MPRLoadTexture  (MPRHandle_t hRC, 
                 UInt16      bitsPixel, 
                 UInt16      mip, 
                 UInt16      width, 
                 UInt16      rows, 
                 Int16       TexWidthBytes, 
                 UInt        ID, 
                 UInt        chroma,
                 UInt8*      TexBuffer);

UInt __stdcall 
MPRNewTexturePalette (MPRHandle_t hRC, 
                      UInt16      PalBitsPerEntry,
                      UInt16      PalNumEntries);

UInt __stdcall 
MPRLoadTexturePalette (MPRHandle_t hRC,
                       UInt        ID,
                       UInt16      info,
                       UInt16      PalBitsPerEntry,
                       UInt16      index,
                       UInt16      entries,
                       UInt8*      PalBuffer);

width and height by the way are read from the global texture list:

00538920         mov     eax, ds:GlobalTextureList_48h_72_dec_entries
00538925         shl     esi, 3
00538928         add     eax, esi
00538934         lea     ecx, [eax+2Ch]
005389C8         mov     edx, ecx
005389CA         call    texture_cpp_MPRNewTexture

[thunderchero]

lets look at options lol

GIF, BMP, APL and TGA

APL -> no longer used, can't even find software to save/convert as .APL
GIF -> we know pro's and con's
BMP -> from what I understand no transparent color so billboards would not work
TGA -> question here is, would code read 0 x 0 pixel color as transparent? and would it render/map texture properly?