Extending the shiplist

Extending the shiplist; support/discussion/questions

Moderator: thunderchero

User avatar
EnPhreg
Lieutenant-Commander
Lieutenant-Commander
Posts: 130
Joined: Thu Jul 10, 2008 2:00 am

Re: Extending the shiplist

Post by EnPhreg »

so i've tested it. the values for number_of_cubes are a limitation value for possible cubes on map like expected.
and also the difficulty check for number of cubes in game are a limitation.
and for this difficulty check there is also a value for level 5 (impossible).
check found in trek.exe at: 0x4EA3B (default 10). tested it with vanilla (if value set to 1, no new cube spawns if a already exisiting cube assimilates a world, cause the limit is 1 and 1 cube is already in the game, tested it several times)

so i think SCT's assumption for no check on difficulty 5 wasn't right so far for this check.
Difficulty case switch 1-5 (5=impossible)
cmp number_of_cubes -> 1 / 2 / 4 / 8 / no check -> (jg ->exit no cube)
so, there are two checks for the possible number of borg cubes in the game.
number_of_cube check and the difficulty check.
User avatar
Spocks-cuddly-tribble
Code Master
Code Master
Posts: 1883
Joined: Sun Apr 27, 2008 2:00 am

Re: Extending the shiplist

Post by Spocks-cuddly-tribble »

outdated/removed
Last edited by Spocks-cuddly-tribble on Sun Mar 24, 2024 4:02 pm, edited 1 time in total.
I don't know how many bugs is too many but that point is reached somewhere before however many in BotF is.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7848
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Extending the shiplist

Post by thunderchero »

After reading though this topic, I wanted to ask does UE Extending the shiplist code need to be updated?

these 2 posts by sct has id's not edited by UE when Extending the shiplist.
viewtopic.php?f=194&t=115&p=19062#p19062
viewtopic.php?p=50815#p50815

also I don't think the code locations UE edits for Extending the shiplist have been documented.

after Extending the shiplist by 1 ship.
here is the changes UE made

Code: Select all

after Extending the shiplist by 1
0x00049da5 74 83 e8 00 90 90 e9
0x00049fc5 74 83 e8 00 90 90 e9
0x0004e878 74 83 e8 00 90 90 90 90 90 90 90
0x0004eedb 74
0x0004f01b 77
0x0004f05a 76
0x0004f0db 75
0x0004f121 7a
0x0004f173 7d
0x0004f1b2 79
0x0004f1f7 78
0x0004f232 7b
0x0004f357 74 83 e8 00 90 eb
0x0006ec08 80
0x0006ec0a 7e 74
0x0006ec66 80
0x0006ec68 7e 75
0x0006ef89 7e
0x0006f02f 7e
0x000f31cc 90 90 90 90 90 90
0x000f31d5 23
0x000f323b 2b
0x000f3241 1c
0x000f3247 1c
0x000f324d 1c

vanilla code
0x00049da5 73 66 3d 09 00 0f 86
0x00049fc5 73 66 3d 09 00 0f 86
0x0004e878 73 66 3d 09 00 0f 87 d3 00 00 00
0x0004eedb 73
0x0004f01b 76
0x0004f05a 75
0x0004f0db 74
0x0004f121 79
0x0004f173 7c
0x0004f1b2 78
0x0004f1f7 77
0x0004f232 7a
0x0004f357 73 66 3d 09 00 76
0x0006ec08 83
0x0006ec0a 7d 7d
0x0006ec66 83
0x0006ec68 7d 7c
0x0006ef89 7d
0x0006f02f 7d
0x000f31cc 80 7e 4c 01 75 19
0x000f31d5 1a
0x000f323b 1b
0x000f3241 22
0x000f3247 29
0x000f324d 2a
User avatar
Spocks-cuddly-tribble
Code Master
Code Master
Posts: 1883
Joined: Sun Apr 27, 2008 2:00 am

Re: Extending the shiplist

Post by Spocks-cuddly-tribble »

thunderchero wrote: Tue Jan 11, 2022 3:47 pmdoes UE Extending the shiplist code need to be updated?
You should include: At 0xD49C, 0x2A736 and 0x34DE6 change: 52 73 to: 28 24.

0x4E698 (loc_44F295) -> Cube ID (block cube check)

The combat.bin value should be left alone.
Last edited by Spocks-cuddly-tribble on Sun Mar 24, 2024 4:01 pm, edited 1 time in total.
I don't know how many bugs is too many but that point is reached somewhere before however many in BotF is.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7848
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Extending the shiplist

Post by thunderchero »

what about this?

UE edit these values but viewing with ida it show 0 switches for jump table?

Code: Select all

Monster switches:

0x4E878 -> Monster system attack (borg, crystal & tarellian)
44F472 mov ax, [eax+28h]
44F476 sub eax, 24h

0x4F357 -> monster race-IDs (+cube count) via ship-IDs
44FF51 mov ax, [edx+28h]
44FF55 sub eax, 24h

NOTE jump tables for switch statement (order of addresses) have to be addapted at 44F3F0 & 44FED0 each:

(old order via shiplist.sst-ID)
0 - Borg Cube (73)
1 - Calamarain (74)
2 - Chodak (75)
3 - Crystal Entity (76)
4 - Combat Drone (77)
5 - Edo Guardian (78 )
6 - Gomtuu (79)
7 - Husnock (7A)
8 - Orbital Battery (7B)
9 - Tarellian (7C)

(new order via monster race-ID)
0 - Borg Cube 24
1 - Crystal Entity 25
2 - Chodak 26
3 - Calamarain 27
4 - Gomtuu 28
5 - Tarellian 29
6 - Edo Guardian 2A
7 - Combat Drone 2B
8 - Husnock 2C
9 - (redundant)
User avatar
Spocks-cuddly-tribble
Code Master
Code Master
Posts: 1883
Joined: Sun Apr 27, 2008 2:00 am

Re: Extending the shiplist

Post by Spocks-cuddly-tribble »

It's an alternative approach to make these code locations independent from ship IDs i.e. shiplist size. Current UE implementation also works.

Look at the jump tables (double-click or scroll on top of subroutines) they work just fine (unless UE corrupts code).

Also you might delete the confusing/wrong spam about number of cube checks (your ignored reply is correct).
I don't know how many bugs is too many but that point is reached somewhere before however many in BotF is.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7848
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Extending the shiplist

Post by thunderchero »

Spocks-cuddly-tribble wrote: Tue Jan 11, 2022 9:36 pm Look at the jump tables (double-click or scroll on top of subroutines) they work just fine (unless UE corrupts code).
when you load edited trek with ida it says;

Code: Select all

AUTO:0044F483 25 FF FF 00 00                      and     eax, 0FFFFh     ; switch 0 cases
AUTO:0044F488 FF 24 85 F0 F3 44 00                jmp     off_44F3F0[eax*4] ; switch jump
AUTO:0044F488                                     sub_44F42C endp
AUTO:0044F488
then the rest of the code that was in sub_44F42C is in a new sub named "sub_44F48F"
User avatar
Spocks-cuddly-tribble
Code Master
Code Master
Posts: 1883
Joined: Sun Apr 27, 2008 2:00 am

Re: Extending the shiplist

Post by Spocks-cuddly-tribble »

No idea why the IDA analysis fails in this case.

E.g. OllyDbg analysis failed for any (working) exploit code conflicting with the old relocation table.
I don't know how many bugs is too many but that point is reached somewhere before however many in BotF is.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7848
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Extending the shiplist

Post by thunderchero »

The reason I am asking, I have always suspected random/borg error (either corrupting save or ctd) and noticed it more on mods with extended shiplist.

the last error/ctd I looked at debugger stopped in sub 44F42C, tethys narrowed it down to error with doomsday (cube).

since UE should be updated with new locations should alternative approach (race id) be used instead in some locations?
and is there any other location that UE does not edit currently?
User avatar
Spocks-cuddly-tribble
Code Master
Code Master
Posts: 1883
Joined: Sun Apr 27, 2008 2:00 am

Re: Extending the shiplist

Post by Spocks-cuddly-tribble »

thunderchero wrote: Tue Jan 11, 2022 11:59 pmThe reason I am asking, I have always suspected random/borg error (either corrupting save or ctd) and noticed it more on mods with extended shiplist.
I don't recall this issue in vanilla or ECM (except tactical combat interface glitches).

thunderchero wrote: Tue Jan 11, 2022 11:59 pmthe last error/ctd I looked at debugger stopped in sub 44F42C, tethys narrowed it down to error with doomsday (cube).
Sounds like a monster system attack modification bug, some very old suggestions at the start of this topic are untested (and/or are maybe conflicting with newer patches below): viewtopic.php?p=18268#p18268
thunderchero wrote: Tue Jan 11, 2022 11:59 pmsince UE should be updated with new locations should alternative approach (race id) be used instead in some locations?
This depends on personal preference (more work for a confusing one time change vs. a simple change each time), as I said it shouldn't be necessary.
Last edited by Spocks-cuddly-tribble on Sun Mar 24, 2024 4:00 pm, edited 2 times in total.
I don't know how many bugs is too many but that point is reached somewhere before however many in BotF is.
User avatar
Tethys
Past Administrator
Past Administrator
Posts: 2392
Joined: Fri Jul 18, 2008 2:00 am
Location: Your mom's bed ;)
Contact:

Re: Extending the shiplist

Post by Tethys »

Spocks-cuddly-tribble wrote: Wed Jan 12, 2022 7:56 am
thunderchero wrote: Tue Jan 11, 2022 11:59 pmthe last error/ctd I looked at debugger stopped in sub 44F42C, tethys narrowed it down to error with doomsday (cube).
Sounds like a monster system attack modification bug, some very old suggestions at the start of this topic are untested (and/or are maybe conflicting with newer patches below): viewtopic.php?p=18268#p18268
This is what I suspected also, but TC and I decided instead to look into why the particular subroutine appeared broken in our mods. I have a free day tomorrow, I will check out the supernova effect for Doomsday and see if this is the issue (of which we seem to be in agreement).

Edit: I found the solution to my immediate problem. It remains to be seen whether this persist in other mods.
Not for the weak of heart...
Galaxies MOD v0.4.0 <--- GALM/Galaxies Mod latest version
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3196
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: Extending the shiplist

Post by Flocke »

Spocks-cuddly-tribble wrote: Tue Jan 11, 2022 8:19 pm
thunderchero wrote: Tue Jan 11, 2022 3:47 pmdoes UE Extending the shiplist code need to be updated?
You should include: At 0xD49C, 0x2A736 and 0x34DE6 change: 52 73 to: 28 24.
0x4E698 (loc_44F295) -> Cube ID (block cube check)
Added, including another integrity check for when the BORG cube ID got changed.
Spocks-cuddly-tribble wrote: Fri Apr 02, 2010 7:08 am and set controlling race in shiplist.sst for monsters accordingly i.e. 0x24 +0-9.
This however is not checked and not supported by UE yet, and it is not set in shiplist.sst for any of the mods.
The race ID of the ship types is always 0x24 = BORG. Therefore I fear the event checks will also work for other monsters now.
And by my expectation, changing it will break the monster race agent.

Edit:
Spocks-cuddly-tribble wrote: Fri Apr 02, 2010 7:08 am We can use instead: cmp word [eax+28h], 24h // (works with any shiplist) :)
:arrow: At 0xD49C, 0x2A736 and 0x34DE6 change: 52 73 to: 28 24.

Explanation:
Ds: 5B2344 [GShipList]
Offset 0x0028 to 0x0029 = race-ID (even for monster unique, but not in shiplist.sst order !)
Offset 0x0052 to 0x0053 = shiplist.sst-ID
This makes more sense. I checked my sample game7.sav from posting.php?mode=quote&p=54002
It has one Chodak and that one indeed has race id 0x26 = 38 and not 36 = BORG.
So patch should be fine.

Spocks-cuddly-tribble wrote: Fri Apr 02, 2010 7:08 am Number of ships:

0x6EC0A (sub_46F7D0) -> redundant
replace at 0x6EC08 -> 83 F8 7D 7D 5E with 90 90 90 90 90

0x6EC68 (loc_46F85D) -> also redundant (at least in my tests, due to shiprace check)
replace at 0x6EC66 -> 83 FA 7D 7C with 90 90 90 EB
0x6EC0A indeed is just a redundant safeguard check, 0x6EC68 however makes sure to not read beyond the ship list end.
When you read beyond the data end, by luck of course the data happens to fail one of the other checks, but it's undefined behaviour.

Although it is just one of many locations that need to be fixed, I overworked the previous patch to now set and compare some integer value limit.

Code: Select all

NAME: UE ship build loop patch
DESC: replaces signed byte by integer comparison
AUTHOR: Flocke
URL: https://www.armadafleetcommand.com/onscreen/botf/viewtopic.php?p=61650#p61650

>> 0x0006EC04 89 44 24 10 83 F8 7D 7D 5E 31 C0 89 E2 8A 44 24 10
>> 0x0006EC5D 8B 54 24 10 42 89 54 24 10 83 FA 7D 7C A2

<< 0x0006EC04 90 90 3D 7D 00 00 00 7D 5E 90 90 89 E2 89 44 24 10
<< 0x0006EC5D 8B 44 24 10 40 3D 7D 00 00 00 7C A6 90 90
First check jumps to 2nd check, which reuses the "mov [esp+10h], eax" instruction from first check.
To activate without exceeding the limit, both segments need to be manually patched in segments view.

Spocks-cuddly-tribble wrote: Fri Apr 02, 2010 7:08 am 0x6EF89 (loc_46FB6A) -> redundant
replace at 0x6EF87 -> 66 3D 7D 00 74 with 90 90 90 90 EB

0x6F02F (loc_46FC0A) -> redundant
replace at 0x6F02D -> 66 3D 7D 00 74 with 90 90 90 90 EB
These are two safeguard checks for the ship count in shiptech.sst and shiplist.sst.
You may call them redundant, but they provide proper error notification for when trek.exe is not properly patched to match stbof.res modded ship counts.
They are both already set by UE, plus they are short type values, so not limiting the moddable ship number.

Spocks-cuddly-tribble wrote: Tue Jan 11, 2022 8:19 pm I'm not sure about the purpose of sub_48B470.
Spocks-cuddly-tribble wrote: Wed Jan 12, 2022 7:56 am Maybe including values in sub_48B470 helps with this? There is a max ship ID +1 check, so max number of ships should be FEh.
...
thunderchero wrote: Tue Jan 11, 2022 11:59 pmand is there any other location that UE does not edit currently?
I can't tell for sure, as I said sub_48B470 is unclear.
Cancel that, for all references to sub_48B470 (Monster__ship_race_IDs), you find that first the lexicon term 320h is looked up.
That string reads "%s{{, %s} and %s}", and guess what:
0x7B = '{'
0x7D = '}'
0x25 = '%'
0x73 = 's'
No BORG (0x73), no Crystal Entity (0x25), no orbital battery (0x7B) and no max ship ID limit (0x7D).
It is a plain text parsing routine! :lol:

In my lexicon lookup you can find that it's both used in combat screen and event summary:
viewtopic.php?p=55644#p55644
Last edited by Flocke on Sun Mar 24, 2024 9:37 am, edited 1 time in total.
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3196
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: Extending the shiplist

Post by Flocke »

Flocke wrote: Thu Mar 14, 2024 12:26 am 4. BOP trek.exe reports fixed ship numbers - only check if segments are loaded:

Code: Select all

Error at segment 0x0006EC08: Number of registered ships is 148, but should be 168. (fixed)
Error at segment 0x0006EC66: Number of registered ships is 148, but should be 168. (fixed)
When you check stbof.res, there indeed are 168 ships. Missing are the minor race ships and monsters.
By forum search I found Gowron nopped the code. Adding back the limit check however shouldn't cause any issues.
viewtopic.php?p=5261#p5261
For 0x6EC66 he missed to update the limit.
While I investigated and fixed missing Borg ID updates, I found that Gowron actually set that limit with intent.
It exactly matches the number of the BOP major empire ships. And in BOP you can't build any minor ships anyway.
Gowron wrote: Wed Sep 10, 2008 1:51 pm This works well so far, but only for the major empires.
Remaining Problem: The minor race ships are still missing (i.e. they're not showing up on the map). I guess there's an extra loop for them somewhere.
Gowron wrote: Thu Sep 11, 2008 9:44 am
DCER wrote:The check is at 0x6EC08, but it's jge (jump if greater or equal), I turned that into je (jump if equal) and fixed the cmp instruction, like you showed and minors have ships now.
Thank you very much :)
Hm, I had expected a loop, but it just seems to be some sort of safeguard check. I've overwritten it with 90's, and it worked great when I tested it. All minor race ships were there, and I fought a battle against the sheliak without any problems :)
I wonder how this might have fixed the minor races, when the other limit still limits to the major empire ships.
But I guess it simply resolved build screen crash and minors were not affected.

Anyway, there's no reason to limit the loop on major empire ships. When at some point the unsigned limit is overcome, like posted above, UE now has a proper fix for these both cmp checks in place. :wink:
User avatar
Spocks-cuddly-tribble
Code Master
Code Master
Posts: 1883
Joined: Sun Apr 27, 2008 2:00 am

Re: Extending the shiplist

Post by Spocks-cuddly-tribble »

Flocke wrote: Sun Mar 24, 2024 7:35 am
Spocks-cuddly-tribble wrote: Fri Apr 02, 2010 7:08 amand set controlling race in shiplist.sst for monsters accordingly i.e. 0x24 +0-9.
This however is not checked and not supported by UE yet, and it is not set in shiplist.sst for any of the mods.
The race ID of the ship types is always 0x24 = BORG. Therefore I fear the event checks will also work for other monsters now.
Yes, this overcomplicated approach is not recommended.
And the patch compatibility with other patches changing the jump tables would be a nightmare.... :mad:
Flocke wrote: Sun Mar 24, 2024 7:35 amAnd by my expectation, changing it will break the monster race agent.
No, it won't. By default, the controlling race in shiplist.sst has no effect for monsters (only player UI and major/minor ship weapon labels).
But doesn't matter, see above. :wink:

Flocke wrote: Sun Mar 24, 2024 7:35 am0x6EC68 however makes sure to not read beyond the ship list end.
When you read beyond the data end, by luck of course the data happens to fail one of the other checks, but it's undefined behaviour.
Indeed, good thinking. There is two cases where it might fail:

- removing all minor/monster ships from shiplist (have the monsters shiptech.sst entries?)
- using "building minor ships after membership" patch

BUT, easiest way to make it work would be just one extra single byte edit: :wink:
0046F81C  jz   short loc_46F85D (inc ship ID check next or exit) -> loc_46F86B EXIT

Flocke wrote: Sun Mar 24, 2024 7:35 amCancel that, for all references to sub_48B470 (...) It is a plain text parsing routine! :lol:
Thank you, I wasn't able the pin that code down. So I remove it from my posts and IDA database. :up:
I don't know how many bugs is too many but that point is reached somewhere before however many in BotF is.
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3196
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: Extending the shiplist

Post by Flocke »

Spocks-cuddly-tribble wrote: Sun Mar 24, 2024 3:50 pm Indeed, good thinking. There is two cases where it might fail:

- removing all minor/monster ships from shiplist (have the monsters shiptech.sst entries?)
- using "building minor ships after membership" patch

BUT, easiest way to make it work would be just one extra single byte edit: :wink:
0046F81C  jz   short loc_46F85D (inc ship ID check next or exit) -> loc_46F86B EXIT
Nice trick to depend on
46F970         mov     ax, ds:shipNum_Max_ID
As a programmer I however learned that true while loops are bad coding style. :razz:
I'm happy with the current patch I implemented. :cool:
Post Reply

Return to “Extending the shiplist”