Extending the shiplist

[Gowron]

Did anybody actually test this with more than just a handful ships? I'm asking because the monster list error still showed up after changing all of the values mentioned here

The first problem is that the single-byte Borg Cube IDs are signed.
This won't affect the game if one adds 12 or less ships. But after adding the 13th ship, the Borg Cube ID is 0x80, and this translates into -128 when the game does a monster list check, resulting in a crash.

To solve this, I've split the subtraction into two subtractions. Instead of subtracting the whole Borg Cube ID at once (which would not work, since it's too large for a signed byte), I'd first subtract the old ID and then subtract the difference.

This uses up additional space, which can be provided by discarding the jump condition. As an example, here's the old and new code for the ID at 0x49DA5:

old code (positions 0x49DA3 - 0x49DAF):

Code: Select all

83 E8 73           // sub eax, 0x73
66 3D 09 00        // cmp ax, 0x0009
0F 86 F5 00 00 00  // jbe ...

new code (positions 0x49DA3 - 0x49DAF):

Code: Select all

83 E8 73           // sub eax, 0x73
83 E8 XX           // sub eax, 0xXX
90 90              // nop nop
E9 F5 00 00 00     // jump ...

(where XX is the difference between the new Borg Cube ID and the old one)


Note that the new jump statement must end at the same position as the old one, because the adresses are not absolute, but relative.

Also note that the ID at 0x4E878 is different because there's a "ja" (jump if above) instead of the usual "jbe" (jump if below or equal). Instead of discarding the jump condition, just remove the entire jump statement



The second problem is that the game will ignore ships with an ID of 125 or more, except for monsters. The ships will just be treated as unavailable and not appear in the F4 menu. Adding 20 ships would leave all minor races without any ships. I'm currently working on this issue. At least this error does not crash the game, so it's easier to examine

[DCER]

Also note that the ID at 0x4E878 is different because there's a "ja" (jump if above) instead of the usual "jbe" (jump if below or equal). Instead of discarding the jump condition, just remove the entire jump statement

Won't this break the switch statement?

[Gowron]

The following solution seems to solve the second issue, at least for the major empires. There's a loop where ship after ship is checked, and if the ID is less than 125, then the loop is entered again (with the ID increased by 1). Else the game stops checking ships. To support more that 128 ships, the "jl" (jump if lower) command, which (due to its structure) can only work for a maximum of 128 IDs, has to be replaced by a "jne" (jump if not equal) command, which can work for up to 255 IDs.

old code (0x6EC66-0x6EC6A):

Code: Select all

83 FA 7D     // cmp edx, 0x7D
7C A2        // jl ...

new code (0x6EC66-0x6EC6A):

Code: Select all

80 FA XX     // cmp dl, 0xXX
75 A2        // jne ...

with "XX" being the new shiplist size.

Instead of edx, I only use dl. I'll explain it with an example. Say the new shiplist size is 0x90. If you compare this to edx, then the 0x90 is treated as 0xFFFFFF90, because the size of edx is 4 bytes. And 0xFFFFFF90 will never match the current ship ID. But if you compare 0x90 to dl, it's still treated as 0x90, because the size of dl is only 1 byte.

This works well so far, but only for the major empires.


Remaining Problem: The minor race ships are still missing (i.e. they're not showing up on the map). I guess there's an extra loop for them somewhere.

Also note that the ID at 0x4E878 is different because there's a "ja" (jump if above) instead of the usual "jbe" (jump if below or equal). Instead of discarding the jump condition, just remove the entire jump statement

Won't this break the switch statement?

Yep, but the "jump if above" would only be done if the game encountered an invalid monster ID. So as long as the shiplist is all right, this won't change anything. It just removes a redundant statement (that would only lead to a crash anyway).

[DCER]

Found the second check thanks to IDA Pro, and already tested it, works great. Gowron if you're not using that tool yet, you should, the graph view will make your work a lot easier.

The check is at 0x6EC08, but it's jge (jump if greater or equal), I turned that into je (jump if equal) and fixed the cmp instruction, like you showed and minors have ships now.

Gowron, you've solved another one

[Gowron]

The check is at 0x6EC08, but it's jge (jump if greater or equal), I turned that into je (jump if equal) and fixed the cmp instruction, like you showed and minors have ships now.

Thank you very much
Hm, I had expected a loop, but it just seems to be some sort of safeguard check. I've overwritten it with 90's, and it worked great when I tested it. All minor race ships were there, and I fought a battle against the sheliak without any problems

Found the second check thanks to IDA Pro, and already tested it, works great. Gowron if you're not using that tool yet, you should, the graph view will make your work a lot easier.

I already tried various disasssemblers in the past, including IDA. IIRC I discarded it when it refused to install under a power user account, demanding an admin account instead (wtf...).

But I gave it another try today. It turned out to be quite confusing. Could you please tell me how I can display the corresponding opcodes in line with the assembler code?

I'm afraid IDA seems to be a bit unclear, and sometimes very annoying. At least it was able to disassemble a few small code segments that the disassembler I'm working with could not decode completely. So I think I'll keep IDA and use it just for these segments

Gowron, you've solved another one

Let's test and see

[DCER]

I already tried various disasssemblers in the past, including IDA. IIRC I discarded it when it refused to install under a power user account, demanding an admin account instead (wtf...).

But I gave it another try today. It turned out to be quite confusing. Could you please tell me how I can display the corresponding opcodes in line with the assembler code?

I'm afraid IDA seems to be a bit unclear, and sometimes very annoying. At least it was able to disassemble a few small code segments that the disassembler I'm working with could not decode completely. So I think I'll keep IDA and use it just for these segments

There should be an IDA view tab showing the assembly code and a hex view tab showing the byte data for that code. It's annoying that in IDA sometimes those can get out of sync slightly. I haven't yet discovered how if at all it can display the opcodes and arguments separately.

The really useful thing is the graph view in the IDA view, you can switch to it by right clicking and then selecting Graph view, the opposite of Text view. It shows the flow of execution for the code.

Another useful feature is the Jump to xref to operand -> It'll display and take you to functions calling the function or operand you have highlighted.

It's confusing to work with I agree, but it has it's uses

[Gowron]

@Gowron


I've made a 254 shiplist-file (max) modified as you suggested and
File: ..\..\source\game\monster.c, Line: 143, Not a monster I've heard of

Code: Select all

83 E8 73           // sub eax, 0x73 (changed from F5 [Jshipedit modify this value])
83 E8 XX           // sub eax, 0xXX (set to 0x82 [0xF5-0x73])
90 90              // nop nop 
E9 F5 00 00 00     // jump ...

The 0x82 is the problem here. The "sub" command would interpret it as "-126" (dec). The value must not exceed 0x7F. You could, however, increase the other one, so this should work:

Code: Select all

83 E8 76     // sub eax, 0x76
83 E8 7F     // sub eax, 0x7F

But I've never tested a shiplist that was that large. There might be other problems as well.

0x4E878 == 0xF5

Again, this will be treated as a negative number, because it's used with the "sub" command.

and what shall i remove????

There's a "ja" (jump if above, i.e. jump if invalid) statement which should be removed:

Code: Select all

asm_offset  opcodes               asm_code
-----------------------------------------------------
:0044F467 8B4206                  mov eax, dword[edx+06]
:0044F46A C1F810                  sar eax, 10
:0044F46D E81EB2FFFF              call 0044A690
:0044F472 668B4052                mov ax, word[eax+52]
:0044F476 83E873                  sub eax, 073   // change this
:0044F479 663D0900                cmp ax, 0009   // remove this
:0044F47D 0F87D3000000            ja 0044F556    // remove this
:0044F483 25FFFF0000              and eax, 0000FFFF
:0044F488 FF2485F0F34400          jmp dword[4*eax+0044F3F0]

stored the new shipfile @
http://www.4shared.com/file/49703002/a1 ... pedit.html

I'm sorry, but I cannot find a shiplist, shiptech or shipdesc file in the .zip archive located there.

[Spocks-cuddly-tribble]

Ok I was wrong, 0x4E698 gets called even when no cubes spawn. And it seems to be reading the empsInfo file. I'm not sure what this does anymore. Runs through successfully and no cubes appear.

A really enigmatic value, it seems. I can't pin it down either.

You were right DCER

I was able to run some tests using a savegame where a cube appears two turns later. I manipulated the cube building cost, maintenance cost and sub_44F264. This is a last check in oder to block the cube under certain circumstances.
As you said sub_44F264 is used when a cube spawns for the first time, but not when copies are made after system assimilation. ->set eax to 1 means cube OK -> else block cube.
In conclusion: 0x4E698 is just used to read ship building cost (per ID) for the block cube check.
Every other ship ID won't cause errors, but reads building cost from wrong ship for calculation, see below.


Involved variables & subroutines:

5A2924: labled in trek.exe as some kind of victory variable?

5B186B: BORG_OFF_ON

5A2B44: difficulty_level

sub_46F9C0 -> read_ship (from shiplist for specific ID)

5791B8: Borg_multiplier (8 byte) -> default 1.25

58FDE4: Borg_timer

5A2918: Turn_Number

58FDE0: number_of_cubes



I think this is going on there:

Check 5A2924: skips the borg on/off test if not zero

Difficulty case switch 1-5 (5=impossible)
cmp number_of_cubes -> 1 / 2 / 4 / 8 / no check -> (jg ->exit no cube)

else

compare empire fleet cost from EmpsInfo with cube buildcost * Borg_muliplier
-> if cube buildcost too large ->(loop test for all empires) -> exit no cube

else

test Borg_timer (first cube) -> if yes -> cube OK (and set borg_timer = turn_number)

else

if Turn_Number larger than Borg_Timer+(difficulty dependant): 400 / 300 / 200 / 100 / 50 (dec)
->cube OK (and set borg_timer = turn_number)

else ->exit no cube

Code: Select all

number_of_cubes checks:
44F324      cmp    ds: 58FDE0, 1
44F337      cmp    ds: 58FDE0, 2
44F34A      cmp    ds: 58FDE0, 4
44F35D      cmp    ds: 58FDE0, 8

delay values for Borg_timer:
44F32D      mov    ebx, 190h
44F340      mov    ebx, 12Ch
44F353      mov    ebx, 0C8h
44F366      mov    ebx, 64h
44F370      mov    ebx, 32h

NOTE: Savegame issues with Borg data: viewtopic.php?p=51581#p51581

Cube limits after system assimilation: viewtopic.php?p=27201#p27201

[DCER]

Good work, Spocks-cuddly-tribble!

Interesting stuff. Why did they have to make it this complicated?

[thunderchero]

The following solution seems to solve the second issue, at least for the major empires. There's a loop where ship after ship is checked, and if the ID is less than 125, then the loop is entered again (with the ID increased by 1). Else the game stops checking ships. To support more that 128 ships, the "jl" (jump if lower) command, which (due to its structure) can only work for a maximum of 128 IDs, has to be replaced by a "jne" (jump if not equal) command, which can work for up to 255 IDs.

Is 255 still max ship IDs? I am getting close and still not done

thunderchero

[Spocks-cuddly-tribble]

In connection with my random events project, I had another look at monsters:

Borg Cube IDs (all jnz)

40E099 cmp word [eax+52h], 73h // 0xD49D
42B333 cmp word [eax+52h], 73h // 0x2A737 (AI empire reactions to Borg presence)
4359E3 cmp word [eax+52h], 73h // 0x34DE7

We can use instead: cmp word [eax+28h], 24h // (works with any shiplist)

At 0xD49C, 0x2A736 and 0x34DE6 change: 52 73 to: 28 24.

Explanation:

Ds: 5B2344 [GShipList]
Offset 0x0028 to 0x0029 = race-ID (even for monster unique, but not in shiplist.sst order !)
Offset 0x0052 to 0x0053 = shiplist.sst-ID





On this occasion, I also had another look at extending the shiplist phase II.


Word instead of single-byte registers - not very complicated (excepting possible issues with ship-ID "FF") - but all in all ~200 code sequences would have to be rewritten:

sub_46F940 (read_shiptech_sst) -> 15 calls
sub_46F9C0 (read_shiplist_sst) -> 155 calls

So even with a very optimistic assumption of 5 minutes per code location...

Thereby the following deliberations become more or less moot.




Monster switches:

0x4E878 -> Monster system attack (borg, crystal & tarellian)
44F472 mov ax, [eax+28h]
44F476 sub eax, 24h

0x4F357 -> monster race-IDs (+cube count) via ship-IDs
44FF51 mov ax, [edx+28h]
44FF55 sub eax, 24h

NOTE jump tables for switch statement (order of addresses) have to be addapted at 44F3F0 & 44FED0 each:

(old order via shiplist.sst-ID)
0 - Borg Cube (73)
1 - Calamarain (74)
2 - Chodak (75)
3 - Crystal Entity (76)
4 - Combat Drone (77)
5 - Edo Guardian (78 )
6 - Gomtuu (79)
7 - Husnock (7A)
8 - Orbital Battery (7B)
9 - Tarellian (7C)

(new order via monster race-ID)
0 - Borg Cube 24
1 - Crystal Entity 25
2 - Chodak 26
3 - Calamarain 27
4 - Gomtuu 28
5 - Tarellian 29
6 - Edo Guardian 2A
7 - Combat Drone 2B
8 - Husnock 2C
9 - (redundant)



0x49DA5 (loc_44A99F) -> monster torpedoes
0x49FC5 (loc_44ABBF) -> monster phaser

Source of ship-ID is shiplist.sst for both: -> mov ax, [ecx+66h] -> sub eax, 0x73
This should work instead: mov ax, [ecx+118h](or mov ax, bx) -> sub eax, 0x24
and set controlling race in shiplist.sst for monsters accordingly i.e. 0x24 +0-9.
So monsters would be just like all the other ships - see also: Weapon labels of ships/ Beams & Torpedoes

Here it's faster to keep the old order (e.g. 25=Calamarain, not Crystal Entity).




Remaining monster-IDs in trek.exe (still need to be adapted for each resizing of shiplist)


0x4E698 (loc_44F295) -> Cube ID (block cube check)


Sub_44FAA4 (add_new_Monster) - ship-IDs (&name) via monster race-IDs

0x4EEDB - Borg Cube ID (73)
0x4F0DB - Calamarain ID (74)
0x4F05A - Chodak ID (75)
0x4F01B - Crystal Entity ID (76)
0x4F1F7 - Combat Drone ID (77)
0x4F1B2 - Edo Guardian ID (78 )
0x4F121 - Gomtuu ID (79)
0x4F232 - Husnock ID (7A)
0x4F173 - Tarellian ID (7C)




Number of ships:

0x6EC0A (sub_46F7D0) -> redundant
replace at 0x6EC08 -> 83 F8 7D 7D 5E with 90 90 90 90 90

0x6EC68 (loc_46F85D) -> also redundant (at least in my tests, due to shiprace check)
replace at 0x6EC66 -> 83 FA 7D 7C with 90 90 90 EB

0x6EF89 (loc_46FB6A) -> redundant
replace at 0x6EF87 -> 66 3D 7D 00 74 with 90 90 90 90 EB

0x6F02F (loc_46FC0A) -> redundant
replace at 0x6F02D -> 66 3D 7D 00 74 with 90 90 90 90 EB


EDIT:

47983C cmp edi, 7Dh -> number of ships (unused) - See combat.bin.

[EnPhreg]

can somebody clarify what this check is for?? i mean the number_of_cubes check. what it checks? the number of cubes already in the game?

58FDE0: number_of_cubes

I think this is going on there:

Check 5A2924: skips the borg on/off test if not zero

Difficulty case switch 1-5 (5=impossible)
cmp number_of_cubes -> 1 / 2 / 4 / 8 / no check -> (jg ->exit no cube)

[thunderchero]

Difficulty case switch 1-5 (5=impossible)
cmp number_of_cubes -> 1 / 2 / 4 / 8 / no check -> (jg ->exit no cube)

the way I read it would be
difficulty simple = 1 borg max
difficulty easy = 2 borg max
difficulty normal = 4 borg max
difficulty hard = 8 borg max
difficulty impossible = no check borg max

[EnPhreg]

but there exists already a check for the max borg cubes, i.e. if X borg cube in the game -> no new cube will spawn.

Code: Select all

No new cube if number of cubes already in game:
44F660                 mov     ecx, 2
44F79F                 mov     ecx, 3
44F7B2                 mov     ecx, 4
44F7C5                 mov     ecx, 8
44F7DB                 mov     ecx, 10h

so maybe number_of_cubes mean simultaniously spawning cubes? is it possible that more cube than 1 spawns at the same time???

[thunderchero]

it might be difference between new borg via timer and spawned borg from assimilation of a system?