New installer Beta available 11-20-23

[Flocke]

example; first thing to select would be resolution/display mode (window/full) this would be used for all mods installed during that install run.

Looks good, but what if mod doesn't support selected resolution?
You might default to the next highest resolution ofc, but when 1920 x 1080 got selected, 1366 x 768 might not be supported by the monitor.

And still I'm missing the lazy select all mods option.

[Flocke]

To make sure I didn't miss any obvious issues, I reinstalled all mods with latest installer and run UE integrity checks.
Here is what I found:

1. In AAM and AAML stbof.res, UE reports:

Code: Select all

Warning: in cbboom.hob: Unknown data detected that is exceeding the expected TotalSize: 11472 bytes
Warning: in cgboom.hob: Unknown data detected that is exceeding the expected TotalSize: 6256 bytes
Warning: in csboom.hob: Unknown data detected that is exceeding the expected TotalSize: 5336 bytes
Warning: in fbboom.hob: Unknown data detected that is exceeding the expected TotalSize: 11472 bytes
Warning: in fgboom.hob: Unknown data detected that is exceeding the expected TotalSize: 6256 bytes
Warning: in fsboom.hob: Unknown data detected that is exceeding the expected TotalSize: 5336 bytes
Warning: in hbboom.hob: Unknown data detected that is exceeding the expected TotalSize: 11472 bytes
Warning: in hgboom.hob: Unknown data detected that is exceeding the expected TotalSize: 6256 bytes
Warning: in hsboom.hob: Unknown data detected that is exceeding the expected TotalSize: 5336 bytes

Checking the hex code, I found that the whole data got duplicated.
The first four bytes is a dword value, that tells on file size.
At least for fbboom.hob data is identical. The others I didn't check.

2. In AAM, BOP, Vanilla and stbof.res of most other mods, there is some SwitchList overlapping detected:

Code: Select all

Warning: in hs_a.hob: SwitchList overlapping detected: 4 bytes.
Warning: in hs_b.hob: SwitchList overlapping detected: 4 bytes.
Warning: in hs_c.hob: SwitchList overlapping detected: 4 bytes.
Warning: in ks_a.hob: SwitchList overlapping detected: 4 bytes.
Warning: in ks_b.hob: SwitchList overlapping detected: 4 bytes.
Warning: in ks_c.hob: SwitchList overlapping detected: 4 bytes.

It might not have much relevance to BotF. To BotF that very same value however is read for two different uses.
It likely got messed by hex-editing the texture count.
see viewtopic.php?p=61387#p61387

3. BOP trek.exe reports a few minor race end building issues that are fixed:

Code: Select all

Error at segment 0x0018DDAC: Minor race end building id [96] doesn't match detected main building end [91]. (fixed)
Error at segment 0x0018DDAC: Minor race end building id [106] doesn't match detected main building end [98]. (fixed)
...

This is about trek.exe segment 0x0018DDAC, which was missed by BOP.
See viewtopic.php?p=57327#p57327

As I found, BOP did not change any of the trek.exe MinorRaceBldUpgradeIDs starting at 0x0018EDE8 ( asm_590FE8, viewtopic.php?f=264&t=2299 ), they are still set to vanilla values.
When UE saves any Edifice changes, it overwrites these ids and fills any missing values repeating the last upgrade ID.

the minor race EL table 0x0018EDE8 (asm_590FE8) and the building limits at 0x0018DDAC (asm_58FFAC)?

As told: 0x0018EDE8 (asm_590FE8) -> shared main building code (not minor)
building limits at 0x0018DDAC (asm_58FFAC) -> minor fake upgrade -> main buildings min-max IDs -> yes looks like a BoP bug (not analysed at the time)

MinorRaceBldUpgradeIDs got relabeled to MinorRaceMainBldIDLimits by the way.

Not sure on what actual effect it has, but to implement a special BOP exception here, is no good option.
So far none of the code checks are actually mod dependent. There is not even a mod detection implemented with UE.

Given that for type 5 buildings, the upgrades got disabled, this change should have no effect or at best fix some faulty minor race evolution upgrades.
Therefore, unless any issues are reported, best simply patch BOP to resolve this issue.

4. BOP trek.exe reports fixed ship numbers - only check if segments are loaded:

Code: Select all

Error at segment 0x0006EC08: Number of registered ships is 148, but should be 168. (fixed)
Error at segment 0x0006EC66: Number of registered ships is 148, but should be 168. (fixed)

When you check stbof.res, there indeed are 168 ships. Missing are the minor race ships and monsters.
By forum search I found Gowron nopped the code. Adding back the limit check however shouldn't cause any issues.
viewtopic.php?p=5261#p5261
For 0x6EC66 he missed to update the limit.

Number of ships:

0x6EC0A (sub_46F7D0) -> redundant
replace at 0x6EC08 -> 83 F8 7D 7D 5E with 90 90 90 90 90

0x6EC68 (loc_46F85D) -> also redundant (at least in my tests, due to shiprace check)
replace at 0x6EC66 -> 83 FA 7D 7C with 90 90 90 EB

0x6EF89 (loc_46FB6A) -> redundant
replace at 0x6EF87 -> 66 3D 7D 00 74 with 90 90 90 90 EB

0x6F02F (loc_46FC0A) -> redundant
replace at 0x6F02D -> 66 3D 7D 00 74 with 90 90 90 90 EB


EDIT:

47983C cmp edi, 7Dh -> number of ships (unused) - See combat.bin.

Well, DCER opted to update instead of nopping the limits. If preferred I can change to nop the code or accept both.
Since updating the UE code also means additional work, I however see little reason not to add the updated limit back.
0x6EF87 and 0x6F02D (UE shipNum1&2) aren't nopped by Gowron either.

5. UCW trek.exe errors:

Code: Select all

Warning at segment 0x0018E880: House of Gowron ship map range 'Short' (2) at level 5 is lower than at level 4 (7).
Warning at segment 0x0018E880: House of Gowron ship map range 'Medium' (4) at level 5 is lower than at level 4 (8).
Warning at segment 0x0018E880: House of Gowron ship map range 'Long' (6) at level 5 is lower than at level 4 (9).
Warning at segment 0x0018E880: Romulan ship map range 'Short' (1) at level 9 is lower than at level 8 (10).
Warning at segment 0x0018E880: Romulan ship map range 'Medium' (2) at level 9 is lower than at level 8 (12).
Warning at segment 0x0018E880: Romulan ship map range 'Long' (3) at level 9 is lower than at level 8 (15).

This looks to be misaligned by some unsupported code extension. I'll have to take a closer look and likely update UE code to support trek.exe ship range settings for extended tech levels.

6. UCW stbof.res errors:

Code: Select all

Warning: in converse.bin: Duplicate reference to voice list 15160 detected by voice group 10.
Warning: in converse.bin: Duplicate reference to voice list 15128 detected by voice group 16.
...
Info: in converse.bin: The following voice lists aren't mapped by any voice group:
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, ...

Usually each voice entry has it's own recordings. For UCW however the House of Gowron and Duras share same recordings. Therefore, when you check english.snd or german.snd, there are many unused entries. I might look into reducing the warnings and just tell the number of duplicates and missing entries. In addition I think it should be info only, no need to warn on integrity.

7. Duplicate ship model name trek.exe errors (MUM,...):

Code: Select all

Warning at segment 0x0018F7FC: Duplicate ship model name 'OPN' detected. Renaming to 'NBT'. (fixed)
Warning at segment 0x0018F7FC: Duplicate ship model name 'OPN' detected. Renaming to 'BMF'. (fixed)

Even vanialla trek.exe still has a duplicate HM1 ship model entry of same model name. Given that the ship models are identified by the slot name, the names should be unique. There is no reason to keep the bug.


-------------------------------------------------------------------

8. GALM 0.4.0 trek.exe issues:

Code: Select all

Validation error at segment 0x00074827 [ shipNameCode ]: ♦ Read unknown code modification

The 0x00074827 shipNameCode segment validation fails due to some unsupported changes I need to check.

Code: Select all

Error at segment 0x0018DDAC: Minor race start building id [217] doesn't match detected main building start [199]. (fixed)
Error at segment 0x0018DDAC: Minor race end building id [217] doesn't match detected main building end [199]. (fixed)

first data complex is classified as research group, see below

9. GALM 0.4.0 stbof.res issues:
to just list a few more important ones...

Code: Select all

Error: in edifice.bst: The industry [89-89] and food [97-98] buildings must have no intermediates but disabled non-buildable structures.
Error: in edifice.bst: The industry [134-134] and food [143-144] buildings must have no intermediates but disabled non-buildable structures.

The slots in between are reused for energy structures. Trek.exe however doesn't support this, because sub_486550 increments the ecx register, to loop through both the industry and food building ids.

Main industry and food buildings of a race also need to be grouped together (industry first, food second).

Code: Select all

Error: in edifice.bst: For empire 2 there is another main research building [217] found.

The previous one is first data complex, erronously set to main research group.

Code: Select all

Error: in edifice.bst: The technology map must be increased to support the max assigned tech level 15.
It currently is limited to 11 technology levels at maximum.

Edifice has a tech level mapping, used for the tech database and possibly AI upgrades?
It must be increased to support additional tech levels. Can be fixed with UE building stats tools.

Code: Select all

Error: in edifice.bst: The Upgrade id 334 of building 290 exceeds the building number 334. (unset)

The Interspatial Power Matrix is set to upgrade building 334, which doesn't exist.

Code: Select all

Exception:
java.io.IOException: No main builings found for filter group main intel building

I'll need to add some check to disable intel stucture updates. Not sure what problems occure ingame when there are no intelligence buildings.

Not as many issues I expected.

[thunderchero]

To make sure I didn't miss any obvious issues, I reinstalled all mods with latest installer and run UE integrity checks.
Here is what I found:

thanks so much, next time I will "load all" before integrity check

I just wanted to give quick response to issues

1 & 2. hobs, files will be fixed
3. will get patch file to correct "BOP minor race end building issues"
4. will get patch file to correct "BOP trek.exe reports fixed ship numbers"
5. UCW trek.exe errors, IIRC this due to shared map range
viewtopic.php?p=24888#p24888
6. UCW stbof.res errors: as long as UE does not try to fix "non issue" ....
7. Duplicate ship model name trek.exe errors (MUM,...): OPN = open
8 & 9. GALM 0.4.0 trek.exe issues: I have some files already I sent to tethys but have not heard from him

[thunderchero]

when looking at BOP trek changes I ran into this conflict (not sure what integrity check is flagging code)

at 0017eaec

800 and 1024 GUI
0017eae0h: 63 00 00 00 25 73 2E 74 67 61 00 00 25 64 00 00 ; c...%s.tga..%d..

1366 and 1920 GUI
0017eae0h: 63 00 00 00 25 73 5F 62 2E 74 67 61 00 25 64 00 ; c...%s_b.tga.%d.

the offset for %d has been changed at
#change offset for %d
0x000d1ef2 ec -> ed

edit; also this code change is crashing when entering ship build screen
>> 0x0006ec08 90 90 90 90 90
>> 0x0006ec66 80
>> 0x0006ec68 94 75

<< 0x0006ec08 83 f8 a8 7d 5e
<< 0x0006ec66 83
<< 0x0006ec68 a8 7c

[Flocke]

thanks so much, next time I will "load all" before integrity check

I guess I should make it the default, to load all segments and only skip textures.
With added options in the settings panel.

1 & 2. hobs, files will be fixed
3. will get patch file to correct "BOP minor race end building issues"
4. will get patch file to correct "BOP trek.exe reports fixed ship numbers"

great!

5. UCW trek.exe errors, IIRC this due to shared map range
viewtopic.php?p=24888#p24888

correct, thanks for the link

6. UCW stbof.res errors: as long as UE does not try to fix "non issue" ....

it's been a little many messages and already fixed

7. Duplicate ship model name trek.exe errors (MUM,...): OPN = open

I guessed so, but it is still a valid graphics name and I rather don't want to patch the UE gfx name generators for it.
By some of the many ship packs it might actually be in use.
I could write some exception to e.g. detect "___", "\0\0\0", "---" or "???" slots as empty and update slots that become unused.

8 & 9. GALM 0.4.0 trek.exe issues: I have some files already I sent to tethys but have not heard from him

I always find myself back fixing new UE issues for GALM compatibility
Still there are a few issues that need to be fixed on the mod itself.

9. GALM 0.4.0 stbof.res issues:

Code: Select all

Error: in edifice.bst: For empire 2 there is another main research building [217] found.

The previous one is first data complex, erronously set to main research group.

Code: Select all

Exception:
java.io.IOException: No main builings found for filter group main intel building

I'll need to add some check to disable intel stucture updates. Not sure what problems occure ingame when there are no intelligence buildings.

These actually are the same issue. I tested it ingame and for trek.exe it is still intelligence buildings.
The error is correct and must be fixed on the mod.

For UE I will however fix, that other changes still can be applied. Else the popup which asks to proceed doesn't make much sense.

Code: Select all

Validation error at segment 0x00074827 [ shipNameCode ]: ♦ Read unknown code modification

The 0x00074827 shipNameCode segment validation fails due to some unsupported changes I need to check.

This actually is caused by closely following instructions of SCT for adapting the non-appendix groups:
viewtopic.php?p=30019#p30019
I'll be fixing UE to properly detect and allow set those values.

the offset for %d has been changed at
#change offset for %d
0x000d1ef2 ec -> ed

I will take a look.

[thunderchero]

I guessed so, but it is still a valid graphics name and I rather don't want to patch the UE gfx name generators for it.
By some of the many ship packs it might actually be in use.
I could write some exception to e.g. detect "___", "\0\0\0", "---" or "???" slots as empty and update slots that become unused.

no need I will adjust any mod that still has duplicate

[Flocke]

when looking at BOP trek changes I ran into this conflict (not sure what integrity check is flagging code)

at 0017eaec

800 and 1024 GUI
0017eae0h: 63 00 00 00 25 73 2E 74 67 61 00 00 25 64 00 00 ; c...%s.tga..%d..

1366 and 1920 GUI
0017eae0h: 63 00 00 00 25 73 5F 62 2E 74 67 61 00 25 64 00 ; c...%s_b.tga.%d.

Thanks, I already enlarged the read string length to 12 bytes, but missed that %d gets sneakingly erased.
I'll fix to write back full data string this case and limit editing by %d.
Actually I might do so by default for all the string values, and add some settings option to do the cleanup.

edit; also this code change is crashing when entering ship build screen
>> 0x0006ec08 90 90 90 90 90
>> 0x0006ec66 80
>> 0x0006ec68 94 75

<< 0x0006ec08 83 f8 a8 7d 5e
<< 0x0006ec66 83
<< 0x0006ec68 a8 7c

I see problem. jge is a signed comparison. For unsigned comparison it must be replaced with jnb/jae.
<< 0x0006ec08 83 f8 a8 73 5e

I'll fix code and nop if 256 is exceeded.

I could write some exception to e.g. detect "___", "\0\0\0", "---" or "???" slots as empty and update slots that become unused.

no need I will adjust any mod that still has duplicate

That wouldn't be much of a problem. I'd just hate to write some integrity check exclusion for some identifier that might actually be used.
Samples above by UE aren't considered valid - at least not for the name generator.

I remember you asked for an integrity check that fixes duplicates.

[thunderchero]

I remember you asked for an integrity check that fixes duplicates.

your memory is better than mine

just a quick check and I have not found a single trek.exe that dose not need a patch of some kind (179 in total)

even ECM needs "minMinorsNumber" update?

[Spocks-cuddly-tribble]

nop if 256 is exceeded.

Without hundreds of, partly major, code changes trek.exe cannot support ship, station, or monster IDs larger than byte, so UE should not allow this and issue a warning (IIRC thunderchero was shocked UE supported this, when analysing shiplist mess-ups by noobs).

Also I feel a bit alienated since ECM has no new error logs, so I want you guys to screw someting up for me, right now (discreetly)...

EDIT: That was fast:

even ECM needs "minMinorsNumber" update?

Can you show code in question? I ommited two minor race code deviation fixes, since they turned out useless (related to max number +1 issue i.e. trek.exe table for 30 needs value 31). They were referenced in MUM dev and UE topics.

[thunderchero]

Can you show code in question?

Code: Select all

NAME: UE integrity check
DESC: min minors
AUTHOR: 
URL: 

>> 0x000b1734 94
>> 0x000b1738 a2
>> 0x000b173c a9
>> 0x000b1744 b0
>> 0x000b1748 b7
>> 0x000b174c be
>> 0x000b17a3 09
>> 0x000b17aa 0f
>> 0x000b17b1 06
>> 0x000b17b8 0c
>> 0x000b17bf 18
>> 0x000b17c5 00 00
>> 0x000b17ca 00 00


<< 0x000b1734 a2
<< 0x000b1738 a9
<< 0x000b173c b0
<< 0x000b1744 b7
<< 0x000b1748 be
<< 0x000b174c c5
<< 0x000b17a3 03
<< 0x000b17aa 09
<< 0x000b17b1 0f
<< 0x000b17b8 06
<< 0x000b17bf 0c
<< 0x000b17c5 b8 18
<< 0x000b17ca 59 c3

[Spocks-cuddly-tribble]

Thanks, I see, sub_4B2350 Minors_Set_min change shared code switch to custom values.
There is many unfavorable shared code switches in the galaxy generation codes, UE has auto-fix options for custom values.
ECM base version with vanilla maps does not need this (since pure vanilla ) , but might be part of ECM larger maps patch, I don't recall right now.

[thunderchero]

I see problem. jge is a signed comparison. For unsigned comparison it must be replaced with jnb/jae.
<< 0x0006ec08 83 f8 a8 73 5e

even using 73 still crashes in ship build screen.

also this crash has a strange crash log

File: scrllbar.c, Line: 288, minValue < maxValue

I am wondering if unknown code is also effecting changed code?

[Studmaster]

Anything I can do after seeing all this for my mod in development to at least fix some of these issues?

[thunderchero]

Anything I can do after seeing all this for my mod in development to at least fix some of these issues?

you can do some testing of development version of UE

main page with compile instructions
https://gitlab.com/stbotf/ultimate-editor

current test version "improve_shipgfx_export"
https://gitlab.com/stbotf/ultimate-editor/-/branches

[Studmaster]

Anything I can do after seeing all this for my mod in development to at least fix some of these issues?

you can do some testing of development version of UE

main page with compile instructions
https://gitlab.com/stbotf/ultimate-editor

current test version "improve_shipgfx_export"
https://gitlab.com/stbotf/ultimate-editor/-/branches

What kind of testing in particular?