Branch/Commits testing of latest UE code

Ultimate Editor; support/discussion/questions

Moderator: thunderchero

User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7940
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Branch/Commits testing of latest UE code

Post by thunderchero »

Branch tested on improve_shipgfx_export

Commits; "SMHelper: fix ship preview image name"

all basic tests went very well testing on vanilla, still need to test on update Commits and test on other mods

testing of adding tech levels did not go as well,
tested on vanilla 1366 x 768 windowed

1. added pre-edited techtree.tec and techdesc.tec (20 tech levels from MUM)
tech.zip
(15.46 KiB) Downloaded 9 times
2. in edit/technology/tecnology fields "trek.exe tech levels (max 42)" change 11 -> 21
3. saved with UE
crash on game load

issue I found is missing code changes, and does not like not changing this also;
structures.jpg
structures.jpg (186.16 KiB) Viewed 499 times
when tech levels are increased I think this should be sneaked in at max tech level. (first option) to prevent crashes

after I allowed it to changes to max tech level. I checked my MUM "20 tech level patch" and "EdificePatcher changes patch"
EdificePatcher changes patch matched 100%
but mum 20 tech level patch did not match, here is omitted changes my patch is showing
note; have not had time to investigate each code location (some may be tech ship range?)
not_changed.jpg
not_changed.jpg (70.61 KiB) Viewed 499 times
important note; If I install my 20 tech level patch at this point, game will crash. this change is cause of crash at 0x00530a3
I will update this post if I find time to track down locations in image

-- = current code (but not compared to vanilla)
<< = changes my patch would make

EDIT;
here is some of the notes I have

# EdificePatcher code changes (you might write code differently? single byte vs 4 byte?)
>> 0x0004f543 44
>> 0x0004f546 59 5b c3 00 00 00

<< 0x0004f543 84
<< 0x0004f546 00 00 00 59 5b c3

# this is code change that causes crash if my patch is added
talked about here; viewtopic.php?p=36816&hilit=0x530A3#p36816
>> 0x530A3 4D
<< 0x530A3 7F

#Tech limit for the tech screen display: (not for the number of displayed techs!)
>> 0x7B5D0 0A
<< 0x7B5D0 14

>> 0x7B5D8 0A
<< 0x7B5D8 14

#tech screen display
>> 0x7B603 01 C3
<< 0x7B603 90 90

#Tech limit for bonus ranges effective
>> 0x7B771 0A
<< 0x7B771 14

>> 0x7B776 0A
<< 0x7B776 14

#bonus ranges effective
>> 0x7B79B 01 C1
<< 0x7B79B 90 90
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3256
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: Branch/Commits testing of latest UE code

Post by Flocke »

thunderchero wrote: Sat Mar 30, 2024 8:46 pm Commits; "SMHelper: fix ship preview image name"
When you reference commits, best name the commit SHA number:
6481f35d87f520664621bdf890bb9ce900c904b2
Or write the complete perma link:
https://gitlab.com/stbotf/ultimate-edit ... e900c904b2
But yes, there is a commit message search I never used before. :mrgreen:

thunderchero wrote: Sat Mar 30, 2024 8:46 pm issue I found is missing code changes, and does not like not changing this also;
when tech levels are increased I think this should be sneaked in at max tech level. (first option) to prevent crashes
The message is a bit confusing, agreed. When I implemented it, UE had no trek.exe patch for the tech levels yet.
All that is patched in the building screen is stbof.res files.
Which is what it warns about.

The popup has 3 options:
  • first is to fix the edifice tech tree table to the max available tech level from technology fields view
  • second is to not ask again but automatically increase or decrease based on used and available tech levels
    e.g. when you switch to tech fields view and add another few tech levels and then increase building tech levels again
  • third is to keep the vanilla edifice.est tech field limits, which becomes visible in tech database where higher technology entries are truncated
Given that by the time there was no trek.exe patch implemented, and once I implemented it, it was still experimental and hidden in technology fields view, I wanted to leave option to keep entries truncated so trek.exe doesn't need to be patched.
As far I recall, by the time there were multiple mods that didn't properly update trek.exe or had custom code changes.

But when it is fixed and approved, I could sneak in trek.exe changes and update the popup text.

thunderchero wrote: Sat Mar 30, 2024 8:46 pm after I allowed it to changes to max tech level. I checked my MUM "20 tech level patch" and "EdificePatcher changes patch"
EdificePatcher changes patch matched 100%
And I hoped so much that it works! Now I remember why it's still been in experimental features. :lol:
Guess I never found time to actually test changes myself.

thunderchero wrote: Sat Mar 30, 2024 8:46 pm # EdificePatcher code changes (you might write code differently? single byte vs 4 byte?)
Another small display glitch of UE I just fixed.
broken-tech-total.jpg
broken-tech-total.jpg (29.2 KiB) Viewed 454 times
It was edited correct, just the default value comparison was broken.

thunderchero wrote: Sat Mar 30, 2024 8:46 pm >> 0x0004f543 44
>> 0x0004f546 59 5b c3 00 00 00
<< 0x0004f543 84
<< 0x0004f546 00 00 00 59 5b c3
That's the UE BldTechReq2 patch. Like stated in UE segment description, unless the signed byte limit is exceeded (25 tech levels * 4 + 24 = 0x7C) only the tech level value is updated. If signed byte limit is exceeded, the full patch is applied.
( The description actually stated 28 tech levels was maximum. This was a discription typo glitch. The code actually auto-applies the patch when the computed signed byte value '0x7F' is exceeded. )

thunderchero wrote: Sat Mar 30, 2024 8:46 pm # this is code change that causes crash if my patch is added
talked about here; viewtopic.php?p=36816&hilit=0x530A3#p36816
>> 0x530A3 4D
<< 0x530A3 7F
This is covered by UE TechTotal patch. (11 * 7 = 77 = 0x4D)
When unsigned byte limit (0xFF) is exceeded, the list is truncated to 0xFF tech level entries.

From what I found in IDA database, this initializes a tech database hash table and allocates memory for each of the tech database entries.
It is called by the startup routine and probably loads entries from stbof.res.

Increasing the list might result in a crash, but I think the error is somewhere else.

thunderchero wrote: Sat Mar 30, 2024 8:46 pm #tech screen display
>> 0x7B603 01 C3
<< 0x7B603 90 90

#bonus ranges effective
>> 0x7B79B 01 C1
<< 0x7B79B 90 90
These both are not supported by UE yet. They force to share the ship range bonus tables for the five major empires.
As long as limits are in place, it should not be relevant.

thunderchero wrote: Sat Mar 30, 2024 8:46 pm #Tech limit for the tech screen display: (not for the number of displayed techs!)
>> 0x7B5D0 0A
<< 0x7B5D0 14

>> 0x7B5D8 0A
<< 0x7B5D8 14

#Tech limit for bonus ranges effective
>> 0x7B771 0A
<< 0x7B771 14

>> 0x7B776 0A
<< 0x7B776 14
That's all ship tech bonus range limits for the UE MapRangeTechBonuses table at 0x18E880. (IDA: tech_bonus_Range)
As far I recall, when limits are not updated, later tech research has no effect on bonus ranges.
If increased however the table must be shared and patched like above.

This patch however should be independent from the tech level patch.
It can be found in UE trek.exe "Ship Map Ranges" view.
Adding shared bonus range patch support already is on my list. :smile:


Back to the crash, I tried changes and got following crash log:

Code: Select all

-------------------------------------
ST:BOF Mon Apr  1 08:22:31 2024

Version Under Test: 72
File: ..\..\source\game\setup\civstart.c, Line: 223, farm != NULL
Initialize State: 31
Player Empire: 1	Starting Seed: 1711956151
Galaxy shape: 2	Galaxy Size: 1
The windows event log further reports:

Code: Select all

Name der fehlerhaften Anwendung: trek.exe, Version: 0.0.0.0, Zeitstempel: 0x37965563
Name des fehlerhaften Moduls: trek.exe, Version: 0.0.0.0, Zeitstempel: 0x37965563
Ausnahmecode: 0xc0000005
Fehleroffset: 0x00024527
ID des fehlerhaften Prozesses: 0x0x3268
Startzeit der fehlerhaften Anwendung: 0x0x1DA83FCEA390D61
Pfad der fehlerhaften Anwendung: C:\Spiele\BotF\Vanilla\trek.exe
Pfad des fehlerhaften Moduls: C:\Spiele\BotF\Vanilla\trek.exe
I will look into it later the day. But it seems that there is some misaligned tech database read that fails to find a farm entry.
Might also be that buildings are missing from edifice.est. :roll:

edit:
a different crash I encountered when only adding another computer and sociology level and updating trek.exe but not edifice:

Code: Select all

Ausnahmecode: 0xc0000005
Fehleroffset: 0x00050320
Last edited by Flocke on Mon Apr 01, 2024 8:47 am, edited 1 time in total.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7940
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Branch/Commits testing of latest UE code

Post by thunderchero »

Flocke wrote: Mon Apr 01, 2024 5:57 am When you reference commits, best name the commit SHA number:
6481f35d87f520664621bdf890bb9ce900c904b2
Or write the complete perma link:
https://gitlab.com/stbotf/ultimate-edit ... e900c904b2
:up:
Flocke wrote: Mon Apr 01, 2024 5:57 am The message is a bit confusing, agreed. When I implemented it, UE had no trek.exe patch for the tech levels yet.
popup was not confusing at all, just without some type of change game will crash
thunderchero wrote: Sat Mar 30, 2024 8:46 pm (some may be tech ship range?)
before I even looked at changes that was what I expected
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3256
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: Branch/Commits testing of latest UE code

Post by Flocke »

thunderchero wrote: Mon Apr 01, 2024 8:46 am popup was not confusing at all, just without some type of change game will crash
Confusing myself at least. :lol:

thunderchero wrote: Mon Apr 01, 2024 8:46 am
thunderchero wrote: Sat Mar 30, 2024 8:46 pm (some may be tech ship range?)
before I even looked at changes that was what I expected
Understood, but great that you posted it, so I don't have to search forum to fix that one, too.
( it's actually been next on my list after the system traits to fix UCW integrity check warnings )


I kept testing tech levels a bit.

Code: Select all

File: ..\..\source\game\setup\civstart.c, Line: 223, farm != NULL
Was reported before by exceL:
"Apparently, changing the primitive farm's tech level or production cost caused the error."
viewtopic.php?p=20542#p20542

I did not mess with the primitive farms, but some other energy food structures. Needs more testing to what is allowed and what must not be changed.
When key variables are found, I can patch those as well. Otherwise, for problematic buildings found, I at least can add a warning and some integrity check.

Code: Select all

Ausnahmecode: 0xc0000005
Fehleroffset: 0x00050320
This exactly is crash you reported before patching edifice.bst in the building screen.
This indeed must be auto-patched or warned as well. :up:

From testing I will also change that added tech levels equals for all the tech fields.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7940
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Branch/Commits testing of latest UE code

Post by thunderchero »

I did noticed the in ship stats the "Function" is no longer using lexicons
ultra.jpg
ultra.jpg (158.97 KiB) Viewed 429 times
here is older version using lexicons
old2.jpg
old2.jpg (161.09 KiB) Viewed 429 times
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3256
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: Branch/Commits testing of latest UE code

Post by Flocke »

thunderchero wrote: Mon Apr 01, 2024 2:34 pm I did noticed the in ship stats the "Function" is no longer using lexicons
That's not correct. It now matches the ingame lexicon values found in the ship database.
I updated it when during testing of the Lexicon mappings I found that UE messed values of mutliple categories.
See https://gitlab.com/stbotf/ultimate-edit ... c4d3f66ff1

For the monsters there is no lexicon entry. UE previously mapped it to "Alien" but BOTF rather uses the term "Monster". See monster.lst of see savegames or "monster.c" used for trek.exe exceptions.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7940
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Branch/Commits testing of latest UE code

Post by thunderchero »

I noticed the ship database right after I posted. either works just noticed change.

I still need to grab latest files to test, hope to do that soon.

could hero marker be added to ship stats? maybe under special ship check box?
without patch having set has no effect from my tests
User avatar
Spocks-cuddly-tribble
Code Master
Code Master
Posts: 1939
Joined: Sun Apr 27, 2008 2:00 am

Re: Branch/Commits testing of latest UE code

Post by Spocks-cuddly-tribble »

thunderchero wrote: Tue Apr 02, 2024 10:39 pmcould hero marker be added to ship stats? maybe under special ship check box?
Problem is the average Joe user might be overwhelmed by special settings for countless special patch features he don't knows or even cares about?
And since ID table has to be edited manually in that patch anyway, hex-editing the corresponding markers in shiplist shouldn't be a problem?
I don't know how many bugs is too many but that point is reached somewhere before however many in BotF is.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7940
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Branch/Commits testing of latest UE code

Post by thunderchero »

commit sha = 394142e0d4f3c5a5f619f01c47b3fce98ea0e4df

while doing some integrity checks I saw this one from Building mod 1.9.4
b194.jpg
b194.jpg (92.89 KiB) Viewed 362 times
not sure what can or should be done, Joker removed those monsters from ship list. :shock:
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7940
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Branch/Commits testing of latest UE code

Post by thunderchero »

commit sha = 394142e0d4f3c5a5f619f01c47b3fce98ea0e4df

when editing race name in race info and if the order of race in minor race attitude differs example; Mintakans, Ullians, Bolians at end of list.
it will create a duplicate entry in minor race attitude.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7940
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Branch/Commits testing of latest UE code

Post by thunderchero »

commit sha = 394142e0d4f3c5a5f619f01c47b3fce98ea0e4df

simple extract then add back of same vanilla model will give texture "X" of hob could not be loaded
happening only on vanilla ship model export?
new models exported no issues
ship pack made prior not seeing this issue.

User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7940
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Branch/Commits testing of latest UE code

Post by thunderchero »

commit sha = 394142e0d4f3c5a5f619f01c47b3fce98ea0e4df

when adding an ani I get this message
ani.jpg
ani.jpg (138.09 KiB) Viewed 328 times
I get this message on ani I just created or working ani just extracted from res.
it also looks like it adds file but no longer displays properly
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7940
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Branch/Commits testing of latest UE code

Post by thunderchero »

commit sha = 394142e0d4f3c5a5f619f01c47b3fce98ea0e4df

vanilla 1366 x 768
once alt res is opened,
1. when file is selected it takes 4 - 5 sec for preview to load.
2. if you click on multiple files one by one UE will not respond until previews catch up
3. when "load all" is selected it take almost 3 min to load them, but once files are loaded issues 1 and 2 are gone.

edit; I noticed on saved game game info screen displays strange text
text.jpg
text.jpg (317.69 KiB) Viewed 120 times
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3256
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: Branch/Commits testing of latest UE code

Post by Flocke »

thunderchero wrote: Fri Apr 05, 2024 1:29 pm not sure what can or should be done, Joker removed those monsters from ship list. :shock:
I could improve the integrity check to handle this, but when ingame it is broken anyhow, not sure whether to not rather patch them back to the mod. :roll:
thunderchero wrote: Sat Apr 06, 2024 1:51 pm when editing race name in race info and if the order of race in minor race attitude differs example; Mintakans, Ullians, Bolians at end of list.
it will create a duplicate entry in minor race attitude.
This has been a strange tripple-bugged bug. The aiminor code failed to lookup the minor race name, because already since ancient times DCER implemented to use a binary search algorithm that depends on the entries to be sorted. The Mintakans however come last and are not listed in alphabetical order. So already in UE 0.7.2dev5c it actually was bugged and added a new entry.

For a dirty fix, DCER implemented to use the integrity checks and remove entries that are missing from race.lst plus add back missing ones.
It's however still been noticable that not found minor races always were appended to the end of the list instead of keeping them in place.

The visible duplication however came by introducing the new DataString type, which is meant to reduce sneaking changes by writing back the full memory string data. With that change I missed that for aiminor always the first entry was copied by value, which means it became the very same data instance for both entries. And when the new name was set, it overwrote both values.

The dirty fixture then added back another acamarian entry to the end of the list. But it is optimized to first search for unused entries and so found the actual Mintakan entry one slot before the new end, that then was overwritten. 8O

It's fixed now, plus a similar issue for german major empire home systems, that aren't sorted alphabetically either.

thunderchero wrote: Sat Apr 06, 2024 10:29 pm I get this message on ani I just created or working ani just extracted from res.
it also looks like it adds file but no longer displays properly
Fixed. UE failed to properly load added animations, because the file was still treated like generic data.
If saved and reloaded it most likely even worked. But I removed the broken GenericFile inheritance and now it displays without error.

thunderchero wrote: Sat Apr 06, 2024 9:09 pm simple extract then add back of same vanilla model will give texture "X" of hob could not be loaded
happening only on vanilla ship model export?
new models exported no issues
ship pack made prior not seeing this issue.
Good finding! I found the cause but havn't fixed it yet.
When some existing model is replaced, all its graphics are first removed and in texture.lst they are marked with the 'X'.
Ahead of it, the import routine however builds a hash list of present textures, which is not updated.

When then it comes to importing the textures, the hash value is checked and it is found that the texture already is present so the import is skipped.
The very same bug already is present in UE 0.7.2dev5c, just noone noticed it before - or at least it never got fixed. :eek:
lost-texture-reimport.jpg
lost-texture-reimport.jpg (14.19 KiB) Viewed 53 times
thunderchero wrote: Sun Apr 07, 2024 10:28 am once alt res is opened,
1. when file is selected it takes 4 - 5 sec for preview to load.
2. if you click on multiple files one by one UE will not respond until previews catch up
3. when "load all" is selected it take almost 3 min to load them, but once files are loaded issues 1 and 2 are gone.
Yeah, I noticed the blocking load delay as well. It even became worse than with UE 0.7.2dev5c, but I think that is related to the improved preview scaling algortithm. The images are large and the used algorithm now is more quality but less performance optimized.

I might add some image size checks to decide on preview algorithm options.
In addition I'll look into making it a non-blocking background task that will be stopped whenever the selection changes.

thunderchero wrote: Sun Apr 07, 2024 10:28 am edit; I noticed on saved game game info screen displays strange text
That's a character encoding issue. For me it shows correct.
Might be caused by the file download or the used compiler or something.
Not on my priority list.
Last edited by Flocke on Sat Apr 13, 2024 8:30 pm, edited 1 time in total.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7940
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Branch/Commits testing of latest UE code

Post by thunderchero »

commit sha = 45d9421d788c7249378b9018f911c000d1b90b58

This one I don't understand

Building mod 1.9.4 1366 x 768 (tested on unreleased installer)
Warning: in edifice.bst: Image size 60x50 of gspdf_60.tga doesn't match hsenergy.wdf placeholder 2464 with 120x100 pixels.
gspdf*.tga's is for "Economic Exchange" in orbital battery slot (even uses OB flag)

placeholder 2464 is set to 120x100 image used in game is 120x100 gspdf.tga not gspdf_50.tga (with border)

Economic Exchange gives +1% credits and is not buildable, only given at start of game.

EDIT; might be related issue? but doubt it
when I was looking at structures I noticed a strange effect (all mods tested)
in video you will notice the tga name changes on each selection except orbital battery?
Post Reply

Return to “Ultimate Editor”