adapting UE for structures above tech ten

[thunderchero]

Hi Everyone,

I was wondering if there has been any progress adapting UE to allow QuasarDonkey EdificePatcher?

[Flocke]

best make a list of what mod compatibility issues to resolve
right now I'm having a look into GALM compatibility
UDM would be next, but I fear there are many more issues than with GALM

[Tethys]

best make a list of what mod compatibility issues to resolve
right now I'm having a look into GALM compatibility
UDM would be next, but I fear there are many more issues than with GALM

Is that an attack or a compliment

Buildings above 10 is a GALM feature too Flocke, I just never implemented it because of the hassle of edifice patching when a value need to be edited. Every above 10 building is available at 10.

Think the most notable mod compatibilities are the taskforce patch, extended build queue, and the edifice patch. I think there may be a 25 tech level mod out there too.

Addition of wdf objects by type (no scrollbars since we don't understand them yet?) like add picture placeholder, add textbox, add listbox, etc... Stellar object would be nice to have add buttons. But now I am just being greedy and I can probably look into the simple wdf and stellar things eventually.

[thunderchero]

best make a list of what mod compatibility issues to resolve

BOP - issue with how mod uses code changes for dilth?

here is the patch to allow editing (remove to edit with UE, re-install to play)

Code: Select all

NAME: BOP reset to allow UE
AUTHOR: thunderchero
DESC: This patch will reset BOP to allow UE
DESC: patch file name: bop_reset.patch
URL: 
TAG: 

# >>  = vanilla/original value

# <<  = new value




>> 0x0505d1 85 C0 75 0C BA 06 00 00 00 89 C8 E8 0F 4C FF FF
>> 0x0505e1 8B 84 24 62 01 00 00 BA 06 00 00 00


<< 0x0505d1 66 8B 84 24 60 01 00 00 E8 A6 94 FE FF 89 C2 90
<< 0x0505e1 8B 84 24 62 01 00 00 90 90 90 90 90

UCW, MUM UM5, and soon Galaxy mod use structures above tech 10 (use Edifice Patcher)

UDM, UCW, UM5 and GALM use 18 fleet taskforce patch (saved game issues only)

[Flocke]

I think I have the UE save game editing fixed now for GALM and UDM3.
I also fixed multiple issues corrupting vanilla save games.

The main save game related fixes are:

• when ordInfo ended with entries smaller than 84 bytes
  84 bytes is both the military order and the economic order size including the order header size.
  There however exist some yet not analysed smaller entries that can crash the game if missing.
  That bug dates back to some version before UE 0.7.2dev5c and simply ignored to read those entries.
• moving re-grouped task forces
  When after turn start the player re-groups task forces and saves, only GWTForce is updated ( refer viewtopic.php?f=74&t=4154 ).
  - Since UE 0.8.0 however the sector positions were read from GTForceList, and it completely messed from which file to read and save.
  - Further I found that the ship positions need to be kept in sync for both the files, and therefore GTForceList task forces need to be re-grouped by UE to match moved player re-grouped task forces and those with changed ownership.
  - Additionally I found there exist negative temporary task force ids, that must not be included for the task force counter and may not be used for changing ownership to some ai player / or may only be used for GWTForce and eventually require some event flag or something.
• task force patch detection
  So far UE completely messed task forces when the 18 task force patch was applied, cause the ship ids values were changed from 4bytes to two bytes each. I now implemented some auto-detection that checks for invalid -1 ids (FF FF) and additionally checks whether the found ids actually are listed by GShipList and for GWTForce whether they list the same task force id.
  From testing this seems to work fine, although I'll add another check to make sure there is no deviation but all are read the same.
• extended build queue detection
  Yet UE already read excessive system data by always searching for the next valid system entry, Which checks for the expected system id plus some constraints like planet count and stuff. Excessive data then was appended to last system entry. That already worked pretty reliable, but I enhanced that some little to actually detect the patch and properly read the build queue entries.
• Integration check fixes
  There were some false detections and issues regarding the integration checks as well. I currently can't tell what exactly I fixed but have to back-check my commits. It now however should work properly and list no false detections.

If anyone is interested to test, changes can be found on:
https://gitlab.com/stbotf/ultimate-edit ... _detection

Would be great to get some confirmation also regarding further compatible and incompatible mods and remaining issues.
I might also release a dev version soon.

Regarding Edifice I took a look as well, but havn't fixed it yet. From what I found it is the Edifice file itself for which UE hard-coded the written tech tree table for 10 tech levels. The fix for it I'd however like to keep separate cause current branch already has 505 commits (with 20240 lines added and 8649 lines removed) and mainly focuses on save game improvements.

[thunderchero]

I think I have the UE save game editing fixed now for GALM and UDM3.
I also fixed multiple issues corrupting vanilla save games.

Great job
I will give it a try as soon as I can

Regarding Edifice I took a look as well, but havn't fixed it yet. From what I found it is the Edifice file itself for which UE hard-coded the written tech tree table for 10 tech levels. The fix for it I'd however like to keep separate cause current branch already has 505 commits (with 20240 lines added and 8649 lines removed) and mainly focuses on save game improvements.

I think UE works for tech tree table above tech 10 since UE allows setting of structure above tech 10.

would it be possible to add a Edifice patch detection? if new format reset to original format then on close reset to new format if it detects structures above tech 10?

edit; don't forget we do have source code for Edifice patcher
files can be found in this post
viewtopic.php?p=53975#p53975

edit 2; I tested on new UDM game after creating fleets and turn process without error. I also looked at lovetrek save game he posted last week and ran integrity check and uploaded fixed saved game for him to test.

I also looked at my latest game I just completed of MUM it showed only a few errors (testing new DxWnd proxy)

Code: Select all

Warning: in AgtSh0: Found shipId 2011 that is missing from GShipList. (removed)
Warning: in TskSh3133: Found shipId 2011 that is missing from GShipList. (removed)
Warning: in AIShpUnt: Found shipId 614 that is missing from GShipList. (removed)
Warning: in AIShpUnt: Found shipId 2011 that is missing from GShipList. (removed)

something that might be helpful when these type of errors display could it also display controlling race of shipid?

[Flocke]

I think UE works for tech tree table above tech 10 since UE allows setting of structure above tech 10.

Well, allowing to set the tech level is something I already patched with 0.8.0. And Lathon here did some fixes regarding Edifice as well.
I tested it yesterday and energy buildings showed up fine. For main building I however faced some issues. Well, actually I'd have to re-test it since there was a bug with the tech descriptions that I fixed right then.

That bug btw was introduced by myself with UE 0.8.0, cause I misunderstood some "magic" named initialization routine that abused some other routine that was meant for a complete different use .

Nonetheless in code of the Edifice.java source I find hard coded tech level values that enforce tech levels 1-10. Havn't decrypted yet what it's for, but from what it seems it writes some index table to the Edifice file. It's almost sure this breaks something.

would it be possible to add a Edifice patch detection? if new format reset to original format then on close reset to new format if it detects structures above tech 10?

I guess so, but to not enforce a specific trek.exe format I'd rather try to keep it independent from trek.exe if possible.
Instead there simply could be some auto-detection dependent on the already added structures, and if there are none above tech 10 it could raise a confirmation popup when you change to a higher level.

I havn't looked at the format changes yet, but I might also add some options menu to configure the format.

edit; don't forget we do have source code for Edifice patcher
files can be found in this post
viewtopic.php?p=53975#p53975

thanks, I remember so

edit 2; I tested on new UDM game after creating fleets and turn process without error. I also looked at lovetrek save game he posted last week and ran integrity check and uploaded fixed saved game for him to test.

that one I wanted to check on next and if I can figure it out, also add further integrity checks that might help

I also looked at my latest game I just completed of MUM it showed only a few errors (testing new DxWnd proxy)

Code: Select all

Warning: in AgtSh0: Found shipId 2011 that is missing from GShipList. (removed)
Warning: in TskSh3133: Found shipId 2011 that is missing from GShipList. (removed)
Warning: in AIShpUnt: Found shipId 614 that is missing from GShipList. (removed)
Warning: in AIShpUnt: Found shipId 2011 that is missing from GShipList. (removed)

something that might be helpful when these type of errors display could it also display controlling race of shipid?

Sure, great idea! Those warnings btw I noticed quite often as well, and it never seemed to have any effect. These occure when right that turn some ship was lost. The ai then auto-adjusts the next turn. Therefore they can simply be ignored.

[thunderchero]

Hi Flocke,

here is a messed up save if you are interested in looking at it.

Mirror universe mod 1366 x 768 beta 3

I was redeploying tt's and not sure what happened when I saved, but 1 fleet of 9 is listed as "attack system eta 2 turns"

this is happening at sector 39 - 2 chalnoth system

game crashes if I try to stop movement by setting fleet to engage then assault system

game1.sav

(336.02 KiB) Downloaded 44 times

[Flocke]

here is a messed up save if you are interested in looking at it.
game crashes if I try to stop movement by setting fleet to engage then assault system

Thanks alot, that messed my weekend plans!

That save gave some great insight and helped to identify and fix some more UE save game bugs.
Current version now is capable to identify and fix that error. Please check that it works, I only tested one turn.

For reference, the crash.log states:

Code: Select all

Version Under Test: 72
File: ..\..\source\game\ship.c, Line: 968, taskForce != NULL
Initialize State: 31
Player Empire: 4	Starting Seed: 1650067517
Galaxy shape: 1	Galaxy Size: 2

The save is broken in that GWTForce lists two redeployed task forces with same temporary id.
trek.exe failed here to iterate the task forces, since it looks up the task forces by id and expected one more.

The second task force further overwrote the OrdInfo order of your other task force, which is why it says "eta 2 turns".

The UE bugsfix for that save now reads:

errors.jpg (24.25 KiB) Viewed 2466 times

Any more such issues or may I release UE 0.8.1?


---
Btw I also did some asm debug analysis to learn more on the issue:

merge_task_forces.jpg (77.2 KiB) Viewed 2466 times

Actually this is not for above error, but for:

Code: Select all

File: gdllist.c, Line: 534, gdlList != NULL

This was raised when I used previous UE integrity checks in try to fix the other issue.
Here trek.exe crashed in remove_ship_c_AlienTFInfo when it tried to merge & reassign the redeployed temporary task forces and ships were missing from GWTForce. This was caused by UE cause there I mapped the task forecs by id as well, and it simply dropped the previous task force when it encountered the duplicate id.

[thunderchero]

Current version now is capable to identify and fix that error. Please check that it works, I only tested one turn.

sorry, real life will prevent testing until next week. But it was funny, I just changed orders to "avoid" and let that taskforce go where it wanted and once it arrived gave them new order without issue.

[Spocks-cuddly-tribble]

Thanks alot, that messed my weekend plans!

Be careful, I'd bet soon thunderchero wants you to fix the funniest variant of this issue: Phantom copies of ships (crash when entering tactical combat).

Trek.exe's flawed temporary task force ID feature, you identified a month back, explains quite a lot vanilla task force / saved game corruptions and inconsistent AI reactions to changed task forces / orders in saved games: viewtopic.php?p=55981#p55981

Workaround recommendation was always to avoid regrouping task forces when loading a saved game i.e. to wait till next turn and do it before saving.

Even worse was regrouping task forces in a saved game, saving changes at the same turn and loading the multiple changed task forces game again (maybe even repeated).

Most problematic was oftentimes removing the last ship from a task force i.e. dissolving the temporary task force (with already corrupted data in the saved game files).

[Flocke]

well, I only accept already corrupted save games
to fix the actual bug is something I'd be asking the omniscient SCT for

@TC next week is fine, I still have a few minor things to address and am not in need to push out a new official version to proceed on other stuff either, it's just a tagged binary afterall

[Spocks-cuddly-tribble]

well, I only accept already corrupted save games

You can bet the phantom duplicates of said ships will be represented as messed up fleet files. Most likely they will list the same gshiplist ID(s) in multiple task forces.

[Flocke]

I am looking forward to sample files! To fix this should not be much of an issue now that I already dealt with those temporary fleet ids.

[Spocks-cuddly-tribble]

Quick testing this I wasn't able to replicate the tactical combat crash, just phantom ships in the player task force ships screen (auto-corrects next turn, except for some messed up orders).

Most experiments ended with a task force NULL crash (e.g. duplicating the same cruiser 7 times):

File: ..\..\source\game\ship.c, Line: 968, taskForce != NULL


The uploaded vanilla save shows an example of a messed up player display (actually there is only two cruisers):