Ultimate Editor

Ultimate Editor; support/discussion/questions

Moderator: thunderchero

User avatar
Lathon
Lieutenant-Commander
Lieutenant-Commander
Posts: 116
Joined: Thu Oct 15, 2009 2:00 am

Re: Ultimate Editor

Post by Lathon »

thunderchero wrote: Sun Jun 13, 2021 9:09 pm just a side note it looks like you are testing on mirror universe mod. (uncivil wars might be better for testing?)

MUM has issues with UE with main structures, but should not cause crash that I know of though.

what happens is when UE reads main structures it wants to auto adjust the upgrade table. Dcer had it setup for either no upgrades structures or 9 upgrade structures. Since MUM has 5 upgrade structures the "structure_table.patch" needs to be applied.

structure_table.patch

the other issue MUM has is;
after resetting with the EdificePatcher.exe, sets the wrong last main research structure (I think EdificePatcher.exe just uses the last ID of Edifice, not the last research structure) since my minor structures are last on structure list. so you will need to apply "Main_Research first and last structure IDs" patch

Main_Research_first_and_last_structure_IDs.patch
Ah okay. know this now, I can retest it. I was using Vanilla for most of my testing, but I noticed the MUM mod had the change too.
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3197
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: Ultimate Editor

Post by Flocke »

I decided to go ahead and release UE 0.8.0 now, check:
https://gitlab.com/stbotf/ultimate-editor/-/releases

From my testing it looks to be stable and I hope I worked out all that I broke.
With the many changes and code cleanup I however easily might have missed something, so as usual, keep a backup of stbof.res and trek.exe if your're going to touch it, and keep a backup of every save game you are going to touch!

I now am going to focus on other things for a while, and then likely the next year want to look into my BotF development framework project again that is left on hold way too long for all the awesome possibilities we have there.
With the UE save game fixes and sav file analysis I however did some great progress on the fundamentals and I hope many of it can be applied to the running game as well.

Thx to all support and I hope you all can make good use of the changes and start participate in future improvements! :D
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3197
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: Ultimate Editor

Post by Flocke »

thunderchero wrote: Tue Apr 20, 2021 9:57 pm here is edifice.bst with and without patch. (from mirror universe mod)
edifice.zip
these files have completely different formats
Like noted in viewtopic.php?p=56210#p56210 the file format is same, beside that the supported tech levels got doubled from 10 to 20. (or 11 to 21 if including the separators from viewtopic.php?f=261&t=1597).
thunderchero wrote: Tue Apr 20, 2021 9:57 pm I think I have all the trek.exe changes documented but some overlap with the tech patch.
Given the sparse documentation, here the actually edited max tech level values with annotations determined from the EdificePatcher source code:
// tech requirements entry size of the tech map
>> 0x0004f52b 89 d0 c1 e0 04 01 d0
<< 0x0004f52b 69 c2 80 00 00 00 90
// 69 c2 [dword = (21 tech levels * 6 bytes) + 2 byte separator] nop

// code nops
>> 0x0004f538 c1 e0 02
<< 0x0004f538 90 90 90

// first number of buildings per tech level file offset
>> 0x0004f541 66 8B 44 42 44 59 5b c3 00 00 00
<< 0x0004f541 66 8B 84 42 6C 00 00 00 59 5B C3
// 66 8B 84 42 [dword = (21 tech levels * 4 bytes) + 24 bytes file offset (0x18)] 59 5B C3

// ignore list offset (8bytes header + tech map size)
>> 0x0004fa5a f4 01 00 00 // 492 + 8 bytes
<< 0x0004fa5a 98 03 00 00 // 912 + 8 bytes

// 11 to 21 tech levels updates
>> 0x0004f51f 0b 
>> 0x0004f822 0b
>> 0x000d0d1e 0b
>> 0x000d2c70 0b
>> 0x000d577a 0b
<< 0x0004f51f 15
<< 0x0004f822 15
<< 0x000d0d1e 15
<< 0x000d2c70 15
<< 0x000d577a 15

The EdificePatcher further patches the max building indexes at following offsets:
0x4f8e5, 0x4f90e, 0x4fad8, 0x53105,
0x851c3, 0x85296, 0x85345, 0x853d0,
0x8544f, 0x85475, 0x85926, 0x85f39,
0x8602b, 0x86240, 0x862e3, 0x8636e,
0x86394, 0x86755, 0x8680f, 0x868ec,
0x869cd, 0x86abd

And the max building numbers at:
0xaddc, 0xcb55, 0x39e65, 0xf38c9

For unmodded BotF the max building number is 13Eh = 318, while the max index is 13Dh = 317 (for the 318 building indexes 0 to 317).
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7849
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Ultimate Editor

Post by thunderchero »

Flocke wrote: Sun Apr 24, 2022 9:52 am Like noted in viewtopic.php?p=56210#p56210 the file format is same, beside that the supported tech levels got doubled from 10 to 20. (or 11 to 21 if including the separators from viewtopic.php?f=261&t=1597).
the edificepatcher does not double tech level it can set to any value example; uncivil wars uses 25 tech levels

just on a side note I am so tired I can't even see straight to read most posts and comprehend them completely. this last month has been rough.
User avatar
Spocks-cuddly-tribble
Code Master
Code Master
Posts: 1884
Joined: Sun Apr 27, 2008 2:00 am

Re: Ultimate Editor

Post by Spocks-cuddly-tribble »

Flocke wrote: Sun Apr 24, 2022 9:52 amthe file format is same
With 'different formats' thunderchero means the trek.exe code needs to be patched and a human user needs to know about the changes to read the file propperly.

Your nitpicking insist of restricting the term format/structure change to certain types of changes (not a problem or a cause for a dispute :wink: ).

Flocke wrote: Sun Apr 24, 2022 9:52 am// ignore list offset (8bytes header + tech map size)
>> 0x0004fa5a f4 01 00 00 // 492 + 8 bytes
<< 0x0004fa5a 98 03 00 00 // 912 + 8 bytes
Doesn't this imply the ignore list is relocated and extended or is this just the starting position? viewtopic.php?p=56210#p56210

EDIT: Even if it's not extended yet, using the known pointer locations it should be easy to move the list to any location in the file (e.g. the end).

EDIT2: Moving the list reqires code changes:

Code: Select all

AUTO:00450659                 add     eax, 1F4h // base pointer

AUTO:0045066B                 imul    ebx, eax, 30h  //  48 bytes each

AUTO:00450689                 add     eax, 0F0h // added to ignore list pointer for start of building data?
I don't know how many bugs is too many but that point is reached somewhere before however many in BotF is.
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3197
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: Ultimate Editor

Post by Flocke »

Spocks-cuddly-tribble wrote: Sun Apr 24, 2022 11:45 am Your nitpicking insist of restricting the term format/structure change to certain types of changes (not a problem or a cause for a dispute :wink: ).
Well, yes, but what I wanted to point out, is that it indeed is identical except the tech maps tech level count.
Spocks-cuddly-tribble wrote: Sun Apr 24, 2022 11:45 am Doesn't this imply the ignore list is relocated and extended or is this just the starting position?
It is just the starting position and it is not extended by the EdificePatcher. The patcher even warns on extending the building count further than what that list permits. And I didn't find any reference in the forums for extending it yet either.
Spocks-cuddly-tribble wrote: Sun Apr 24, 2022 11:45 am EDIT: Even if it's not extended yet, using the known pointer locations it should be easy to move the list to any location in the file (e.g. the end).
Well, that would indeed change the file data structure! :lol:
Further it is not about moving that list, but to lengthen the 48byte bitwise building flags to allow for more buildings than 48*8bit=384.

Regarding UE, the problem with it is that how can it be determined from the edifice.bst file itself without having to look up changes in trek.exe, which I'd like to avoid to not enforce a specific data format on the trek.exe file.

Edit: Well, the expected size for the ignore list can be calculated to either be the default 240bytes or 6 empires * rounded up ( number of buildings / 8 ), where the 6th empire likely is the minors. The expected tech level count on the other hand can be determined from techtree.tec.
Those numbers then can be checked against the combined tech map and ignore list size, since the expected building and address list size can be determined by the building count and substracted from file size.
The chance of failure is rather low so I'll likely implement it that way. Patching trek.exe I however rather leave out of UE.

Edit2: Something must be wrong, either it is indeed 5 empires * 384 buildings, or it is 6 empires including minors * 320 buildings max, like what the up rounded default building number of 318 does indicate! :shock:

Edit3: No, by the hex data it is obvious that it is 5 empires * 48bytes by default:
ignore_list.jpg
ignore_list.jpg (61.34 KiB) Viewed 1899 times
User avatar
Spocks-cuddly-tribble
Code Master
Code Master
Posts: 1884
Joined: Sun Apr 27, 2008 2:00 am

Re: Ultimate Editor

Post by Spocks-cuddly-tribble »

I wasn't expecting the code is that extension friendly. Just a silly wording in my edit2. Should be: Is easy to extend without relocation.

The list is 48*5=240 bytes (ALL building bits for all 5 empires). You stated this by yourself :?:

You just need to change two values in trek.exe to extend it (see my above code 30h * 5 = F0h).

For UE detection/calculations just read: edifice.bst Offset 0x0000 to 0x0003 [number of buildings] / 8 (round up)

I don't know about the tech files.
I don't know how many bugs is too many but that point is reached somewhere before however many in BotF is.
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3197
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: Ultimate Editor

Post by Flocke »

Spocks-cuddly-tribble wrote: Sun Apr 24, 2022 2:37 pm For UE detection/calculations just read: edifice.bst Offset 0x0000 to 0x0003 [number of buildings] / 8 (round up)
For vanilla BotF that would be 318 / 8 = 39,75 so round about 40 bytes.
But when adjusting trek.exe is that easy, I use that formula and set a minimum of 48 bytes. :up:

As a fallback, when file size indicates so, I'll make it stick to 48 bytes but issue a warning to remind patch trek.exe.

I should find time to implement this next weekend.
User avatar
Spocks-cuddly-tribble
Code Master
Code Master
Posts: 1884
Joined: Sun Apr 27, 2008 2:00 am

Re: Ultimate Editor

Post by Spocks-cuddly-tribble »

Before wasting your time, is there even need for more than 384 buildings (+66 based on vanilla 318)?

Even some extended tech level mods use reduced upgrades lowering the total count even further.

And from what I remember adding more than 8 main building upgrades (default T1-9) is a tremendous code and edit mess. So it's only about extra special structures.
I don't know how many bugs is too many but that point is reached somewhere before however many in BotF is.
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3197
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: Ultimate Editor

Post by Flocke »

Spocks-cuddly-tribble wrote: Sun Apr 24, 2022 3:40 pm Before wasting your time, is there even need for more than 384 buildings (+66 based on vanilla 318)?
Having opened the possibility is enough reason. It's easy to implement and I need to touch the code anyhow. :wink:
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7849
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Ultimate Editor

Post by thunderchero »

Just want to say "Thank you" for all your efforts.

sorry I can't be of more help on some of your projects.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7849
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Ultimate Editor

Post by thunderchero »

I just found a bug?

312555, // 0x0004C4EB - martial law buildings - fileMARTIAL_LAW_BLD3

if structure is above structure id 127 value will be read as negative and structure will not be removed from list when added to build queue.

This is not an error by UE, but we might what to add a warning?

question is how can I fix issue in MUM without rearranging structure list? looks like I will be adding code to relocation table area.
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3197
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: Ultimate Editor

Post by Flocke »

thunderchero wrote: Sun Apr 24, 2022 4:57 pm Just want to say "Thank you" for all your efforts.
Thanks, and ofc thanks to all your amazing efforts and all time present service too!

When I find time I have alot in plan. But it doesn't help to imagine, it's alot of work ahead.
And soon I need to care more on RL again so the framework thing has to wait.
thunderchero wrote: Sun Apr 24, 2022 8:50 pm if structure is above structure id 127 value will be read as negative and structure will not be removed from list when added to build queue.
Weird, even unmodded vanilla BotF has 318 structures, how can that fail? :shock:
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7849
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Ultimate Editor

Post by thunderchero »

Flocke wrote: Mon Apr 25, 2022 2:51 am
thunderchero wrote: Sun Apr 24, 2022 8:50 pm if structure is above structure id 127 value will be read as negative and structure will not be removed from list when added to build queue.
Weird, even unmodded vanilla BotF has 318 structures, how can that fail? :shock:
the 3 other locations are 4 byte but that location is a single byte
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3197
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: Ultimate Editor

Post by Flocke »

thunderchero wrote: Mon Apr 25, 2022 6:17 am the 3 other locations are 4 byte but that location is a single byte
By the asm checks it is not about the id becoming negative, but that the game specifically handles the inquisition and trade goods different and looks them up by id. cmp dx is a 16bit comparison, so -32768 to 32767, which should be sufficient.
building id checks.jpg
building id checks.jpg (80.61 KiB) Viewed 1825 times
Post Reply

Return to “Ultimate Editor”