Ultimate Editor

[Flocke]

The way this is set up means there's no way to 'import' the spread sheet data into UE. Every field (thousands) have to entered by hand one by one. Errors are inevitable. I had thought I had done a better job of avoiding them.

QD created a tool some time ago, but don't know if his spreadsheet would work or not.
"Tool for editing and importing the ship list to stbof.res"
viewtopic.php?p=37706#p37706

I have made a tool in excel that uses some base values and generates the stats for the ships based on race, ship type and tech levels (like Mentat's, but simpler and friendlier ). The tool can also import the htlm ship list from UE on a different page and compare the UE values with the generated values and flags the differences so one can go back to UE and fix the mistype.

If one of you Java guys can add the import feature and state the list format for input, it will save the moder days.

Adding some advanced CSV Import- & Export option to UE ihas become one of the core features implemented with the upcoming version of UE.

In private I rather use Libre Office, but I tested and checked on the Ship_List_Modifier2.1.xlsm Excel file from adi and made sure to match same data format, with a few columns added. It asks for permission to execute Excel scripts, but beside that it's auto-recorded and pretty messed, the code looks fine to me.

The xlsm is included with upcoming UE or can be downloaded from viewtopic.php?t=2710
If scripts aren't relevant, the exported CSV of course can be viewed and edited with any CSV editing tool or text editor.

Most importantly I also implemented to import it back to the game!

So it's again about some import / export feature you are missing. Either import/export the german descriptions or all the other data I guess.

Note that it's not only about two description strings and the pointer for shipdesc.sst but also various edits in lexicon.dic and trek.exe for monsters.

you already can copy over the german shipdesc.sst file and then fix the text offsets with the bottom right description fixture tools in ship stats view of UE.

The english/german sorting in the ECM shiplist(s).sst might differ? I haven't tested any german BotF version in years, so I couldn't even determine the right vanilla descriptions to match them with the right ships without studying german vanilla before.
Plus I remember more than 20 years ago I edited some german vanilla ship descriptions since they were confusing/incorrect...

To help on translations, I added additional import options, see middle screenshot in above posting:

ship_import_export.jpg (69.07 KiB) Viewed 441 times

These allow to skip overwrite ship names and descriptions, so you can export -> import data from another stbof.res file that uses different language.

Plus if ship sorting is messed, like when you try to import back translations only from a previous version, and you are too lazy to manually match all the ship ids, you can try the match by gfx function.

In addition, you can also selectively import ships, but they still must match by id or gfx.
Or you can limit to import existing ships only, to just update the data stats.

Furthermore I improved trek.exe and stbof.res ship id updates. And when it's about the lexicon.dic, to simplify editing it has an import / export option for a while already. I even added CSV import & export options to the "Group Edit Of Building Stats"-GUI. Yet missing is tech tree and races.

I am sure there is much what still can be improved on the import / export options. But before I continue to invest time, I think it needs some testing and feedback. Maybe you guys have additional suggestions. I am sure you have.

[thunderchero]

quick question, when I un-check/check boxes in this section it make no changes
is it not possible to edit?

check.jpg (266.67 KiB) Viewed 425 times

Note; so far no issues noticed (downloaded/compiled Monday, ‎March ‎4, ‎2024, ‏‎7:48:12 AM)

[Flocke]

quick question, when I un-check/check boxes in this section it make no changes
is it not possible to edit?

I missed to update the HOB file on flag changes. Should be fixed.
Don't ask for what effect these flags have. All untested.

[thunderchero]

Don't ask for what effect these flags have. All untested.

when I saw flags I thought about testing plasma hob, but as a single billboard it will never display correctly from front or rear angle.

[Spocks-cuddly-tribble]

I thought about testing plasma hob, but as a single billboard it will never display correctly from front or rear angle.

IMHO that's just a minor graphic inconvenience we could live with. There is far worse multifactorial issues with many wrong and incomplete codes:

- reference scale sometimes volatile depending on clicked object
- for asymmetric pics heading/orientation volatile depending on clicked object
- wrong animation of missing shots
- phaser code trajectory animation still traces moving targets
- at least one billboard orientation value volatile/flawed (sideways issue)
- Ok-ish visuals would require dual shot animation (Defiant/Klingon BOPs)

[Flocke]

can u change the wormhole stuff it doesn't seem to work connecting wormholes doesn't seem to work in game or idk maybe it takes a turn to work but the save says it's good and I mean new wormholes not old already set wormholes

I'm sorry for late response, but I got sick on weekend and still don't feel good. Only this evening I managed to check back on your posts.
I will look into connecting wormholes when I'm getting better again, they have been working before.

I now checked on the wormholes. Connecting wormholes in latest build works fine for me and I can't remember that I fixed anything on it:

wormhole-endpoint.jpg (37.79 KiB) Viewed 354 times

I sent a scout through the wormhole and it moved to the configured endpoint as expected.
Both wormholes were added with UE.

passed_wormhole.jpg (45.5 KiB) Viewed 354 times

[Flocke]

I added the wrong outpost to a wrong ppls colony by accident and the game crashed but then fixed it and was fine

Adding outposts of another race works fine as well.

hijacked_outpost.jpg (139.35 KiB) Viewed 349 times

I can even add own outposts to other race systems and it doesn't crash:

ferg_sol_outpost.jpg (60.88 KiB) Viewed 349 times

qo'nos_combat.jpg (67.19 KiB) Viewed 349 times

And when I change ownership to another race, it doesn't crash either.

Either the error report was tested with an old release or there are other steps involved needed for reproduction.

[Flocke]

Not sure you need this for save game editing:
GTForceList/GWTForce [dword +0x54] = num way points (number of green arrows displayed in gridmap) sectors to move destination @ +0x30

So far I assumed it to be a military order id since it is always same for assigned militar orders at 0x34+0x24 ( viewtopic.php?p=54174#p54174 ), but it isn't unique so I always was in doubt.

I checked on GTForceList/GWTForce @ 0x54 now. How can that be the number of way points?

militaryOrderId.jpg (280.76 KiB) Viewed 336 times

that is checked on game7.sav I used for savegame analysis, uploaded in:
viewtopic.php?p=54002#p54002

and matches the military order ids over here:

0x24-0x27 unk2: Unknown, but for ordInfo the same as GTForceList / GWTForce 0x54-0x57. Looks like some counted identifier, but isn't listed by any other file.

e.g. in sector D16 there are 12 single cruiser fleets moving just one sector south east to Ariannus
they all have different values in GTForceList / GWTForce 0x54-0x57

TFID: 23 04 00 00 -> 0x54: 08 D3 00 00
TFID: EC 04 00 00 -> 0x54: FE D2 00 00
TFID: DA 04 00 00 -> 0x54: 07 D3 00 00
TFID: E3 04 00 00 -> 0x54: 06 D3 00 00
TFID: E4 04 00 00 -> 0x54: 05 D3 00 00
TFID: E6 04 00 00 -> 0x54: 02 D3 00 00
TFID: E7 04 00 00 -> 0x54: 01 D3 00 00
TFID: E8 04 00 00 -> 0x54: 00 D3 00 00
TFID: E9 04 00 00 -> 0x54: FF D2 00 00

it obviously is a counted dword value, for task force 89 04 00 00 you even find EF ED 01 00 @ 0x54

[Spocks-cuddly-tribble]

it obviously is a counted dword value, for task force 89 04 00 00 you even find EF ED 01 00 @ 0x54

Sorry for the fuss, looks like an error in my old database. Was mislabeled as distance (num sectors/waypoints) in map movement codes, but indeed is an item ID. Sometimes set to zero if a sector destination is reached.

There might be a special case where it's short term misused for player ship UI purposes, but I can't find this code right now.

[Spocks-cuddly-tribble]

Hi Flocke, I have a hard time understanding UE edifice.bst code: https://gitlab.com/stbotf/ultimate-edit ... ifice.java

But I love the gitlab 'Blame' button. I blamed the rain on you and got one hour sunshine, enough for a relaxed 10km run.
No idea how you coded this one, but much appreciated.

To the point, I'm about to torture you with another messy patch.
It reads the empire output value of the trade goods building in edifice.bst as percentage factor for the industry output as credits bonus.


Questions:


- by default edifice.bst output values of the bonus type shipyard (builds ships) and trade goods (+ credits 04) are unused, except for wrong GUI displays:

trade_goods.png (63.14 KiB) Viewed 235 times

I assume UE still allows editing them (important for patches)?


- UE allows editing trade goods building ID 0x12 and edits three trek.exe values for it? (I cant see them in java file, but know them from IDA)

Does it still limit trade goods to one shared ID for all empires (important for a patch using the fixed ID -> no real structInfo entry)?


- There is a reference to UnrestBonus BONUS/BONUS_2 (word 92h +2*race index?) -> cf. Structure of edifice.bst: viewtopic.php?p=21827#p21827

Trek.exe code for effect of unrest orders (emergency morale programs) use the regular output values (word 86h +2*race index)

Is there any confirmation of UnrestBonus / BONUS_2 (word 92h +2*race index), or is this an unfounded speculation?


- There is many trek.exe codes for +7A 'SystemPopCap requirement (cumulative)' & +7E 'Support cost in credits per turn'.

I can't see them referenced (editing them might become useful)? viewtopic.php?p=30656#p30656

[Flocke]

Hi Flocke, I have a hard time understanding UE edifice.bst code: https://gitlab.com/stbotf/ultimate-edit ... ifice.java

You went a bit deep into history commits.
Best look at latest version in improve_shipgfx_export branch. And don't ask how the ship graphics feature relate to the edifice buildings.
https://gitlab.com/stbotf/ultimate-edit ... ifice.java

There (and already on master branch) I added alot of helpful code documentation. But the main part you possibly missed is that actually you wanted to look at Building.java found in the data subfolder:
https://gitlab.com/stbotf/ultimate-edit ... lding.java

Java is a (not so modern anymore) object oriented language. Edifice therefore is just a storage of the buildings and stuff.

I assume UE still allows editing them (important for patches)?

In Building.java you find there is a setBonusType and a setBonusValue routine. The default web editor sadly doesn't show code references, from where it is called.
Of course you could download the whole code and do a multi-file search to find all usages of the string.
Or better install some Java IDE to browse code references.
But yes, I didn't remove it.

- UE allows editing trade goods building ID 0x12 and edits three trek.exe values for it? (I cant see them in java file, but know them from IDA)

The trek.exe update is hidden in BuildingTools.updateExe(), called from finalWarning call when the BuildingGUI is closed:
https://gitlab.com/stbotf/ultimate-edit ... java#L1248
https://gitlab.com/stbotf/ultimate-edit ... .java#L196
There are however no trek.exe updates for the bonus as far I can see. When you tell on the addresses, I can check what segments are registered for it and whether there are any method references that update the value.

Does it still limit trade goods to one shared ID for all empires (important for a patch using the fixed ID -> no real structInfo entry)?

Watching the UE building bonus screen, there is one type selection and 6 bonus values for the different empires (and minors I guess).
UE doesn't change whole trek.exe code to add in race individual bonus type support.
https://gitlab.com/stbotf/ultimate-edit ... g.java#L96

Is there any confirmation of UnrestBonus / BONUS_2 (word 92h +2*race index), or is this an unfounded speculation?

It is not documented in the code, but when you check the UI screen, it sais "Bonus to unrest threshold".
For e.g. the private farms it is set to 1 and forced labor farms is set to -1 for all the empires.

- There is many trek.exe codes for +7A 'SystemPopCap requirement (cumulative)' & +7E 'Support cost in credits per turn'.
I can't see them referenced (editing them might become useful)? viewtopic.php?p=30656#p30656

support-cost.jpg (26.12 KiB) Viewed 215 times

Thanks for the hint, I add it to my list...

[Spocks-cuddly-tribble]

Best look at latest version in improve_shipgfx_export branch.

I thought it automatically displays the newest code you are working on if I start from the main page: https://gitlab.com/stbotf/ultimate-editor
So how do I know whether the latest version is a dev or branch?
And thanks for the other hints to Java structure and reading options.

But yes, I didn't remove it.

Very nice!

There are however no trek.exe updates for the bonus as far I can see. When you tell on the addresses, I can check what segments are registered for it and whether there are any method references that update the value.

By default just used for GUI cosmetics:

004F995C         mov     eax, 12h                          ; (trade goods)
004F9A11         mov     eax, 12h                          ; trade goods
004F9AFD         mov     eax, 12h

To be clear, for my purposes it's better if UE doesn't change this fixed ID!
It's referenced in my new patch, so UE changed IDs require loading them from one of the above locations by asm pointer.

Watching the UE building bonus screen, there is one type selection and 6 bonus values for the different empires (and minors I guess).
UE doesn't change whole trek.exe code to add in race individual bonus type support.

Expected and recommended, sole purpose would be different pictures / description texts for the empires (c.f. the five emergency morale structures).

Base output value depends on current system owner, the sixth minor value is used in very few cases (OB & Shield Generator when attacking independent minor systems), but even those default codes are flawed with wrong tech levels or sometimes missing.

It is not documented in the code, but when you check the UI screen, it sais "Bonus to unrest threshold".
For e.g. the private farms it is set to 1 and forced labor farms is set to -1 for all the empires.

Found GUI threshold quick thanks to your lexicon.dic list: viewtopic.php?p=55648#p55648

004F38DC         mov     eax, [ebp+42h]
004F38DF         xor     edx, edx
004F38E1         sar     eax, 10h                          ; current moral value
004F38E4         mov     dx, [ecx+2F0h]                    ; base morale (when owned by empire X)
004F38EB         cmp     eax, edx
004F38ED         jge     GUI_no_unrest_effect

Still no hint of [edifice+92h +2*race index]. Same for effect of unrest orders. Smells like an unused feature...
So is this supposed to affect the systInfo base morale value or part of code for negative morale events?

---

OT: Just learned something new about assembler coding. Negative esp locations are volatile, since the virtual memory usage of the OS might repurpose them during runtime. Most of the time they work perfectly fine, until they don't, leading to really wired stuff....
Have to fix my new '+ ship support bonus' accordingly.

[Flocke]

I thought it automatically displays the newest code you are working on if I start from the main page: https://gitlab.com/stbotf/ultimate-editor

No, what is displayed is the main branch where all developers of the project merge their development once a feature is complete and reviewed by DCER.

Usually a project has multiple development branches that are worked on in parallel. Of course that can cause conflicts, but more important is that during development of a feature people don't mess each other changes. Therefore the development is split to branches and at some point it is merged back. That there is a branch worked on for more than a few weeks is rather unusual.

(Well, not that unusual I guess. At work I currently deal with a project that was split years back and now they try to merge it again. Given the huge development gap, that idea soon was dropped and now it is worked on in parallel to fulfill urgent customer needs... )

Since there are no other active developers and I didn't want to bug DCER all the time with small bug fixes and incomplete changes, I kept working on same old improve_shipgfx_export branch you can select in the top left.

To see current active development branches, open left sidebar and open the Code -> Branches view:
https://gitlab.com/stbotf/ultimate-editor/-/branches

By default just used for GUI cosmetics:

004F995C         mov     eax, 12h                          ; (trade goods)
004F9A11         mov     eax, 12h                          ; trade goods
004F9AFD         mov     eax, 12h

In UE code you need to search for the file offsets. With a quick full-text search for 'F8D' I found they are all used:

trade-goods.jpg (41.99 KiB) Viewed 163 times

Searching for 'SD_BldId_TradeGoods' I found they are set by the updateTrekEconomy routine of the BuildingTools, which is called when the Building GUi is closed - like I alread explained:
https://gitlab.com/stbotf/ultimate-edit ... .java#L247
https://gitlab.com/stbotf/ultimate-edit ... .java#L270

To be clear, for my purposes it's better if UE doesn't change this fixed ID!
It's referenced in my new patch, so UE changed IDs require loading them from one of the above locations by asm pointer.

Bad luck for you, already UE 0.7.2dev5c set those values and I won't remove it:
https://gitlab.com/stbotf/ultimate-edit ... java#L1878
https://gitlab.com/stbotf/ultimate-edit ... .java#L354

So is this supposed to affect the systInfo base morale value or part of code for negative morale events?

Never gave it a test myself. But searching forums, use of those values never was clear:
viewtopic.php?p=46902#p46902
Even in initial post of DCER it is marked with a question mark, and there's no confirmation posted.
viewtopic.php?p=21827#p21827
And UE 0.7.2dev5c simply names it 'BONUS_2'.
https://gitlab.com/stbotf/ultimate-edit ... g.java#L66
Only by the GUI label it is named to be unrest bonus, like if he still wasn't clear on it:
https://gitlab.com/stbotf/ultimate-edit ... java#L1633
https://gitlab.com/stbotf/ultimate-edit ... ties#L1001
Therefore it's most likely just been a wild guess based on the building values set.

OT: Just learned something new about assembler coding. Negative esp locations are volatile, since the virtual memory usage of the OS might repurpose them during runtime. Most of the time they work perfectly fine, until they don't, leading to really wired stuff....

I didn't know negative values was possible!
It should always be positive, because it is stack memory that needs to be allocated and adds up to the esp stack pointer.
If negative, it should access stack values from some of the calling functions I guess.

I did a quick search:
"the stack pointer points to the bottom of the stack since x86 stacks build from high address values to lower address values"
https://stackoverflow.com/questions/139 ... they-point

So a negative value points to some yet not allocated value that might be overwritten by the stack usage of any call that does follow.

[Spocks-cuddly-tribble]

a project that was split years back and now they try to merge it again. Given the huge development gap, that idea soon was dropped and now it is worked on in parallel to fulfill urgent customer needs...

Yeah, I know that kind of stuff. A few years back I had a glance at the software the german MDK (medical institution) from the nineties patched for modern hardware/OS. The bugs reminded me on BotF....

With a quick full-text search for 'F8D' I found they are all used:

As told, expected from three unspecified value references in the UE java page. So I was surprised when you said you see no trek.exe updates for trade goods. Albeit the misunderstanding is obvious, since I talked about the edifice.bst ID, not the bonus type 04 +credits (for trade goods).

-> I'll update my patch from: push 12h & pop eax -> mov eax, [4F995D]

Just for the record, in the shiplist topic you said removing custom checks and depending on other code locations' limits/values is bad coding style....

use of those values never was clear: ... and there's no confirmation posted.
Therefore it's most likely just been a wild guess based on the building values set.

There was a very old building editor from Jokers' page. I guess DCER just copied the nonsense from there. Anyway, since UE supports editing the values, how about just changing UE GUI label to e.g. 'unused values' and update the description once we developed a useful patch using them?

"the stack pointer points to the bottom of the stack since x86 stacks build from high address values to lower address values"
So a negative value points to some yet not allocated value that might be overwritten by the stack usage of any call that does follow.

Yes, and that's the way I used it. The calling sub loads a negative esp location which should still hold the (yet not overwritten) data from the just returning sub. And that worked flawless in tests.
Yet, that code causes stange/sporadic/unexplainable issues. Some hardcore nerds told the esp stack in the dynamic/virtual memory allocation of modern OSs' is not like the human mind expects a static linear line. So once released by add esp,x - pop - retn there is a remote possiblity the data and/or pointer is no more valid (depending on OS and programming of the application). From the applications' point of view, think of it as an area outside of the definded/known universe. At some point, the laws of physics might not persist with a certain/low probability.
So, in the end, your thinking might be valid and I have never seen any default asm code using negative esp pointers:

It should always be positive, because it is stack memory that needs to be allocated

That released negative esp memory oftentime persists might have someting to do with stuff like min page size for re-use of released esp areas?
But I still hope another issue causes my problem. With the negative esp data recycling trick we could scrap quite allot redundant trek.exe code.

[Flocke]

Just for the record, in the shiplist topic you said removing custom checks and depending on other code locations' limits/values is bad coding style....

When the limit is known, while(true) { } loops are considered bad coding style, because programmers often fail to properly check termination for all possible cases, and because not every time it is obvious from reading the code.
Instead it is preferred to write for (int i = 0; i < limit; ++i) { } loops, or the modern foreach ( x : collection ) { } version.

Anyway, since UE supports editing the values, how about just changing UE GUI label to e.g. 'unused values' and update the description once we developed a useful patch using them?

When it is confirmed by ingame testing, I update the description. Given that the value might not be read by the UI, for proper testing I'd say that you need to enable random events, and activate energy buildings that massively add negative unrest bonus to see whether colonies might rebel.

But I still hope another issue causes my problem. With the negative esp data recycling trick we could scrap quite allot redundant trek.exe code.

From wikipedia ( https://en.wikipedia.org/wiki/Stack-bas ... allocation ):

If, however, the data needs to be kept in some form, then it must be copied from the stack to the heap before the function exits. Therefore, stack based allocation is suitable for temporary data or data which is no longer required after the current function exits.

This is by convention. From what I read stack implementation and limits is OS, language and machine dependent. For Windows C & C++ programs usually the stack is pre-allocated and has a fixed size of 1MB. But even then, from what I know you run into CPU optimization issues. The CPU might optimize to already start execution of next function call and stop when some parameter value is needed that isn't set yet.
Therefore, stick to the convention and don't mess with invalidated stack pointers!