Adding more minor races

General Modding Information/Questions; support/discussion/questions

Moderator: thunderchero

Forum rules
:idea: Please search before starting new topic. :idea:
There is a good chance it has already been asked.
Post Reply
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7849
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Adding more minor races

Post by thunderchero »

Hi Everyone,

well I gave it an honest effort, here are the issues I encountered.

my first attempt was increasing to 60 minors (30 new), but structures bitmask had issues. so reduced to 58 minors (28 new) + 5 majors. this looks to be max.

stbof.res files edited;

race.rst added 28 new minors
racedesc.rst added 28 new descriptions
planet.pst added/edited 28 new planet names
aiminor.wtf added 28 new minor race attitudes
edifice.bst added 28 new minor structure (cloned current structures)
edifdesc.bst added 28 new structure descriptions (cloned current structures)
aibldreq.bin 28 new structures added for minor structures
racetech.tec 280 new entries added 10 per race
shiplist.sst edited to move monster race id (no new ships added)
bldset1-5.bin UE edited these file
starname.bin added with more star names

trek.exe changes
moved race names list at 0x1746e8 to relocation table
moved race name table at 0x189440 to relocation table
changed/tested over 1000 code locations for
23h (35) race list total
24h (36) start of monster ids
1Eh (30) total minors (not sure if this is even used)

results;
I never saw a single new minor on map :mad: but in diplomacy screen and summary I had 58 entries, but names were gibberish (see obstacle 2-3 for why).

4 major obstacles to over come I noticed so far. (I am sure there would be more)

1. sub section 48A02C meet race dialog screen. this has 3 major issues, increasing lexicon entries, increasing english.snd entries and expanding code area. but this sub section might be able to be no-op?

2. sub section 48AF20 Lexicon_GetRaceOrAlienName, main issue, expanding code area but lexicon entries could be shared?

3. sub section 48ACDC Lexicon_GetRaceName, main issue, expanding code area but lexicon entries could be shared?

4. sub section 4C22F8 minor special structure ids, only issue, expanding code area.

is it possible? yes but would require dll injection or other method to expand code areas.

thunderchero
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7849
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Adding more minor races

Post by thunderchero »

Should this topic be re-visited now that we have code space?

Note; I still have all the old files from 4 years ago... :shock:
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3197
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: Adding more minor races

Post by Flocke »

thunderchero wrote: Wed May 19, 2021 9:18 pm Should this topic be re-visited now that we have code space?

Note; I still have all the old files from 4 years ago... :shock:
Only in case you are happy to plague yourself with the countless code references you will have to battle.
On the other hand, having asked that question, you probably can't stop yourself thinking of it without of having tried. :mrgreen:

For the limit to keep in mind is that in many locations races are stored as bit flags, so multiple races can be stored.
E.g. in gameInfo there are 8bytes used for the flags including minor and major races, so max 64 races in total.
If you wanted to increase to even more races, all those locations had to be identified and patched for a larger data space.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7849
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Adding more minor races

Post by thunderchero »

Flocke wrote: Thu May 20, 2021 11:50 am For the limit to keep in mind is that in many locations races are stored as bit flags, so multiple races can be stored.
E.g. in gameInfo there are 8bytes used for the flags including minor and major races, so max 64 races in total.
If you wanted to increase to even more races, all those location had to be identified and patched for a larger data space.
already noticed this issue lol
thunderchero wrote: Tue May 30, 2017 10:24 am my first attempt was increasing to 60 minors (30 new), but structures bitmask had issues. so reduced to 58 minors (28 new) + 5 majors. this looks to be max.
I was looking at old files and noticed the position I moved the race list to 4 years ago, it sits perfect 16 bytes after the minors on left data in new data area. what are the chances of that....

even current version of UE will display all my changes but throws exception in "Building stats"
error.jpg
error.jpg (23.25 KiB) Viewed 2547 times
but will still open and will even display all races in special building tab
special.jpg
special.jpg (197.24 KiB) Viewed 2547 times
but will only allow assigning first 30 minors since code does not exist.
User avatar
Spocks-cuddly-tribble
Code Master
Code Master
Posts: 1883
Joined: Sun Apr 27, 2008 2:00 am

Re: Adding more minor races

Post by Spocks-cuddly-tribble »

With your dedication it might be possible, but prepare for weeks/months of work.


As told, extending the 64-bit flags is possible but beyond madness (relocate, expand and change structure of many BotF dynamic core and saved-game files) i.e. 64-5=59 minors max for now (you said for buildings flags it's max 58 for some reason?).



Your '4 major obstacles' are actually just peanuts.

sub_48A02C MeetRaceDialog
sub_48ACDC GetRaceName
sub_48AF20 GetRaceOrAlienName
sub_4C22F8 Minor_race_blg_IDs

You don't need to relocate them. Just use easier code i.e. overwrite the large jump tables with index tables for two byte ID entries:


sub_48ACDC GetRaceName

0048ACDC cmp ax, 22h // +1 per new minor

at 0048AD04
mov ax, word[48AC50+2*eax]
nop
jmp Lexicon_Term

at [48AC50 +2*raceID] set TWO byte lexicon.dic entry numbers (up to 70 slots)


sub_48AF20 GetRaceOrAlienName

0048AF20 cmp ax, 2Bh // +1 per new minor

at 0048AF48
mov ax, word[48AE70+2*eax]
nop
jmp Lexicon_Term

at [48AE70+2*race] set TWO byte lexicon.dic entry numbers (up to 88 slots including monsters)


sub_4C22F8 Minor_race_blg_IDs

004C22FB cmp ax, 1Dh // +1 per new minor

at 004C230A
mov ax, word [4C2280+eax*2]
retn

at [4C2280+2*MinorIndex(MinorID-5)] set TWO byte building IDs (up to 60 slots)

or if you keep building IDs in order [X (default 38h) + (minor index)] just use:

004C230A add eax, X (first minor building ID)
retn


sub_48A02C MeetRaceDialog (sound files converse.bin english/german.snd I'd just set to a placeholder sound ID for all new minors)

eax = player empire ID
edx = meet race ID

0048A02C cmp dx, 2Ch // -> set to last/greatest minor race ID

at 48A036
cmp dl,5
jl skip next 3 instructions (goto xor eax,eax)
sub dl,4
imul edx,edx,5
jmp skip 2bytes (goto cmp al,4)
xor eax,eax
cmp al,4
ja 48A030
mov ax,[489D20+eax*2+edx*2]
jmp Lexicon_Term

at [489D20+2*MeetEmpireID] set TWO byte lexicon.dic entry numbers for empires
at [489D2A+10*MinorIndex(MinorID-5)+2*PlayerEmpireID] set TWO byte lexicon.dic entry numbers (5 per minor for player empires 0-4)




Place minors on map -> max number of random minors +5 (+1 per new minor race)

004B20D6 mov eax, 23h

004B1ED6 mov ebx, 23h

004B3FFD cmp bx, 23h



alienInfo & RToSInfo (maybe even more) -> extend and relocate to new dynamic area with static address (create new trek.exe dynamic code section)

All references in trek.exe and many patches (not only the base address! due to the static location some data is loaded directly e.g. 5A202C address alienTFInfo in alienInfo)

Note: You must understand the difference between dynamic data with fixed address and length (e.g. empsInfo) vs. files with dynamic address and size (e.g. systInfo)!



[systInfo+4Ch], 23h // system owner 23h = placeholder for all none empire systems

Most you don't want to change this placeholder 23h. Unless there is a specific reason for it.

Note: You must understand each 23h code location to decide this. You can't just change them by try and error!



Race IDs in ship files e.g.:

GTForceList = GWTForce +34h
GShipList+28h (2 byte word)

23h can be minor or monster checks
24h+ is most Borg & Monster checks

Agian you must understand each code location. You said you adapted all monster race IDs and checks? There is many in trek.exe and patches, some can cause combat crashes. Remember Edo God & Combat Drone are there even without random events.



After this there will be more issues....
I don't know how many bugs is too many but that point is reached somewhere before however many in BotF is.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7849
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Adding more minor races

Post by thunderchero »

Spocks-cuddly-tribble wrote: Fri May 21, 2021 3:30 pm As told, extending the 64-bit flags is possible but beyond madness (relocate, expand and change structure of many BotF dynamic core and saved-game files) i.e. 64-5=59 minors max for now (you said for buildings flags it's max 58 for some reason?).
let me rephrase my statement "this was the highest/safest value I tested"
(i tested 65, 55, 63 total races) why I tested 63 instead of 64 I don't remember.
Spocks-cuddly-tribble wrote: Fri May 21, 2021 3:30 pm alienInfo & RToSInfo (maybe even more) -> extend and relocate to new dynamic area with static address (create new trek.exe dynamic code section)

All references in trek.exe and many patches (not only the base address! due to the static location some data is loaded directly e.g. 5A202C address alienTFInfo in alienInfo)

Note: You must understand the difference between dynamic data with fixed address and length (e.g. empsInfo) vs. files with dynamic address and size (e.g. systInfo)!
Just a quick look tonight and if I understand location of dynamic area correctly I found the "loaded_race_rst" would also be added to that list?
or if this area is increased it would throw off any offset to new code section?

just this alone I think will prevent me from any further attempts.
Spocks-cuddly-tribble wrote: Fri May 21, 2021 3:30 pm With your dedication it might be possible, but prepare for weeks/months of work.
I would guess more in terms of years.....

I am still amazed the GUI of the build queue is working.
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3197
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: Adding more minor races

Post by Flocke »

I just noticed another thing to care, although that might already be known.
By the UE code, I always thought all the monsters use race id 36, but that's not the case.
In fact at least for GTForceList & GWTForce, they use the race ids 36 to 45, in the order of the UE ship stat entries. :roll:
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7849
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Adding more minor races

Post by thunderchero »

yes I have used the Trek.java as a reference many time for code locations.
User avatar
Spocks-cuddly-tribble
Code Master
Code Master
Posts: 1883
Joined: Sun Apr 27, 2008 2:00 am

Re: Adding more minor races

Post by Spocks-cuddly-tribble »

thunderchero wrote: Fri May 21, 2021 9:37 pmif I understand location of dynamic area correctly I found the "loaded_race_rst" would also be added to that list?
No, you still don't get it. But to be fair many of my old labels are confusing. They are remains from a time when I still learned asm basics.

592BF8: race.rst should be adr_race.rst not loaded_race_rst (it's a pointer to a variable location/size file, not the base address itself).

Look in IDA at 592BF8. Next pointer reference is 592C04 planet_pst_load1.

0x592C04-0x592BF8=0xC (hardly enought for a loaded file....) EDIT: Plus the clue 'stbof.res file' most means dynamic position/size.

Fixed files are easier to read and use in patches, but very hard to extend depending on number of default references in source code.

Flocke wrote: Sat May 22, 2021 6:08 amfor GTForceList & GWTForce, they use the race ids 36 to 45, in the order of the UE ship stat entries.
Are you sure about the order? Click the small arrow link for more info:
Spocks-cuddly-tribble wrote: Fri Apr 02, 2010 7:08 amDs: 5B2344 [GShipList]
Offset 0x0028 to 0x0029 = race-ID (even for monster unique, but not in shiplist.sst order !)

Also most mods will still miss some monster and max ship IDs: viewtopic.php?p=50815#p50815
I don't know how many bugs is too many but that point is reached somewhere before however many in BotF is.
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3197
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: Adding more minor races

Post by Flocke »

Spocks-cuddly-tribble wrote: Sat May 22, 2021 11:41 am
Flocke wrote: Sat May 22, 2021 6:08 amfor GTForceList & GWTForce, they use the race ids 36 to 45, in the order of the UE ship stat entries.
Are you sure about the order? Click the small arrow link for more info:
Spocks-cuddly-tribble wrote: Fri Apr 02, 2010 7:08 amDs: 5B2344 [GShipList]
Offset 0x0028 to 0x0029 = race-ID (even for monster unique, but not in shiplist.sst order !)
Well it matches with the Chodak id of my save7.sav test save, but I didn't check other monsters. :shock:
For now I fixed it by adding an out of bounds check to UE where it failed to determine the race name.
Post Reply

Return to “General Modding Information/Questions”