Adding more minor races
Moderator: thunderchero
Forum rules
Please search before starting new topic.
There is a good chance it has already been asked.
Please search before starting new topic.
There is a good chance it has already been asked.
- thunderchero
- Site Administrator aka Fleet Admiral
- Posts: 7962
- Joined: Fri Apr 25, 2008 2:00 am
- Location: On a three month training mission, in command of the USS Valiant.
Adding more minor races
Hi Everyone,
well I gave it an honest effort, here are the issues I encountered.
my first attempt was increasing to 60 minors (30 new), but structures bitmask had issues. so reduced to 58 minors (28 new) + 5 majors. this looks to be max.
stbof.res files edited;
race.rst added 28 new minors
racedesc.rst added 28 new descriptions
planet.pst added/edited 28 new planet names
aiminor.wtf added 28 new minor race attitudes
edifice.bst added 28 new minor structure (cloned current structures)
edifdesc.bst added 28 new structure descriptions (cloned current structures)
aibldreq.bin 28 new structures added for minor structures
racetech.tec 280 new entries added 10 per race
shiplist.sst edited to move monster race id (no new ships added)
bldset1-5.bin UE edited these file
starname.bin added with more star names
trek.exe changes
moved race names list at 0x1746e8 to relocation table
moved race name table at 0x189440 to relocation table
changed/tested over 1000 code locations for
23h (35) race list total
24h (36) start of monster ids
1Eh (30) total minors (not sure if this is even used)
results;
I never saw a single new minor on map but in diplomacy screen and summary I had 58 entries, but names were gibberish (see obstacle 2-3 for why).
4 major obstacles to over come I noticed so far. (I am sure there would be more)
1. sub section 48A02C meet race dialog screen. this has 3 major issues, increasing lexicon entries, increasing english.snd entries and expanding code area. but this sub section might be able to be no-op?
2. sub section 48AF20 Lexicon_GetRaceOrAlienName, main issue, expanding code area but lexicon entries could be shared?
3. sub section 48ACDC Lexicon_GetRaceName, main issue, expanding code area but lexicon entries could be shared?
4. sub section 4C22F8 minor special structure ids, only issue, expanding code area.
is it possible? yes but would require dll injection or other method to expand code areas.
thunderchero
well I gave it an honest effort, here are the issues I encountered.
my first attempt was increasing to 60 minors (30 new), but structures bitmask had issues. so reduced to 58 minors (28 new) + 5 majors. this looks to be max.
stbof.res files edited;
race.rst added 28 new minors
racedesc.rst added 28 new descriptions
planet.pst added/edited 28 new planet names
aiminor.wtf added 28 new minor race attitudes
edifice.bst added 28 new minor structure (cloned current structures)
edifdesc.bst added 28 new structure descriptions (cloned current structures)
aibldreq.bin 28 new structures added for minor structures
racetech.tec 280 new entries added 10 per race
shiplist.sst edited to move monster race id (no new ships added)
bldset1-5.bin UE edited these file
starname.bin added with more star names
trek.exe changes
moved race names list at 0x1746e8 to relocation table
moved race name table at 0x189440 to relocation table
changed/tested over 1000 code locations for
23h (35) race list total
24h (36) start of monster ids
1Eh (30) total minors (not sure if this is even used)
results;
I never saw a single new minor on map but in diplomacy screen and summary I had 58 entries, but names were gibberish (see obstacle 2-3 for why).
4 major obstacles to over come I noticed so far. (I am sure there would be more)
1. sub section 48A02C meet race dialog screen. this has 3 major issues, increasing lexicon entries, increasing english.snd entries and expanding code area. but this sub section might be able to be no-op?
2. sub section 48AF20 Lexicon_GetRaceOrAlienName, main issue, expanding code area but lexicon entries could be shared?
3. sub section 48ACDC Lexicon_GetRaceName, main issue, expanding code area but lexicon entries could be shared?
4. sub section 4C22F8 minor special structure ids, only issue, expanding code area.
is it possible? yes but would require dll injection or other method to expand code areas.
thunderchero
- thunderchero
- Site Administrator aka Fleet Admiral
- Posts: 7962
- Joined: Fri Apr 25, 2008 2:00 am
- Location: On a three month training mission, in command of the USS Valiant.
Re: Adding more minor races
Should this topic be re-visited now that we have code space?
Note; I still have all the old files from 4 years ago...
Note; I still have all the old files from 4 years ago...
- Flocke
- BORG Trouble Maker
- Posts: 3258
- Joined: Sun Apr 27, 2008 2:00 am
- Location: Hamburg, Germany
- Contact:
Re: Adding more minor races
Only in case you are happy to plague yourself with the countless code references you will have to battle.thunderchero wrote: ↑Wed May 19, 2021 9:18 pm Should this topic be re-visited now that we have code space?
Note; I still have all the old files from 4 years ago...
On the other hand, having asked that question, you probably can't stop yourself thinking of it without of having tried.
For the limit to keep in mind is that in many locations races are stored as bit flags, so multiple races can be stored.
E.g. in gameInfo there are 8bytes used for the flags including minor and major races, so max 64 races in total.
If you wanted to increase to even more races, all those locations had to be identified and patched for a larger data space.
- thunderchero
- Site Administrator aka Fleet Admiral
- Posts: 7962
- Joined: Fri Apr 25, 2008 2:00 am
- Location: On a three month training mission, in command of the USS Valiant.
Re: Adding more minor races
already noticed this issue lolFlocke wrote: ↑Thu May 20, 2021 11:50 am For the limit to keep in mind is that in many locations races are stored as bit flags, so multiple races can be stored.
E.g. in gameInfo there are 8bytes used for the flags including minor and major races, so max 64 races in total.
If you wanted to increase to even more races, all those location had to be identified and patched for a larger data space.
I was looking at old files and noticed the position I moved the race list to 4 years ago, it sits perfect 16 bytes after the minors on left data in new data area. what are the chances of that....thunderchero wrote: ↑Tue May 30, 2017 10:24 am my first attempt was increasing to 60 minors (30 new), but structures bitmask had issues. so reduced to 58 minors (28 new) + 5 majors. this looks to be max.
even current version of UE will display all my changes but throws exception in "Building stats"
but will still open and will even display all races in special building tab
but will only allow assigning first 30 minors since code does not exist.
- Spocks-cuddly-tribble
- Code Master
- Posts: 1955
- Joined: Sun Apr 27, 2008 2:00 am
Re: Adding more minor races
With your dedication it might be possible, but prepare for weeks/months of work.
As told, extending the 64-bit flags is possible but beyond madness (relocate, expand and change structure of many BotF dynamic core and saved-game files) i.e. 64-5=59 minors max for now (you said for buildings flags it's max 58 for some reason?).
Your '4 major obstacles' are actually just peanuts.
sub_48A02C MeetRaceDialog
sub_48ACDC GetRaceName
sub_48AF20 GetRaceOrAlienName
sub_4C22F8 Minor_race_blg_IDs
You don't need to relocate them. Just use easier code i.e. overwrite the large jump tables with index tables for two byte ID entries:
sub_48ACDC GetRaceName
0048ACDC cmp ax, 22h // +1 per new minor
at 0048AD04
mov ax, word[48AC50+2*eax]
nop
jmp Lexicon_Term
at [48AC50 +2*raceID] set TWO byte lexicon.dic entry numbers (up to 70 slots)
sub_48AF20 GetRaceOrAlienName
0048AF20 cmp ax, 2Bh // +1 per new minor
at 0048AF48
mov ax, word[48AE70+2*eax]
nop
jmp Lexicon_Term
at [48AE70+2*race] set TWO byte lexicon.dic entry numbers (up to 88 slots including monsters)
sub_4C22F8 Minor_race_blg_IDs
004C22FB cmp ax, 1Dh // +1 per new minor
at 004C230A
mov ax, word [4C2280+eax*2]
retn
at [4C2280+2*MinorIndex(MinorID-5)] set TWO byte building IDs (up to 60 slots)
or if you keep building IDs in order [X (default 38h) + (minor index)] just use:
004C230A add eax, X (first minor building ID)
retn
sub_48A02C MeetRaceDialog (sound files converse.bin english/german.snd I'd just set to a placeholder sound ID for all new minors)
eax = player empire ID
edx = meet race ID
0048A02C cmp dx, 2Ch // -> set to last/greatest minor race ID
at 48A036
cmp dl,5
jl skip next 3 instructions (goto xor eax,eax)
sub dl,4
imul edx,edx,5
jmp skip 2bytes (goto cmp al,4)
xor eax,eax
cmp al,4
ja 48A030
mov ax,[489D20+eax*2+edx*2]
jmp Lexicon_Term
at [489D20+2*MeetEmpireID] set TWO byte lexicon.dic entry numbers for empires
at [489D2A+10*MinorIndex(MinorID-5)+2*PlayerEmpireID] set TWO byte lexicon.dic entry numbers (5 per minor for player empires 0-4)
Place minors on map -> max number of random minors +5 (+1 per new minor race)
004B20D6 mov eax, 23h
004B1ED6 mov ebx, 23h
004B3FFD cmp bx, 23h
alienInfo & RToSInfo (maybe even more) -> extend and relocate to new dynamic area with static address (create new trek.exe dynamic code section)
All references in trek.exe and many patches (not only the base address! due to the static location some data is loaded directly e.g. 5A202C address alienTFInfo in alienInfo)
Note: You must understand the difference between dynamic data with fixed address and length (e.g. empsInfo) vs. files with dynamic address and size (e.g. systInfo)!
[systInfo+4Ch], 23h // system owner 23h = placeholder for all none empire systems
Most you don't want to change this placeholder 23h. Unless there is a specific reason for it.
Note: You must understand each 23h code location to decide this. You can't just change them by try and error!
Race IDs in ship files e.g.:
GTForceList = GWTForce +34h
GShipList+28h (2 byte word)
23h can be minor or monster checks
24h+ is most Borg & Monster checks
Agian you must understand each code location. You said you adapted all monster race IDs and checks? There is many in trek.exe and patches, some can cause combat crashes. Remember Edo God & Combat Drone are there even without random events.
After this there will be more issues....
As told, extending the 64-bit flags is possible but beyond madness (relocate, expand and change structure of many BotF dynamic core and saved-game files) i.e. 64-5=59 minors max for now (you said for buildings flags it's max 58 for some reason?).
Your '4 major obstacles' are actually just peanuts.
sub_48A02C MeetRaceDialog
sub_48ACDC GetRaceName
sub_48AF20 GetRaceOrAlienName
sub_4C22F8 Minor_race_blg_IDs
You don't need to relocate them. Just use easier code i.e. overwrite the large jump tables with index tables for two byte ID entries:
sub_48ACDC GetRaceName
0048ACDC cmp ax, 22h // +1 per new minor
at 0048AD04
mov ax, word[48AC50+2*eax]
nop
jmp Lexicon_Term
at [48AC50 +2*raceID] set TWO byte lexicon.dic entry numbers (up to 70 slots)
sub_48AF20 GetRaceOrAlienName
0048AF20 cmp ax, 2Bh // +1 per new minor
at 0048AF48
mov ax, word[48AE70+2*eax]
nop
jmp Lexicon_Term
at [48AE70+2*race] set TWO byte lexicon.dic entry numbers (up to 88 slots including monsters)
sub_4C22F8 Minor_race_blg_IDs
004C22FB cmp ax, 1Dh // +1 per new minor
at 004C230A
mov ax, word [4C2280+eax*2]
retn
at [4C2280+2*MinorIndex(MinorID-5)] set TWO byte building IDs (up to 60 slots)
or if you keep building IDs in order [X (default 38h) + (minor index)] just use:
004C230A add eax, X (first minor building ID)
retn
sub_48A02C MeetRaceDialog (sound files converse.bin english/german.snd I'd just set to a placeholder sound ID for all new minors)
eax = player empire ID
edx = meet race ID
0048A02C cmp dx, 2Ch // -> set to last/greatest minor race ID
at 48A036
cmp dl,5
jl skip next 3 instructions (goto xor eax,eax)
sub dl,4
imul edx,edx,5
jmp skip 2bytes (goto cmp al,4)
xor eax,eax
cmp al,4
ja 48A030
mov ax,[489D20+eax*2+edx*2]
jmp Lexicon_Term
at [489D20+2*MeetEmpireID] set TWO byte lexicon.dic entry numbers for empires
at [489D2A+10*MinorIndex(MinorID-5)+2*PlayerEmpireID] set TWO byte lexicon.dic entry numbers (5 per minor for player empires 0-4)
Place minors on map -> max number of random minors +5 (+1 per new minor race)
004B20D6 mov eax, 23h
004B1ED6 mov ebx, 23h
004B3FFD cmp bx, 23h
alienInfo & RToSInfo (maybe even more) -> extend and relocate to new dynamic area with static address (create new trek.exe dynamic code section)
All references in trek.exe and many patches (not only the base address! due to the static location some data is loaded directly e.g. 5A202C address alienTFInfo in alienInfo)
Note: You must understand the difference between dynamic data with fixed address and length (e.g. empsInfo) vs. files with dynamic address and size (e.g. systInfo)!
[systInfo+4Ch], 23h // system owner 23h = placeholder for all none empire systems
Most you don't want to change this placeholder 23h. Unless there is a specific reason for it.
Note: You must understand each 23h code location to decide this. You can't just change them by try and error!
Race IDs in ship files e.g.:
GTForceList = GWTForce +34h
GShipList+28h (2 byte word)
23h can be minor or monster checks
24h+ is most Borg & Monster checks
Agian you must understand each code location. You said you adapted all monster race IDs and checks? There is many in trek.exe and patches, some can cause combat crashes. Remember Edo God & Combat Drone are there even without random events.
After this there will be more issues....
I don't know how many bugs is too many but that point is reached somewhere before however many in BotF is.
- thunderchero
- Site Administrator aka Fleet Admiral
- Posts: 7962
- Joined: Fri Apr 25, 2008 2:00 am
- Location: On a three month training mission, in command of the USS Valiant.
Re: Adding more minor races
let me rephrase my statement "this was the highest/safest value I tested"Spocks-cuddly-tribble wrote: ↑Fri May 21, 2021 3:30 pm As told, extending the 64-bit flags is possible but beyond madness (relocate, expand and change structure of many BotF dynamic core and saved-game files) i.e. 64-5=59 minors max for now (you said for buildings flags it's max 58 for some reason?).
(i tested 65, 55, 63 total races) why I tested 63 instead of 64 I don't remember.
Just a quick look tonight and if I understand location of dynamic area correctly I found the "loaded_race_rst" would also be added to that list?Spocks-cuddly-tribble wrote: ↑Fri May 21, 2021 3:30 pm alienInfo & RToSInfo (maybe even more) -> extend and relocate to new dynamic area with static address (create new trek.exe dynamic code section)
All references in trek.exe and many patches (not only the base address! due to the static location some data is loaded directly e.g. 5A202C address alienTFInfo in alienInfo)
Note: You must understand the difference between dynamic data with fixed address and length (e.g. empsInfo) vs. files with dynamic address and size (e.g. systInfo)!
or if this area is increased it would throw off any offset to new code section?
just this alone I think will prevent me from any further attempts.
I would guess more in terms of years.....Spocks-cuddly-tribble wrote: ↑Fri May 21, 2021 3:30 pm With your dedication it might be possible, but prepare for weeks/months of work.
I am still amazed the GUI of the build queue is working.
- Flocke
- BORG Trouble Maker
- Posts: 3258
- Joined: Sun Apr 27, 2008 2:00 am
- Location: Hamburg, Germany
- Contact:
Re: Adding more minor races
I just noticed another thing to care, although that might already be known.
By the UE code, I always thought all the monsters use race id 36, but that's not the case.
In fact at least for GTForceList & GWTForce, they use the race ids 36 to 45, in the order of the UE ship stat entries.
By the UE code, I always thought all the monsters use race id 36, but that's not the case.
In fact at least for GTForceList & GWTForce, they use the race ids 36 to 45, in the order of the UE ship stat entries.
- thunderchero
- Site Administrator aka Fleet Admiral
- Posts: 7962
- Joined: Fri Apr 25, 2008 2:00 am
- Location: On a three month training mission, in command of the USS Valiant.
Re: Adding more minor races
yes I have used the Trek.java as a reference many time for code locations.
- Spocks-cuddly-tribble
- Code Master
- Posts: 1955
- Joined: Sun Apr 27, 2008 2:00 am
Re: Adding more minor races
No, you still don't get it. But to be fair many of my old labels are confusing. They are remains from a time when I still learned asm basics.thunderchero wrote: ↑Fri May 21, 2021 9:37 pmif I understand location of dynamic area correctly I found the "loaded_race_rst" would also be added to that list?
592BF8: race.rst should be adr_race.rst not loaded_race_rst (it's a pointer to a variable location/size file, not the base address itself).
Look in IDA at 592BF8. Next pointer reference is 592C04 planet_pst_load1.
0x592C04-0x592BF8=0xC (hardly enought for a loaded file....) EDIT: Plus the clue 'stbof.res file' most means dynamic position/size.
Fixed files are easier to read and use in patches, but very hard to extend depending on number of default references in source code.
Are you sure about the order? Click the small arrow link for more info:
Spocks-cuddly-tribble wrote: ↑Fri Apr 02, 2010 7:08 amDs: 5B2344 [GShipList]
Offset 0x0028 to 0x0029 = race-ID (even for monster unique, but not in shiplist.sst order !)
Also most mods will still miss some monster and max ship IDs: viewtopic.php?p=50815#p50815
I don't know how many bugs is too many but that point is reached somewhere before however many in BotF is.
- Flocke
- BORG Trouble Maker
- Posts: 3258
- Joined: Sun Apr 27, 2008 2:00 am
- Location: Hamburg, Germany
- Contact:
Re: Adding more minor races
Well it matches with the Chodak id of my save7.sav test save, but I didn't check other monsters.Spocks-cuddly-tribble wrote: ↑Sat May 22, 2021 11:41 amAre you sure about the order? Click the small arrow link for more info:Spocks-cuddly-tribble wrote: ↑Fri Apr 02, 2010 7:08 amDs: 5B2344 [GShipList]
Offset 0x0028 to 0x0029 = race-ID (even for monster unique, but not in shiplist.sst order !)
For now I fixed it by adding an out of bounds check to UE where it failed to determine the race name.