Turn processing patch (CLOSED)

General Modding Information/Questions; support/discussion/questions

Moderator: thunderchero

Forum rules
:idea: Please search before starting new topic. :idea:
There is a good chance it has already been asked.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7851
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Turn processing patch (CLOSED)

Post by thunderchero »

thunderchero wrote: Sat Mar 03, 2018 12:25 am Next I want to run an unedited from turn 1 to see how it will advance that way. but this will take much longer to do
well after 290 turns I stopped testing
final.jpg
final.jpg (455.22 KiB) Viewed 7644 times
As I feared reducing range had too much of a negative effect on AI

at turn 194 a minor found me (never happened in prior tests)
by turn 247 TPT was 2:44 sec (and kept increasing) over 3:30 after turn 290
by turn 290 all AI races were either at war or had some kind of treaty (all 4 race knew other 3 races)
turn 290 AI system count was about the same 25 - 35 systems each
turn 290 Federation was only a few turn away before I was going meet them.
turn 290 AI ships count was 250 - 700 each race

so in conclusion to editing range, if you don't mind crippling the AI this patch would decrease TPT greatly. But in my opinion it is not worth it, especially for UDML with the improved AI changes.

So we are still hoping for a direct fix to the problem (still unknown exact location)

code to close borders for non aggression, friendship, affiliation (this would be a usable change with less AI impact)
at 0x5e3bc change 0F 84 -> 4F 02
User avatar
mullet
Cadet 4th Year
Cadet 4th Year
Posts: 18
Joined: Sat Feb 10, 2018 10:30 pm

Re: Turn processing patch (CLOSED)

Post by mullet »

You know......... i actually started a game with reduced ranges. It makes some areas inaccessible without daisy chaining star-bases, but, only a few systems tend to be that impossible.

I personally didn't find the AI any more crippled than it usually is. It limits war strategy to: packing forces into narrow points of articulation, but, its still pretty ok to pass time.

I didn't use the range patch, but rather: cut the ranges in UE, which would be a great way to do a multiplayer. just decide what factions you and friend(s) want to play, and, cut the ranges of the others.

I guess in that scenario: if someone allied with an AI controlled faction, they could move inside your range bubble, and, you could get some long turns from that.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7851
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Turn processing patch (CLOSED)

Post by thunderchero »

crippled may have not been best term, from my tests without patch what they could do in 200 turns would now take 450 + turns with range patch. So mainly it is just delaying AI advancement. But AI will still be strong when you meet them with patch.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7851
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Turn processing patch (CLOSED)

Post by thunderchero »

Hi Everyone,

I came across an interesting sub-section AI_X_4189D0, at 0x17f4c 418B4C if you no-op the jump statement 75 89 -> 90 90 it will decrease TPT.
on my vanilla saved game 14 sec to 7 sec
on my UDML saved game 3:50 to 2:55

The interesting part is AI is still issuing ship movement orders as well as all other AI orders, :shock: but ship movement and orders changed when patched (see images below).

Since this saved game the AI has zero human interaction, I also tested consistency of AI movements.
The AI ship movements did not vary though 10 turns on 3 separate tests. (Un-patched or patched)
So AI tends to be consistent in its movements as long as human player does not interact. (I have always suspected this)

The jump statement I changed is what I call "a loop" if "x" is not meet it will run though same section of code again and again until "x" is meet.

Now I have no idea what "x" is in this sub-section. and it maybe only a small part of our over all issue. But I found it very interesting, since most of my other tests like this would CTD or kill AI or have no effect on TPT.

unpatched (after 10 turn from saved point)
un-patched.jpg
un-patched.jpg (290.04 KiB) Viewed 7602 times
patched (after 10 turn from saved point)
patched.jpg
patched.jpg (290.05 KiB) Viewed 7602 times
I noticed when patched AI built outposts? So did the no-op of jump statement change AI agenda?

Edit; this post was only an observation, and not intended as a patch.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7851
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Turn processing patch (CLOSED)

Post by thunderchero »

I tested no-op above on new game, it was interesting....

this was a quick test of 200 turn no action by me as human player.

I observed the AI;
build ships
build structures
explore
declare war
member minors
use intel
increase research
use diplomacy

AI did not;
assault systems
colonize systems
and strangely enough did not build outposts? (editing mid game there must have been a demand for them?)

I could not confirm but suspect no AI vs AI combat, but I could start combat with a close by minor.
User avatar
Martok
Rear-Admiral
Rear-Admiral
Posts: 1208
Joined: Thu May 01, 2008 2:00 am

Re: Turn processing patch (CLOSED)

Post by Martok »

Odd.
"Evil is easy, and has infinite forms." -- Pascal
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7851
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Turn processing patch (CLOSED)

Post by thunderchero »

Martok wrote: Wed Mar 14, 2018 7:16 pmOdd.
not at all, when you no-op jump statements strange/different behavior is expected.

I was doing this in an attempt to narrow down code location of long turn processing time. it was never meant as a fix.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7851
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Turn processing patch (CLOSED)

Post by thunderchero »

Hi Everyone,

This is a question to the coders out there,

I still think Smartheap is limiting memory and from looking at ida database I still see sections for System_Memory_* but most are not used.
I always suspected that smartheap was added during beta testing after beta testers systems kept running out of memory. I just hope they left enough code to reset back to use system memory.
examples;
System_Memory_Free 00492E60
System_Memory_Realloc 00492E90
System_Memory_AllocClear 00492EA0
System_Memory_AllocFromNetworkPool 00492ED0
System_Memory_AllocFromUIPool 00492EF0
System_Memory_AllocFromTacticalPool 00492F10
System_Memory_AllocFromAIPool 00492F30
System_Memory_AllocFromDBPool 00492F50
System_Memory_AllocFromSoundPool 00492F70
System_Memory_AllocFromStrategicPool 00492F90
System_Memory_FreeAIPool 00492FB0
System_Memory_InitAIPool 00492FE0
System_Memory_FreeDBPool 00493020
System_Memory_InitDBPool 00493050
System_Memory_FreeUIPool 00493090
System_Memory_InitUIPool 004930C0
System_Memory_FreeStrategicPool 00493100
System_Memory_InitStrategicPool 00493130
System_Memory_FreeSoundPool 00493170
System_Memory_InitSoundPool 004931A0
System_Memory_FreeNetworkPool 004931E0
System_Memory_InitNetworkPool 00493210
System_Memory_FreeTacticalPool 00493250
System_Memory_InitTacticalPool 00493280
System_Memory_FreeMemory 004932C0
System_Memory_InitializeMemory 00493310
System_Memory_ErrorHandler 004933D0
System_Memory_GetErrorString 004933E0

so my question is, could calls for smartheap be switched back to use system memory? Then smartheap would be ignored.

BTW, just for giggles I sent an email to MicroQuill creator of smartheap and they said only way to fix issue would be with source code and update to latest version of smartheap. (that did not help...) I hope we can prove them wrong.
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3197
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: Turn processing patch (CLOSED)

Post by Flocke »

Given we know the interface, I'm sure it can be changed. QuasarDonkey already prooved it can be changed. But it'll be some work with unknown outcome.
From what I have read SmartHeap often was used for debugging the memory usage to e.g. figure memory leaks, but it doesn't fix them by itself.
So likely they switched to SmartHeap to figure the issues they had but limited the memory usage in some upper layer.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7851
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Turn processing patch (CLOSED)

Post by thunderchero »

While I am very limited in my ability to read assembly code, but I swear I see duplicate code sections that QD has marked as (Void) that look vary similar as the code for ship movement that use smartheap?

Also with all the hidden/broken features that have been found it makes me ask, were they hidden/broken or edited/omitted to decrease memory usage?
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7851
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Turn processing patch (CLOSED)

Post by thunderchero »

something I just noticed every time I run debug with ida database I get this message

Debugged application message: Invalid address specified to RtlGetUserInfoHeap( 02A50000, 7FEA3000 )

I tried to do a web search and did not find much other than it might be a call from ntdll.dll

any ideas? or is it just nothing. but every time I see "heap" I think memory.

I also notice this on "total memory allocated" (on first load of game and first thing log writes)

bytes pages size description
--------- ----- ---- --------------------------------------------
14393344 1757 8192 allocating memory for b-tree...
10936320 1335 8192 allocating memory for virtual array...
262144 32 8192 allocating memory for name pointers...
-----------------------------------------------------------------
25591808 total memory allocated

b-tree? saved file?
virtual array? stbof.res files?
name pointers? loaded races?

that is only 25.591808 mb lol

Edit; I did a search for 00 20 00 00 (8192 size) and found 6 locations in memory sub-sections and changed to 00 80 00 00

and I got a message I have never seen before;
out.jpg
out.jpg (30.63 KiB) Viewed 7430 times
funny thing is look at title "Smartheap library"

retry or ignore buttons would not allow to continue and abort closed app.

Note; I have 16Gb of memory and since I only increased x4 total memory allocated should have been 102.367232 so I had plenty......

563ADD 0x162edd
563AD5 0x162ed5
55575A 0x154b5a
549F5F 0x14935f
549E52 0x149252
54524E 0x14464e
User avatar
mullet
Cadet 4th Year
Cadet 4th Year
Posts: 18
Joined: Sat Feb 10, 2018 10:30 pm

Re: Turn processing patch (CLOSED)

Post by mullet »

your way beyond my pay grade now. Good luck to you sir.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7851
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Turn processing patch (CLOSED)

Post by thunderchero »

mullet wrote: Tue Mar 20, 2018 10:08 pm your way beyond my pay grade now. Good luck to you sir.
mine also...
but just as a note; I tend to edit posts like the one above.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7851
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Turn processing patch (CLOSED)

Post by thunderchero »

Hi Everyone,

I have an observation and a question for everyone.

first, I noticed in the 1.0.2 "update" readme.doc

in "Version 1.0.1 Changes" section this statement;

"Turn processing time has been improved."

it make me wonder if that improvement could be increased, and how was it improved?

so my question, anyone have an original 1.0.0 version cd? lowest I have is version 1.0.1
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7851
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Turn processing patch (CLOSED)

Post by thunderchero »

Hi Everyone,

I did some profiling with Code XL of the vanilla saved game I am currently using to see hotspots (called most/used most time).

Code: Select all

Function	Module	Timer
trek.exe!0x47aedb	trek.exe	1040 * part of no-op sub-section Game_Map_PathFind_47AE40
trek.exe!0x47ae10	trek.exe	813 * part of no-op sub-section Game_Map_PathFind_47AE40
trek.exe!0x51b02a	trek.exe	788 * part of gHeap_SortRange
trek.exe!0x51b044	trek.exe	734 * part of gHeap_SortRange
trek.exe!0x449c00	trek.exe	481 * called high below when no-op Game_Ship_ShipCompareFunc
trek.exe!0x47ae19	trek.exe	441 * part of no-op sub-section Game_Map_PathFind_47AE40
trek.exe!0x51b033	trek.exe	422 * part of gHeap_SortRange
trek.exe!0x51b04d	trek.exe	342 * part of gHeap_SortRange
trek.exe!0x51b00f	trek.exe	313 * part of gHeap_SortRange
trek.exe!0x47ae11	trek.exe	283 * part of no-op sub-section Game_Map_PathFind_47AE40
trek.exe!0x51b018	trek.exe	282 * part of gHeap_SortRange
trek.exe!0x51abc2	trek.exe	250 * part of gHeap_PopMax
trek.exe!0x47ae22	trek.exe	206 * part of no-op sub-section Game_Map_PathFind_47AE40
trek.exe!0x47a5ac	trek.exe	190 * called 3 times by no-op  Game_Map_GridMap_GetEntry
trek.exe!0x47aef8	trek.exe	186 * part of no-op sub-section Game_Map_PathFind_47AE40
trek.exe!0x449c03	trek.exe	184 * called high below when no-op Game_Ship_ShipCompareFunc
trek.exe!0x51abe3	trek.exe	173 * part of gHeap_PopMax
trek.exe!0x47af0c	trek.exe	172 * part of no-op sub-section Game_Map_PathFind_47AE40
trek.exe!0x47ae31	trek.exe	143 * part of no-op sub-section Game_Map_PathFind_47AE40
trek.exe!0x51ae87	trek.exe	139 * part of gHeap_Push
trek.exe!0x47a5a9	trek.exe	108 * part of no-op sub-section Game_Map_PathFind_47AE40
trek.exe!0x51ae90	trek.exe	108 * part of gHeap_Push
trek.exe!0x47aeff	trek.exe	100 * part of no-op sub-section Game_Map_PathFind_47AE40

with 0x7a2ba 47AEBA no-op tpt 2 sec

Function	Module	Timer
trek.exe!0x449c00	trek.exe	538 * hotspot above Game_Ship_ShipCompareFunc
trek.exe!0x449c03	trek.exe	100 * hotspot above Game_Ship_ShipCompareFunc
trek.exe!0x47ae10	trek.exe	62 * hotspot above sub-section Game_Map_PathFind_47AE40
trek.exe!0x448d00	trek.exe	47
trek.exe!0x402432	trek.exe	46
trek.exe!0x54ef71	trek.exe	44
trek.exe!0x51a81b	trek.exe	42
trek.exe!0x47aedb	trek.exe	39
trek.exe!0x51a826	trek.exe	39
trek.exe!0x51d6d9	trek.exe	37
trek.exe!0x51b044	trek.exe	36
trek.exe!0x402433	trek.exe	35
most of the main hotspots were in same sub-section or called by that subsection or part of gHeap calls

this is info only, But I found it interesting

thunderchero
Post Reply

Return to “General Modding Information/Questions”