Have long-turn-loading times been fixed?

Important info on site change.

Moderator: thunderchero

User avatar
Firefly
Ensign
Ensign
Posts: 44
Joined: Mon Jan 15, 2018 1:11 pm

Have long-turn-loading times been fixed?

Post by Firefly »

Haven´t play Birth or any mods in a while...

Did you guys fix the bug that makes the memory leak and make turns take longer and longer in mid/late game?

Heared it might be hardcoded and unfixable... but its been a few years and you guys are still active, is there a solution by now?

OR is there a mod or Birth clone, that does not have that problem?

thx.
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7848
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Have long-turn-loading times been fixed?

Post by thunderchero »

sorry no fix for long turn processing time, It is not a memory leak. it is more of a poorly coded AI.

it does help to play small maps.

as for clone, you can check out Supremacy.
User avatar
Firefly
Ensign
Ensign
Posts: 44
Joined: Mon Jan 15, 2018 1:11 pm

Re: Have long-turn-loading times been fixed?

Post by Firefly »

thx for the quick replay. i will check out supremacy
User avatar
Firefly
Ensign
Ensign
Posts: 44
Joined: Mon Jan 15, 2018 1:11 pm

Re: Have long-turn-loading times been fixed?

Post by Firefly »

Last update is from 2008 and it says its just a sneak peak...

is it still under construction? An active project?
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7848
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Have long-turn-loading times been fixed?

Post by thunderchero »

files were last updated right before Christmas

viewtopic.php?f=300&t=3068#p40799
User avatar
Firefly
Ensign
Ensign
Posts: 44
Joined: Mon Jan 15, 2018 1:11 pm

Re: Have long-turn-loading times been fixed?

Post by Firefly »

cool. hope the project will be completed.

Without the long-turn bug, it would be the better birth.
User avatar
mullet
Cadet 4th Year
Cadet 4th Year
Posts: 18
Joined: Sat Feb 10, 2018 10:30 pm

Re: Have long-turn-loading times been fixed?

Post by mullet »

I picked up playing BOTF after a long while, and, i ; of course, remembered the famous turn lag issue, but, with all the time passed, i fooled myself into thinking i could just suck it up and cope with waiting for turns to pass.....

What makes it so frustrating, and, the reason early attempts at diagnosing / combating were focused on a "memory leak" is:

In the first few turns, the turn processing time is virtually instant. If you blink: you would miss the transition entirely. Its a fraction of a second.

Then you get a few turns that are noticeably long , but, tolerable, and, then: it evolves into unplayable.

Well, its definitely not a memory leak, which is unfortunate, because: there are workarounds that could control its effects if it was.

As i read from a windows support site: "the application fails to release memory when no longer needed. " "Windows will free all memory allocated by the application on process termination" This means that: if nothing else: closing the game and reloading the save would clear it up for a bit. Any turn time reduction caused by doing this is, negligible to non existent.

What i read a lot in researching any hope of solution is things like: "smaller maps ... fewer units" , and, this is true, as well as a vague hint of where this scourge of BOTF is stemming from, but, it saddens me to see those comments because:

There are people who put a lot of time, love, and, effort into mods that allow for larger maps and more units. There are a few mods with large variants, and, it really sucks that these great improvements are pretty much rendered moot, because, no one will be able to finish a full game on large map on large mods.

While i'm ranting: This game is practically 20 years old. Even if your playing on what today would be considered an older or slower machine, its something that far and wide absolutely blows away anything available in 1999. What I've noticed from reading all the old threads on this issue: more powerful computers don't equal to faster turn processing.

You could be using duel Xeon E7s and, still have 3 minute turn wait after 300 turns; the same result as a Celeron D single core. What an insidious bug!

My computer meets the spec requirements to run GTA5; a game that would be science fiction by 1999 standards, and, it can't finish a single turn in this 2D map strategy game in under two minutes!!!!!

Its so frustrating, because: the typical issue with a PC game is something related directly to the visuals/animations. Either: a newer game challenging the computers capability, or, an older game having issues from being designed for a lesser color palette or resolution.

thundercho- " poorly coded AI. " This kind of makes more sense, in a way. I imagine a computer players behavior in a game like this would be dictated by directives along the lines of: " do *action* if *condition* unless *additional condition* then do *other action* ....." Its turn based, so: the four computers are doing their turns after yours in an order.....

so: the AI runs one player turn, and, swaps to the next player, and, its making "decisions" based on the conditions of its own created circumstance...aaand,..... its creating circumstances of going back and re-evaluating the rigid directive against the conditions.

Has anyone ever tried a five way multiplayer of an XL map with several hundred turns in a LAN party situation? If so: how much if any turn lag then?

Wouldn't such a ritual of the AI creating and resolving its own paradoxes wrap up faster on a machine with faster processing than anyone ever thought would be available for it?......or.......

Does the game limit its system resource use inherently? If so: is it possible to raise that cap?
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7848
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Have long-turn-loading times been fixed?

Post by thunderchero »

As far as I know no one has attempted a 5 player MP game to avoid AI lag (but should work). but firefly just reported single player game human vs single AI at turn 150 with only a 5 sec turn processing time.

over the years and testing this issue, I have used code to stop AI calculations completely and turn processing was instantaneous. One other code change was to move AI turn processing to the beginning of next turn, this also made turn time instantaneous, but what was funny was you could see AI test each ship and each system like, "should I move here" or "should I move here" and it would do this for every sector it could see for every ship one by one. This was interesting to watch, but threw combat out of sync and would cause game to crash on next turn.

Edit;
I have no proof of this, but I always suspected the original version given to beta testers had an issue with "running out of memory" late in the game. So the developers used "smartheap" to dump memory that it would normally use to increase turn processing.

So I concluded the AI needs to read info over and over to keep memory usage below 16 MB due to smartheap
User avatar
mullet
Cadet 4th Year
Cadet 4th Year
Posts: 18
Joined: Sat Feb 10, 2018 10:30 pm

Re: Have long-turn-loading times been fixed?

Post by mullet »

Hmmmmmmm...... "So I concluded the AI needs to read info over and over to keep memory usage below 16 MB due to smartheap"

It sounds like your really close on this actually. This would explain why higher spec computers have no effect.

so, if there was a way to let it access more RAM, that would improve and/or solve the issue?

Making the AI "smarter" is one thing, but, this seems like the key to finally tackling this problem.

I know people on here have been off and on trying to correct turn lag for almost a decade. Does anybody even know where this "smartheap" restriction is located?
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7848
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Have long-turn-loading times been fixed?

Post by thunderchero »

I hope it is not smartheap, since it is part of every sub-section that deals with memory.

The main reason I suspect it is smartheap, a couple years ago I contacted one of the original beta testers and he told me during first testing, they would run out of memory all the time. but I have never heard of anyone ever running out of memory.
User avatar
mullet
Cadet 4th Year
Cadet 4th Year
Posts: 18
Joined: Sat Feb 10, 2018 10:30 pm

Re: Have long-turn-loading times been fixed?

Post by mullet »

I looked up "smartheap" and, what i guess is: there was / is a big memory leak, and, it was the easiest way to deal with it.

Has any one tried running smart heap on BOTF, and, changing its setting? or, is it a one way street thing, and, i could only reduce the RAM cap lower?

OMFG , maybe not, since: they are asking ridiculous prices for a program that has not been updated since 07....
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3196
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: Have long-turn-loading times been fixed?

Post by Flocke »

if it's so slow cause it needs to read back the data from disk all the time, then installing to some virtual ram disk should do the trick.
try e.g. http://www.freewarefiles.com/SoftPerfec ... 94028.html ( last free version ) or https://www.starwindsoftware.com/high-p ... k-emulator and see what effect it has

if it's really due to smartheap and all smartheap does is wrap the "malloc/operator new" memory allocation, then it should not be too complicate to replace
never looked at that part of the code yet

edit: in System_Memory_InitializeMemory at asm 0x49333C / trek.exe offset 0x9273C the memory looks to be initialized to 10MB ( B8 00 00 A0 00 ), did anyone ever try to increase that value? say 100 MB = B8 00 00 40 06 or even 1GB = B8 00 00 00 40?
edit6: hm, reading up the documentation a little more as well as the ida asm code, I guess this is rather a check for whether 10MB could be allocated or fail. There doesn't seem to be a fixed smartheap memory limit

edit2: further there are the different pool allocations like System_Memory_InitAIPool calling MemPoolInit all with a fixed size of 32770 = 32KB (+2Byte) it seems. Might be a good idea to increase these ones as well.
System_Memory_InitAIPool: asm 0x492FE3 / trek.exe offset 0x923E3 try change 68 02 80 00 00 to e.g. 68 00 00 A0 00, the 10MB from above
System_Memory_InitDBPool: asm 0x493053 / trek.exe offset 0x92453 try change 68 02 80 00 00 to e.g. 68 00 00 A0 00
System_Memory_InitUIPool: asm 0x4930C3 / trek.exe offset 0x924C3 try change 68 02 80 00 00 to e.g. 68 00 00 A0 00
System_Memory_InitStrategicPool: asm 0x493133 / trek.exe offset 0x92533 try change 68 02 80 00 00 to e.g. 68 00 00 A0 00
System_Memory_InitSoundPool: asm 0x4931A3 / trek.exe offset 0x925A3 try change 68 02 80 00 00 to e.g. 68 00 00 A0 00
System_Memory_InitNetworkPool: asm 0x493213 / trek.exe offset 0x92613 try change 68 02 80 00 00 to e.g. 68 00 00 A0 00
System_Memory_InitTacticalPool: asm 0x493283 / trek.exe offset 0x92683 try change 68 02 80 00 00 to e.g. 68 00 00 A0 00

edit3: well, looking at MemInitDefaultPoo that seems to be flags not the pool size
edit4: also refer the api docs at http://www.microquill.com/kb/01_intro.html / http://www.microquill.com/kb/04_api.html#memalloc
edit5: a quick websearch reveals the flag values at https://github.com/eezstreet/JediKnight ... SMRTHEAP.H and http://www.codelooker.com/codea/1121054 ... EAP.H.html in whatever projects that may be..
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7848
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Have long-turn-loading times been fixed?

Post by thunderchero »

:idea: I am not going to pretend I understand any of this.. but for reference don't forget about falcon source code has smartheap files.
User avatar
Lathon
Lieutenant-Commander
Lieutenant-Commander
Posts: 116
Joined: Thu Oct 15, 2009 2:00 am

Re: Have long-turn-loading times been fixed?

Post by Lathon »

Flocke wrote: Tue Feb 13, 2018 4:37 am if it's so slow cause it needs to read back the data from disk all the time, then installing to some virtual ram disk should do the trick.
try e.g. http://www.freewarefiles.com/SoftPerfec ... 94028.html ( last free version ) or https://www.starwindsoftware.com/high-p ... k-emulator and see what effect it has

if it's really due to smartheap and all smartheap does is wrap the "malloc/operator new" memory allocation, then it should not be too complicate to replace
never looked at that part of the code yet

edit: in System_Memory_InitializeMemory at asm 0x49333C / trek.exe offset 0x9273C the memory looks to be initialized to 10MB ( B8 00 00 A0 00 ), did anyone ever try to increase that value? say 100 MB = B8 00 00 40 06 or even 1GB = B8 00 00 00 40?
edit6: hm, reading up the documentation a little more as well as the ida asm code, I guess this is rather a check for whether 10MB could be allocated or fail. There doesn't seem to be a fixed smartheap memory limit

edit2: further there are the different pool allocations like System_Memory_InitAIPool calling MemPoolInit all with a fixed size of 32770 = 32KB (+2Byte) it seems. Might be a good idea to increase these ones as well.
System_Memory_InitAIPool: asm 0x492FE3 / trek.exe offset 0x923E3 try change 68 02 80 00 00 to e.g. 68 00 00 A0 00, the 10MB from above
System_Memory_InitDBPool: asm 0x493053 / trek.exe offset 0x92453 try change 68 02 80 00 00 to e.g. 68 00 00 A0 00
System_Memory_InitUIPool: asm 0x4930C3 / trek.exe offset 0x924C3 try change 68 02 80 00 00 to e.g. 68 00 00 A0 00
System_Memory_InitStrategicPool: asm 0x493133 / trek.exe offset 0x92533 try change 68 02 80 00 00 to e.g. 68 00 00 A0 00
System_Memory_InitSoundPool: asm 0x4931A3 / trek.exe offset 0x925A3 try change 68 02 80 00 00 to e.g. 68 00 00 A0 00
System_Memory_InitNetworkPool: asm 0x493213 / trek.exe offset 0x92613 try change 68 02 80 00 00 to e.g. 68 00 00 A0 00
System_Memory_InitTacticalPool: asm 0x493283 / trek.exe offset 0x92683 try change 68 02 80 00 00 to e.g. 68 00 00 A0 00

edit3: well, looking at MemInitDefaultPoo that seems to be flags not the pool size
edit4: also refer the api docs at http://www.microquill.com/kb/01_intro.html / http://www.microquill.com/kb/04_api.html#memalloc
edit5: a quick websearch reveals the flag values at https://github.com/eezstreet/JediKnight ... SMRTHEAP.H and http://www.codelooker.com/codea/1121054 ... EAP.H.html in whatever projects that may be..
Just for the hell of it, I increased the value to 100MB, but it didn't do any good. I still had slow turns after about turn 150.

Interesting enough though, if you change it to 1 GB, you will get an Out of memory error when you try launching the game.
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3196
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: Have long-turn-loading times been fixed?

Post by Flocke »

I guess there is some other limit in code then that is checked during ai calculation or generally prior to memory allocation to not exceed the *assumed* memory limit. On the other hand I can't remember ever having checked what memory botf actually consumes.
Post Reply

Return to “Site Information”