Memory Leak: Solution?

General Modding Information/Questions; support/discussion/questions

Moderator: thunderchero

Forum rules
:idea: Please search before starting new topic. :idea:
There is a good chance it has already been asked.
Post Reply
User avatar
Tethys
Past Administrator
Past Administrator
Posts: 2404
Joined: Fri Jul 18, 2008 2:00 am
Location: Your mom's bed ;)
Contact:

Memory Leak: Solution?

Post by Tethys »

So I was listening to the Dev Interview recently and when Bill talked about the memory leak a little bit, I got to thinking. From what he said, it leaks into the operating system. This happens with other games as well, particularly older ones, and the reasoning was that the computer's own uptime creates bugs when dealing with floating points maximum and minimum thresholds. Rounding errors, dropped zeroes, eventually creep and grow into bigger and bigger discrepancies with regards to accuracy. For example, in many places in the exe SCT has labeled allocfp which if I understand correctly allocates the floating point for whatever process is being created.

The clever solution was to use an uptime faker.

I am now wondering if something like this is possible for BotF, which might actually solve the memory leak by faking the computer uptime every time BotF is initiated. This would be best for people like me who leave their computer running for months without a shutdown. I have access to an uptime faker program, but not sure what would be needed to have it working for BotF. Perhaps only the faker.exe or ini needs to be placed in the games working directory? :?:
Not for the weak of heart...
Galaxies MOD v0.4.0 <--- GALM/Galaxies Mod latest version
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3256
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: Memory Leak: Solution?

Post by Flocke »

It is very unlikely the uptime itself is responsible to this. :lol:

Very true however is, that old days C++ programming required to carefully release allocated memory, which most programmers were lazy about.
Furthermore the memory becomes fragmented over time, which makes re-allocation less efficient. And with smartheap used, it is kept for later use instead of actually releasing it.

In addition, restarting the game also clears any event history, that might not be fully saved.

For actually reducing the memory usage, the responsible routines will have to be identified and need to be reworked. Then ofc there might be some stupid AI routine that keeps iterating some floating point numbers and keeps allocating memory. But then again, it is not related to the system timer, but to some stored values used by the algorithm.
User avatar
Studmaster
Lieutenant-Commander
Lieutenant-Commander
Posts: 100
Joined: Sat Oct 25, 2008 2:00 am

Re: Memory Leak: Solution?

Post by Studmaster »

I have seen a lot of changes over the past 15 years in this game. Witnessing the birth of new ships,bigger resolutions,bug fixes ect. Are we saying we are getting closer to a impasse in what can or cannot be changed anymore?
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3256
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: Memory Leak: Solution?

Post by Flocke »

Let's say, it remains a lot of work....
User avatar
Tethys
Past Administrator
Past Administrator
Posts: 2404
Joined: Fri Jul 18, 2008 2:00 am
Location: Your mom's bed ;)
Contact:

Re: Memory Leak: Solution?

Post by Tethys »

Flocke I think you misunderstood perhaps? I will try to explain better. When a computer is left running for a long period of time, the computer uptime becomes greater. When the uptime (in ms) exceeds the floating point maximum, weird stuff happens. Especially in the memory. If we look at the trek exe imports list it should give clues to Kernel32.dll being used, within the system time is being imported. Using an uptime faker for, lets say, Star Trek Bridge Commander, has proven effective in reducing crashes related to high uptime, by faking the computer uptime to 0 for the application being used, in this case Star Trek Bridge Commander (placing the faker into the directory of the exe file). This has a similar, if not the same effect as rebooting the PC every time you load the program. Floating points can therefore be calculated correctly and memory instances within the program itself become far more reliable.

It won't solve the root of the issue but it would be a good potential workaround I would imagine, if it can be explored.
Not for the weak of heart...
Galaxies MOD v0.4.0 <--- GALM/Galaxies Mod latest version
User avatar
Flocke
BORG Trouble Maker
BORG Trouble Maker
Posts: 3256
Joined: Sun Apr 27, 2008 2:00 am
Location: Hamburg, Germany
Contact:

Re: Memory Leak: Solution?

Post by Flocke »

Tethys wrote: Fri Mar 29, 2024 4:08 pmWhen the uptime (in ms) exceeds the floating point maximum, weird stuff happens.
It might be that some stupid games fail to compute elapsed time by this. As long there is no evidence, I however very much doubt that it is of any relevance to BOTF. And it for sure is no reasonable explanaition to memory leaks.

Anyhow, give it a try and keep your system up and game running without completing any turns.
If your system is stable, doesn't go sleeping, and you don't keep switching views or issue other load dependent computions, the game should keep running for quite a while. :idea:
User avatar
thunderchero
Site Administrator aka Fleet Admiral
Site  Administrator aka Fleet Admiral
Posts: 7940
Joined: Fri Apr 25, 2008 2:00 am
Location: On a three month training mission, in command of the USS Valiant.

Re: Memory Leak: Solution?

Post by thunderchero »

I still say the simplest way to decrease turn processing (this is not a memory leak issue) would be to add multi thread AI turn processing

4 separate threads running at same time processing each major AI race turn at the same time.
problem might be getting all 4 threads to write into saved game without errors

My main question is with the CPU speed we are running now a days why is it not running faster.
game runs just as slow on Pentium I processor as the best processor you can get now. (code limits?)
User avatar
Tethys
Past Administrator
Past Administrator
Posts: 2404
Joined: Fri Jul 18, 2008 2:00 am
Location: Your mom's bed ;)
Contact:

Re: Memory Leak: Solution?

Post by Tethys »

thunderchero wrote: Fri Mar 29, 2024 6:08 pm I still say the simplest way to decrease turn processing (this is not a memory leak issue) would be to add multi thread AI turn processing

4 separate threads running at same time processing each major AI race turn at the same time.
problem might be getting all 4 threads to write into saved game without errors

My main question is with the CPU speed we are running now a days why is it not running faster.
game runs just as slow on Pentium I processor as the best processor you can get now. (code limits?)
Yeah, seems there is a bit of a roadblock when it comes to CPU speed vs BotF turn processing. I bet it has something to do with multiplayer and the advent of gigabit ethernet during the time of the game's development. My guess is that there are bottlenecks preventing the turn processing from completing quickly (probably force limited in one way or another via network code, CPU calculating too quickly for networking ?). I should have some free time upcoming, I could investigate, but not sure where to start in that regard...

It could be directed to use only x amount of CPU due to the technology of the time (I doubt they thought ahead to scale AI code to CPU gens as they are released). That would be a best case scenario, though, as it could be easily changed to accommodate the CPUs of later gens via increases in CPU allotment.

I have some ideas for controlled testing? Starting seed, single player > multiplayer 2 people > multiplayer 5 people. No human interaction, gauging ONLY AI turn procedures (press turn over and over). Is there noticeable decrease of turn time WRT how many AI are present in the game? I am thinking YES (I suspect 5 player MP will turn far better than 1 or 2 player + 3 or 4 AI). Followup questions come to mind: Do all the AI process simultaneously, or are they processed in some sequence ? (most likely Card, Fed, Fer, Kli, Rom). Is it possible to supervise a game completely controlled by the AI, all 5 races? What would be the turn processing in this case? Would the game even be able to handle 5 AI? Something tells me it is only programmed with 4 in mind, as there would never be a case where we see 5 AI all at once (a barrier to adding 6th major race?), so that would scale down from 4-0. We tested turning off ? the AI, ? or some of the components of? There was a decrease in turn times? I can't remember right now. Seems relevant though. More brains we have better the chance of success :)
Not for the weak of heart...
Galaxies MOD v0.4.0 <--- GALM/Galaxies Mod latest version
User avatar
Studmaster
Lieutenant-Commander
Lieutenant-Commander
Posts: 100
Joined: Sat Oct 25, 2008 2:00 am

Re: Memory Leak: Solution?

Post by Studmaster »

Tethys wrote: Tue Apr 02, 2024 6:25 pm
thunderchero wrote: Fri Mar 29, 2024 6:08 pm I still say the simplest way to decrease turn processing (this is not a memory leak issue) would be to add multi thread AI turn processing

4 separate threads running at same time processing each major AI race turn at the same time.
problem might be getting all 4 threads to write into saved game without errors

My main question is with the CPU speed we are running now a days why is it not running faster.
game runs just as slow on Pentium I processor as the best processor you can get now. (code limits?)
Yeah, seems there is a bit of a roadblock when it comes to CPU speed vs BotF turn processing. I bet it has something to do with multiplayer and the advent of gigabit ethernet during the time of the game's development. My guess is that there are bottlenecks preventing the turn processing from completing quickly (probably force limited in one way or another via network code, CPU calculating too quickly for networking ?). I should have some free time upcoming, I could investigate, but not sure where to start in that regard...

It could be directed to use only x amount of CPU due to the technology of the time (I doubt they thought ahead to scale AI code to CPU gens as they are released). That would be a best case scenario, though, as it could be easily changed to accommodate the CPUs of later gens via increases in CPU allotment.

I have some ideas for controlled testing? Starting seed, single player > multiplayer 2 people > multiplayer 5 people. No human interaction, gauging ONLY AI turn procedures (press turn over and over). Is there noticeable decrease of turn time WRT how many AI are present in the game? I am thinking YES (I suspect 5 player MP will turn far better than 1 or 2 player + 3 or 4 AI). Followup questions come to mind: Do all the AI process simultaneously, or are they processed in some sequence ? (most likely Card, Fed, Fer, Kli, Rom). Is it possible to supervise a game completely controlled by the AI, all 5 races? What would be the turn processing in this case? Would the game even be able to handle 5 AI? Something tells me it is only programmed with 4 in mind, as there would never be a case where we see 5 AI all at once (a barrier to adding 6th major race?), so that would scale down from 4-0. We tested turning off ? the AI, ? or some of the components of? There was a decrease in turn times? I can't remember right now. Seems relevant though. More brains we have better the chance of success :)
I have been playing my beta "Generations" mod with no minors and no real issues with turn speed even at 300ish. Seems like more races the slower it becomes later in the game. Most likely due to processing for everyone doing stuff and getting bigger.

No idea if this was ever discussed but is there a way to eliminate some of this chunky stuff through some kind of optimizations on the races end. Some kind of auto stuff maybe causing a issue.
User avatar
Tethys
Past Administrator
Past Administrator
Posts: 2404
Joined: Fri Jul 18, 2008 2:00 am
Location: Your mom's bed ;)
Contact:

Re: Memory Leak: Solution?

Post by Tethys »

Yes, it has been discussed and there are a combination of factors at play WRT to high turn times. First and foremost is the number of ships in the galaxy. More ships = higher turn times. Solution = eliminate ships through battle, or with UE savegame editing (not recommended). Next, in my opinion, would be the maximum number of traversible sectors for the AI empire - INCLUDING - affiliations/alliances for shared and extra/open territories. But it's a one-two punch because also the ship speeds and overall ship ranges come into play as well (all ships set to SHORT range with 1 sector range should perform better than all ships set to LONG with 1 sector range since it would not need to calculate for each range perhaps?). Next, systems held should play a factor as well, as the AI is constantly (every turn) updating the build queue because it can buy anything 1 turn without monetary impact. Finally, ship maintenance costs play a small role as well. Higher maintenance costs will prevent the empires from building so many ships, so raising them should have a positive impact on turn times by preventing the overconstruction of ships in the galaxy. I might have missed something, I feel like I missed something. Yes I did: Map size. Very large maps coincide with very large areas of traversible sectors; smaller maps have faster turn times.
Not for the weak of heart...
Galaxies MOD v0.4.0 <--- GALM/Galaxies Mod latest version
Post Reply

Return to “General Modding Information/Questions”