Memory Leak: Solution?

[Tethys]

So I was listening to the Dev Interview recently and when Bill talked about the memory leak a little bit, I got to thinking. From what he said, it leaks into the operating system. This happens with other games as well, particularly older ones, and the reasoning was that the computer's own uptime creates bugs when dealing with floating points maximum and minimum thresholds. Rounding errors, dropped zeroes, eventually creep and grow into bigger and bigger discrepancies with regards to accuracy. For example, in many places in the exe SCT has labeled allocfp which if I understand correctly allocates the floating point for whatever process is being created.

The clever solution was to use an uptime faker.

I am now wondering if something like this is possible for BotF, which might actually solve the memory leak by faking the computer uptime every time BotF is initiated. This would be best for people like me who leave their computer running for months without a shutdown. I have access to an uptime faker program, but not sure what would be needed to have it working for BotF. Perhaps only the faker.exe or ini needs to be placed in the games working directory?

[Flocke]

It is very unlikely the uptime itself is responsible to this.

Very true however is, that old days C++ programming required to carefully release allocated memory, which most programmers were lazy about.
Furthermore the memory becomes fragmented over time, which makes re-allocation less efficient. And with smartheap used, it is kept for later use instead of actually releasing it.

In addition, restarting the game also clears any event history, that might not be fully saved.

For actually reducing the memory usage, the responsible routines will have to be identified and need to be reworked. Then ofc there might be some stupid AI routine that keeps iterating some floating point numbers and keeps allocating memory. But then again, it is not related to the system timer, but to some stored values used by the algorithm.

[Studmaster]

I have seen a lot of changes over the past 15 years in this game. Witnessing the birth of new ships,bigger resolutions,bug fixes ect. Are we saying we are getting closer to a impasse in what can or cannot be changed anymore?

[Flocke]

Let's say, it remains a lot of work....

[Tethys]

Flocke I think you misunderstood perhaps? I will try to explain better. When a computer is left running for a long period of time, the computer uptime becomes greater. When the uptime (in ms) exceeds the floating point maximum, weird stuff happens. Especially in the memory. If we look at the trek exe imports list it should give clues to Kernel32.dll being used, within the system time is being imported. Using an uptime faker for, lets say, Star Trek Bridge Commander, has proven effective in reducing crashes related to high uptime, by faking the computer uptime to 0 for the application being used, in this case Star Trek Bridge Commander (placing the faker into the directory of the exe file). This has a similar, if not the same effect as rebooting the PC every time you load the program. Floating points can therefore be calculated correctly and memory instances within the program itself become far more reliable.

It won't solve the root of the issue but it would be a good potential workaround I would imagine, if it can be explored.

[Flocke]

When the uptime (in ms) exceeds the floating point maximum, weird stuff happens.

It might be that some stupid games fail to compute elapsed time by this. As long there is no evidence, I however very much doubt that it is of any relevance to BOTF. And it for sure is no reasonable explanaition to memory leaks.

Anyhow, give it a try and keep your system up and game running without completing any turns.
If your system is stable, doesn't go sleeping, and you don't keep switching views or issue other load dependent computions, the game should keep running for quite a while.

[thunderchero]

I still say the simplest way to decrease turn processing (this is not a memory leak issue) would be to add multi thread AI turn processing

4 separate threads running at same time processing each major AI race turn at the same time.
problem might be getting all 4 threads to write into saved game without errors

My main question is with the CPU speed we are running now a days why is it not running faster.
game runs just as slow on Pentium I processor as the best processor you can get now. (code limits?)

[Tethys]

I still say the simplest way to decrease turn processing (this is not a memory leak issue) would be to add multi thread AI turn processing

4 separate threads running at same time processing each major AI race turn at the same time.
problem might be getting all 4 threads to write into saved game without errors

My main question is with the CPU speed we are running now a days why is it not running faster.
game runs just as slow on Pentium I processor as the best processor you can get now. (code limits?)

Yeah, seems there is a bit of a roadblock when it comes to CPU speed vs BotF turn processing. I bet it has something to do with multiplayer and the advent of gigabit ethernet during the time of the game's development. My guess is that there are bottlenecks preventing the turn processing from completing quickly (probably force limited in one way or another via network code, CPU calculating too quickly for networking ?). I should have some free time upcoming, I could investigate, but not sure where to start in that regard...

It could be directed to use only x amount of CPU due to the technology of the time (I doubt they thought ahead to scale AI code to CPU gens as they are released). That would be a best case scenario, though, as it could be easily changed to accommodate the CPUs of later gens via increases in CPU allotment.

I have some ideas for controlled testing? Starting seed, single player > multiplayer 2 people > multiplayer 5 people. No human interaction, gauging ONLY AI turn procedures (press turn over and over). Is there noticeable decrease of turn time WRT how many AI are present in the game? I am thinking YES (I suspect 5 player MP will turn far better than 1 or 2 player + 3 or 4 AI). Followup questions come to mind: Do all the AI process simultaneously, or are they processed in some sequence ? (most likely Card, Fed, Fer, Kli, Rom). Is it possible to supervise a game completely controlled by the AI, all 5 races? What would be the turn processing in this case? Would the game even be able to handle 5 AI? Something tells me it is only programmed with 4 in mind, as there would never be a case where we see 5 AI all at once (a barrier to adding 6th major race?), so that would scale down from 4-0. We tested turning off ? the AI, ? or some of the components of? There was a decrease in turn times? I can't remember right now. Seems relevant though. More brains we have better the chance of success

[Studmaster]

I still say the simplest way to decrease turn processing (this is not a memory leak issue) would be to add multi thread AI turn processing

4 separate threads running at same time processing each major AI race turn at the same time.
problem might be getting all 4 threads to write into saved game without errors

My main question is with the CPU speed we are running now a days why is it not running faster.
game runs just as slow on Pentium I processor as the best processor you can get now. (code limits?)

Yeah, seems there is a bit of a roadblock when it comes to CPU speed vs BotF turn processing. I bet it has something to do with multiplayer and the advent of gigabit ethernet during the time of the game's development. My guess is that there are bottlenecks preventing the turn processing from completing quickly (probably force limited in one way or another via network code, CPU calculating too quickly for networking ?). I should have some free time upcoming, I could investigate, but not sure where to start in that regard...

It could be directed to use only x amount of CPU due to the technology of the time (I doubt they thought ahead to scale AI code to CPU gens as they are released). That would be a best case scenario, though, as it could be easily changed to accommodate the CPUs of later gens via increases in CPU allotment.

I have some ideas for controlled testing? Starting seed, single player > multiplayer 2 people > multiplayer 5 people. No human interaction, gauging ONLY AI turn procedures (press turn over and over). Is there noticeable decrease of turn time WRT how many AI are present in the game? I am thinking YES (I suspect 5 player MP will turn far better than 1 or 2 player + 3 or 4 AI). Followup questions come to mind: Do all the AI process simultaneously, or are they processed in some sequence ? (most likely Card, Fed, Fer, Kli, Rom). Is it possible to supervise a game completely controlled by the AI, all 5 races? What would be the turn processing in this case? Would the game even be able to handle 5 AI? Something tells me it is only programmed with 4 in mind, as there would never be a case where we see 5 AI all at once (a barrier to adding 6th major race?), so that would scale down from 4-0. We tested turning off ? the AI, ? or some of the components of? There was a decrease in turn times? I can't remember right now. Seems relevant though. More brains we have better the chance of success

I have been playing my beta "Generations" mod with no minors and no real issues with turn speed even at 300ish. Seems like more races the slower it becomes later in the game. Most likely due to processing for everyone doing stuff and getting bigger.

No idea if this was ever discussed but is there a way to eliminate some of this chunky stuff through some kind of optimizations on the races end. Some kind of auto stuff maybe causing a issue.

[Tethys]

Yes, it has been discussed and there are a combination of factors at play WRT to high turn times. First and foremost is the number of ships in the galaxy. More ships = higher turn times. Solution = eliminate ships through battle, or with UE savegame editing (not recommended). Next, in my opinion, would be the maximum number of traversible sectors for the AI empire - INCLUDING - affiliations/alliances for shared and extra/open territories. But it's a one-two punch because also the ship speeds and overall ship ranges come into play as well (all ships set to SHORT range with 1 sector range should perform better than all ships set to LONG with 1 sector range since it would not need to calculate for each range perhaps?). Next, systems held should play a factor as well, as the AI is constantly (every turn) updating the build queue because it can buy anything 1 turn without monetary impact. Finally, ship maintenance costs play a small role as well. Higher maintenance costs will prevent the empires from building so many ships, so raising them should have a positive impact on turn times by preventing the overconstruction of ships in the galaxy. I might have missed something, I feel like I missed something. Yes I did: Map size. Very large maps coincide with very large areas of traversible sectors; smaller maps have faster turn times.