Page 2 of 4

Posted: Fri Oct 03, 2008 12:38 pm
by Malvoisin
Despite your love/hate-relationship with bote (:lol:) I bet it would have been helpful if the network cpp files were included in the repository wouldn't it. CBot made a very fine and stable network library and now don't tell me you don't like that ;).

Posted: Fri Oct 03, 2008 1:30 pm
by Flocke
So you finally found out about, hehe.

No, that's not the point, I would want to do my own anyway cause of the NAT punch through technic (based upon raknet maybe). But I'm currently just trying to establish a layer between trek.exe and directplay and although I've had widely success, botf still crashes after establishing the TCP connection and before showing up next screen.

With Dependency Walker I currently get this (after hundrets of lines):

First chance exception 0xC0000005 (Access Violation) occurred at address 0x00000000.
Second chance exception 0xC0000005 (Access Violation) occurred at address 0x00000000.
Exited "TREK.EXE" (process 0xAA8) with code -1073741819 (0xC0000005).

So it seems as if there's a NULL pointer somewhere, hopefully I'll find it.
If I'm able to solve this, I should pass to the next screen and should be mostly finished with the layer.
But, this may be a big problem... we'll see.
I just have to give it a try, that's what I think about.
Would be great if botf would have such a "stable" networkengine as bote, although still without of NAT punchthrough and stuff. But painfully, it doesn't and most people I met on warzone have at least big troubles getting botf to work without of buggy bypass.

In just a few days, I'm already that far...
Nearly completely finished the layer and still with the opinion that it is possible... but sometimes with a vanishing belief in beeing capable of this.

So keep laughing about me, your chances aren't that bad that I'll fail. :? ;)

Posted: Fri Oct 03, 2008 1:41 pm
by Malvoisin
the NAT punch through technique is the one cbot mentioned which allows you to bypass router port forwarding and make that one obsolete? IIRC he said it was on his list but it's a little painful to implement.

well, stabbing in an undocumented binary like trek.exe inherently has the danger of leading nowhere but I wish you luck with it. btw. I wasn't laughing at the attempt..yet :lol: (just kidding ;))

Posted: Fri Oct 03, 2008 1:59 pm
by Flocke
I know you were kidding hehe, but pittily it's really not easy cause I'm not able to debug trek.exe and just see where the error occures.

Yes, NAT punch through is a feature to bypass routers and firewalls without of needing to redirect every message and data through a server like with normal bypass.
NAT punch through is already implemented in raknet as far I know and I believe it would be a pain to do it by yourself like CBot might intend to do.
Even a more stable bypass than with warzone might be enough for botf, but I don't want to fall in same traps as warzone did and redevelope a directplay lobby. This wouldn't make sense to me, but the alternative with replacing the whole directplay dependance seemed to be a great possibility to me.
First I just researched a little about dlls, just for fun, but with getting to know more and more about, I really got that strange idea to just give it a try.

If I get the layer to work and you've a working NAT punch through or bypass feature, we might think about combining them, thx for your offer, hehe. =)

Posted: Tue Oct 07, 2008 6:25 am
by Flocke
It turned out that I'm hunting a ghost... :(

BotF still crashes after having initialized the connection with that damn NULL pointer somewhere. I've checked every piece of my dlls, developed my own testprograms, protocoled every message that comes in and goes out and... nothing, all is fine. My librarys do exact what has been done before by ole32.dll and dplayx.dll.
So what's wrong?
I've checked trek.exe with Dependancy Walker again and again, saw many DLL_THREAD_ATTACH that didn't occure before, thought as many times before that this might be the problem, cause the MPR97.dll always returns 0 on that, telling that the attachment failed and it doesn't have thread support, but then I've renamed ole32.dll and dplayx.dll, placed them in botf folder and got exact the same stuff without a crash.
So Dependancyy Walker didn't help and my code has been fine and all ideas I got about possible problems turned out to be none.
Crash still there.
I managed to get the gdb debugger to run, being part of gcc/MinGW btw my Code::Blocks installation. And what I got in the call stack is this:

#0 00000000 ??() (??:??)
#1 0054BA10 gen!DevGetDeviceResolution() (??:??)
#2 00000000 ??() (??:??)
#3 00000001 ??() (??:??)
#4 0054B9E0 gen!DevGetDeviceResolution() (??:??)
#5 005A0730 gen!DevGetDeviceResolution() (??:??)
#6 00000000 ??() (??:??)
#7 00000000 ??() (??:??)
#8 00000000 ??() (??:??)
#9 00000001 ??() (??:??)
#10 00000028 ??() (??:??)
#11 00519BFF gen!DevGetDeviceResolution() (??:??)
#12 005A2928 gen!DevGetDeviceResolution() (??:??)
#13 00000001 ??() (??:??)
#14 000A07EC ??() (??:??)
#15 0050E1DA gen!DevGetDeviceResolution() (??:??)
#16 005B1A91 gen!DevGetDeviceResolution() (??:??)
#17 00000001 ??() (??:??)
#18 000A07EC ??() (??:??)
#19 0006FA20 ??() (??:??)
#20 00541100 gen!DevGetDeviceResolution() (??:??)
#21 00000000 ??() (??:??)
#22 00220000 ??() (??:??)
#23 010D05D0 ??() (??:??)
#24 0000000A ??() (??:??)
#25 0054F29C gen!DevGetDeviceResolution() (??:??)
#26 00000028 ??() (??:??)
#27 0051D868 gen!DevGetDeviceResolution() (??:??)
#28 005A29A0 gen!DevGetDeviceResolution() (??:??)
#29 00000000 ??() (??:??)

It is telling that the crash occured in a function being called by DevGetDeviceResolution() inside the "gen" library. And with Dependancy Walker it turned out that gen.mpr is used by MPR97.dll and MPR565.dll and itself depending on DDRAW.dll. DDRAW.dll, the DirectDraw library being used for 2D rendering in the old days, itself is using D3DIM.dll, missing the D3DTextureUpdate function (just like with normal botf).
I dunno if that might be a problem, but it wouldn't be an anser to the crash, that only occures with my library replacements.
Interestingly DirectDraw is based upon the COM interface as well as DirectPlay.
The COM interface gives the possibility for inter process communication and I just don't know what's hidden by microsoft behind it.

It is not crashing within the network code but it seems to crash whenever botf tries to render the next screen inside DirectDraw.

So as I said this is a ghost and I've tried everything possible, no ideas left with.
Beside one... replacing DirectDraw or to say MPR97.dll and MPR565.dll, the graphics engine of BotF. With some luck this might solve the crash. But I won't start another thread about implementing a new graphics engine and you havn't to expect anything soon. Now you can laugh. :lol: :cry: :lol: :cry:

All the critics seem to be right again and I'm currently not in mood to start another project the like. I've spent alot time on this project and at least learned alot about dlls and stuff, so it's not completely wasted and who knows, perhaps the ghost might be catched some day.

If there's interest in the code or the current crashing version, just ask for.

What can I say? Big Sorry, with all that success I've been absolutely sure to get it work else I wouldn't have posted anything. And I'm still that sure it must work, it just doesn't make sense to me. Microsoft must have burrowed some real crazy pitfalls in it's COM, that's the only thing I can think of.

Posted: Tue Oct 07, 2008 6:31 am
by Malvoisin
that's why open-source is so cool :lol:

nevertheless your audio engine project seems interesting, take on with that I'd say. Something to replace FMOD or OpenAL would be nice for bote :lol:.

Posted: Tue Oct 07, 2008 6:39 am
by Flocke
Hm, yeah, my AudioEngine is nearly done (and it's based upon OpenAL, not replacing it), I just have to give it the last few beautifications. It is already supporting wave, ogg vorbis and mp3 and all the core stuff, but still needing a better interface and refinement in the resource manager and the node system for 3D audio and stuff.

I think you're fine with FMOD anyway.

Give me a dark cell in some kind of cellar to rest in, that's how I feel right now... would have preferred to loose the tourney. :roll: *snif*

Posted: Tue Oct 07, 2008 8:59 am
by jigalypuff
3d engine is the falcon 4.0 one, it uses Msvcrt.dll and i found out while looking for a way to update it that you can in fact delete that dll and botf still runs lol, I`ll keep poking about with regards to the engine and hopfully find a way to update it.

Posted: Tue Oct 07, 2008 9:29 am
by Flocke
As you can read here: when loading dlls without of a specified path, applications always try to locate it in the application path first, later in the windows\system32 path and cause there is another msvcrt.dll with probably a newer version but providing same functionality, it's no problem to delete msvcrt.dll from botf directory.
I used this behavior to get my own replacements of ole32.dll and dplayx.dll to be loaded by botf and as told, this worked without of any problem after having assured that the functions are mapped right with Dependancy Walker.
(just needed to rename the library and therefore hexedit trek.exe to prevent it being loaded by other dlls depending on ole32 as well)

I may give it a try as well.. after having overcome my flop. :roll:

Posted: Tue Oct 07, 2008 1:27 pm
by Spocks-cuddly-tribble
Flocke wrote:All the critics seem to be right again
I think you've misunderstood.8O

We all appreciate what (and that :!:) you've tried 8)

Thanks for your effords.:)

PS: It looks funny that your call stack was so............confused^^

Posted: Thu Oct 09, 2008 8:59 am
by Flocke
Thx, at least someone trying to console me! :)
[size=0]PS: Found out that the call stack got messed cause I've been using the wrong calling convention - strange that it didn't crash any further - should have thought about after the call stack problems with creating the lobby - I feel that dumb now having even edited the dplay headers to remove the stdcall to get right functionnames with gcc. Please, let noone read this! xD[/size]

Posted: Thu Oct 09, 2008 11:12 am
Wow, I'm impressed. Very interesting research. Sorry you got stuck in the end, but still this info might be useful in the future and maybe someone will get it to work someday.

Posted: Fri Oct 10, 2008 7:08 pm
by Flocke
Incredible NEWS!
First pre-prep-real-phat-est version release:

This version is the finished basic layer, only protocoling some few stuff yet.
Everything protocoled gets saved in a file named "MP DPNetworkEmulation-Log.txt" in same path as trek.exe.
This version isn't intended to do further analysing of BotF's mp or anything yet. The log isn't very meaningful so far anyway. It'll tell you alot false positive errors, that occure during calling directplay cause BotF is using these in a try and error manner and even DirectPlay itself is designed this way.
This version is just intended to show that it is working!! No theory anymore but practice. A new Networkengine is to be expected, but I can't tell when. :)

This version doesn't improve botf in any way yet, it may even cause more crashes. Also, as you might have expected, I don't give any assurances or guarantees on it's behavior. Its use is on your own risk and will ever be.

Anyway, as you trust me, I'm absolutely sure that it wouldn't cause any damage and there is no spyware, malware, trojan or anything the like included. (Well, you have to trust any modder that changes executable binary files like trek.exe or dlls or has it's own installer. Just prefer to make things clear in this bureaucratic country.)

Install Instructions:
This version doesn't have an installer and you need to alter trek.exe with an hexeditor like HxD!
But it's an easy task everyone is able to do, even if knowing nothing about hexediting and stuff.
First save your current version of trek.exe so you have a backup before hexediting anything!
You just need to open trek.exe with the hexeditor, search for "ole32.dll", replace it with "dpemu.dll" and afterwards replace "dplayx.dll" with "dplemu.dll", that's it. They should only be found once in trek.exe so with HxD you can just use the "Replace..." or in german "Ersetzen..." function in the "Search"/"Suchen" tab. Just a matter of some few seconds.
Of course you need to place the new dlls in the BotF execution path.
Here you get them:
DirectPlayEmulation (ole32.dll):
DirectPlayLobbyEmulation (dplayx.dll):
Now BotF should work as normal but log some additional stuff to the log file in mp. Not much yet but it'll become more soon.

How come:
As I've told within my previous post, I've finally found my big damn fault I knew it was there but didn't found, lost all ideas and already gave up about.
Well, yeah, you didn't read, you shouldn't read! :lol: It's hidden with a size of zero, but you'll see with quoting. ;)
I found it while analysing some assemblercode for a new graphics engine. While analysing, I found some strange stack calls and with doing some search in the internet I read about different calling conventions. I've found out alot about how to implement a new graphics engine, but I'm not sure if it will be possible to develope a new one as well, cause the used graphics engine is from MicroProse and it isn't documented anywhere - no parameters, no classes, hidden functionpointers, nothing.
But with having read about the calling conventions of __cdecl and __stdcall it came clear to me that I've done a stupid fault. So stupid that I even got a little ashamed. :oops:
As told somewhere I had removed the __stdcall prefixes in the directplay and ole32 headers to get the right functionnames with gcc, cause with __stdcall gcc is adding an '@' and the size of the parameters to the functionnames and botf wouldn't find them (or I would have had to hex that to trek.exe as well but I did want to get the same names as there have been before).
Unfortunately this corrupted the call stack, cause with __stdcall the called functions are responsible for clearing the parameters from stack instead of the calling ones.

Such an easy thing and with having read about I even remembered having fallen in this trap some years ago. This happens cause of some little minor interpretation difference of gcc and msvc.
To solve the problem I changed back to __stdcall and hexedited dpemu.dll to get the right names, but this hasn't been a problem and it's just an adapter to dpnemu.dll anyway.
Damn, I still can't believe I didn't think about this before - just didn't came to my mind.

But however, now it is working and I wish you some fun with it.
Greetz, Flocke 8)

p.s.: Ha! Now I'll laugh! :lol: :lol: :lol: :lol: :lol: :lol: :lol:

Posted: Fri Oct 10, 2008 7:30 pm
by jonboylondon
Congratulations Flocke.... Im pleased to see your efforts have not been in vain.!


Posted: Sat Oct 11, 2008 7:32 am
by jigalypuff
the grapics engine (3d) is the falcon 4.0 one, and i believe it is open source now.
Well done btw on your breakthrough.