MP packet size

[thunderchero]

Hi Everyone,

As some of you has noticed I have been looking at some older coding projects.

when the host sends a saved game to the joining player seems will take for ever. larger saved games may take up to 2 min or more.

QD did do basic labeling on "packetDataSize" 0054DF90

these packets are sent back and forth all the time during game play.
examples;
packets have to be sent to joining player for configuration
packets have to be sent to host player when joining player selects race
packets have to be sent to joining player when host player selects race
packets have to be sent to joining player when save game is sent
needless to say they are sent all the time updating changes.

I started off testing just to see if I could see the packet size joining player was actually was receiving. so installed a network monitor on joining system. packet after packet size is 4Eh (could we have made it any smaller)

Just searching for "4Eh" I noticed i was in "packetDataSize" 0054DF90
I see several "4Eh"

Code: Select all

0054E0A7 8B 4C 24 4E             mov     ecx, [esp+4Eh]
0054E0D5 8B 4C 24 4E             mov     ecx, [esp+4Eh]
0054E29B 8B 4C 24 4E             mov     ecx, [esp+4Eh]
0054E17E 8B 44 24 4E             mov     eax, [esp+4Eh]

but I also see several like this

Code: Select all

AUTO:0054E0B7 89 44 24 48             mov     [esp+48h], eax

this might be a little more than I thought, love they only gave single byte values lol

I did put a breakpoint at 0054E0A7 and loaded MP game.
joining player selects race looped though breakpoint 5 - 6 times
when I sent saved game it looped though breakpoint 200 - 500 times (held alt-f7 for 30+ sec)
so no wonder it takes so long sending so many small packets.
I kind of understand most of us was limited to dial-up connections back then.

I still need to go back to see what all the code is doing or if I am even in right place lol

[trevtones]

Does this mean we could potentially figure out a way to have faster load times when sending the save game? A min or 2 or even 5 is not the end of the world but when you're in one of those "crashy" mp games this could make a very nice difference!

[thunderchero]

Does this mean we could potentially figure out a way to have faster load times when sending the save game?

After my first run though of code it looks like more of a mess than I was hoping for. Plus the "dplayx.dll" might just override any changes we make in trek.exe Debug using GR to activate dplayx might not be possible. I have been testing currently using direct IP connection. next time I look at code I will be checking if dplayx.dll is even loading using direct IP?

so like many other project this one might end up on back burner... but I will see if any progress can be made.

[Flocke]

so like many other project this one might end up on back burner... but I will see if any progress can be made.

You might finally delve into programming and revive my old dpnemu project: viewtopic.php?t=418
Plus you might also find some better working wrappers on the web theses days. My first search however didn't bring up anything useful.

[Spocks-cuddly-tribble]

Increasing package size might come with other issues, if not discreetly packaged with better working wrappers:

non_discreet_packaging.jpg

[Flocke]

I started off testing just to see if I could see the packet size joining player was actually was receiving. so installed a network monitor on joining system. packet after packet size is 4Eh (could we have made it any smaller)

Just searching for "4Eh" I noticed i was in "packetDataSize" 0054DF90
I see several "4Eh"

Code: Select all

0054E0A7 8B 4C 24 4E             mov     ecx, [esp+4Eh]
0054E0D5 8B 4C 24 4E             mov     ecx, [esp+4Eh]
0054E29B 8B 4C 24 4E             mov     ecx, [esp+4Eh]
0054E17E 8B 44 24 4E             mov     eax, [esp+4Eh]

mov ..., [esp + 4Eh] is just some arbitrary value read from esp offset 4Eh.
Furthermore BOTF will rather allocate the packet content while directplay wraps it in a package. So the effective data size you are looking for, rather is even smaller.

By accident I found that in loc_54DA56 and loc_54DA99 there is a max packet size check for 200h.
The data however seems to be sent by a thread at asm_54DBCE and read from a critical section initialized at asm_54DBC4.

It makes sense to process network traffic asynchronously. At asm_51B47C and asm_51B4B0 that critical section queue is initialized with just 10h bytes.
Increasing it however should have no effect. At e.g. asm_51B434 it can be seen it is processed like a message list. So the allocated 16 bytes just refer to the first list entry. It most likely is a FIFO list where you add messages to be sent to one end while processing and sending them from the other.

Code: Select all

AUTO:0051B42F 83 C0 08                add     eax, 8          ; -> Add 8 bytes to the current itereated list entry pointer. Most pointing to some entry or packet data class.
AUTO:0051B432 89 F2                   mov     edx, esi        ; -> pointing to the data structure to be read (or written?), e.g. systInfo
AUTO:0051B434 8B 48 F8                mov     ecx, [eax-8]    ; -> Reverse incrementing the eax pointer but read next list entry pointer for the next iteration.
AUTO:0051B437 FF D7                   call    edi             ; -> Call DPMPQueu_566C70, passed in at asm_566C58.
AUTO:0051B43D 89 C8                   mov     eax, ecx        ; -> Set eax to next list entry pointer.
AUTO:0051B43F 85 C9                   test    ecx, ecx
AUTO:0051B441 75 EC                   jnz     short           ; -> Loop until all the list entries are processed.

AUTO:00566C7F E8 3C 76 FE FF          call    DPMPPckt_54E2C0

of DPMPQueu_566C70 seems to be just a check. It copies 2Ch bytes to local stack area but drops it again before return.

Instead the list entry data is processed by:

Code: Select all

AUTO:00566C8C 8B 46 04                mov     eax, [esi+4]
AUTO:00566C8F E8 4C 71 FE FF          call    DPMPThrd_54DDE0 ;

That routine however only seems to copy 2 bytes from a list entry to destination data structure. Without further debugging and an in depths analysis I fear I can't make much sense of it.

Given modern system speed, the queue however might just be processed such fast, that there never are more than one entry to send. While in early times the network connections were slow enough to fill the queue with more data before it was sent.

Edit: as I found, "AUTO:0054DA70 call DPMPThrd_54D810" uses the Windows PostMessage API. This indicates that DPMPThrd_54DB00 uses the Winows event system to synchronize other threads with incoming network data, likely by passing a shared data pointer through lParam, read by DPMPThrd_54DE10. The bottleneck not necessarily might be the network connection, but it might also be how the Windows event notifications are used. This actually might also cause sync errors if not implemented properly.

Received network messages then are processed by "AUTO:0050A6E0 MP_stuff__error_s". It then calls Net_519920 to enter the shared queue section in DPMPThrd_54DC60 and read back the data.

[thunderchero]

no wonder every time I look at code, I just shake my head.

[Flocke]

no wonder every time I look at code, I just shake my head.

Yeah, and I still keep editing my post, reversing parts of my analysis.

Ok, above post already is long enough and I think keeps valid now.
I wondered alot about that seemingly redundant repne movsd data copying for retrieving just a single data value in DPMPThrd_54DDE0.
I however guess this was just some kind of poorly implemented concurrent access protection, that has no use here. Might also be an implementation leftover before they implemented that critical section guards.

For a similar routine we can see it reads back the memory size to allocate from the 2Ch sized PostMessage message data from:

Code: Select all

AUTO:00566B53 E8 E8 72 FE FF          call    DPMPThrd_54DE40 ; Call Procedure
-> AUTO:0054DE5F 8B 44 24 14             mov     eax, [esp+14h]
AUTO:00566B5B 0F BF F8                movsx   edi, ax
...
AUTO:00566B80 8D 04 FD 00 00 00 00    lea     eax, ds:0[edi*8]   ; -> eax = edi * 8
AUTO:00566B87 E8 44 C3 F2 FF          call    System_Memory_AllocFromNetworklPool 

The size to allocate looks to be compressed by a factor of 8.

[Flocke]

I further kept analysing this. Turns out that triggered by Windows event messages, c_Dbg_R1_2 is called to read back packet data from the allocated shared gdlList memory buffer and with the call to Gbg_R2 converts the packet data back to actual game data messages being sent.

c_Dbg_R1_2 retrieve game message:

Code: Select all

AUTO:0054DC93 89 C7                   mov     edi, eax        ; -> previous packet data pointer? probably used to continue receiving partial message data.
AUTO:0054DC95 89 D6                   mov     esi, edx        ; -> the Gbg_R2 game message data structure to be filled
AUTO:0054DC97 68 00 86 68 00          push    offset stru_688600 ; lpCriticalSection ; -> critical section entered for concurrent message buffer access
AUTO:0054DC9C 2E FF 15 AC 44 57 00    call    cs:__imp_EnterCriticalSection
AUTO:0054DCA3 89 DA                   mov     edx, ebx        ; -> total pool member count
AUTO:0054DCA5 89 F8                   mov     eax, edi        ; -> previous packet data pointer?
AUTO:0054DCA7 E8 A4 8D 01 00          call    Dbg_R1          ; -> collect gdlList packet data, used by the windows event messages
AUTO:0054DCAC 89 F2                   mov     edx, esi        ; -> the game message data structure to be filled
AUTO:0054DCAE E8 1D 00 00 00          call    Gbg_R2          ; -> convert packet to game message data

Dbg_R1 collecting gdlList packet data:

Code: Select all

AUTO:00566A53 83 EC 1C                sub     esp, 1Ch        ; -> [b]1Ch temporary packet structure size allocated from local memory stack[/b]
AUTO:00566A81 89 E3                   mov     ebx, esp        ; -> local stack pointer passed as packet data struct to be filled
AUTO:00566A83 BA 30 6B 56 00          mov     edx, offset DPMPQueu_566B30 ; -> passes the DPMPQueu_566B30 callback function
AUTO:00566A88 A1 FC 0D 5A 00          mov     eax, ds:dword_5A0DFC ; -> the gdlList to process
AUTO:00566A8D E8 6E 49 FB FF          call    gdlList_ForEach__edx_edi__MultiSub

gdlList_ForEach__edx_edi__MultiSub gdlList callback iteration:

Code: Select all

AUTO:0051B407 89 DE                   mov     esi, ebx        ; -> the data struct to be filled, cancel ID to check for
…
AUTO:0051B42F                         call_function_eax__p8:  ; -> iterates the gdlList items
AUTO:0051B42F 83 C0 08                add     eax, 8          ; -> item address = entry address + 8
AUTO:0051B434 8B 48 F8                mov     ecx, [eax-8]    ; -> read next gdlList entry address for next iteration
AUTO:0051B432 89 F2                   mov     edx, esi        ; -> the data struct to be filled, cancel systInfo
AUTO:0051B437 FF D7                   call    edi             ; -> calls DPMPQueu_566B30 for each gdlList item until the full ist is processed

DPMPQueu_566B30 network buffer allocation:

Code: Select all

AUTO:00566B38 89 D1                   mov     ecx, edx        ; ->  data struct to be filled
...
AUTO:00566B53 E8 E8 72 FE FF          call    DPMPThrd_54DE40 ; ->  By the memory copy below, reads the total pool member count.
-> AUTO:0054DE5F 8B 44 24 14          mov     eax, [esp+14h]  ; -> the total pool member count sent
AUTO:00566B5B 0F BF F8                movsx   edi, ax
...
AUTO:00566B80 8D 04 FD 00 00 00 00    lea     eax, ds:0[edi*8] ; -> eax = edi * 8: total edi pool member count * 8 bytes pool member size
AUTO:00566B87 E8 44 C3 F2 FF          call    System_Memory_AllocFromNetworklPool
AUTO:00566B8C 89 41 10                mov     [ecx+10h], eax  ; -> allocated total network pool Memory, member count * 8 bytes
...
AUTO:00566BB6 89 79 0C                mov     [ecx+0Ch], edi  ; -> writes the total member count to the temporary packet struct

Dbg_R1 allocated buffer usage:

Code: Select all

AUTO:00566A92 83 7C 24 10 00          cmp     dword ptr [esp+10h], 0 ; -> check network pool memory allocation
...
AUTO:00566A9B A1 FC 0D 5A 00          mov     eax, ds:dword_5A0DFC   ; -> the gdlList to process
AUTO:00566AA4 E8 57 50 FB FF          call    gdlList_FirstItem ; -> returns first gdlList item address
-> AUTO:0051BB05 8B 10                mov     edx, [eax]      ; -> read gdlList entry address
-> AUTO:0051BB0F 8D 42 08             lea     eax, [edx+8]    ; -> eax = edx + 8 bytes offset for the item address
AUTO:00566AAF                         loc_566AAF:             ; -> iterates the gdlList items
-> AUTO:00566AB1 89 E2                mov     edx, esp        ; -> the local stack allocated temporary packet data struct
-> AUTO:00566AB3 E8 E8 4C FB FF       call    gdlList_NextItem ; -> get next gdlList item pointer
-> AUTO:00566ABA 89 C8                mov     eax, ecx        ; -> current gdlList item
-> AUTO:00566ABC E8 2F FE FF FF       call    DPMPQueu_5668F0 ; -> copy network allocated packet data from gdlList members

DPMPQueu_5668F0 copy one allocated pool member and remove from gdlList:

Code: Select all

AUTO:0056692C E8 8F 79 FE FF          call    DPMPPckt_54E2C0 ; -> check for packet content changes?
...
loc_566961:
AUTO:00566961 A1 18 86 68 00          mov     eax, ds:_numMembers
AUTO:00566966 3B 41 0C                cmp     eax, [ecx+0Ch]  ; -> checks the total allocated memory pool member count
...
AUTO:00566976 8B 79 10                mov     edi, [ecx+10h]  ; -> read network allocated memory address
AUTO:0056697C B9 08 00 00 00          mov     ecx, 8          ; -> !! hardcoded copy 8 bytes to the network allocated memory buffer
AUTO:00566981 01 C7                   add     edi, eax        ; -> align for memory member offset
AUTO:00566986 C1 E9 02                shr     ecx, 2          ; -> convert bytes to dword size
AUTO:00566989 F2 A5                   repne movsd             ; -> copy ecx dwords from esi to edi
...
AUTO:0056699B 47                      inc     edi             ; note, that the member count Increment is only 1

Dbg_R1 return to c_Dbg_R1_2:

Code: Select all

AUTO:00566AE4 8B 44 24 10             mov     eax, [esp+10h]  ; -> return allocated packet data
->
AUTO:0054DC95 89 D6                   mov     esi, edx        ; -> some other data structure the packet data gets copied to
AUTO:0054DCAC 89 F2                   mov     edx, esi        ; -> packet data to fill by the call to Gbg_R2
AUTO:0054DCAE E8 1D 00 00 00          call    Gbg_R2          ; -> continues to copy around data before leaving the critical section

[Flocke]

Interestingly, in DPMPThrd_54D810, which writes to the gdlList and notifies the game on new messages with the PostMessage routine, all calls to DPMPThrd_54DE10 only allocate two bytes (or 8 byte member chunks?) of data from the network memory pool:

Code: Select all

AUTO:0054D91C B9 02 00 00 00          mov     ecx, 2 ; -> two bytes to allocate from network pool
AUTO:0054D92A 89 4C 24 28             mov     [esp+38h+var_10], ecx ; -> cached
AUTO:0054D933 8B 54 24 28             mov     edx, [esp+38h+var_10] ; -> restored
AUTO:0054D939 E8 72 93 01 00          call    DPMPQueu_566CB0 ; -> passed as size to allocate the memory
...
AUTO:0054D94A 68 02 04 00 00          push    402h            ; Received network message type
AUTO:0054D956 2E FF 15 2C 46 57 00    call    cs:__imp_PostMessageA ; window event notification

Like if each received gdlList item passed via the event notification is cut to 2 byte pieces and like if that's not enough of a mess, at the start of DPMPThrd_54D810 enters a critical section each iteration of the outer call. Factoring both the tiny buffer allocated gdlList event messages with entering and leaving the critical section all the time, might very well explain the slow processing time. This in turn would both explain the small network packets being sent and the slow transfer speed.

Whenever the critical section is entered, the thread is paused and must wait until other threads finished access plus some little delay. This would also explain why the CPU load is low. New incoming packets on the other hand might soon be dropped, because the network buffer runs full and they can't be processed fast enough. Dropped packets then cause further delay, because they need to be re-sent!

This is alot of guesses of course. I neither have analysed the sending side, nor have found the actual network packet size yet, nor did I try any debugging of all this. Any case, looks like a very awkward network implementation

Edit: Ok, there are other packet receival routines, so it might be some special pachets being handled there, that just don't need any further data.

Code: Select all

AUTO:0054DAC7 8B 5C 24 0C             mov     ebx, [esp+0Ch]
AUTO:0054DACB E8 A0 FC FF FF          call    DPMPThrd_54D770

Is limited to a packet size of 200h.

[Flocke]

Further analysing the code, I found:

Code: Select all

AUTO:00401E2C B9 F0 14 40 00          mov     ecx, offset call_mudd_cheat_effects ; -> this actually is the registered WndProc window event message callback, see WNDCLASSA at https://learn.microsoft.com/de-de/windows/win32/api/winuser/ns-winuser-wndclassa
AUTO:00401E36 89 44 24 0C             mov     [esp+0Ch], eax  ; -> cbClsExtra
AUTO:00401E3A 89 44 24 10             mov     [esp+10h], eax  ; -> cbWndExtra
AUTO:00401E3E 89 5C 24 14             mov     [esp+14h], ebx  ; -> hInstance
AUTO:00401E43 89 54 24 08             mov     [esp+8], edx    ; -> style
AUTO:00401E47 89 4C 24 0C             mov     [esp+0Ch], ecx  ; -> lpfnWndProc
AUTO:00401E4B 2E FF 15 18 46 57 00    call    cs:__imp_LoadIconA ; load "TREK_ICON"
AUTO:00401E58 89 44 24 1C             mov     [esp+1Ch], eax  ; -> hIcon
AUTO:00401E5C 2E FF 15 14 46 57 00    call    cs:__imp_LoadCursorA ; -> load lpCursorName 7F00h
AUTO:00401E63 BA 3C AE 58 00          mov     edx, offset WindowName ; "Star Trek: Birth of the Federation"
AUTO:00401E68 89 44 24 18             mov     [esp+18h], eax  ; -> hCursor
AUTO:00401E6C 89 E0                   mov     eax, esp        ; -> local stack allocated WNDCLASSA
AUTO:00401E6E 89 74 24 1C             mov     [esp+1Ch], esi  ; -> hbrBackground
AUTO:00401E72 50                      push    eax             ; lpWndClass (WNDCLASSA)
AUTO:00401E73 89 54 24 24             mov     [esp+24h], edx  ; -> lpszMenuName
AUTO:00401E77 89 54 24 28             mov     [esp+28h], edx  ; -> lpszClassName
AUTO:00401E7B 2E FF 15 34 46 57 00    call    cs:__imp_RegisterClassA ; -> register the window class to Windows

WndProc (call_mudd_cheat_effects) checks the received window message type:

Code: Select all

AUTO:0040159A 81 F9 00 04 00 00       cmp     ecx, 400h       ; -> network message type
AUTO:004015A0 73 42                   jnb     short loc_4015E4 ; -> not below
AUTO:004015E4 76 18                   jbe     short loc_4015FE ; -> below or equal
AUTO:0040160E E8 CD 90 10 00          call    MP_stuff__error_s ; -> handle 400h or 402h ecx network message
AUTO:0050A702 31 D2                   xor     edx, edx        ; -> for 402h messages, the edx content is reset

The messages received are read by:

Code: Select all

AUTO:0050A704 E8 17 F2 00 00          call    Net_519920      ; -> receive message
AUTO:0050A710 66 3D 74 00             cmp     ax, 74h ; 't'   ; -> compare message content type
AUTO:0050A88D 66 3D 89 00             cmp     ax, 89h         ; -> check to display "Receiving saved game …"
AUTO:0050A897 E8 34 27 00 00          call    Comms_ConnInit_50CFD0 ; -> selects what message to Display, based on whether being the host

Net_519920 enters a critical section and calls DPMPQueu_566C30 to receive any message.

Code: Select all

AUTO:0054DC77 E8 B4 8F 01 00          call    DPMPQueu_566C30 ; receive next message
…
AUTO:00566C56 89 E3                   mov     ebx, esp        ; pass the local stack allocated message data structure
AUTO:00566C58 BA 70 6C 56 00          mov     edx, offset DPMPQueu_566C70 ; the passed in callback function I already refered to
AUTO:00566C5D A1 FC 0D 5A 00          mov     eax, ds:dword_5A0DFC ; gdlList
AUTO:00566C62 E8 99 47 FB FF          call    gdlList_ForEach__edx_edi__MultiSub ; process gdlList to re-assemble the received message data
…
AUTO:00566C67 8B 44 24 04             mov     eax, [esp+4]    ; -> return read message type

By Comms_ConnInit_50CFD0 I interestingly found, that the host message for sending the save game is triggered by sub_50DA70 (AI_takeover_not_host?_timer_ordinfo_mil_order_2):

Code: Select all

AUTO:0050DAF1 E8 7A F5 FF FF          call    Sending_Receiving_saved_game ; -> display "Sending saved game …"
…
AUTO:0050DBA0 E8 4B AC F2 FF          call    LoadGame        ; -> after send completed, load the game

It is called from the sub_4C16E0 button callback (set_player_emp_bit_start_avi_movies proc):

Code: Select all

AUTO:004C17BC E8 EF 87 05 00          call    Net_519FB0      ; -> seems to check for whether the network connection got initialized
AUTO:004C17C1 83 F8 01                cmp     eax, 1
AUTO:004C17C6 E8 D5 F4 04 00          call    Comms_X_510CA0  ; -> seems to start the network transfer
…
AUTO:004C1837 E8 34 C2 04 00          call    AI_takeover_not_host?_timer_ordinfo_mil_order_2 ; call to display the "Sending save game …" popup

Comms_X_510CA0 checks the player empire bitmask and interacts with direct play to send network messages

Code: Select all

AUTO:00510CBE 42                      inc     edx             ; -> iterate all player empires and increase edx count if not current player empire
…
AUTO:00510CC6 E8 45 89 00 00          call    Net_519610      ; -> interacts with direct play to allocate and send the message packets

[Flocke]

To bring some more light to the code, I now had a look at the DirectPlay interfaces.

----
The most interesting first: In the middle of the packet send routine I found a Sleep call.
When all CPUs were slow and had just one core, this likely was necessary to prevent blocking the render thread and others.
Nowerdays the Sleep call most likely just delays the whole send process and can be nopped.

Or at least the idle time can be set to 0, which is a special value, that allows the OS to
reassign CPU cycles to another process, but immediately returns when there is no other work.
In addition, for any values between 1-20 milliseconds, the Sleep timer is not much accurate.
And sending masses of tiny packets, it cumulates and can heavily impact the transfer speed!

1000 packets of 4Eh bytes TC reported -2Ch header size then already makes
1-2 seconds delay per 34 kb data transfer...

Actually ALL the Sleep calls should be checked! If they also called it to delay turn processing,
no surprise it is that slow!
----

c.f. IDirectPlay3:

Code: Select all

#define INTERFACE IDirectPlay3
DECLARE_INTERFACE_( IDirectPlay3, IDirectPlay2 )
{
    /*** IUnknown methods ***/
    00: STDMETHOD(QueryInterface)       (THIS_ REFIID riid, LPVOID * ppvObj) PURE;
    04: STDMETHOD_(ULONG,AddRef)        (THIS)  PURE;
    08: STDMETHOD_(ULONG,Release)       (THIS) PURE;
    /*** IDirectPlay2 methods ***/
    0C: STDMETHOD(AddPlayerToGroup)     (THIS_ DPID, DPID) PURE;
    10: STDMETHOD(Close)                (THIS) PURE;
    14: STDMETHOD(CreateGroup)          (THIS_ LPDPID,LPDPNAME,LPVOID,DWORD,DWORD) PURE;
    18: STDMETHOD(CreatePlayer)         (THIS_ LPDPID,LPDPNAME,HANDLE,LPVOID,DWORD,DWORD) PURE;
    1C: STDMETHOD(DeletePlayerFromGroup)(THIS_ DPID,DPID) PURE;
    20: STDMETHOD(DestroyGroup)         (THIS_ DPID) PURE;
    24: STDMETHOD(DestroyPlayer)        (THIS_ DPID) PURE;
    28: STDMETHOD(EnumGroupPlayers)     (THIS_ DPID,LPGUID,LPDPENUMPLAYERSCALLBACK2,LPVOID,DWORD) PURE;
    2C: STDMETHOD(EnumGroups)           (THIS_ LPGUID,LPDPENUMPLAYERSCALLBACK2,LPVOID,DWORD) PURE;
    30: STDMETHOD(EnumPlayers)          (THIS_ LPGUID,LPDPENUMPLAYERSCALLBACK2,LPVOID,DWORD) PURE;
    34: STDMETHOD(EnumSessions)         (THIS_ LPDPSESSIONDESC2,DWORD,LPDPENUMSESSIONSCALLBACK2,LPVOID,DWORD) PURE;
    38: STDMETHOD(GetCaps)              (THIS_ LPDPCAPS,DWORD) PURE;
    3C: STDMETHOD(GetGroupData)         (THIS_ DPID,LPVOID,LPDWORD,DWORD) PURE;
    40: STDMETHOD(GetGroupName)         (THIS_ DPID,LPVOID,LPDWORD) PURE;
    44: STDMETHOD(GetMessageCount)      (THIS_ DPID, LPDWORD) PURE;
    48: STDMETHOD(GetPlayerAddress)     (THIS_ DPID,LPVOID,LPDWORD) PURE;
    4C: STDMETHOD(GetPlayerCaps)        (THIS_ DPID,LPDPCAPS,DWORD) PURE;
    50: STDMETHOD(GetPlayerData)        (THIS_ DPID,LPVOID,LPDWORD,DWORD) PURE;
    54: STDMETHOD(GetPlayerName)        (THIS_ DPID,LPVOID,LPDWORD) PURE;
    58: STDMETHOD(GetSessionDesc)       (THIS_ LPVOID,LPDWORD) PURE;
    5C: STDMETHOD(Initialize)           (THIS_ LPGUID) PURE;
    60: STDMETHOD(Open)                 (THIS_ LPDPSESSIONDESC2,DWORD) PURE;
    64: STDMETHOD(Receive)              (THIS_ LPDPID,LPDPID,DWORD,LPVOID,LPDWORD) PURE;
    68: STDMETHOD(Send)                 (THIS_ DPID, DPID, DWORD, LPVOID, DWORD) PURE;
    6C: STDMETHOD(SetGroupData)         (THIS_ DPID,LPVOID,DWORD,DWORD) PURE;
    70: STDMETHOD(SetGroupName)         (THIS_ DPID,LPDPNAME,DWORD) PURE;
    74: STDMETHOD(SetPlayerData)        (THIS_ DPID,LPVOID,DWORD,DWORD) PURE;
    78: STDMETHOD(SetPlayerName)        (THIS_ DPID,LPDPNAME,DWORD) PURE;
    7C: STDMETHOD(SetSessionDesc)       (THIS_ LPDPSESSIONDESC2,DWORD) PURE;
    /*** IDirectPlay3 methods ***/
    80: STDMETHOD(AddGroupToGroup)      (THIS_ DPID, DPID) PURE;
    84: STDMETHOD(CreateGroupInGroup)   (THIS_ DPID,LPDPID,LPDPNAME,LPVOID,DWORD,DWORD) PURE;
    88: STDMETHOD(DeleteGroupFromGroup) (THIS_ DPID,DPID) PURE;	
    8C: STDMETHOD(EnumConnections)      (THIS_ LPCGUID,LPDPENUMCONNECTIONSCALLBACK,LPVOID,DWORD) PURE;
    90: STDMETHOD(EnumGroupsInGroup)    (THIS_ DPID,LPGUID,LPDPENUMPLAYERSCALLBACK2,LPVOID,DWORD) PURE;
    94: STDMETHOD(GetGroupConnectionSettings)(THIS_ DWORD, DPID, LPVOID, LPDWORD) PURE;
    98: STDMETHOD(InitializeConnection) (THIS_ LPVOID,DWORD) PURE;
    9C: STDMETHOD(SecureOpen)           (THIS_ LPCDPSESSIONDESC2,DWORD,LPCDPSECURITYDESC,LPCDPCREDENTIALS) PURE;
    A0: STDMETHOD(SendChatMessage)      (THIS_ DPID,DPID,DWORD,LPDPCHAT) PURE;
    A4: STDMETHOD(SetGroupConnectionSettings)(THIS_ DWORD,DPID,LPDPLCONNECTION) PURE;
    A8: STDMETHOD(StartSession)         (THIS_ DWORD,DPID) PURE;
    AC: STDMETHOD(GetGroupFlags)        (THIS_ DPID,LPDWORD) PURE;
    B0: STDMETHOD(GetGroupParent)       (THIS_ DPID,LPDPID) PURE;
    B4: STDMETHOD(GetPlayerAccount)     (THIS_ DPID, DWORD, LPVOID, LPDWORD) PURE;
    B8: STDMETHOD(GetPlayerFlags)       (THIS_ DPID,LPDWORD) PURE;
};

https://github.com/google/sagetv/blob/m ... de/dplay.h
https://github.com/github/VisualStudio/ ... st/dplay.h

This is of course not DirectPlay 6A or what they used for BOTF, but the DirectPlay interfaces build up on each other, so the method sequence should match. Somewhere I have downloaded the actual DirectPlay 6 SDK a long while ago, but would need to search for it.
Plus I only found methods called from the DirectPlay 3 interface.


The following both are server side send routines I found for:

DPRESULT Send(DPID idFrom, DPID idTo, SendFlags dwFlags, LPVOID lpData, DWORD dwDataSize);

This I could determine from calculating the ds:DPMPInterface offset of 68h being called.
When you check the IDirectPlay interfaces, it is the 27th method, that is method 0 + 26 * 4 Bytes = 104 = 68h

By checking them all, I could also verify that tis is the sole packet send routine,
and I also found that the Receive counter part is located at:

AUTO:0054BC96 FF 57 64                call    dword ptr [edi+64h]

Code: Select all

AUTO:0054BC21 51                      push    ecx            ; DWORD dwDataSize
AUTO:0054BC22 53                      push    ebx            ; LPVOID lpData
AUTO:0054BC23 6A 01                   push    1              ; SendFlags dwFlags
AUTO:0054BC25 52                      push    edx	     ; DPID idTo, the receiver ID
AUTO:0054BC26 A1 30 07 5A 00          mov     eax, ds:DPMPInterface
AUTO:0054BC2B 56                      push    esi            ; DPID idFrom, the sender ID
AUTO:0054BC2C 8B 38                   mov     edi, [eax]
AUTO:0054BC2E 50                      push    eax            ; this-pointer of the IDirectPlay interface instance
AUTO:0054BC2F FF 57 68                call    dword ptr [edi+68h] ; IDirectPlay::Send
AUTO:0054BC32 89 C2                   mov     edx, eax

Code: Select all

AUTO:0054BC55
AUTO:0054BC55                         loc_54BC55:
AUTO:0054BC55 51                      push    ecx            ; DWORD dwDataSize
AUTO:0054BC56 53                      push    ebx            ; LPVOID lpData
AUTO:0054BC57 6A 01                   push    1              ; SendFlags dwFlags
AUTO:0054BC59 6A 00                   push    0	             ; DPID idTo, the receiver ID, with 0 most likely being the server ID
AUTO:0054BC5B A1 30 07 5A 00          mov     eax, ds:DPMPInterface
AUTO:0054BC60 56                      push    esi            ; DPID idFrom, the sender ID
AUTO:0054BC61 8B 10                   mov     edx, [eax]
AUTO:0054BC63 50                      push    eax            ; this-pointer of the IDirectPlay interface instance
AUTO:0054BC64 FF 52 68                call    dword ptr [edx+68h] ; Indirect Call Near Procedure
AUTO:0054BC67 89 C2                   mov     edx, eax
AUTO:0054BC69 EB CA                   jmp     short loc_54BC35 ; Jump

The packet data size here is set at:

Code: Select all

AUTO:0054DFB0 BA 2C 00 00 00          mov     edx, 2Ch        ; -> packet header size
AUTO:0054DFB5 8B 44 24 78             mov     eax, [esp+78h]  ; -> arg_0 total data size to send
AUTO:0054DFB9 66 89 54 24 58          mov     [esp+58h], dx   ; -> locally cache the packet header size WORD
AUTO:0054DFC0 B9 D2 01 00 00          mov     ecx, 1D2h       ; -> the max packet data size: 1D2h = 466 bytes
                                                              ;    ( by asm_0054E094: 466 data bytes + 44 byte header + 2 bytes = 512 bytes total )
AUTO:0054DFC8 F7 F9                   idiv    ecx             ; -> devide total eax data size by the max packet data size
AUTO:0054DFCA 40                      inc     eax             ; -> increment the packet count by one for the remaining data
AUTO:0054DFCB 89 44 24 30             mov     [esp+30h], eax  ; -> locally cache the packet count to send

AUTO:0054DFCF 8B 44 24 30             mov     eax, [esp+30h]  ; -> redundantly read back the packet count to send
AUTO:0054DFD3 BB D2 01 00 00          mov     ebx, 1D2h       ; -> the max packet data size
AUTO:0054DFD8 48                      dec     eax             ; -> decrement packet count by 1 for full data packets
AUTO:0054DFD9 0F AF C3                imul    eax, ebx        ; -> multiply eax by the max packet data size
AUTO:0054DFDC 8B 54 24 78             mov     edx, [esp+78h]  ; -> arg_0 total data size to send
AUTO:0054DFE0 29 C2                   sub     edx, eax        ; -> subtract eax for final packet data size remainder
AUTO:0054DFE2 89 5C 24 64             mov     [esp+64h], ebx  ; -> cache full packet data size
AUTO:0054DFE6 89 54 24 60             mov     [esp+60h], edx  ; -> cache final packet data size
...
AUTO:0054E01F 8B 7C 24 30             mov     edi, [esp+30h]  ; -> read packet count to send
AUTO:0054E035 89 54 24 40             mov     [esp+40h], edx  ; -> initialize current packet number to 0
AUTO:0054E042 89 54 24 34             mov     [esp+34h], edx  ; -> reset sent packet count
AUTO:0054E052 8D 47 FF                lea     eax, [edi-1]    ; -> reduce by 1 for final packet number
AUTO:0054E055 89 44 24 38             mov     [esp+38h], eax  ; -> save final packet number
...
AUTO:0054E059 8B 44 24 40             mov     eax, [esp+40h]  ; -> read current packet number
AUTO:0054E05D 3B 44 24 38             cmp     eax, [esp+38h]  ; -> read final packet number
AUTO:0054E061 0F 8D E8 01 00 00       jge     loc_54E24F      ; -> check whether it's the final packet to send
AUTO:0054E067 8B 44 24 64             mov     eax, [esp+64h]  ; -> full packet data size if not
AUTO:0054E24F 8B 44 24 60             mov     eax, [esp+60h]  ; -> final packet data size if reached
AUTO:0054E06B 66 89 44 24 50          mov     [esp+50h], ax   ; -> locally cache the packet data size WORD
...
AUTO:0054E08C 8B 44 24 50             mov     eax, [esp+50h]  ; -> the packet data size
AUTO:0054E090 03 44 24 58             add     eax, [esp+58h]  ; -> add packet header size
AUTO:0054E094 05 02 00 00 00          add     eax, 2          ; -> add another two bytes for checksum or whatever
AUTO:0054E099 66 89 44 24 5C          mov     [esp+5Ch], ax   ; -> locally cache the total packet size to send
...
AUTO:0054E0FB 8B 44 24 5A             mov     eax, [esp+5Ah]  ; -> read back the total packet size, but read 16bits ahead of [esp+5Ch]
AUTO:0054E103 C1 F8 10                sar     eax, 10h        ; -> shift by 16bit to read the WORD value from [esp+5Ch]:
                                                              ;    Since we are in a little endian world, with [esp+5Ah] the low value bits
                                                              ;    from [esp+5Ch] were read into the high value bits of eax.
                                                              ;    The right shift ensures to only read a word value to 32bit eax.
AUTO:0054E106 B9 2C 00 00 00          mov     ecx, 2Ch ; -> the header size or data offset
AUTO:0054E10B E8 C0 4D F4 FF          call    System_Memory_AllocFromNetworklPool ;
...
AUTO:0054E161 8B 4C 24 5A             mov     ecx, [esp+5Ah]  ; -> read back the total packet size
AUTO:0054E16F C1 F9 10                sar     ecx, 10h        ; -> align for [esp+5Ch] WORD value
AUTO:0054E172 E8 99 DA FF FF          call    DPMP_54BC10     ; -> call to try send the message and repeat by the surrounding loop
-> AUTO:0054BC21 51                   push    ecx             ; -> read back dwDataSize posted above
-> AUTO:0054BC2F FF 57 68             call    dword ptr [edi+68h] ; -> call DirectPlay->Send
...
AUTO:0054E1CD 6A 0A                   push    0Ah             ; -> 10 dwMilliseconds to idle
AUTO:0054E1CF 8B 6C 24 44             mov     ebp, [esp+44h]  ; -> read current packet number [esp+40h], offset by the push
AUTO:0054E1D3 2E FF 15 94 45 57 00    call    cs:__imp_Sleep  ; -> Idle at least the pushed milliseconds before sending next packet.
                                                              ;    Note that this potentially can cause a huge bottleneck.
AUTO:0054E1DA 8B 44 24 34             mov     eax, [esp+34h]  ; -> read sent packet count
AUTO:0054E1DE 8B 54 24 30             mov     edx, [esp+30h]  ; -> read final packet number
AUTO:0054E1E2 45                      inc     ebp             ; increment current packet number
AUTO:0054E1E3 40                      inc     eax             ; increment packet count sent
AUTO:0054E1E4 89 6C 24 40             mov     [esp+40h], ebp  ; save current packet number
AUTO:0054E1E8 89 44 24 34             mov     [esp+34h], eax  ; save packet count sent
AUTO:0054E1EC 39 D5                   cmp     ebp, edx        ; compare current with final packet number
AUTO:0054E1EE 0F 8C 65 FE FF FF       jl      loc_54E059      ; if not reached, loop sending more packets

The fact that thunderchero experiences even saller packets, might be related to how DirectPlay actually handles delivering them.
It very well might split them up and re-package them. It however also might be, that the outer calls already pass in such small data pieces to send.

Plus note, that DirectPlay has a throttling mechanism, so the awkwardly slow message handling on the receiver side might very well impact DirectPlay packet delivery as well.

The DirectPlay protocol's throttling mechanism guarantees that the client will not receive messages faster than they can be handled.

c.f. https://documentation.help/DirectPlay/o ... kusage.htm


P.S.: Note that I still didn't do any debugging or testing on it. It is all poor code analysis.

[thunderchero]

you know I am not sure how to process any of this... (insert spinning head emoji)

but my confusion says it does not look very good for patch.

[Flocke]

you know I am not sure how to process any of this... (insert spinning head emoji)
but my confusion says it does not look very good for patch.

With what I've found, I think it looks very good! Already figured much more than I expected! But I have to do some testing next.

I am still not sure by what call path the savegame data transfer is processed. But I already setup a local copy for testing the network traffic, started a multiplayer savegame with decent load time even on the local machine, and managed to attach the IDA debugger to the network processing.

Later the day I hope to find more on it. And with some luck, aready nopping the sleep call will help.

Here my confusion on the call graph for savegame data transfer. It is really hard to figure call connections between distant routines. Even the proximity browser only finds one of the many call connections there:

save call graph.jpg

[Spocks-cuddly-tribble]

I am still not sure by what call path the savegame data transfer is processed.

I don't recall specifics right now, but TC and me had to extend a MP array for a patch where is composes the different player only datas (extending build queue?). ASM patch notes might give a hint for MP special data transfer.