Assembler Offsets

[Gowron]

Since the focus of our BotF modding research is more and more moving from the hex level to the assembler level, and since using both asm and hex offsets can become confusing, I'll stick to the asm offsets in all new modding topics/articles from now on.

The asm offsets can easily be translated to hex offsets and vice-versa, as there's a one-to-one correlation between them:

Code: Select all

assembler offsets   trek.exe hex offsets  displacement
----------------------------------------------------------
401000-573600        000400-172A00         400C00     // code area
576000-5A1600        173E00-19F400         402200     // data area
68A000-69DA00        19F400-1B2E00         4EAC00     // data area

To switch between assembler and hex offsets, simply add/subtract the corresponding displacement value

[Flocke]

The asm offsets may depend on the disassembler (and it's version) you're using.
If using them, at least link a free download plz and always refer to this thread for conversion

[Gowron]

The asm offsets may depend on the disassembler (and it's version) you're using.

The asm offsets are explicitely stored in trek.exe for each absolute address, so they have to be unique. IMHO it's not possible that you get different asm offsets from two x86 disassembler programs (as long as they're working correctly ^^).

[Scatter]

i have absolutely no idea how to search for asm offsets in neo.

is it likely to be called something else?

[Gowron]

You need a disassembler like IDA to view the asm statements.
It will be a bit difficult at first, but it will pay off

[Scatter]

it's ok. just noticed the newest ue has an offset converter...

[Glorfindel7]

OK, so I downloaded IDA, opened Trek.exe and found the relevant address (e.g. 004B5175) and hex input (8D 42 01). I know what I want to change it to - just no idea of how to do it.

Be gentle with me: last time I did this kind of stuff, we were still using punch cards and IBM main frames.

[Tethys]

i think IDA is just a disassembler program, u need something like HxD to physically edit the exe (iirc)

[Glorfindel7]

Yup, downloaded UltraEdit and after several failures trying to do clever stuff, I discovered that one can just overwrite the desired hex digits. Thanks for your help.

[Spocks-cuddly-tribble]

Here are some links wrt assembler you might find useful (for a first steps guide, see below):


For IDA-Pro hints (recommended for code analysis) see: http://www.hex-rays.com/idapro/


OllyDbg from http://www.ollydbg.de/ will re-write the hexcode for you!



Instruction Set: http://www.husseinsspace.com/teaching/u ... haptwo.htm (F3 -> enter asm-command)


Assembly Tutorial

x86 Assembly Guide (Registers/Memory and Addressing/Instructions/Calling Convention) http://www.cs.virginia.edu/~evans/cs216/guides/x86.html

FLOATING POINT CALCULATIONS IN ASSEMBLY http://www.programmers-corner.com/tutorial/31



- German pages -

Assemblerbefehle http://andremueller.gmxhome.de/befehle.html

Assembler Informationen http://runtime-basic.net/Assembler:Assembler


8086/88 Assembler Befehlsreferenzen http://www.i8086.de/asm/8086-88-asm.html

Assembler Crashkurs (Register/Speicher/Stack) http://www-ivs.cs.uni-magdeburg.de/bs/l ... bler.shtml

Assembler (80x86 Prozessor)-Programmierung http://de.wikibooks.org/wiki/Assembler_ ... lsreferenz

Wie dividiere ich mit großen Zahlen (zwei Register)? http://dcla.rkhb.de/div64.html

[Dr_Breen]

Are there any other disassemblers you would recommend? IDA costs a lot and ollydbg does, according to the author, not run correctly on a win 64 system. or am i wrong and it does run fine but cannot debug 64 bit tasks?

edit:

well Ollydbg seems to run fine on 32-bit tasks

[Spocks-cuddly-tribble]

Here are the "avoid nerd stuff" ASM Hints & Tips:



1. Code analysis -> IDA-Pro (the free version is more than adequate for BotF)


On the first time opening resp disassembling a file, IDA-Pro baffles new users with a confusing amount of esoteric windows for a lot of needlessly complicated nerd stuff. Just close them. In the beginning you only need:

-> disassemly view (aka "IDA View-A")

-> hexadecimalview (aka "Hex View-A")

IDA-Pro won't display the opcode (i.e. the hex repesentation) in a line with the asm-instruction, so in "IDA(or Hex) View-A" window -> Right click -> Syncronise with "Hex(or IDA) View-A"


Navigation:

(G) or -> Jump -> Jump to adress [enter asm address]

For most objects: [mouse hovering = preview pop-up] resp. [double click = goto]

[click on dynamic var, subroutine or location] -> (X) or Right click -> Jumpt to xref to operant -> in pop-up "xrefs to..." [double click = goto]


As I suggested HERE, the more identified and labeled dynamic data & sub's, the easier to understand the code (for known data see: Modding Index -> Tutorials: -Coding and read posts in recoding section).

To rename them in IDA-Pro: [click on dynamic var, subroutine or location] -> (N) or Right click -> Rename [enter new label (has to be unique)]

Also you can add own notes in the disassemly view via -> Edit -> comments


What a code area could look like after this:

Code: Select all

AUTO:0044F439                 call    GetTaskForceEntry
AUTO:0044F43E                 mov     edx, eax
AUTO:0044F440                 add     eax, 38h
AUTO:0044F443                 call    GetSectorLstEntry
AUTO:0044F448                 mov     [esp+18h], eax
AUTO:0044F44C                 mov     eax, [eax+0Ah]
AUTO:0044F44F                 sar     eax, 10h
AUTO:0044F452                 xor     ebp, ebp
AUTO:0044F454                 cmp     eax, 0FFFFFFFFh
AUTO:0044F457                 jz      short MonsterSwitchViaShipID
AUTO:0044F459                 imul    eax, 328h
AUTO:0044F45F                 mov     ebp, ds:adr_systInfo
AUTO:0044F465                 add     ebp, eax
AUTO:0044F467
AUTO:0044F467 MonsterSwitchViaShipID:
AUTO:0044F467                 mov     eax, [edx+6]
AUTO:0044F46A                 sar     eax, 10h
AUTO:0044F46D                 call    GetGShipListEntry
AUTO:0044F472                 mov     ax, [eax+52h]
AUTO:0044F476                 sub     eax, 73h

Sometimes it's a bit confusing at first. E.g. following code actually reads the difficulty level (located immediately afterwards from the minor race setting, see info on dynamic variables):

Code: Select all

AUTO:0044C1E0                 mov     eax, ds:MinorRacesSetting+2
AUTO:0044C1E5                 sar     eax, 10h

Local variables: (current subroutine only)

Code: Select all

AUTO:00467316                 fmul    [esp+420h+var_10C]   // local variables (default)

AUTO:00467316                 fmul    dword ptr [esp+314h]   // standard asm repesentation

To switch between local-variable / standard-asm display -> [click on asm-instruction] -> press (K)

Whilst local variables view makes it easier to keep track of stack variables[esp+], since they can be labeled, the standard asm representation is still needed for recoding(cf. #2/3) and advanced analysis(for an example see HERE).




2. Thinking up the code changes


First you need to find the(all) responsible code sequence(s). In the majority of cases there is way too much code to change (e.g. adding new minors = 2 years+ fulltime task).

As an easy example we take: liberating foreign home systems (fix)

Here I just searched for the number of the lexicon.dic entry(606) in IDA-Pro -> (Alt+I) or Search -> immediate value (best -> Find all occurences -> navigate via occurences window)

Sub_48B2E0(load lexicon.dic entry number[eax]) is known from recoding posts, so it's fast to find:

Code: Select all

AUTO:004EBCB6                 mov     eax, 25Eh
AUTO:004EBCBB                 call    Lexicon_Term

Not too far above from this we find the switch statements for the liberation pop-up:

Code: Select all

AUTO:004EBC5A wish_to_liberate_?:                     ; CODE XREF: System_attack_GUI+619
AUTO:004EBC5A                 jnz     no_free_choice
AUTO:004EBC60                 cmp     byte ptr [ebp+30h], 3  // see systInfo
AUTO:004EBC64                 jnz     no_free_choice
AUTO:004EBC6A                 mov     bl, [ebp+4Ch]   // see systInfo
AUTO:004EBC6D                 cmp     bl, 23h
AUTO:004EBC70                 jz      no_free_choice
AUTO:004EBC76                 xor     eax, eax
AUTO:004EBC78                 xor     edx, edx
AUTO:004EBC7A                 mov     al, bl
AUTO:004EBC7C                 mov     dx, [ebp+44h]  // see systInfo
AUTO:004EBC80                 cmp     eax, edx
AUTO:004EBC82                 jz      no_free_choice

We need an additional check & switch for conquered home systems aka [systInfo+30h=6] i.e. 6 extra bytes at 4EBC64, but we can't simply insert more commands into trek.exe, because the length of the code must stay the same. So we have to determine the easiest/fastes way to optimise the syntax for the desired effects.

It's technically possible to use distant code, data from loaded res.files or even external (and more modern) code with newer techniques. But this doesn't mean it becomes less work. The best replacement code is as short, clear and minimally invasive as possible, whilst achieving the desired results.

The better the understanding of a code sequence and the BotF core files, the less work is required for manipulations (for unknown asm-commands: see links above, recoding examples and use google).


Coming back to the example, here I 'redirected' the jumps at 4EBC5A & (former)-4EBC70 to (former)-4EBC64 resp 4EBC82 (i.e. 6-byte long become 2-byte short = 8 bytes free). This was possible due to the matching jump targets and conditions (ungainly but fast to do).

Take another look at the unmodded code above:
4EBC5A -> 4EBC64
4EBC70 -> 4EBC82

Alternatively, one could remove/overwrite the owner-inhabitants check at 4EBC80 (in theory redundant, but in event of bugs...), but I'd always suggest the shortes possible replacement code.


Writing the new code to notepad: (optional in preparation for OllyDbg cf. #3)

Code: Select all

at 4EBC5A: 
jnz (skip next 3 statements i.e. goto address of "jnz 4EBD44") -> 2byte/nop
cmp     byte ptr [ebp+30h], 3
jz (skip next 2 statements i.e. goto address of "mov bl, [ebp+4Ch]") -> 2byte/nop
cmp     byte ptr [ebp+30h], 6
jnz     4EBD44
mov     bl, [ebp+4Ch]
cmp     bl, 23h
jz      4EBC82
nop...
4EBC76  xor  eax, eax  // remains unchanged

Take note:

Since the targets of the first two jump instructions are part of the new code, one would have to calculate the addresses. However, just typing in nop's in length of the instructions and going back once OllyDbg displays the target locations is faster and less error-prone.

Also note that all "xrefs to..."(cf. #1) have to be adapted when changing start point of subroutines and locations(i.e. targets of jumps and calls) !


Here is how OllyDbg displays the above code example: (je = jz)

Code: Select all

004EBC5A     75 0A           JNZ SHORT trek.004EBC66
004EBC5C     807D 30 03      CMP BYTE PTR SS:[EBP+30],3
004EBC60     74 0A           JE SHORT trek.004EBC6C
004EBC62     807D 30 06      CMP BYTE PTR SS:[EBP+30],6
004EBC66     0F85 D8000000   JNZ trek.004EBD44
004EBC6C     8A5D 4C         MOV BL,BYTE PTR SS:[EBP+4C]
004EBC6F     80FB 23         CMP BL,23
004EBC72     74 0E           JE SHORT trek.004EBC82
004EBC74     90              NOP
004EBC75     90              NOP

3. Rewriting the code -> OllyDbg


Start OllyDbg.exe -> Open trek.exe

(Ctrl+G) or Right click -> Go to -> Expression -> [enter asm address]

Doubleclick first code line you want to change and in "Assemble at ..." [enter new asm statement]

After this, line gets red, indicating that command is modified.


For documentation purpose:

[Mark modified code lines] (Ctrl+C) or Right click -> Copy -> to clipboard (& paste in your documentation file)

Column width is adjustable via mouse (e.g. for opcode length display)


Save code changes to exe:

Right click -> Copy to executable -> All modifications (ignore any strange warnig messages, just continue)

In pop-up "Copy to executable file" -> Copy all

In pop-up window with changed code (it's just a documentation!) -> Right click -> Save file [under different name e.g. trek1.exe]

Now just change the file names (e.g. trek.exe to trekold.exe and trek1.exe to trek.exe) and test your new code.


Pitfalls:

For some nerdish reasons OllyDbg sometimes...

- fails to read (i.e. display) its own code modifications correctly when analysing changed files (see hints by Flocke -> below)

- doesn't understand some asm-statements (e.g. fnstsw ax), fastest solution: in IDA-Pro (Alt+T) or Search -> text [enter statement] -> in OllyDbg look at the found asm-address (it's FSTSW AX)

- doesn't accept its own syntax (remove all labels e.g. call trek.00123456 -> call 123456)

- uses opcode different from BotF defaults. For most, it doesn't matter (e.g. xor commands), but sometimes might de-optimise length of code.

If everything else fails just type nop's & afterwards insert opcode via hex-editor.

[Flocke]

Great article!

- fails to read (i.e. display) its own code modifications correctly when analysing changed files (therefore the documentation hint above)

That's really a bit odd with OllyDbg. Seems like it's always referring to the stored original analysed code structure. I currently don't have it installed, but you can also remove analysis (or reanalyse?) selected lines. That's what I mostly do to view changed asm code.

the length of the code must stay the same. So we have to determine the easiest/fastes way to optimise the syntax for the desired effects.

Well, you mention it in the side notes, but just for clarification, if you're using a foisted dll for coding, you don't have this restriction. I told how to do this with gcc and mingw allowing for C++ and inline-assembler coding, other languages and compilers might be possible. It requires some work to setup the development environment, but once it's done it's a great benefit not to be limited by space in trek.exe and allows for alot new abilities like complex decisions and more structured coding.
And coding in a high level language like C++ is actually alot easier than in low level assembler.
One thing I'd like to see is the possibility to activate/deactivate or configure code fixes, e.g. using an ini file. Easy in C++, not applicable in pure asm.
So in case someone is going to learn all these asm coding aspects, make sure you also learn about some C++ coding, at least a few, and give it a try. Don't be scared by the complexity of C++, with some few old-style C coding you can achieve alot and if you're more happy with asm, use inline assembler.

[Spocks-cuddly-tribble]

Great article!

Glad you like it.

- fails to read (i.e. display) its own code modifications correctly when analysing changed files (therefore the documentation hint above)

That's really a bit odd with OllyDbg. Seems like it's always referring to the stored original analysed code structure. I currently don't have it installed, but you can also remove analysis (or reanalyse?) selected lines. That's what I mostly do to view changed asm code.

Multiple sources e.g. OllyDbg still uses unused/disabled code as reference for startingpoint of instructions, misinterpreting code areas as hieroglyphic-like ASCII texts. Might be that can be prevented via studying the manual, some protocol settings or one of the other trivias that no one don't gives a damn about...

the length of the code must stay the same. So we have to determine the easiest/fastes way to optimise the syntax for the desired effects.

Well, you mention it in the side notes, but just for clarification, if you're using a foisted dll for coding, you don't have this restriction. I told how to do this with gcc and mingw allowing for C++ and inline-assembler coding, other languages and compilers might be possible. It requires some work to setup the development environment, but once it's done it's a great benefit not to be limited by space in trek.exe and allows for alot new abilities like complex decisions and more structured coding.
And coding in a high level language like C++ is actually alot easier than in low level assembler.
One thing I'd like to see is the possibility to activate/deactivate or configure code fixes, e.g. using an ini file. Easy in C++, not applicable in pure asm.
So in case someone is going to learn all these asm coding aspects, make sure you also learn about some C++ coding, at least a few, and give it a try. Don't be scared by the complexity of C++, with some few old-style C coding you can achieve alot and if you're more happy with asm, use inline assembler.

It is self-evident that the benefit, of an even only partial code-upgrade, increases with the complexity and quantity of the added, bypassed or replaced codes. Also needless to say projects like optimising tactical combat, main AI control or improving diplomacy options, shouldn't even been started without this option (that are long-time projects with a minimum average of 2-3 hours per week).

However, above instructions obviously are a first steps guide on how to start with simple editings (resp achieving significant results with max. 0.5h/week). The core message behind the comment was: Be realistic & disciplined. If you're still struggling with simplest things then don't waste time with triggering a snowball effect of additional required modifications.

I just added some missing noob hints to navigation & command labels, before I forget all due to retiring ^^.

[Flocke]

first steps guide on how to start with simple editings

Agreed, but in all those years I spent learning, one major realizing was that it always helps to first get an overview.
SCT, you fixed lots of bugs by asm coding. I'm not sure how many easy asm only ones are left, but I'm sure there are many easy things that could be done with help of some C++ coding. Don't mind me for mentioning this again.
Edit: Your article is more than just a collection of some asm hints & tips, it's an introduction to asm 'code fixing', a first steps guide as you said, so I felt it shouldn't miss this info, that's all.

Edit2: I got an idea for an easy example. E.g. one could implement an automatic population reassign for preventing starvation. Work of just a few hours or less once everything is setup (on turn end, loop all systems, check for food, reassign people if necessary, proceed with original turn end function). With some more effort this could be extended to replace the auto build by an 'auto control' to make it actually useful and make huge maps with countless star systems 'playable'. Dependent on how complex one would want the new algorithm to be this could be either a 0.5h/week minimal effort short time task or a major effort long time project. In both cases I'm sure it would soon result in noticable improvements and the challenge to start off isn't too high.