“Stage e№” – An Introductory Guide To Reperverse-Engineering Visual Novels

I’ve written this for one of my online friends back in 2010. Despite its age, even 14 years later it doesn’t look dated – IDA’s interface is just as horrible, WinAPI is just as stable and basic x86 assembler is no different from what it was 40 years ago. Reverse-engineering fundamentals did not change either – and probably never will.

This 30-page guide is intended to give an all-round insight into how RCE is performed. It requires no special knowledge except only for more or less common mechanics of how a computer works (but this does not mean it will be easy). It’s a big plus if you can program a little but you don’t have to be able to read assembly code – you’ll learn this and more things as you progress through these pages…

…If your motivation is a real thing, that is. People often ask me to teach them how to hack a Visual Novel but so far I don’t know of anybody from that group who has completed this guide. In fact, I’m not even sure if those people have started it at all. Well, it’s their call… but maybe you are different? O_~

Foreword

Before we begin, let me tell you a few words about how it all started for me…

I became interested in Visual Novels a few years before I became interested in hacking them for translation purposes – which happened near September of 2008 when I suddenly realized that I could see through that gibberish coming out of OllyDbg, that same assembly code that was previously scaring me to faints. Later I have set up a page with a dozen of VN tools I’ve made. With this tutorial I want to give you a head start, something I clearly lacked back then – and also prove that RCE is hard, but not off-limits to mere mortals.

And with this, the game was afoot

Part 1 – The Bruteforce

First thing you need is the test script interpreter from here. Extract the EXE and DAT files somewhere but don’t look at the source code until we’re done (it’s written in Delphi 7).

Second thing you need is a hex editor. If you don’t have one I suggest 010 Editor (a trial will do for now) – I used to use WinHex but it sucks when working in Japanese locale, becoming virtually unusable. 010 Editor is an amazing powerhouse for all things related to binary data.

Now we’re all set for the first part.

So, do you see that runme.dat in the Scenario Runner’s directory? That’s a “scenario” that it runs. Launch the EXE file. The scenario is simple, as well as the interpreter itself – it outputs a string, then asks you a question and depending on your answer it will either ask you the same question again or output a message and exit.

Your job in this tutorial is to change the first string it outputs – that’s what we do when we need to translate a game. And the scenario should remain workable!

We’ll go brute force for the start. Open runme.dat in a hexed. You can clearly see some strings like “Make me laugh!” and others. What if we change one of them?

Let’s edit the first string making it “Do not cry…” – and as you will notice we have an extra “!” left from the old string so we’ll just go ahead and delete it (too bad for the string). The file will become 1 byte shorter.

Orange – edited bytes. Note that the final “!” is gone.
Editing the string, the “rectilinear” apprach

Save your changes and run the EXE. Oh-huh, we get some exception and also we see that it output some “☺” before dying – while it’s good to stay positive it’s not exactly what we hoped for :)

We probably forgot to update something, for example, a string length. Naturally, the app should know how long the string is. How would it do that?

I’m sure you can look around and find where the problem is on your own. Riiiight?


Here’s what we have got: 0E 00 44 6F 20 6E 6F 74 20 63 72 79 2E 2E 2E.

Those 0E 00 bytes look exactly like the string’s length, don’t they (blue selection on the above screenshot)? We need to do a quick conversion to check this supposition: in 010 Editor, hit F11 (Tools | Base Converter) and enter 0E in the hex field – it’s 14 in decimal notation. Actually, you could also do this by setting the caret before 0E and looking in the Inspector panel, under Unsigned Short

You can also use the lightweight Notepad 2e for Windows which can not only convert the bases but also calculate entire expressions, encode strings, etc.

Oh, and I forgot to tell you an important rule: before touching any files always make sure to back up the original versions in case you mess them up.

So now update that length byte and re-run the script. Wow, that’s cool, the script is working! Job done 8)

…Well, not exactly so as it turns out after a bit of investigation – the script works normally only if you pick the second choice on the following branch, otherwise the program crashes. It says that the bytecode is corrupted… Oh noes, did it die?

I’ll now allow you to explore the script with the hexed so you can try to find where the trouble is. It’s fine if you can’t fix it yet (else you wouldn’t be needing this tutorial) but avoid blindly following into my steps – poking around on your own will accelerate your learning. When you’ve used up all your mental powers – carry on to the most fascinating part, and one that’s a pluck for most acolytes – debugging.


Part 2 – IDA Pro & The Assembler

A debugger is an incredible tool that allows us to read other people’s minds… even if figuratively. But, it does allow us to literally read a program’s code and memory which is a good compromise, eh?

For this part we need IDA Pro (version 6 is better if you can get it). Unpack it somewhere (it works without installation) and Open our ScenarioRunner.exe (IDA might show a “friendly start-up dialog” but being friendly doesn’t go well with being IDA so just skip it). After picking the EXE file IDA will display a dialog box about its file type; we don’t need it – hit Enter.

After a few seconds IDA will have finished disassembling the EXE. You’ll notice this – the bulb icon on the right of the third panel row from the top changes color from yellow to green.

So, we should be at public start right now. If for some reason you’re not there, hit G and type “start”, then hit Enter.

We’ll now jump straight into the action – let’s run the program from within IDA and see how it works. Unlike Olly, IDA has several debuggers but most of the time you will need just one – select Local Win32 Debugger from the menu and press F9 to run it. (In IDA Pro version 5 you don’t need to select it because Local Win32 Debugger is the only one it supports.)

Nothing extraordinary happened. The program runs as it does without a debugger – and that’s exactly what we want since we can peek at its inner mechanics while it’s running around peacefully… muhaha >:3

Beeル Break: A One-Page Intro to the Assembly Language

Here I present some very fundamental info, the bare minimum. You will learn the rest of asm as/if you keep going.

You might already know that any CPU has registers which acts like memory slots (RAM) but much faster. A “register” can hold a 32-bit value. The registers we will need for our purposes (omitting x86_64 registers and specialized registers like FPU’s):

  • EAX, EBX, EDX, ESI, EDI, EBP – general-purpose registers (exact use depends on a compiler). Consider them as simple variables (as in math – X, Y, etc.).
  • ECX – normally used as a counter in cycles: for (ECX = 0; ECX < length; ECX++) { ... }
  • ESP – “Stack Pointer”, points to the top of a thread’s stack. A stack grows every time a function is called and shrinks every time a function returns. Functions put their own variables there (in addition to the registers). Stack values on addresses < (less than) ESP are unused (free).
  • EIP – “Instruction Pointer”. Points to the next instruction to be executed by the CPU.

Most registers can be directly changed by instructions like MOV (“move”) except EIP which is changed by J*, CALL, RET, etc.

All these registers are prefixed with “E” for a reason: each “E*” (“Extended”) register holds a 32-bit value. In fact, you can address registers ending on “X” (which probably also meant “eXtended” in even older times) as 16-bit and 8-bit: EAXAX (16-bit) → AL & AH (8-bit). If AL is “L” and AH is “H” then the layout of bits of a register (one letter – one bit) is like this: HHHHHHHH LLLLLLLL. Changing Low/High parts of a register doesn’t affect its other parts. Note that you can’t access top 16-bit register of “E*X” in this fashion.

Constructions like EAX:EDX create 64-bit “super-registers” but you will rarely see them – certainly not in this project and not in most VN engines.

Now for the basic assembly instructions:

MOV dst, src

Copies src into dst. MOV EAX, 2 is the same as EAX = 2; in C. Most operations that have the form of OP reg, otherReg (like this MOV) put the result back into reg thus modifying its content: reg = reg [op] otherReg;.

That said, some assemblers (notably on *nix) render arguments in reversed order: MOV src, dst. Windows assemblers tend to use Intel’s notation so we won’t be concerned about this here.

MOV/LEA dst, [src]

[ ] around an operand cause that memory address to be read (like *(src) in C or (src)^ in Pascal). If EDX = 0x401FDE20 then MOV EAX, [EDX+4] sets EAX to whatever DWord is at 0x401FDE24.

LEA does the calculation only, no dereference so LEA EAX, [EDX+4] sets EAX to the number 0x401FDE24 (not to the content at that address). This instruction exists because in x86 asm you cannot really write MOV EAX, EDX+4 – you have to use [ ] to calculate expressions, and using them for MOV automatically implies dereferencing.

PUSH src / POP dst

Puts src into / gets dst from the stack. A “stack” is simply a memory area which is aligned on a 4-byte boundary (i.e. each item in a stack is a DWord that starts on an address which can be divided by 4). The address of the last used stack value (“stack top”) is stored in ESP.

XOR reg, byKey

Boolean XOR: reg = reg ^ byKey;. Has two notable uses: encryption and assigning zero to a register (as anything XOR’ed against itself is 0). The latter is because on the x86 platform it’s faster and more space-efficient than MOV (just 1 byte instead of 2-4 bytes). So you can think of XOR EAX, EAX as of MOV EAX, 0 or EAX = 0;.

AND/OR reg, byReg
ADD/SUB/IMUL/IDIV reg, byReg

Other Boolean (bitwise) and integer arithmetic operators: reg = reg + byReg;.

TEST/CMP reg1, reg2

These two make asm’s conditions tick. It’s hard to explain in brief what exactly they do but usually it’s enough to keep in mind that the result of running these instructions is put into the flag register (technically called EFL) which is then accessed by J*. E.g. if we did TEST EAX, 0 and our EAX was 0 then ZF (zero flag) is set and if one of the following instructions is JZ addr then it will “jump” while if it is some other type of J* then it will be skipped over.

JMP/J* reg/addr

These make execution resume from another location. Used in conjunction with TEST and CMP. You can think of them as of MOV EIP, reg. JMP is an unconditional jump while its other forms (JNZ, JGE, etc. which I call J* here) are conditional jumps testing bits in the flag register, such as ZF (zero flag or zero bit).

CALL reg/addr / RET [bytesToPop]

As their names suggest, they have something to do with function calls. CALL can be thought of as a shortcut for PUSH EIP; JMP addr and RET – for POP EIP. RET can be sometimes written RETN.

Attention: RET xxx has nothing to do with the function’s return value as you might have thought. Functions usually (depending on their “calling convention”) return their result in EAX (note this!). xxx is the number of bytes to pop off the stack (“cleaning the stack” on return). So it’s like: SUB ESP, bytesToPop or like this cycle with POP:

while (bytesToPop > 0) { POP tmp; bytesToPop -= 4; }

If you’re interested why this form exists and why it’s rarely used (particularly in C and C++) read about calling conventionsstdcall (as used in WinAPI) and those of Pascal and C (also see a side note later). But it’s not vital to know in this tutorial.

Back To IDA

This should be enough to get you started. Now we can go back to the disassembly. I’ve uploaded some docs about Intel asm instructions so you can always consult them – they have every existing instruction, literally every one (yes, modern desktop CPUs are woefully complex).

So now we’re on public start. We don’t yet understand anything because everything has meaningless names like CALL sub_40276C – right, “sub_” is a prefix indicating a function (“subroutine”) but what does this name tell us? Nothing, that’s why we should start giving things proper names!

I’ll give you a pointer. Gray lines starting with “;” are comments and IDA (as well as Olly) puts some useful info into those areas. For example, we see this:

    MOV     EDX, offset aScenariorunner   ; "    * * * ScenarioRunner demo... "

Can you guess what is it? It’s a reference to a string (address of its first byte), and “aScenariorunner” is a name that IDA has auto-chosen for this string (if you go to Options | General | Strings you will see the Prefix field which you can change; default is “a”). This particular autogenerated name looks okay so we’ll leave it as it is.

Let’s think about the purpose of the code block where it’s used:

    CODE:00413F8E                 MOV     EDX, offset aScenariorunner
    CODE:00413F93                 CALL    sub_4049E4
    CODE:00413F98                 CALL    sub_4032E4
    CODE:00413F9D                 CALL    sub_40276C

We can make a deduction that one of these functions outputs a line to the console. Which one? Best guess is the first since it seems to accept EDX as an argument and because it’s closer to MOV than other CALLs. However, this might not be the case – maybe sub_4049E4 is only preparing something and its result is passed to a later CALL which is the actual “write-to-console” function.

So let’s find it out. What we need are breakpoints. A breakpoint is just a “point” at which normal program execution will “break” (pause) and the debugger will take control, allowing us to inspect the thing. Both in Olly and IDA breakpoints are set by F2; in Delphi – by F5.

Put a BP on the first function call (sub_4049E4). After you press F2 the line will be highlighted in red.

Now start the program by F9 and IDA will immediately pause on the BP we’ve just set. Look at the Scenario Runner’s console window – confirm that it’s yet empty. Hit F8 now to “walk-over” the current instruction. Now the cursor is on the line with a call to sub_4032E4. Look at the console again – huh, it’s still empty! Then my guess was wrong. But no problem – press F8 again and yup, that second function was indeed the one we were looking for because a line was produced on screen. We must have found the output function! Isn’t that great?

Put the cursor on the second CALL, to anywhere inside the sub_4032E4 text and press N – this opens a rename dialog. Enter some meaningful name for the function, e.g. “WriteLn” as it is called in Delphi and press OK. By the way, I suggest you prepend the names you give with some symbol (I use “$”), so that (1) you can quickly distinguish the names you gave from the autogenerated ones (2) they appear on top of the name list when sorted.

So, I’ve named this function $WriteLn.

We should also take care of sub_4049E4 – although we don’t know what it does we still need to give it some name so when we see it next time we can at least remember that we have already encountered it in this context. Since we don’t know the call’s purpose let’s name it something like $IsCalledBeforeWriteLn – we can always rename it later.

If we don’t do this we’ll most likely end up in a situation when we’re lost in a mess of unnamed functions, although we might have seen many of them – we just don’t recognize cryptic names like sub_40FB68 as something that we already know. Your own names, even if they are just “SpookyGizmo” or “CalledLastNight” make navigating the boundless assembly code more doable and eventually might point you in the right direction.

Now it’s time to take care of the weird IDA Debug workspace that it creates by default. You can customize it as you like. Here’s mine, for example:

My version of IDA’s debug workspace
My version of IDA’s debug workspace

So, we have used one method of determining a subroutine’s purpose – by examining string(s) that it accepts. In fact, strings are like beacons for us reversers in the ocean of asm code – strings are what we see and what connect us to the original program source, which is mangled but thriving deep within the disassembly…

And we’ve also used another method – by setting a BP before a function, stepping over it and looking what has changed after its execution. This doesn’t work always, especially in GUI apps but it’s the shortest way if it does.

You can explore the functions on your own now and when you’re done we’ll begin to search for the actual interpreter’s loop which is like Holy Graal for a VN hacker.


Part 3 – Finding Clues and The Case

Summon us and thou shall see the light for we are the Imported Ones…

True, strings are like beacons but there’s an even better thing – imported functions. They also connect us to the program’s source code, although in a more subtle way than strings because we don’t exactly see them on screen but rather feel them being used somewhere in the core, he-he >:3

The table of imported functions is the number one target for EXE “protectors” – programs implementing various tricks so that debuggers and disasms (which usually come packaged as a single tool) like IDA and Olly won’t see it… at least without some effort. However, our EXE is not protected (and most VNs are not) and so its import table is simply an array of DWords – Pointers to each function’s first instruction (in kernel32.dll or elsewhere) and hence direct arguments for CALLs.

You will notice that almost every system function ends on either “A” or “W” (e.g. TextOutA and TextOutW). “A” stands for ASCII while “W” stands for Unicode (also called “Wide” because each symbol takes up 2 bytes instead of 1). ASCII strings take up less space than Unicode ones but they are only able to represent a limited set of characters which is a common PITA when localizing programs whose authors believed that the only natural language on Earth is Japanese… or English.

Alright, now let’s get more specific to our problem. We need to find a function that is the interpreter’s loop – since generally a script interpreter has a loop which reads an instruction from a script, goes through a (potentially very large) switch..case block, “interpretes” it and… well, we shall see what’s next once we locate that.

To give you an idea here’s a sample interpreter’s loop written in pseudo-code that understands 3 opcodes, each taking a single argument of type “string”:

    function RunScript(script) {
      pos = 0;
      while (StringLength(script) > pos) {
        switch (script[pos++]) {
          case 0:
            WriteConsole("New message: ", ReadStringFrom(script, pos));
            break;
          case 1:
            varName = ReadStringFrom(script, pos);
            SetVar(varName, Random());
            break;
          case 2:
            scriptName = ReadStringFrom(script, pos);
            RunScriptNamed(scriptName);
            break;
          default:
            throw new Exception("Unknown command's opcode.");
        }
      }
    }

So, in short: I suggest that we find all calls to ReadFile, set BPs on them, run the program and watch for something to happen.

Open the Imports tab (Open | Subviews | Imports). We could find the function by looking through the list but a faster way is to type first few characters of the function’s name (simply with that list focused). If you press F1 you’ll get some help on IDA’s lists, they have other handy features like searching by Alt/Ctrl+T.

Press Enter and IDA will transfer you to the tab with the code, to that function’s record in the import table… But we need code that is using it, not just that record so let’s press Ctrl+X… Wait, there’s only one place? That’s strange. Press Enter to go to that part.

Ah, so this is some kind of a wrapper function: JMP DS:__imp_ReadFile. In fact, Delphi tends to use them a lot while Visual Studio’s compiler produces direct jumps. Anyway, we are dealing with Delphi here and we need the actual code so let’s find what refers to this function – again, press Ctrl+X. We got another two matches, great. Go to and set a BP on each of them.

Now, if you were paying attention then you must have noticed that there were in fact two ReadFile entries in the Imports list. This can be – and we’ll have to set BPs on all of them. For me that second entry is also a wrapper and Ctrl+X inside that wrapper reports 1 more call meaning that in total you must have 3 BPs set.

Now we have 3 BPs set. As a reminder, we are looking for a place hinting at some connection with runme.dat. Let’s roll! F9 – and we got the first client!

Among other arguments, ReadFile takes a file handle, a buffer and a number of bytes to read. The first is the best clue we can have since it’s an unique ID connecting with any given file… but to match it to a real file on disk we’d need to observe CreateFile (which takes a file name and returns that unique ID – different on each run!). We could set more BPs on calls to CreateFile, observe when it is called with lpFileName set to …\runme.dat and note down somewhere the file handle it returns (by the way, do you remember that functions usually return their result in the EAX register?)…

Nah, going for CreateFile is too troublesome. We’d like to avoid doing more steps than necessary (since that’s something we can always make up for). First, let’s examine other clues – we have two more useful arguments: bytes to read and buffer. Well, try bytes to read, shall we?

With execution paused on the CALL ReadFile line, examine the series of PUSHes before it – each one is “passing” a parameter (in reverse order, so the PUSH closest to CALL is the left-most argument if you look at the function’s declaration in C or Pascal below). The Stack view (reopen it from the Debugger | Debugger windows menu if it’s not visible) highlights last “passed” parameter (on “stack top”) with a blue line (if you don’t see it then right-click anywhere in this view and call Jump to ESP). The line under the highlighted one is the previous parameter (i.e. the second from the left in a C/Pascal declaration). By matching PUSHes with these lines you can see which arguments were passed to ReadFile. (Later we’ll see how to use IDA’s hints for this.)

    BOOL ReadFile(HANDLE hFile top, LPVOID lpBuffer top-1,
                  DWORD nNumberOfBytesToRead top-2, ...);
Stack structure before a call to ReadFile
Stack structure before a call to ReadFile

My numbers are: hFile = 0x50 (doesn’t tell me much, and yours will be different) and nNumberOfBytesToRead = 0x0143. Let’s hit Shift+/ and open IDA’s calculator, which can be also used for base conversion (although 010 Editor’s or Notepad 2e’s are probably more convenient). Let’s enter 0x0143 – I see that’s 323 in decimal which I suspect is… Hurray! Check the size of runme.dat – it’s exactly 323 bytes. How handy, the program seems to read the entire file into memory.

If you’re getting 0x80, not 0x143 then you have missed the second import entry. Game over, try again!

As for us, we got lucky – our test subject is naive, it’s reading the whole script into the buffer, right into our arms >:3

Now it’s time to track down what it’s gonna do with all that data it has just read.

The program is paused on the call to ReadFile. Before we let it continue, open a new tab with the view of the memory which is going to be filled with data (lpBuffer) after ReadFile returns. IDA offers multiple ways to do that:

  • Locate the expression with the address of that memory that was PUSHed (in our case it’s just the register ESI), right-click on it and call Jump in a new window, or set the cursor on it and hit Alt+Enter.
  • Alternatively, call Jump in a new hex window to get a view very similar to a hexed. This may be more convenient in our case.
  • If you were to do a double click or press only Enter then it would navigate you to that address in the same code tab. If that happened then go back any time by pressing Esc. This is useful when exploring sub-functions.
  • Finally, you can locate the line with this parameter (lpBuffer) in the Stack view, right-click on it and call Follow in disassembly (but this will navigate the current tab instead of opening a new one).
  • …And you could also do that from the context menu of the ESI register in the General registers tab.
Different ways to open a new tab in IDA
Different ways to open a new tab in IDA

Did it? Good, now leave it opened, switch back to the code tab and press F8.

Now IDA is highlighting the line after ReadFile meaning that Windows has read whatever data the program has requested. Switch to the memory tab opened just before – now this is the memory area that has the contents of the file (prior to that it had some junk). Doesn’t it look like our precious scenario bytecode? Compare with what you see in 010 Editor… It sure does!

It’s just about time we use hardware breakpoints to see what the program is doing with this data.

There are two types of breakpoints: hardware (“hwBP”) and software (“swBP”). What we’ve used until now were swBPs and they are triggered when the CPU is about to execute an instruction (in fact, a swBP is simply an asm instruction – INT 03 which the debugger writes over the “real” instruction). HwBPs don’t rely on INT 03 – they are triggered when the CPU executes an instruction that accesses the memory on which any hwBP was previously set (which can happen more than once for a single BP).

Assuming you have that buffer tab active, set the cursor somewhere inside the lpBuffer area (e.g. on its first byte), press F2 – a dialog will appear and IDA will likely have automatically checked the “hardware BP” flag for you. See that Mode is Read (“break when reading this value, not changing it”) and press OK.

We don’t need anything anymore from swBPs set on ReadFile calls so you could remove them – but I suggest only disabling them (from the context menu) so you can get back to them quickly if necessary. You can open the Breakpoints tab by Ctrl+Alt+B, by Debugger | Breakpoints | Breakpoint list or by a button on one of the (many…) toolbars.

Now press F9 and wait until something happens… Here we go – “Hardware breakpoint… has been triggered”. That’s nice, let’s see what we got here…

REPE MOVSD. Well, it might sound scary but it’s simply an asm instruction that copies a block of memory from one location to another. If you want more info consult the Intel docs and search for that instruction.

We can now undertake a challenge of setting BPs on every REPE instruction we get (that’s not the only one, I promise) unless we hit something useful… or run out of hwBPs… but we’ll take another route – press (or even better – hold down) F8 unless we find something of interest. This way (holding F8) we’ll go up the call tree towards the root (which is public start) because we won’t go inside new functions (we’re not holding F7) and will gradually RETurn from all subroutines. On the way we’ll need to be on lookout for something of potential value.

…After five functions or so I got tired of this and I decided to press F9 again – maybe I’d find something in another part faster. Duh, IDA has displayed the same function again, just another branch. I guess I need to be more patient with F8 this time… (Yes, RCE involves a great deal of improvisation and uncertainty.)

Looks like we found a case statement’s graph!
Looks like we found a case statement’s graph!

After a dozen of returns I stumble upon a wide Graph that looks like a switch..case statement. See those boxes going from one root and then joining together on the bottom? If you think about it, that’s exactly how a case statement could be visualized (more on this below).

IDA has even identified this case for us by putting comments like “switch jump” all around the disassembled code. Olly can do this too but not as good.

Could it be the interpreter’s loop we’re looking for? Let’s check the comments and strings we have in this function. Hmm…

Well, so far the code doesn’t tell me much about its purpose. The only thing that looks interesting to me is a referenced string that IDA put in a comment that says “opcode %.2x” (let’s take a note on this – I wonder why we don’t see it in the console?).

A mysterious referenced string

I’ve got an idea: I’ll disable all BPs for now and set one at the beginning of this function… Actually I’ll set it at the case’s beginning – which must be here as suggested by IDA:

    JMP     off_41374A[EAX*4]   ; switch jump

So put a BP on that JMP and hit F9… Let’s look at the EAX register (you can either look at it in General registers window or put a mouse pointer over EAX and wait until IDA shows a hint).

So for me IDA’s hint says that EAX = 0x01. This doesn’t tell us anything, probably yet. However, I have a strong conviction that this is the function we were looking for so before we go deeper™ let’s rename it – I called it “$InterpretInstruction”.

Now roll back up a little and review how EAX gets this value. What we see is this (try to guess what it does before reading on):

    CODE:00413735 CALL    sub_413AA0
    CODE:0041373A XOR     EAX, EAX
    CODE:0041373C MOV     AL, BL
    CODE:0041373E CMP     EAX, 6           ; switch 7 cases
    CODE:00413741 JA      short loc_4137B5 ; default

Firstly, it clears EAX by XOR’ing it against itself like MOV EAX, 0 (likely because it held the return value of the preceding CALL), then it sets EAX’s lower part (AL) to some value of BL (as you remember BL is a low-word of the 16-bit register BX which is in turn part of the 32-bit register EBX). We need to track how BL is set…

Or do we? Let’s take a break and draw a deep breath.

What’s our goal with this quest? We went to change one line in the script but after we did that it started crashing. So we need to (or must, if we’re pressed by the group -_-’) find why.

This kind of self-conrol is important when reversing desktop programs – their code is dozens of Megabytes long, it’s all too easy to get swamped in there. We almost rushed into finding which function sets that BL – but that’s not necessary for us to know. We’re getting sidetracked! The thing we really need to know is what it does with that byte (that is now in AL/EAX), not to find which one of those zillion disassembled functions picked it from the bytecode stream (runme.dat).

So for now one part is done – we have supposedly found the interpreter’s case statement. We can verify this in a few different ways but in this tutorial I’ll show you how Olly shines with its marvellous BP logging facility (IDA has something similar but it involves Python which we really don’t want to do here).


Part 4 – Getting to the Crash Point

And then Olly the Mighty stood up and said: “I am the king of this hill!”.

Get the freeware OllyDbg and load ScenarioRunner.exe into it. That said, I am going to use v1.1 because v2.0 at the time of this writing (2010) still doesn’t have all the features of v1.1.

Since we’ve already analyzed quite a lot of code with IDA this task will be a piece of cake for us. Copy the address that IDA shows in disassembly listing on the left of the JMP instruction (case statement’s start) – for me it’s 00413743. Hit Ctrl+G in Olly and put in there.

Of course, Olly’s code listing is similar to IDA’s but because it doesn’t show the code as a graph we see that the case’s jump table is declared right under the JMP instruction (if you pressed Space in IDA it’d show it too).

    CMP EAX, 6                             ;  Switch (cases 0..6)
    JA  SHORT Scenario.004137B5            ;  <= jump for default case
    JMP DWORD PTR DS:[EAX*4+41374A]
    DD  Scenario.00413766                  ;  ** Switch table used at 00413743 **
    DD  Scenario.00413787
    DD  Scenario.00413790
    DD  Scenario.00413799
    DD  Scenario.004137A3
    DD  Scenario.004137AC
    DD  Scenario.004137C6
    LEA EDX, DWORD PTR SS:[EBP-10]         ;  Case 0 of switch 0041373E
OllyDbg’s breakpoint dialog
OllyDbg’s breakpoint dialog

Now we’re going to set a breakpoint that will log the, presumably, instruction codes (opcodes) that this function is passed. As we’ve already determined, the opcode is stored in EAX.

Select the line with JMP and press Shift+F4 (or right-click, then Breakpoints | Conditional log). Olly has support for complex conditions which is described in detail in its help file. I’ll only show you the basics – the interface is pretty much intuitive anyway (unlike IDA’s, kekeke).

There are 3 radio groups with 3 choices: when to Pause the program, when to Log expression result and when to Log function arguments. Each one can have a value of Never, On condition or Always. By combining these settings we can create very flexible breakpoints.

In our case we only need to log the value of the expression we enter so make your dialog look like one on the screenshot.

Let the program run freely now with F9 (by default when a program is run from the debugger, Olly will pause it at the entry point). The log is populated by lines similar to these:

    00413743   COND: Instruction code = 00000006
    00413743   COND: Instruction code = 00000000
    00413743   COND: Instruction code = 00000001

After that Scenario Runner’s console window presents us with its meaningful question followed by a crash. Perhaps it thinks it could stop us with that? Ha!

Get back to 010 Editor with the runme.dat file opened and carefully review the contents in the beginning:

    0000h: 06 00 0D 00 44 6F 20 6E 6F 74 20 63 72 79 2E 2E  ....Do not cry..
    0010h: 2E 01 29 00 59 6F 75 20 73 65 65 20 61 20 63 6C  ..).You see a cl
    0020h: 6F 75 64 2E 20 57 68 61 74 20 64 6F 20 79 6F 75  oud. What do you
    0030h: 20 74 68 69 6E 6B 20 61 62 6F 75 74 3F 02 21 00   think about?.!.
    0040h: 4E 6F 20 6A 6F 6B 65 2C 20 49 20 74 68 69 6E 6B  No joke, I think

I have highlighted the 4 bytes (06 00 01 02) whose purpose is unknown to us. Everything else is a part of some stringlength bytes (see, their right-most bytes are all 00?) or characters (note how none of those are below 20 and above 7E? Check the ASCII table to understand why). Hmm, in fact… don’t these 4 bytes look familiar? Compare them with Olly’s log messages – they are exactly the same in the exact same order! MAH BOI, we’ve scored a real hit even though we still don’t know what that 02 byte is supposed to mean.

To reiterate, this means that we’ve found the function that executes script instructions – the interpreter’s loop. You can even guess what the 06 opcode does and why it points to the instruction after the case statement – the answer might sound strange at first but don’t worry, strange is the norm :)

Close Olly, we’re getting back to IDA. Reactivate our shenanigans breakpoints on ReadFile, run, then set a hwBP inside the buffer which it reads our runme.dat into, then disable that new BP. In Olly you can disable a BP with Space in the Breakpoints window but in IDA you’ll have to use the context menu, for once.

Now we should wait until the program asks the question – but don’t choose anything when it does. Recall: when we answer “2” it exits correctly while if we choose “1” it crashes. Here goes the question… okay, now we can re-enable our hwBP to see what the app is going to do when we choose “1”. Gotcha, something’s been triggered. What a twist! It’s REPE MOVSD again – this must be our lucky charm :)

Hmm, let’s slide down a bit using F8… code, code, code… aah. Aah! What’s that? It says: “read str of len %d”. Interesting! Let’s slide down more… What? We’re already in $InterpretInstruction? So this parts seems okay. What does it say on the console? Aha, it output a message that we’ll need to try that again. No crash yet. You just wait, machine >:3

$InterpretInstruction returned successfully, nothing got broken so the error doesn’t seem to be in this instruction. That’s not surprising, actually, because why should it break on a simple message output? We probably broke a jump instruction or something more complicated than that.

I’m thinking… What if we set a BP on that case statement in $InterpretInstruction? At least we will learn after which instructions the program breaks.

On my side, this function received the following opcodes: 05 then 00 then 04 – I found this out by simply setting a BP on the case statement and quickly pressing F9. One thing worth our attention is that after 04 IDA says that the program has risen an exception. Must be our target >:3

Whatever you answer to IDA’s question on how to handle the exception the program is done for so we’ll need to restart it – but we will continue for educational purposes (choose to pass the exception to the program). Whoa! It looked like the program’s normal execution got somehow transferred to another place in one instant. Well, we don’t really care unless it works… Uh? Come on, how could we end up in the default case if we did enter the 5th (04)? So the program actually doesn’t run sequentially?

Well, all of this is sure fascinating to know but it doesn’t help us understand why the script is crashing.

Let’s look at the bytecode, maybe it will reveal new secrets to us?

    0100h: 61 67 61 69 6E 21 20 3A 50 0A 04 12 00 00 00 00  again! :P.......
    0110h: 2F 00 57 6F 77 2C 20 74 68 61 74 27 73 20 73 75  /.Wow, that's su

I think we’ve got something here. Disregarding strings, we have some strange numbers (04, then 12 00 00 00 and 00) right after the opcode which writes the message after which the program rises that exception (I suggests we dub 04 as Crashing Opcode™ between ourselves). As for 00 in the end – it must be another opcode, maybe for displaying a message?.. Because look, it has a string length going right after it, and then the message itself. That’s something you should investigate once we’re done. Meanwhile, in between 04 and 01 we have 12 00 00 00 – I wonder if it’s just a coincidence that it looks like a DWord? Need to check this out.

Go back to IDA, open the tab with the bytecode buffer and scroll down where you can see the end of “You’ll need to guess it again! :P” message. We see the same bytes that we saw in the hexed above. Let’s put a hwBP on every of the following bytes!.. Okay, maybe that’s an overkill – we are only concerned about those 12 00 00 00 and in fact we can cover all of them with just one hwBP since a BP can span 1, 2 or 4 bytes (IDA supports other sizes but I personally wouldn’t advise to use them; OllyDbg too supports just those 3 sizes).

Now as the guide is nearing the end you’ll have to work this out on your own. To recap, what you need to do is this: set a hwBP in the bytecode buffer immediately after ReadFile returns, track all copies of that buffer and in the end find a place where the program does something useful with that number (that is, not just copying). Easy, huh?

Of course, the hwBP should be set on that DWord (12 00 00 00) so that we land right on the instruction that has done something with this 32-bit number. I know you can do it, and it took me just a few jumps :)

What will definitely help you is reading those Intel docs on instructions you don’t know – like REPE MOVSD. Don’t worry if you can’t find exactly REPE MOVSD, some other similar instruction (REPE MOVS) will do too since the only thing you need to know is the purpose of that instruction and, in case it copies something somewhere, which registers hold the source and destination addresses.

You can do it! Go-go-go!

Unlimited Crash Works Mode on!

Part 5 – The Final Act

Two crashes for the price of one!

I assume you did your homework and found a place where that number gets involved in some obscure machinations:

    CODE:0041390F loc_41390F:
    CODE:0041390F XOR     ECX, ECX
    CODE:00413911 MOV     EDX, [ESP+0Ch+var_C]  ; <= here it got trapped
    CODE:00413914 MOV     EAX, [EBX+14h]
    CODE:00413917 MOV     EBX, [EAX]
    CODE:00413919 CALL    DWORD PTR [EBX+14h]
    CODE:0041391C JMP     SHORT loc_41392f

Now is the time for a few last tips. First, without love it cannot be seen take a look at EDX’s value – it’s 0x12, exactly that number we were trapping for (12 00 00 00). If MOV’s brackets confuse you then check here again.

I got curious and peeked into the memory at [ESP+0Ch+var_C] (hover over it or use G) – well, it looks nothing like our bytecode from runme.dat, that 0x12 is sitting there all alone and sad. However, it must have been copied from our bytecode because we put hardware breakpoints on places originating from that ReadFile’s buffer and as for why it ended up there… we don’t really care. What’s important is that it’s getting assigned to EDX and then something gets CALLed – we need to know what happens there.

Good for us, the code seems well-written… okay, that might have been a bit biased… Anyway, the functions are small and this one looks particularly small – just step inside it (inside that CALL) with F7 to see for yourself. It consists of only 6 instructions including RET:

  sub_412278 proc near

    SUB     CX, 1
    JB      SHORT loc_412287
    ...
    loc_412287:
    MOV     [EAX+0Ch], EDX
    JMP     SHORT loc_412297
    ...
    loc_412297:
    MOV     EAX, [EAX+0Ch]
    RETN

  sub_412278 endp

The highlighted part is about as a critical needle in the haystack that is a game engine as the interpreter’s loop is: it’s the program’s variable that holds the current position in the bytecode (its “EIP”). This is a core thing that lets us see how the program interprets the bytecode, how it transitions through it, what exactly it does along the way. Ultimately it leads us to specific places where those values are used – because to access something inside the bytecode you need to use a pointer like this one, and that access operation must be quite close to the code that actually uses the accessed value (not merely copies it). Basically, you just put a hwBP on this variable and no longer care about jumping through hoops by trapping ReadFile buffers.

So let’s see what happens when we run the original, unmodified runme.dat. When it reaches 04 12 00 00 00, it jumps to the 18th byte (0x12 = 18) just like asm’s JMP changing EIP. There it reads opcode 01 followed by a string “You see a cloud…” (29 00 59 …).

However, our modified version has made the first string (“Make me laugh!”) shorter by 1 byte. When 04 jumps, the 18th byte happens to be 29 – but it was supposed to be part of the string! So the runner tries to interpret 29 as an opcode and fails.

Woot! With this knowledge we are able to produce a localization tool for this script engine.

One thing you can do after locating this pointer is to log the motions of the script engine through the bytecode. Open OllyDbg and put a Conditional log BP on the location of $ScenarioPos (obtained from IDA) like we already did and let the program run. The last position in the scenario (inside the runme.dat file) before the exception occurs turns out to be 0x12 (which also happens right after the 04 opcode is interpreted) meaning we’ve just confirmed that 12 00 00 00 we saw earlier is an “offset” and 04 is a jump instruction.


True, we could have guessed all of that by simply looking at the bytecode. We could have also figured what kind of offset it was (relative to the current position or absolute from the beginning) without going through all this code hunting. However, in a real VN engine with a ton of code before yourself you won’t always be able to decipher instructions so easily – scenario files can be Megabytes in size, the engine may define a hundred of valid opcodes (ISM SCRIPT of Sisters ~Natsu no Saigo no Hi~ defines 122) and so our guess that 31 32 33 00 is a DWord number valued around 8,000,000,000 might be as good as that it’s 2 Bytes and a Word (0x31 – opcode, 0x32 – variable’s index, and 0x0033 – relative jump distance), or that it’s a null-terminated a string (“123” in ASCII), or that 31 32 is a per-string encryption key while 33 00 is its length, etc.

Before we wrap up, let me quickly outline another way of getting through the “crash point”. Previously, we made a supposition that 12 00 00 00 was indeed an offset (which is a valid supposition you could have made when working with a real VN) and breakpoints helped us locate $ScenarioPos. This was a “forward” approach because we went from the start – from the moment the program obtained the scenario data with ReadFile we were scooping every place using it until arriving at sub_412278. But we could go from the end instead, back-tracking from the point where it was crashing:

The problem:
Our ScenarioRunner.exe is raising an exception on a modified scenario.
-9.
It happens in the default case branch of $InterpretInstruction (we have already reliably identified this function) which means the case can’t handle that value as there is no separate branch for it.
-8.
Since we’re sure that this is the interpreter’s case statement we conclude that this scripting engine can only handle opcodes 00-06 and this “opcode” that it received (0x29 in EAX which we confirm by putting a swBP at the beginning of the default branch) is not an instruction’s code at all.
-7.
Thus we guess that it has read that opcodes from a wrong position, which is the same as saying that we’ve shifted something when we’ve altered a string in the scenario before this position. Obvious stuff so far – but who holds the key how does it determine from where to read?
-6.
We examine how EAX got its value (yes, this is exactly the step I have cautioned against in the beginning) by back-tracking from the case branch upwards – EAXALBLEDX… step out of $InterpretInstructionDL is getting set from some local variable for which the only way to receive a value is via the preceding CALL.
-5.
We set a BP on that unassuming CALL, restart the program, check the variable’s value, step over by F8, check again and confirm that it’s changed. This means the opcode is coming from that CALL.
-4.
Next we continue by F9, note down the opcodes this variable is receiving, observe the crash after it gets 04 (first) then 29 (second), restart, skip until it gets 04 again, press F9 just one more time – it breaks on that CALL but the variable hasn’t been filled with the following “opcode” yet (29). From here, we set a hwBP on the variable’s memory area (it’s a DWord) then chant a prayer and pull the trigger hit F8
-3.
Sure enough, we land in another form of REPEREP MOVSB. What’s worth of noting here is that before F8 the variable’s memory contained some junk but now it’s holding 29 which is exactly the “opcode” causing the crash. I feel with my gut that we’re excruciatingly close to the solution but where precisely did this value come from?
-2.
The Stack view shows a few calls between the “parent” function (with our breakpoint) and this trapped one. From quick glance we don’t see anything promising in the function with REP so we move up the stack, poke around instructions and registers on that level…
-1.
…And claim that we’ve ran out of the midnight oil and go to bed for we have just found variables holding not only the script pointer ($ScenarioPos) but also the pointer to the bytecode buffer itself (after undergoing all the copies!).

It's hard to say which approach is more optimal. There’s no silver bullet. Sometimes setting hwBPs on read buffers will bear fruit quickly (even though it’s done by hand and therefore is tedious). At other times you pin-point the issue faster if you go in reverse (from the last executed opcode) – for example, in our demo I reached the “-1” step within just two minutes. Try to do the same for practice – it’ll be easer than with a real VN because functions in this EXE are extremely small.


Ready to see the answer? It’s in sub_412240 (comments are mine):

    MOV     EDI, [EBX+0Ch]  ; [EBX+0Ch] - location of $ScenarioPos in memory.
    TEST    EDI, EDI        ; Is it <0?
    JL      SHORT loc_4     ; Yes - then jump. (It should always be >= 0.)
    ...
    MOV     EAX, [EBX+4]    ; The pointer to our bytecode buffer!
    ADD     EAX, EDI        ; = buffer's base address + current script position

Should have these two variables appeared separately (like in our first “forward” approach) it would have been harder to recognize their purpose. However, with this last statement everything is coming together. The rest is trivial now that we have identified them.

At last, we’re coming out from the fog… unless we are moving in the opposite direction!
At last, we’re coming out from the fog… unless we are moving in the opposite direction!

Regardless of the way you have arrived at $ScenarioPos, it’s safe to say that the 04 opcode, the most critical line of which is this:

    MOV [EAX+0Ch], EDX    ; $ScenarioPos, 0x12

…is doing nothing other than setting $ScenarioPos to the value of the DWord that it has just read (in our case it’s 12 00 00 000x12). If we needed a final proof that this instruction performed an absolute jump (making the script continue execution from an arbitrary position in the bytecode) then we just got it.

And we’ve modified a string right before the position to where it jumps…

No surprise the program crashes – it doesn’t understand why it’s getting some weird 0x29 “opcode”…

Gimme just a second to fix it!

Switch to the hexed, Ctrl+G to 010Bh, change 12 to 11 and run the program. Muhaha, it now works like a charm, totally disregarding the modification of the first line! >:3

— FUS-RO-DAH!
— FUS-RO-DAH!

Epilogue

You have been through this tutorial. If you managed to finish it and understand the backbone of what we were doing – then kudos! It means you have become part of the 31337… and also that you’re on your own now. You can enlarge your, uh, skillz by messing further with Scenario Runner which has a few hidden Easter eggs. Or, if you feel it’s not exciting enough then pick a VN (ideally one that hasn’t been translated yet) – but be warned that even a tiny VN has a much more complex engine than this demo.

Here’s more stuff you can do with ScenarioRunner.exe for practice:

  • Remember that suspiciously-looking string saying “opcode %.2x”? Actually, if you look over the entries under the Names subview of IDA (click on the Name column to sort the list) you’ll find other interesting strings (they should have been also listed in the Strings subview, Shift+F12 – but IDA doesn’t catch Delphi and/or Unicode strings by default).

    For example, aReadStrOfLenD – what is this for? Why none of them show up in the console output? You can investigate this – maybe it will help you in tracking down crashes in Scenario Runner or give some insight into its core operations, who knows? (Spoiler: indeed it does if you manage to pull this trick with a VN like Haeleth did with RealLive.)

  • You can undertake a challenge of understanding every opcode function. We’ve already got the notion of the 04 opcode and I’m sure you’ve understood a few others but there are at least 3 of them left – and also that strange 06 case branch – what is it used for?
  • Try to translate (change) every other string in runme.dat – there are 6 lines in total, including 2 question strings. We’ve already “translated” the first line. There’s a surprise awaiting you when you modify one of the remaining lines – you’ll need to dig into the disassembly to solve it :)
  • Just for fun, try finding out quickly how to make ScenarioRunner.exe execute an arbitrary scenario file, with a name other than runme.dat.
  • Can you make your own custom scenario (or modify the default one) so that it would do things you want it to do? I made this runme.dat from scratch in a hexed – can you do something similar? It will likely require good knowledge of most opcodes – you will have some fun.
  • And the ultimate challenge – try writing a complete decompiler and compiler of Scenario Runner’s scenario files. Or at least try to make a translation tool like ones on my page (e.g. msdcomp) which can both extract texts (from a script file into a text file) and update texts (inside the script based on lines in a text file). Working out a finished solution like this one is bound to give you one or two level-ups.

But, you know… regardless of what you do from now on, remember just these two things, amirite?

  • Do it only for lulz
  • Lurk more

Thanks for reading!

~ 11 May 2010 & 25 February 2020

Visitors' Comments

Your avatar Note: comments are premoderated

Home page