I stopped learning Japanese last weekend just long enough to work on an idea I’ve had in mind for a couple of years. I enjoy playing through Japanese games as source of study material but dictionary lookups can be a little time consuming. I wanted an application that could read text from a game and provide a means of instant dictionary lookup.
Talking to the Emulator
First things first, I needed some way of communicating with BGB, my Gameboy emulator of choice. I’d recently read though where the author wanted to display the Japanese game text by the side of the English game on SNES games. He had looked into the possibility of communication via scriptable emulators but without much luck and had decided to read the memory of the emulator instead. As I’d seen seen evidence that this approach could work I decided to go down this route.
Where’s the Japanese?
The first idea I had for finding the Japanese text currently displayed on the screen was to look for some kind of unique identifier for the current line, and then reading it from the ROM after having loaded the whole ROM into memory. This is the same technique that is used in the article I previously mentioned. The downside of this is that any placeholders e.g. for a player name, location, character name and so on would be missing and I had decided Pokémon was a good candidate game for my application but didn’t want text that read “[placeholder] sent out [placeholder]. Go [placeholder]!” and “[placeholder] used [placeholder] attack”.
After a bit of thought I came up with the idea of looking for one of the background map areas in the Gameboy’s VRAM (Video RAM). These define which tiled graphics are used to draw a background to the screen by referencing areas of the Gameboy’s character RAM which is where the tiled graphic data is stored. Once discovered the idea was then to read an array of bytes and map the values of the bytes to Japanese characters. This approach will only work providing the tile data always stays in the same place though I’m guessing this will be the case as it would be more bother for the developers to keep moving it around.
Some Background Info
With the Gameboy (and with many consoles of the era) there is no such thing as “text”, there are only tiled graphics so any text you see on the screen is merely a series of tiled graphics strung together. For this reason the Gameboy can’t print “hi”, instead it is told to display two tiles on the screen at a given location and those two tiles contain an image of “h” and “i” characters. The tiles are 8×8 pixels and to the best of my knowledge, most games that deal with a lot of text typically store a single alphanumeric character in a single 8×8 tile. I imagine it would get very messy otherwise. Anyway, let’s take a look at the image below for some clarification.
In BGB’s VRAM Viewer window (pictured above) you can see all of the tiles held in the Character area of the Gameboy’s VRAM (memory area 8000-97FF). Some of these will be used to display what is currently drawn to the screen. In this instance you can see that each hiragana and katakana character, including numerals and special characters, has been loaded into VRAM and occupies a single tile. The fact that the whole character set is present in VRAM is good news as it means they are likely static and don’t move around throughout the game. Not all games work this way however but my idea relies on them staying in the same place so that when I read the reference to tile number B1 (pictured above) in the Gameboy’s background map I can safely assume it is an あ and not some other character. Apart from being reasonably text heavy this is another reason that Pokémon was a good choice.
I’m going to show you what one of the Gameboy’s background maps actually looks like. There are two and they are nothing more than defined areas of VRAM. They start at one address and end at another. According to this Gameboy memory map they exist at 9800-9BFF (background map data area 1) and 9C00-9FFF (background map data area 2). Using BGB’s debugger we can easily find them.
The largest pane in BGB’s Debugger window shows all of the Game Boy’s memory but is focused on a particular area of interest starting at 9D80, placing it somewhere with the second background map memory area 9C00-9FFF. I’ve highlighted the first byte 79. This is a reference to the address of a tile stored in the character (tile) memory area which ranges from 8000-97FF. In this case 79 references memory address 9790. I carried on looking down the list and saw how this same code was also responsible for the picture of Professor Oak currently on my screen which really confirmed my initial assumption that I’d found part of a routine for populating the screen from the background map.
Anyway, look back at BGB’s VRAM Viewer. The large graphic towards the top right shows a visual representation of the 16 bytes of data contained between address 9790 to 979F. 16 bytes is exactly how big a tile is in memory. Back in the memory area of BGB’s Debugger window we can see that the next tile reference is 7A and this is repeated several times until we get to 7B. Now take a look at the bottom row of tiles back over in BGB’s VRAM Viewer window. You should be able to identify tile 79 which is also enlarged at the top right of the window. Next to it on the right sits 7A followed by 7B. You should be able to see how the image on the screen is being mapped now. We have 79 which is the top left corner of the text box you can see in the game screen, followed by several 7A’s which make up the top of the box, followed by 7B which is the top right corner of the box. You can keep on following this until the whole box has been covered including its text.
What’s the Address?
Up until now we’ve been using BGB’s tools to look at the memory of virtualised Gameboy hardware but our application won’t have access to these tools. It will need to look inside BGB’s allocated memory area inside our computers RAM to find the virtualised Gameboy memory areas we are interested in.
We will need another tool to enable us to search inside BGB’s allocated memory. Initially I used OllyDbg but switched to Cheat Engine as I found it a little easier to work with despite it not really being built for this task. With the game open in Cheat Engine I needed something to search for. The text はじめまして！ was currently shown in the game and by using BGB’s VRAM Viewer we can work out this will be stored as tiles CA, BC, D2, CF, BC, C3, and E7 (the BC tile is used for both じ and し but more on this later) and should appear in this order somewhere inside the virtual Gameboy’s background memory, which is somewhere inside BGB’s allocated memory. Now we can start looking for an address to use in our application that will take us straight to the background map memory area within BGB’s allocated memory.
As you can see in the picture above I found two address where the array of bytes I was searching for exists. At first I thought this might be the echo area of RAM. The Gameboy has an area of RAM you’re not supposed to use that just echos the content in C000-DDFF. However, the background maps aren’t stored in that range so I’ve no idea what the second address is for, but it seems to contain a full copy of the background map.
I decided to go with the first address (for a reason I’ll explain later) so now I want to try something, I close BGB, start it back up again, same game, same point in the game, check 0237DDCC1 and I don’t see CA BC D2 CF BC C3 E7 anymore. Not really surprising as every time a program starts it is allocated a different range of memory to run with.
I searched for the array of bytes again a few times. It moved from 0237DDC1 to 0240DDC1 to 023FDDC1. Interestingly the last two byes remain the same which I didn’t expect. I checked BGB’s VRAM Viewer again and found that the array of bytes I was searching for starts from 9DC1 in the virtualised Game Boy’s memory. I guess the developer of BGB has kept the last part DC1 aligned to memory in the same way (by the way, this is why I went with the first address and not the second). I can certainly see how it would be faster instead of mapping onto offset memory values requiring an additional calculation on every access.
Handy as it is that the last part is always DDC1, I now need a way to get the whole address. I decided to see when this area of code was being accessed and attached Cheat Engine’s debugger to BGB’s process. Using “Find Out What Accesses this address”. I threw up a couple of instructions and the first looked quite interesting. I could see that at some point register EAX held the address where my array of bytes started from and that this address had been moved from register EBP into EAX. I hit the “Show Disassembler” button to find out more.
The disassembler shows that the value in register EBP had been created by adding registers AX + EDI together. I decided to toggle a breakpoint on mov eax,ebp which was just after the lea ebp,[eax+edi] instruction so I could take a look at the values of the registers EAX and EDI prior to EAX being modified.
As you can see in the image above. EAX contains 0241C000, EDI contains 00001DC9 and the result of the two added together, 0241DDC9 was in EBP. As I ran the code I saw EAX remained the same and EDI was incremented by one. This was probably part of a routine for drawing a background map to the screen.
After seeing how the address I was interested in was comprised of two addresses being added together, the first of which wasn’t as I ran through it I decided I’d search for an array of bytes containing 0241C000. But first, I was curious about something and decided to poke a little further. Cheat Engine has a “Find out what addresses this instruction access” feature. This gave me a long list of memory addresses that looked like areas of the background map. The first address in the list was 0241DD80. Let’s browse this memory region.
In the middle of the picture you can see an area of memory. At the top our address 0241DD80 with contents 79 (a tile number). Look at BGB’s VRAM Viewer on the left and you’ll see tile 79 is the top left corner of the box that text appears in during the game. This should now look very familiar as is the same thing we were previously looking at in BGB’s VRAM Viewer.
Anyway, back to the task at hand. I searched for an array of bytes containing 02 41 C0 00 but with no luck. Then, remembering how the address was being built in the dissembled code I saw decided to try searching for just 02 41. This produced 71 results even when hitting the “Next Scan” button a few times so I added all the addresses to the addresslist. Closed BGB, opened it back up again. Loaded the same game at the same place. Opened BGB’s process in Cheat Engine making sure I kept the old addresslist. Found the address I was interested in again and noted down the first part of the address which had changed to 02 3C now. I then deleted all of the addresses from the addresslist that didn’t point to 02 3C and repeated this process once more. I wasn’t able to shave off any more addresses the second time so I was left with six addresses that pointed to an array of bytes containing 02 3C, the first half of the address I was interested in.
I went with the first address in my list 004FE295. Now I’d be able to read the first two byes of this address then append DDC1 and this would give me a reliable way of drilling down into the area of memory in the BGB process I’m interested in that would persist across restarts of the software.
Read Between the Lines
I’m ready to start reading tile values and converting them to Japanese text. At first glance it would appear there are two lines of text in Pokémon but there’s actually four. Two lines are used to display the Japanese characters and two lines are used to display the dakuten (dots above Japanese characters a.k.a voiced) and handakuten (circles above Japanese characters a.k.a half-voiced) when in rendaku (voiced characters) form. Take a look at the image below to see how this works.
You can see how the gameboy screen is comprised of 8×8 tiles and how each character takes up one of those tiles but what about the じ and ポ characters? The dakuten and handakuten both sit on their own lines of text for these. This was done because there wasn’t enough room for all of those characters in their rendaku form.
I’d need a way of converting these characters in rendaku form back into their unicode Japanese character equivalent. This was done by processing two lines at a time, tile by tile and checking to see if the rendaku line contained a dakuten or handakuten and then converting this to the appropriate Japanese character.
User Interface
By this point in time my application was just spitting things out the console but it needed a proper interface. The instant lookup dictionary I wanted to use was Rikaichan which I use on a daily basis. This is a web based so I’d need a web based interface to make use of this. After I got a quick HTTP server piping out my data I wrote a quick HTML page and some Javascript to poll my application for changes and update only when a change is found.
Once this was all working I started working on filling my tile number to Japanese character map. Thankfully the Pokémon games don’t use any Kanji apart from 円 for the money so I didn’t have to spend too long getting all the characters and symbols in.
Game Over
I’d now got a working prototype that could assist in my studies. The prototype is written in C# though I’ll admit C++ seems like a better fit for this but I wanted a chance to play around with C# and it seemed like a good language for a quick proof of concept.
I’m yet to discover whether or not the text tiles move around as I’ve not completed the game yet and so can’t say for sure but fingers crossed that this continues to work!
There are a few improvements I’d consider making to my application. In the near future I’d like to rework the code to support other games. In the distant future perhaps I’ll use WebSockets instead of polling as it feels cleaner to me and I might even consider reworking the code to support other emulators.
