PCEngine-FX.com

PCE-FX Homebrew Development => Localizations, Games, Apps, Docs => Topic started by: megatron-uk on 02/05/2014, 01:03 PM

Title: Cyber Knight translation
Post by: megatron-uk on 02/05/2014, 01:03 PM
Can anyone confirm what I believe to be the case with Cyber Knight? That it, its script and artwork assets are stored as compressed in the rom?
I'm having no luck with Tile Layer Pro, TMOD2 or YY-CHR when trying to viewing any sprite or font data.


This thread started out in early 2014 as an attempt to find any English text or assets in the RPG/Strategy game Cyber Knight, it has now turned into a full English translation project.

Translation tools, patches and other assets will be continuously updated on my github page and anyone can download and use them. The project now has a 99% complete Ascii/Hiragana/Katakana translation table, a 99% complete Kanji translation table, working extractor and injector scripts and work has started on writing patches for the game text. These take the form of individual JSON (http://en.wikipedia.org/wiki/JSON) that correspond to a particular section of the rom file. This way we can extract and re-insert seperate sections of text and not have to create a single, giant all-encompassing patch.
The patch files mostly correspond to certain, common sections of the text - all the ship menu structures, all the item names, battle screen dialogue, dialogue for each world/town etc.

https://github.com/megatron-uk/cyberknight-pce (https://github.com/megatron-uk/cyberknight-pce)



Edit (April 18th, 2014):
The battle interface is almost 100% complete - all menu options, status text, messages from the computer etc and all translated. There are some minor spacing and linebreaks to test visually, but the text itself is all done. Unfortunately a side effect of doing more translating of the main game dialogue has broken the scrolling intro - it will need work again (possibly someone to adjust it's location in the rom). Here's a video of several battles on the way to the first main quest in the game (the crashed ship 'Europa'):
http://youtu.be/DVnsNSLQ_Hs

Edit (March 30th, 2014):
Lots more translations done, virtually all weapons/items, character names and now in-game text too. Here's the full English menu structure for the ship:
http://youtu.be/xbZdix18ho8

Edit (March 3nd, 2014):
Scrolling 'the story so far' text that appears after the intro cinema is now translated. Updated patch files now on github.

Edit (March 2nd, 2014):
Intro cinema now fully translated!
http://youtu.be/solMaSfvP1Y

Edit (Febrary 28th, 2014):
Quick example showing some of the translated menu and dialogue: Cyber Knight PC-Engine initial translation test (https://youtu.be/j76cYe1r5H0#)

Edit (Febrary 26th, 2014):
Updated player character name entry screen, started translating npc name lists (incomplete).

Edit (Febrary 25th, 2014):
Main menu screen is now translated (patches/test.json on github) as a test of the extractor and injector utilities.

Edit (Febrary 23rd, 2014):
All code and assets I've produced to extract/inject script data will be available on github here: https://github.com/megatron-uk/cyberknight-pce (https://github.com/megatron-uk/cyberknight-pce)
For now this includes a 99% complete translation table and a 'mostly' working script extractor; which we need more text dialogue location information for!
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/05/2014, 01:44 PM
Hold on, I think I can see partial font data in TLP if I switch to 1BPP format (why?), but it doesn't look right - the fonts are all offset and corrupted, different to any of the other hucard games I've looked at so far.

One good thing about this is that it does have a full western character set (both upper and lower-case) in there though.

Tried Nana too - it also seems to show the font not being byte-aligned.

tlp.png
Title: Re: Cyber Knight assets (tiles/text)
Post by: TurboXray on 02/05/2014, 04:00 PM
Looks like data(a byte or two) is interleaved between some of the font graphics. Look at "C" and "D", appear to be corrupt. Some of the others, while not corrupt looking, are probably offset because of header bytes or control codes (for sequencing through the font?).

Edit:
 Actually, just looking at it - it appear to be compressed via RLE variant. Look at the "C", and look right at the first break. See that pattern of pixel? You see the same thing on the number "1". And the same thing for "D". The byte is probably something unique that doesn't exist in the whole font (graphically) and thus is a control code. When encountered, the following bytes are probably the paramenters for the control code (run length). Pretty poor compression method. They would have been better off using a mask 1bit repeat method. Typical lousy compression schemes uses on hucard games (ugh).

 Anyway, that's my guess.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/05/2014, 06:05 PM
That makes sense Tom.

I'm following some of MooZ's postings here (http://blog.blockos.org/?p=304) where he describes looking for writes to the vram at the locations where the font ends up in order to find out what  code is doing the decompression. According the bg/sprite viewer the font occupies 0x1210 to 0x2ff0 in vram.

I'll see if I can trace what is decompressing and loading the font.

font.png
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/05/2014, 06:45 PM
Well I think I found the code thats clearing vram before the copy:

EAD3    SEI
EAD4    STX $0002
EAD7    STX $0003
EADA    CLI
EADB    DEC $34
EADD    BNE $EAD3

This seems to be running for the entire vram region, so I presume it's clearing it before writing sprite/tile/font data. Still looking for the font decompressor, off to peeps now though.

John
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/06/2014, 05:33 AM
So it appears as though the ASCII used in the game is stored clear in the rom - the phrases I've found in English so-far:

"CYBER KNIGHT" - In the title bar of the (dreadful!) blue start screen (seriously? could they not do a better effort than that?) appears three times in the rom (once in the header, once misspelled as 'CYBER NIGHT' and once more for the title screen) and when edited to other ASCII codes represented by the included font is reflected in that screen.

"MAX!" - Appears in one of the dialogue boxes from the crew during the intro while the ship is under attack and is present in the rom exactly once; having briefly played the SNES translation I think it refers to 'Hyperdrive now charged at MAX!' or something similar.

Judging from the control codes embedded in some of the text strings I'm also guessing that 043c is the flashing 'more text' graphic that appears at the end of dialogue boxes a that sequence is very heavily used.

There's also the name of the ships computer "Mica", which occurs quite regularly throught the rom.

Also another point - when you create a new character the name of the player you create is stored as a plain ASCII or single-char Japanese representation at 0x2400 in main memory.

With that, although we don't have a font to extract (yet) because of the encoding (and, actually as it already has full upper and lower case western characters with punctuation, we wouldn't necessarily need to replace it), I should at least be able to make a start doing a table of characters to font tiles (as at least the western font tiles are sequential).

I guess what I'm trying to say is that although the graphic assets appear encoded in some fashion, the text itself now doesn't appear to be.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/06/2014, 06:41 AM
... and I think that that the code 0x52 switches between character sets - as it occurs right before and right after any ASCII text (when that text has Japanese characters before or after it), so it must be interpreted as a char mode toggle.
Title: Re: Cyber Knight assets (tiles/text)
Post by: TurboXray on 02/06/2014, 11:33 AM
Yeah, if the font has everything you need - you could leave it as is. But you will need a vram dump of it, and need to identify all those japanese characters, so when you dump the game script you'll now what char is one is referring to (I usually do a small table to convert the game JP chars to SJIS and write them to a doc/txt file). JWPCE will read plain text files with SJIS chars (and ascii as well). Just use the .SJIS extension instead of the .TXT
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/06/2014, 11:59 AM
Well I've got the vram screenshot of all the western and single tile Japanese characters and have now divided into a grid so that I can (slowly) go through it and identify all the characters and recheck that key phrases I've been looking at in Windhex start to look correct with the new table added. This bit is going to be slow...

I think the some of the control codes in the text strings are to switch to/from double width/byte characters as the second screen dump from Mednafen, further down the vram memory map, shows some Kanji and 'large' fonts. But it's not sequential like the first font set. You can see that you get the top half of one set of characters in one row, then the top half of the next, then the bottom of the first half, then the bottom of the second set.
Title: Re: Cyber Knight assets (tiles/text)
Post by: TurboXray on 02/06/2014, 12:22 PM
Ahh, I see. There's another approach I've seen over at RHDN. First, you have to find the main print routine and figured out the smallest and largest character values. Then you take that vram dump and write an app that goes through the entire valid character range, and spit out the vram assembled graphics to a bitmap file. Then present the file to RHDN or whatever translation site, and request an identification of said chars.

 Of course, you can do this without writing an app, but it's slower. You save state the emulator, over-write the corresponding text string with sequential values, and take visual snap shots of each set of sequential output to the text box.


 As far as the font itself, assuming you want to change it, you can always expand the rom and write a hook to monitor font decompression routine (because it could be used for more than just the font) and jump to alt code that updates vram with your new font. No need to decompress the old one and re-compress a new one.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/06/2014, 12:41 PM
I would probably just keep the font as-is, but expanding the rom image is a good idea. It would be nice to jazz up that main menu screen and some of the dialogue box bezel images.

To be honest, about 50% of what you've just said is probably over my head :mrgreen: but I'll keep plugging away to create this table in the first instance :)

Edit: actually, just looking at that second set of fonts, I bet it's only used in the in-game menus as it certainly matches the menu options I've seen in the game so-far.
Title: Re: Cyber Knight assets (tiles/text)
Post by: TurboXray on 02/06/2014, 12:52 PM
Have you done this type of translation hacking before? If this is your first project, it just takes a little bit of time to get used to all of this. Lol - after your first project is done, you feel like you can translate any game at that point - haha.

 If the script is uncompressed, then that makes this twice or three times as easy as a compressed script game. I really hate Huffman compression :P Necro-whatever PCE game uses it. I took a look at it for a hacker over at RHDN, because they're trying to translate the game. I really didn't want to write the decompressor... ugh. Dunno - something about huffman (binary trees) that I never liked/clicked with me (LZSS for me alllll day). If you talk to MooZ, see if he's interested in dumping the script to Necro-whatever for PCE. There's already a group waiting on the script. I have my notes, somewhere around here...

 That aside, have you thought about upgrading any of the graphics? They're probably in 2bit/3bit format. If you expand the rom, you could hook the character/sprite upload routine to upload upgraded 4bit versions of the graphics. A LOT of early hucard games used simple 1/2/3bit graphic formats, because there is no decompression resource penalty (planar graphics) and can be uploaded to vram on the fly without decompressing to a buffer first.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/06/2014, 01:05 PM
Quote from: TurboXray on 02/06/2014, 12:52 PMHave you done this type of translation hacking before? If this is your first project, it just takes a little bit of time to get used to all of this. Lol - after your first project is done, you feel like you can translate any game at that point - haha.
I haven't done any hacking before :D

I did a lot of Unix systems programming (C, Posix threads, OS design, that kind of stuff) about 15 years ago at University, but the last proper C/systems-programming was probably about 10 years ago.

The everdrive-fat code is the first bit of decent coding I've done for a long time (and I'm enjoying it again!); I'm a Unix systems manager in my day-job so the kind of stuff I do on a daily basis tends to be shell scripting, Python, Ruby, maybe a bit of Java. But the programming side of it is just a tiny bit (just enough to keep my skills from going rusty) it's mainly managing virtual machines, doing dba work, designing middleware libraries etc. Nothing as low level as this! :lol:

I think this is one that's definitely worth persevering with - the script is already translated via the SNES and everyone really rates the game.

Graphical upgrades would be the icing on the cake - the SNES version already has some better in-game sprites (although the ship navigation screens look nicer on the PCE, imo) and I would oh-so-love to put a snazzy mecha background on that menu screen 8)
Title: Re: Cyber Knight assets (tiles/text)
Post by: Xak on 02/06/2014, 01:57 PM
Very very interesting topic. thank you for posting and keep us updated.
Title: Re: Cyber Knight assets (tiles/text)
Post by: TurboXray on 02/06/2014, 02:23 PM
QuoteI haven't done any hacking before :D
Well, looks like you have a knack for it then.

 Both MooZ and I have a few PCE translation works under our belt. Feel free to ask questions if you get stuck.

QuoteNothing as low level as this! :lol:
It's not so bad. PCE arch' is fairly straight forward/clean compared to some other systems. Mednafen is a pretty powerful debugger, once you get the hang of it.

QuoteGraphical upgrades would be the icing on the cake - the SNES version already has some better in-game sprites (although the ship navigation screens look nicer on the PCE, imo) and I would oh-so-love to put a snazzy mecha background on that menu screen 8)
Let me know when you get to that point. I can definitely help out there (I have a lot of experience with writing hook code for loading alternate stuffs, for PCE related games).
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/06/2014, 04:45 PM
The text at the main title screen seems to be generated with the same/similar display routine as the dialogue in the rest of the game.

The text, as standard, looks like this:

0D 06 43 59 42 45 52 29 4B 4E 49 47 48 54 00
... which displays "CYBER KNIGHT", centrered in the dialogue box.

The first two bytes appear to be control codes and this particular sequence controls how much whitespace is prepended to the text before display.

By changing 0D 06 to 0D 01 and the 0D 15 it moves the text towards the left (1 space instead of 6) or towards the right (15 spaces instead of 6, which is too much, in this case).
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/06/2014, 06:16 PM
Quote from: TurboXray on 02/06/2014, 02:23 PM
QuoteNothing as low level as this! :lol:
It's not so bad. PCE arch' is fairly straight forward/clean compared to some other systems. Mednafen is a pretty powerful debugger, once you get the hang of it.
Oh, I don't know about that :wink: I think Unix with it's flat memory model has spoiled me! The whole bank and memory region register thing baffles me! :lol: I just trust the assembler to do it's job  [-o<

I'll certainly take up your offer when the time comes and I wouldn't be surprised if I post a few more questions relating to the translation process here or on RHDN when I get a little further.
Title: Re: Cyber Knight assets (tiles/text)
Post by: TurboXray on 02/06/2014, 08:13 PM
Quote from: megatron-uk on 02/06/2014, 06:16 PM
Quote from: TurboXray on 02/06/2014, 02:23 PM
QuoteNothing as low level as this! :lol:
It's not so bad. PCE arch' is fairly straight forward/clean compared to some other systems. Mednafen is a pretty powerful debugger, once you get the hang of it.
Oh, I don't know about that :wink: I think Unix with it's flat memory model has spoiled me! The whole bank and memory region register thing baffles me! :lol: I just trust the assembler to do it's job  [-o<
Well, the video/sound arch is. The CPU is accumulator based, so it's gonna be a little bit convoluted in that respect. Though IMO not any more so than x86 banked mode. Actually, less so - but that's probably because I've cut my teeth on 65x. Yeah, flat memory models are nice. But the banking on the 6280 isn't so bad. Better than both the snes and the nes. I usually keep important stuff near, and everything else is far jump/jsr (with a macro, just looks like a regular function call). But that doesn't apply to hacking. You have to learn how the game treats each 8k bank. Sometimes the games use a very clean/organized approach and it's easy to hook into that - other times it's more hack-ish in approach. All depends on how the developer structured their code.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/07/2014, 03:38 AM
I think I've just worked out that all the single-char Japanese characters run on from the ASCII numbering sequentially in Cyber Knight.

For example, look back at the first font image - all the ascii chars are in sequence, and they are represented by standard ascii hex codes in the game rom. Fine so far (but a nice easy way to start!).

Now, look at the font tiles that come at the end of lowercase 'z', a gun, a sword, a right chevron and a red block, followed by the single Japanese characters. If you number these with the ASCII sequence immediately following that of z=0x7a, then you get the gun=0x7b, sword=0x7c and so on. The Japanese characters also continue that sequence sequentially, so 0x81=small_o_symbol, 0x82=left_upper_right_angle and so on.

I haven't gone all the way through the first page of fonts yet, but from those I've tested so far (the main title screen is the easiest way of doing this - simply replace the ASCII codes for CYBER KNIGHT with them and it works 100% so far (ie you get "CYBER <gun><sword><chevron><block>"). If everything is sequential it will make it a lot easier to construct the mapping table.

The image shows the result of changing the ASCII sequence for "CYBER KNIGHT" to the hex codes:
0D 06 7B 7C 7D 7E 81 82 83 84 85 86 87 88 00Which begins after the end of lower-case 'z', and shows that the numbering of the font tiles is contiguous with the ASCII sequence at that point - what a stroke of luck! :)

EDIT: I have to say, every time I look at that dull menu screen I die a little bit inside. It's like they didn't even bother trying.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/07/2014, 04:52 AM
Ok, confirmed. These are the hex codes of the single byte fonts in Cyber Knight. It's very convenient that they're sequentially numbered.
I have a suspicion that the 0x1n codes refer to the dialogue box bezel tiles, but that still needs checking out.

EDIT: Ok, no the bezel tiles don't appear to work in the same way, replacing text with 0x11 (which would be the left hand bezel image according to the numbering sequence used for fonts) causes the game to reset on reaching the menu screen. That's fine, we're not interesting in hacking graphics yet.

(?action=dlattach&topic=16222&attach=1676&image)
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/07/2014, 07:36 AM
I've mapped all of the single character fonts to their ASCII and SJIS equivalents.

Although each character prints correctly when embedded directly within the menu text or one of the introductory cinematic dialogue boxes, the hex representations don't match what is in the dialogue text strings.

E.g

0C BC BD C3 D1 5C C6 20 45 4D 50 C0 DE D2 B0 BC DE 21
Displays the text as shown in the attached image, but the hex codes used to generate that display don't match those that my mapping of hex to font tiles show (other than the plain ASCII characters). Yet if I swap out a normal ASCII hex code (for example the uppercase 'E' at the start of 'EMP') with one of the Japanese charactrs my table says corresponds to 0xfb (a 'square' character) my lookup seems to work correctly, as the second image shows.

So it would seem that dialogue text is a bit weird. The thing is, there are more bytes in the sequence (at least 17, depending on what seems to be the start byte) than needed to represent all the characters (14), so it can't be compressed.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/07/2014, 08:05 AM
Hold on. I've just realised something - all the dialogue I've seen so-far in this game has been printed on two lines.

I wonder if that text is indeed compressed and contains data for both lines, with the top being (I assume) accents or other character modifiers that can be seen in the various screenshots I've posted? I think that the code 0x5c has something to do with it

Yes, I'm almost sure of it.

Let's look at the 'EMP' example:

Byte pos: 0  1  2  3  4  5  6  7  8  9  10 11 12 13 14 15 16 17
Hex vals: 0C BC BD C3 D1 5C C6 20 45 4D 5C C0 DE D2 B0 BC DE 21

Let's presume that the first sequence of bytes deals with both text lines, it then issues a 0x5c at byte 5, which I'm guessing means 'switch to literal one line output', it then prints a 0xC6 which does match my font table, a space, and then the ASCII sequence 'EMP'. At byte position 10 it then issues another 0x5c and presumably switches back to two-line mode so that it can print the remainder; some of which do require output on two lines.
Title: Re: Cyber Knight assets (tiles/text)
Post by: cabbage on 02/07/2014, 08:54 AM
the accents are the 0xDE bytes, which tell the game to stick a ゛ above the last byte/character. it is printed on the row of tiles directly above, hence the double-spaced nature of the text which leaves room for these dakuten

the 0x5C is a control code to swap between katakana/hiragana output
e.g. シ and し are both 0xBC and which one is displayed depends on which character set is currently active
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/07/2014, 09:10 AM
Thanks,

Byte 0xDE is one I couldn't find a match for in JWPCE - however, the character code I've got that matches the first Japanese text shown after the 'P' is 0xE0. And the letter at the start of the string should be a 0x9C, not a 0xBC.

So the katakana/hiragana toggle code is doing something with the relative value of the hex code (-32). The default behaviour appears to be hex - 32, then after 0x5C the address becomes literal. Would that seem to be right?

EDIT: Thanks for the clarification. So it would seem that I need to work out if the default behaviour is for one character set or the other (default based on the above example would be show font for address - 32), then, when I detect an 0x5c, swap to the other, and swap back again when I get another 0x5c.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/07/2014, 09:36 AM
Note sure I've quite got it.... if I work on the assumption that the default mode requires (hex-32) I go from this:

Byte pos: 0  1  2  3  4  5  6  7  8  9  10 11 12 13 14 15 16 17
Hex vals: 0C BC BD C3 D1 5C C6 20 45 4D 5C C0 DE D2 B0 BC DE 21

Giving: しすてむに EMPため―し!

To this:

Byte pos: 0  1  2  3  4  5  6  7  8  9  10 11 12 13 14 15 16 17
Hex vals: 0C 9C 9D A3 B1 5C C6 20 45 4D 5C A0 BE B2 90 9C BE 21

Which would print:
シス』あに EMP せい‐シせ

The first two characters are correct, but the next two are not - the middle is correct, as it uses the literal values after the first 0x5c. But the last sequence is incorrect as well.

Hmm, not as straight forward as I though.

The sequence should be:

Byte pos: 0   1   2   3   4   5   6   7   8   9   10  11  12  13  14  15  16  17
Hex vals: 0C  9C  9D  E3  F1  5C  C6  20  45  4D  5C  E0  DE  F2  90  9C  DE  21
Modifier: 0  -32 -32 +32 +32  0   0   0   0   0   0  +32 +32 +32 -32 -32  0  0

Giving: システムに EMPタメ‐シ!  (and matching the screengrab)
Title: Re: Cyber Knight assets (tiles/text)
Post by: cabbage on 02/07/2014, 10:01 AM
cyber knight does some things in strangely (like so many pce games, hehe)...

your table must have some mistakes, so try working from the data already in the game to rebuild it to match what the game expects.

hiragana and katakana are 1:1, so try making a table (from known data in the rom, not data you have changed to try and figure out what gets displayed from each value) just for hiragana...
then, the katakana part occupies that same space.
シ and し are both 0xBC in the game script. in this case they happen to be in different parts of vram, but that doesn't really matter if you're just trying to make a table
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/07/2014, 10:12 AM
So really I need a table with 3 columns:

hex value, before 0x5c, after 0x5c
Title: Re: Cyber Knight assets (tiles/text)
Post by: cabbage on 02/07/2014, 10:24 AM
are you making a .tbl file for use with romhacking?
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/07/2014, 10:29 AM
That was my eventual aim.
Title: Re: Cyber Knight assets (tiles/text)
Post by: cabbage on 02/07/2014, 10:44 AM
I'm not aware of any elegant way to support both character sets with a single table when they use swapping like this. Hopefully someone could enlighten us but I won't hold my breath hehe!
I know someone (Nightcrawler?) was working on a new table file standard which would allow for table switching, but I don't know if any tools support it yet (e.g. cartographer/atlas/hex editors)

You can just make two table files--one for hiragana and one for katakana--but that doesn't help very much for script dumping/insertion without special tools... hence the appeal of the new table file standard

Really, though, as long as the hiragana and katakana overlap 1:1, a good translator should be able to proceed if the whole script is dumped just using one or the other. The 0x5C would even serve as a flag to alert them to the switch for added context.
So they would be reading along, "blah blah blah japanese text <SWAP>some katakana data, shown as hiragana text<SWAP>some more japanese text blah blah", and just know that the text surrounded by <SWAP>s is meant to be displayed as katakana. It would be read just the same...
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/07/2014, 11:01 AM
Yeah, I think I'll probably have to write my own extraction tool (once I find out how to access all of the script) and a parser to load the 'swap' logic as each string is read - for example, the dual line thing doesn't make any sense as I can just lookup the correct symbol for a character with a dakuten/handakuten when I write the script file to disk (I'm hoping that the SNES translator will be willing to part with a copy of the translated script for that version).
Title: Re: Cyber Knight assets (tiles/text)
Post by: TurboXray on 02/07/2014, 11:16 AM
You already seem proficient in C, you should just write your own. I always write my own extractors and inserters.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/07/2014, 12:48 PM
Quote from: cabbage on 02/07/2014, 10:01 AMyour table must have some mistakes, so try working from the data already in the game to rebuild it to match what the game expects.

hiragana and katakana are 1:1, so try making a table (from known data in the rom, not data you have changed to try and figure out what gets displayed from each value) just for hiragana...
then, the katakana part occupies that same space.
シ and し are both 0xBC in the game script. in this case they happen to be in different parts of vram, but that doesn't really matter if you're just trying to make a table
I think this is the problem - I've got very little to go on, which is why I've been picking out phrases with at least some ASCII codes in them, as without that, I've got nothing to search for in a hex editor.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/08/2014, 05:00 AM
I've been using the menu screen to go through and make a new table. First off I did all combinations from 0x20 - 0xff and it matches my existing table. I've now added the 0x5c control code at the beginning and am doing a second column - so far it does seem to be a 1:1 mapping between Katakana and Hiragana, I think they switch places, but need to complete the table to confirm this.

e.g.
pre-0x5c a 0xFD = ン

after a 0x5c a 0xFD = ん
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/08/2014, 10:28 AM
Ok, I now have a translation of all fonts in the game, both those before a 0x5c control code, and those displayed after a 0x5c sequence.

There are about half a dozen fonts I cannot identify from the very small tiles included in the game, I may need some help with those at some point, but I have around 200 characters identified now.
Title: Re: Cyber Knight assets (tiles/text)
Post by: TurboXray on 02/08/2014, 10:54 AM
Have you figured out the pointer system yet?
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/08/2014, 11:09 AM
Not yet, I haven't really thought about where to start with that yet - Ive just been using some obvious text strings to work out the fonts, but I don't know how to start looking for all the locations of the text.

Any "pointers"? :lol:
Title: Re: Cyber Knight assets (tiles/text)
Post by: TurboXray on 02/08/2014, 11:48 AM
Lol.

If there's a chunk of text, the pointer block might be near by. You can usually identify a pointer block, visually, in a hex editor. The values in the pointer block might be relative, might 10bit, or might be logical (16bit). Or possibly something weird in format.

 Give me an address to some text, and I'll take a look.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/08/2014, 12:03 PM
These are the three main sections I've been using to extract the font details as they have ASCII characters in the display...

Main menu
Description, the main game menu.
Text reads CYBER KNIGHT
ROM location: 0001CB00
0D 06 43 59 42 45 52 20 4B 4E 49 47 48 54 00

Opening Cinematic
One of the first dialogue boxes displayed as the players ship starts to take fire. I would think most text in the game would be in a similar format or pointed to in the same way as this as it seems quite standard with all of the other dialogue box text (0x3c at the start and 0x04 at the end).
Text reads システムに EMPタメ‐シ!
ROM location: 0002A142
3C 0C BC BD C3 D1 5C C6 20 45 4D 50 5C C0 DE D2 B0 BC DE 21 04

Scrolling Intro
The intro text scrolls up the screen once the intro cinematic has finished and before the game begins.
Test begins AD2352.11.21 (and continues over several lines)
ROM location: 0001BAD6
41 44 32 33 35 32 2e 31 31 2e 32 31 00
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/08/2014, 07:29 PM
It's go-to-bed time here, but before I do, someone on RHDN shared a few interesting pieces of information from his looking at Cyber Knight quite a number of years ago:

0xC7EC - font decompression routine
0x2C21F - compressed font (*16:001F logical)
0x28200 PC - some dialogue, pointers seem to be 2-byte, little endian
$CEF5 - dialogue load routine
Title: Re: Cyber Knight assets (tiles/text)
Post by: TurboXray on 02/09/2014, 01:31 AM
Quote from: megatron-uk on 02/08/2014, 07:29 PMIt's go-to-bed time here, but before I do, someone on RHDN shared a few interesting pieces of information from his looking at Cyber Knight quite a number of years ago:

0xC7EC - font decompression routine
0x2C21F - compressed font (*16:001F logical)
0x28200 PC - some dialogue, pointers seem to be 2-byte, little endian
$CEF5 - dialogue load routine
I saw that table. But the text in the beginning of the game appeared to use hard offsets. So I didn't get any break points on that table yet. But I only took a 5 minute glance at it.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/09/2014, 06:25 AM
Looks to be a possibility of a second (slightly bigger) pointer table at 0x2D409 (rom + header) / 0x2D209 (no header).

I can't figure out the address format for either though - relative to start of the table, absolute plus some offset, nor how many entries are supposed to be in the table. More testing to do!

EDIT: Now found what appears to be yet another table at 0x26F4A (no header).
Title: Re: Cyber Knight assets (tiles/text)
Post by: NightWolve on 02/09/2014, 08:54 AM
Quote from: TurboXray on 02/06/2014, 02:23 PM
Quote from: megatron-ukI haven't done any hacking before :D
Well, looks like you have a knack for it then.
Yeah, could've fooled me!! Well, welcome to the fan tran club then, megatron. ;) I thought you were just knee deep in that Everdrive tinkering and what not. I didn't expect to find this all of a sudden from ya! Good luck, at any rate.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/09/2014, 09:18 AM
Ha ha! Thanks :) Still early days yet though.

Right now I'm trying to get my head around where these pointers are actually referencing - I've read David Shadoff's post here (http://www.romhacking.net/forum/index.php?topic=15209.0) and it makes sense, seeing as the PCE only maps an 8k window onto the rom at any point in time.

What I probably need to do is find an in-game piece of text, such as something said by one of the villagers in the first settlement you visit, work out the hex for the text being displayed, find the match in rom, and then try and find it in the CPU logical view in mednafen stepping through just before the dialogue displays.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/09/2014, 10:22 AM
Found what should be a helpful piece of dialogue text to track down what pointers are being used:

ROM position: 0x01DEFE (without header)
Hex sequence: 4D 49 43 41 A2 BB B8 BE DD 5C B4 D8 B1 5C B6 D7 20 CA BD DE DA C3 B2 CF BD A3 08

It's a phrase from MICA, the ships computer that appears whenever you try and wander off just after landing - basically if you go anywhere other than the city or your ship, that phrase appears.

Now I need to step through what happens just after triggering that phrase to see where the pointer to it is being loaded from.

(?action=dlattach&topic=16222&attach=1690&image)
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/09/2014, 10:55 AM
Is there any way of having Mednafen put a breakpoint at logical address? - as I'm having a bit of a pain tracking down where this text string is being loaded from and when the section of rom is being swapped in to be read from.

I'll probably call it enough for the weekend now - I've got to prepare for a job interview tomorrow.
Title: Re: Cyber Knight assets (tiles/text)
Post by: OldMan on 02/09/2014, 11:40 AM
*(page):(address)   => ie, *02:ffef  will break on that exact address only.
(addr)                      => ie         0fff  will break on that address in any page, when mapped in.

The second one will generate a lot more breaks; you have to check the mpr registers to see if it's the page you want.  Try both and see which one you need - thought it's probably the first, at this stage.
Title: Re: Cyber Knight assets (tiles/text)
Post by: TurboXray on 02/09/2014, 12:18 PM
There's also external address breaking that isn't bank mode: so *15:8c44 can be done as *2ac44. And you can do a range as well *2ac44-*2acff. And you can have multiple range reads, but just make sure your 'long' address ranges come last in the window. So something like: 57ff 2066 7ff0-7fff *2ac44-2acff. Another pro tip; you can put more arguments/parameters than what will fit in the text window. Use the arrow keys to scroll back and forth, because it won't show the whole thing.
Title: Re: Cyber Knight assets (tiles/text)
Post by: burn_654 on 02/09/2014, 12:39 PM
When I've been working here and there on Momotarou Katsugeki the script worked much the same way - that control code to switch between hiragana and katakana, and a control code to put dakuten/handakuten on the previous character. What I ended up doing was similar - making a table for each - but there are tools that can make the extraction easier for you. I had good luck extracting snippets (ie, a hex range) with romjuice by putting that control code as my 'switch tables' identifier, hooking up my two tables and it spat out some nice txt dumps. Atlas can do this too (and probably the tool to use for complete dumps) but I found it a bit harder to use for simple tests, it seems intended for script dumps when you have a pointer system figured out. It was just my experience dumping a sentence or two with a table change mechanic worked well in romjuice.

This makes me want to resume my work as well - I'll be following this! For katsugeki I'm still cracking away at trying to figure out how to alter the tilemap for dialog boxes.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/09/2014, 01:57 PM
Thanks for the read-break pointer syntax help! The mednafen docs could do with some extra examples!

But, we have a winner! The image below is with a read break on a single address - *1DEFE - immediately after the dialogue box is shown and before the text is printed, so the pointer must have been fetched just before this point.
Title: Re: Cyber Knight assets (tiles/text)
Post by: NightWolve on 02/09/2014, 02:26 PM
Ah man, I wish we had that debugger (David and I) back when we worked on Ys IV! Looks pretty decent. I used a brute-force method to find a string pointer and I got lucky at about 3000 or so byte changes to the state file per reloading in the YAME emulator! Heh-heh. Did what I had to do, and it managed to work after enough patience!

Basically, I wrote a Perl script that opened the YAME state file at the start of RAM, and it would INC a byte, then pause till I'd hit return. I would ALT+TAB to YAME, press the quick key to load a state file (set to F5), load the menu, see if I changed the text in question, if not, ALT+TAB back to the Perl script now open in a CMD window, press enter, and so then it would DEC the byte that it just changed, move to the next one, INC it, pause, rinse and repeat for hundreds and hundreds and hundreds of times till you find the byte that you're looking for!!! So I found it after like 3 or 4 days.... If not, David would've had to hand trace some assembly for me, but it didn't come to that for this issue.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/09/2014, 02:30 PM
That's hardcore, that is :shock:
Title: Re: Cyber Knight assets (tiles/text)
Post by: TurboXray on 02/09/2014, 05:13 PM
Quote from: NightWolve on 02/09/2014, 02:26 PMAh man, I wish we had that debugger (David and I) back when we worked on Ys IV! Looks pretty decent. I used a brute-force method to find a string pointer and I got lucky at about 3000 or so byte changes to the state file per reloading in the YAME emulator! Heh-heh. Did what I had to do, and it managed to work after enough patience!

Basically, I wrote a Perl script that opened the YAME state file at the start of RAM, and it would INC a byte, then pause till I'd hit return. I would ALT+TAB to YAME, press the quick key to load a state file (set to F5), load the menu, see if I changed the text in question, if not, ALT+TAB back to the Perl script now open in a CMD window, press enter, and so then it would DEC the byte that it just changed, move to the next one, INC it, pause, rinse and repeat for hundreds and hundreds and hundreds of times till you find the byte that you're looking for!!! So I found it after like 3 or 4 days.... If not, David would've had to hand trace some assembly for me, but it didn't come to that for this issue.
That's hardcore old skool!
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/09/2014, 06:15 PM
Ok, I now have two reproducible dialogue examples and have found where the string is located in logical memory for each of them.

The first is the message from MICA which activates if you stray too far on initial landing:

ROM position: 0x01DEFE
Logical position: 5EFE
Hex sequence: 4D 49 43 41 A2 BB B8 BE DD 5C B4 D8 B1 5C B6 D7 20 CA BD DE DA C3 B2 CF BD A3 08

The second is a message from the guards in the first town, move to them and they will activate a sequence where they check whether you are allowed to pass them. After that, this phrase will trigger when you then talk to each guard alternately:

ROM position: 0x01423E
Logical position: 423E
Hex Sequence: C0 C0 DE B2 CF 20 B7 DD D1 C1 AD B3 C3 DE B1 D8 CF BD 08

Now, I can see the text actually mapped in via the logical location that I have the read breakpoints set to (and the hex code matches that I scan in the rom). But I can't work back from that to figure out where the pointer to the string is loaded from.
Title: Re: Cyber Knight assets (tiles/text)
Post by: OldMan on 02/09/2014, 08:40 PM
I assume the read breakpoints are at the actual text addresses.

When you hit the break, write down the pc (program counter) and which page is at that address.
Then set a breakpoint there, and remove the one on the text. If you start again, and go for the same message, you shoud break at the -code- where the address is accessed. From there, it's a matter of looking at the disassembly of the code in mednafen, and seeing what is being done before the text is printed. Most probably, the text is being loaded indirectly (ie, lda  [ax], x) from some address in the zero page. Examine that address, and your text pointer table should be there.

Also keep in mind mednafen has a memory viewer, so you can set it to the zero page and look at the stack to see what routines have been called to get where you are.

Hope that helps.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/10/2014, 03:26 AM
Ok, so taking the guards example; the read breakpoint for the actual text string halts on the following instruction:

CD54: LDA ($30), Y (@ $423E = $C0 ; B1 30)
It appears that 0x00014000 is mapped in at $4000 at that point as the text is visible at $423E.
I clear the read breakpoint on *0001423E and set it for CD54. Stepping through it calls the same line with different offsets, the first one it breaks on is:

CD54: LDA ($30), Y (@ $400B = $00 ; B1 30)
After that it does

CD55: BEQ $CD9F ;F0 47
CD56: CMP #$20  ;C9 20

CD9F: INY ;C8
CDA0: JSR $C287;20 87 C2

C287: TYA ;98
C288: CLC ;18
C289: ADD $30 (@ $2030 = $0B) ;65 30
C28B: STA $30 ;85 30
C28D: BCC $2C91 ;90 02

C291: RTS ;60

CDA3: DEC $34 (@ $2034 = $12) ;C6 34
CDA5: BNE $CD51 ;D0 AA

CD51: LDY #$FF ;A0 FF
CD53: INY ;C8
CD54: LDA ($30),Y (@400C = $1A) ;B1 30

This continues for hundreds and hundreds of cycles, incrementing the address used by the LDA @ PC CD54 by one each time - so many that I just had to hold 'r' until it popped up the dialogue box.

Edit: Looking back at both traces, in each case it seems the operand to load the text string address is sourced from $5E (as both $5E and $5F are being set earlier).
Title: Re: Cyber Knight assets (tiles/text)
Post by: NightWolve on 02/10/2014, 04:55 AM
Quote from: TurboXray on 02/09/2014, 05:13 PMThat's hardcore old skool!
Yeah, a "brute-force linear search," so max search time equals 1-n, thus worst case scenario with 256 KB RAM, the byte that you're looking for is the last one, so I would've had to press "Enter", "Alt+Tab", "F5", etc. each about 262,144 (x3-4) times!!! Heh-heh. No cheating either, you couldn't say INCrement/DECrement 10 bytes at a time in the state file to increase your search range and then test in-game, because you would likely damage computer instructions at the same time and crash it, so it was gonna have to be one byte at a time if you were gonna do it!

Anyway, there goes a "protip" for ya megatron, that is, if for some reason you had no hope of finding something any other way!  :lol: I wanted to find some way to usefully contribute to your thread, so this was the best that I could do! ;)
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/10/2014, 04:56 AM
Quote from: NightWolve on 02/10/2014, 04:55 AMAnyway, there's a "protip" for ya megatron, that is, if for some reason you had no hope of finding something any way else! ;)
:lol: Thanks for that one
Title: Re: Cyber Knight assets (tiles/text)
Post by: OldMan on 02/10/2014, 09:30 AM
It looks like $30 and $31 hold a pointer to the current character to print. The code posted appears to be part of the character-convert code (ie, the stuff that interprets the control characters.)
Note that bne $cd51 - that's probably the jump to re-start the loop with another character.
If you set a break there, you should be able to step and see one character at a time appear.

You say it sets $5e and $5f to the string address earlier. Set a break on that, and see what appears on screen when it breaks. If you see new strings each time, then whatever is being put into $5e and $5f may be your string pointer table.
In any case, you now know where the logic is to interpret the strings. From that you should be able to deduce all the control codes, and what information they use.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/10/2014, 02:45 PM
Thanks again for the info - I'll take a look at that and see if I can find where $5e/$5f are getting set.
Title: Re: Cyber Knight assets (tiles/text)
Post by: TurboXray on 02/10/2014, 02:54 PM
I looked at the code, when in the first town. You're right, there's no pointer table. Not in the traditional sense.

 The text block is parse from the very first string of the block, until it reaches 0x00 (I assume End of String), then decrements the internal counter (which is the string offset mechanism). This is an old (and sloppy) method of text string handling. But on that note, it's easier for you. As in, you don't need to worry about any pointer tables. Simply just replace the strings. Though you'll have to reposition all the proceeding strings, but that's cake.

 You'll just have to locate the block of text. Which, 1) you can do visually if you have a hex editor with custom table/font, 2) could probably do this as you play through the game (find out which bank is loaded at MPR #2 and 3 in the debugger).
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/10/2014, 04:19 PM
Really? It's as straightforward as that?  #-o

I'll have to work backwards from that string I know and find the start - like you say, that won't be difficult (even with the two-char set font table). It looks like there is some extra space to play with (in the case some strings need to be made longer) as there are a block of some 40-odd 0-bytes at the end of that big block of text. This seems to be the case with several other big blocks of text, very useful!

It shouldn't be too difficult to extract those strings now. What would be interesting is to find out what those pointer-table looking structures are though.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/10/2014, 05:01 PM
Actually, it doesn't seem quite as simple. Certain text blocks appear to use 0x00 as delimiters, but others are missing 0x00 and instead use 0x08.... and (at least) the intro cinematic uses a pairing of 0x04 followed by 0x3C (guessing one is end of string, one is start).

I'll need to work out which block uses which delimiter method and dump it using the appropriate method. Not so bad really though.

I wonder if they all use the same 'scan N strings until you get to X' in order to display the correct text though?
Title: Re: Cyber Knight assets (tiles/text)
Post by: TurboXray on 02/10/2014, 05:51 PM
Yes, the routine/code specifically looks for 0x00 or 0x08. I figured you knew what all the control codes do. Is 0x08 a redirection? That's common in game. Strings split and re-continue at other places. It's usually used as a sort of 'compression' for redundant text. BubbleGum Crash game did this.

 If you run out of string space, you can write hook code that uses your own control codes to point to text that's in expanded rom area (again, did this with BGC on PCE).
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/10/2014, 06:13 PM
Quote from: TurboXray on 02/10/2014, 05:51 PMYes, the routine/code specifically looks for 0x00 or 0x08. I figured you knew what all the control codes do. Is 0x08 a redirection?
No, I don't think 0x08 is used like that, at least, I've not seen any examples of it used in that context. That said, I've not gone far into the game to see much text, so it may be feasible.

The control codes I've worked out so far are:

00 - Delimiter type 1
02 - Newline
03 - Newline
04 - Delimiter start type 2
3C - Delimiter end type 2
08 - Delimiter type 3
0D - Pad with N leading spaces

I'm going to spend some time playing with the various 2-byte control codes at the start of some of these blocks - they're a dialogue window property pairing.
Title: Re: Cyber Knight assets (tiles/text)
Post by: TurboXray on 02/10/2014, 07:01 PM
0x08 is wait for button to be pressed, for text to continue.
0x00 is string terminator (end of string).

 But there's a TST #$01, <$60 instruction, right after fetching the byte from the string. Not sure what it does, but it when the condition is met - it uses a different sets of compares. Looks like the game modifies this. Probably from a control code. I put $01 in there and the game put a bunch of blocks on screen.
 

 It's possible the game uses more than one type of text format/routine. I'd concentrate on the town text first, since that'll be the bulk of the game text.

Edited
Title: Re: Cyber Knight assets (tiles/text)
Post by: TurboXray on 02/10/2014, 07:09 PM
All normal control codes appear to be 0x00-0x1f. A few seem to be in the upper range (above 0x7f), but seem to be directly text related (probably japanese accent marks).

 I didn't see anything for 0x3c.

 Read char:
 if >0x20, do...
 else if ==0x5c, do..
 else if <0xDE, do..
else if >=0xE0, do...


 If <0xDE true, then
 if <0xA0, do...
else do...
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/11/2014, 06:20 AM
Yep, 0xDE/0xDF are the accent marks that appear over several characters - I've got them mapped in my table, so rather than printing two characters out when I dump the text, it will print the proper accented character.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/11/2014, 04:15 PM
Quote from: TurboXray on 02/10/2014, 07:01 PM0x08 is wait for button to be pressed, for text to continue.
0x00 is string terminator (end of string).
Ah, interesting. That would explain why certain strings (such as cinematics) don't have it, as they automatically close after a short period of time.
Title: Re: Cyber Knight assets (tiles/text)
Post by: TurboXray on 02/11/2014, 04:36 PM
That's all for the in town text routine. Not sure about the other routines.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/12/2014, 05:40 PM
Quote from: TurboXray on 02/06/2014, 12:52 PMThat aside, have you thought about upgrading any of the graphics? They're probably in 2bit/3bit format. If you expand the rom, you could hook the character/sprite upload routine to upload upgraded 4bit versions of the graphics. A LOT of early hucard games used simple 1/2/3bit graphic formats, because there is no decompression resource penalty (planar graphics) and can be uploaded to vram on the fly without decompressing to a buffer first.
I've been loading up the translated version for the SNES to compare things, and you know what? In terms of graphics it's not significantly better.


Out of all of them, the one area the PCE could definitely do with improvement is the battle screen. The
small sprites could do with being replaced and the ground/grass texture could actually do with having less detail, it's very 'noisy'.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/12/2014, 05:41 PM
The mecha selection screen.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/12/2014, 05:43 PM
Battle screen.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/12/2014, 05:51 PM
Oh, and one area where I think the PCE stands head and shoulders above the SNES is the music score. The hard edged synth style music on the PCE really suits the game, whereas the SNES, although having more realistic sounds, just doesn't seem a right fit.

And the opening cinematic. The SNES tries to pull it off with some scaling routines, but it comes off looking corny compared to the anime-style panels showing the SS-Swordfish and the pursing vessels on the PCE.

The sprites in the viewscreen before the jump drive look much better than the very blocky scaling used on the SNES intro.

Still, the battle map on the SNES (although smaller) is much nicer.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/19/2014, 05:28 PM
I've started work on writing an extractor for the script. It's not anywhere near finished yet, but it can pull sections out and writes a document with the actual characters as found in the font table.

It does basic substitution and uses the correct character based on whether its in pre-font-shift or post-font-shift mode.

In the example I've given an arbitrary address range to translate. It would be useful to firm up the actual ranges used for the various dialogue sections at some point.

EDIT: There's a typo in the extractScript.py file as attached, the SWITCH_MODE byte should be defined simply as "5C", not as hex "\x5C". Change that and the font substitution will work correctly.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/19/2014, 05:48 PM
This slight modification gives a better output.
Bear in mind that this extractor is only dealing with one type of string at the moment - that terminated by a 0x00. There are others (such as the intro text/cinematics and menus etc), and this will only work if you give the start and end address of a known block of text. That's the next key thing, IMO.

#!/usr/bin/env python

"""
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.


extractScript.py
================
A (basic, initial) script extractor for the PC-Engine game 'Cyber Knight'.

In order to run, it requires a headerless copy of the Cyber Knight ROM file
and a complete translation table (distributed with this programe).

John Snowdon <john@target-earth.net>
"""

import struct
import binascii

ROM_NAME = "Cyber Knight (J).pce"
TABLE_NAME = "CyberKnightTranslation.csv"
OUT_NAME = "out.sjs"
METHOD_1 = 1
METHOD_2 = 2
METHOD_3 = 3
SWITCH_MODE = '5C'

def load_table():
"""
load_table - load the translation table.
The translation table is a tab delimited data file
with the following columns:
hex code, actual char pre-0x5c byte, char set type (A/K/H/S), post-0x5c byte, char set type, notes

where A/S/H/K = ASCII, Symbol, Hiragana, Katakana
pre-0x5c = the character shown if the byte come before a 0x5c control byte
post-0x5c = the character shown if the byte comes after a 0x5c control byte
"""
trans_table = {}

f = open(TABLE_NAME, "r")
for line in f:
columns = line.split('\t')
byte_code = columns[0].replace('"', '')
trans_table[byte_code] = {}
trans_table[byte_code]["byte_code"] = byte_code
trans_table[byte_code]["pre_shift"] = columns[1].replace('"', '')
trans_table[byte_code]["pre_shift_type"] = columns[2].replace('"', '')
trans_table[byte_code]["post_shift"] = columns[3].replace('"', '')
trans_table[byte_code]["post_shift_type"] = columns[4].replace('"', '')
trans_table[byte_code]["notes"] = columns[5].replace('"', '')
f.close()
return trans_table


def translate_string(byte_sequence, trans_table):
"""
translate_string - construct the actual text, using multi-byte characters
where appropriate, that represent the hex codes found in the rom.
e.g. 0x1A 0x5F 0x76 0x61 0x62 0x63 0x64 0x65 0x00 = <control><control>vabcde<end>
"""

# method1 has two leading control bytes and a null byte as terminator
byte_sequence["text"] = []
if (byte_sequence["method"] == METHOD_1):
previous_b = ""
switch_mode = False
for i in range(2, len(byte_sequence["bytes"]) - 1):
b = str(binascii.hexlify(byte_sequence["bytes"][i])).upper()
if switch_mode:
if b == SWITCH_MODE:
switch_mode = False
else:
if b in trans_table.keys():
byte_sequence["text"].append(trans_table[b]["post_shift"])
else:
# warning - byte sequence not in table
print "WARNING: Untranslated byte <%s>" % b
byte_sequence["text"].append("<%s>" % b)
else:
if b == SWITCH_MODE:
switch_mode = True
else:
if b in trans_table.keys():
byte_sequence["text"].append(trans_table[b]["pre_shift"])
else:
# warning - byte sequence not in table
print "WARNING: Untranslated byte <%s>" % b
byte_sequence["text"].append("<%s>" % b)
return byte_sequence


def method1(ROM_NAME, rom_start_address, rom_end_address):
"""
method1 - extract text from a given byte range using
the notation of 2 control bytes, a variable number of
text bytes and then a single null closing byte.
e.g. 0x1A 0x2B 0x60 0x61 0x62 0x63 0x64 0x65 0x00
"""

ttable = load_table()

f = open(ROM_NAME, "rb")
f.seek(rom_start_address, 0)
rom_addr = rom_start_address

byte_strings = []
byte_sequence = {}
byte_sequence["start_pos"] = rom_addr
byte_sequence["bytes"] = []
byte_sequence["size"] = 0
byte_sequence["method"] = METHOD_1

while (rom_addr <= rom_end_address):
# Read a byte from the file at the current position
try:
byte = struct.unpack('c', f.read(1))[0]
if byte != "\x00":
# Add the byte
byte_sequence["bytes"].append(byte)
else:
# Add the end byte and record the string
byte_sequence["bytes"].append(byte)
byte_sequence["size"] = len(byte_sequence["bytes"])

# Generate the actual text string (which we will print for translation)
byte_sequence = translate_string(byte_sequence, ttable)

# Record the data
byte_strings.append(byte_sequence)

# Start a new byte sequence
byte_sequence = {}
byte_sequence["start_pos"] = rom_addr
byte_sequence["bytes"] = []
byte_sequence["size"] = 0
byte_sequence["method"] = METHOD_1

# Increment position ID
rom_addr += 1
except:
pass
f.close()
return byte_strings

def method2(ROM_NAME, rom_start_address, rom_end_address):
"""
method2 - extract text from a given byte range using
the notation of each string being wrapped in a single
control byte to start, and a single control byte to end.
e.g. 0x3C 0x60 0x61 0x62 0x63 0x64 0x65 0x04
"""
pass


byte_strings = method1(ROM_NAME, 0000, 500000)
f = open(OUT_NAME, "w")
for b in byte_strings:
f.write("Position: %s\n" % b["start_pos"])
f.write("Method: %s\n" % b["method"])
f.write("Length: %s\n" % b["size"])
f.write("Raw:\n")
for c in b["bytes"]:
f.write(str(binascii.hexlify(c)))
f.write(' ')
f.write('\n')
f.write("Text:\n")
for c in b["text"]:
f.write(c)
f.write('\n\n')
f.close()
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/22/2014, 04:23 AM
Here's the latest version of the script extractor. I've made it neater, moved the definition of the dialogue ranges up to the configuration options at the top of the file and have a help screen for the various input/output file options.

Run in its current form, it should generate an output file like this:

[
    {
        "block_range" : "0x1c87e-0x1c90d",
        "block_description" : "Main menu text and configuration options, start, continue, load, stereo/mono etc",
        "position" : "0x1c87e",
        "method" : 3,
        "start_bytes" : [],
        "raw_size" : 39,
        "raw" : ["20","ca","bc","de","d2","b6","d7","02","20","c2","c2","de","b7","b6","d7","02","20","5c","b8","db","b0","dd","5c","bb","b2","be","b2","02","20","b6","dd","b7","ae","b3","be","af","c3","b2","00"],
        "raw_text" : " はじめから\n つづきから\n クロ‐ンさいせい\n かんきぉうせってい",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x1c87e-0x1c90d",
        "block_description" : "Main menu text and configuration options, start, continue, load, stereo/mono etc",
        "position" : "0x1c8a5",
        "method" : 3,
        "raw_size" : 27,
        "raw" : ["20","5c","ca","de","af","b8","b1","af","cc","df","d2","d3","d8","02","20","ba","b0","c4","de","5c","c6","ad","b3","d8","ae","b8","00"],
        "raw_text" : " ックアッメモリ\n コ‐にうりぉく",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x1c87e-0x1c90d",
        "block_description" : "Main menu text and configuration options, start, continue, load, stereo/mono etc",
        "position" : "0x1c8c0",
        "method" : 3,
        "raw_size" : 26,
        "raw" : ["20","5c","d2","af","be","b0","bc","de","5c","bf","b8","c4","de","02","20","5c","bb","b3","dd","c4","de","d3","b0","c4","de","00"],
        "raw_text" : " メッセ‐そくど\n サウンモ‐",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x1c87e-0x1c90d",
        "block_description" : "Main menu text and configuration options, start, continue, load, stereo/mono etc",
        "position" : "0x1c8da",
        "method" : 3,
        "raw_size" : 23,
        "raw" : ["20","20","b5","bf","b2","20","20","cc","c2","b3","20","20","ca","d4","b2","20","20","5c","c0","b0","ce","de","00"],
        "raw_text" : "  おそい  ふつう  はやい  タ‐",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x1c87e-0x1c90d",
        "block_description" : "Main menu text and configuration options, start, continue, load, stereo/mono etc",
        "position" : "0x1c8f1",
        "method" : 3,
        "raw_size" : 14,
        "raw" : ["20","20","5c","bd","c3","da","b5","20","20","d3","c9","d7","d9","00"],
        "raw_text" : "  ステレオ  モノラル",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x1c87e-0x1c90d",
        "block_description" : "Main menu text and configuration options, start, continue, load, stereo/mono etc",
        "position" : "0x1c8ff",
        "method" : 3,
        "raw_size" : 15,
        "raw" : ["0d","06","43","59","42","45","52","20","4b","4e","49","47","48","54","00"],
        "raw_text" : "<pad><06>CYBER KNIGHT",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x1defc-0x1e0a5",
        "block_description" : "Unknown, possible ship dialogue for first world",
        "position" : "0x1defc",
        "method" : 1,
        "start_bytes" : ["1a","5f"],
        "raw_size" : 32,
        "raw" : ["1a","5f","4d","49","43","41","a2","bb","b8","be","dd","5c","b4","d8","b1","5c","b6","d7","20","ca","bd","de","da","c3","b2","cf","bd","a3","08","1b","01","00"],
        "raw_text" : "MICA『さくせんエリアから はずれています』<wait><1B><01>",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x1defc-0x1e0a5",
        "block_description" : "Unknown, possible ship dialogue for first world",
        "position" : "0x1df1c",
        "method" : 1,
        "start_bytes" : ["10","34"],
        "raw_size" : 27,
        "raw" : ["10","34","03","a2","b6","b2","be","b7","20","bd","d9","d3","c9","ca","20","c5","c6","d3","20","c5","b2","bf","de","21","a3","08","00"],
        "raw_text" : "\n『かいせき するものは なにも ないぞ!』<wait>",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x1defc-0x1e0a5",
        "block_description" : "Unknown, possible ship dialogue for first world",
        "position" : "0x1df37",
        "method" : 1,
        "start_bytes" : ["10","35"],
        "raw_size" : 37,
        "raw" : ["10","35","a2","b9","b6","de","ca","20","c5","b5","af","c0","dc","d6","a1","03","b1","cf","d8","20","d1","c1","ac","a6","20","bc","c1","ac","20","c0","de","d2","d6","21","a3","08","00"],
        "raw_text" : "『けがは なおったわよ゛\nおまり むちを しち だめよ!』<wait>",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x1defc-0x1e0a5",
        "block_description" : "Unknown, possible ship dialogue for first world",
        "position" : "0x1df5c",
        "method" : 1,
        "start_bytes" : ["10","35"],
        "raw_size" : 29,
        "raw" : ["10","35","a2","ba","da","b6","de","20","b1","c5","c0","c0","c1","c9","20","5c","b8","db","b0","dd","ba","b0","c4","de","5c","d6","a1","a3","00"],
        "raw_text" : "『これが おなたたちの クロ‐ンコ‐よ゛』",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x1defc-0x1e0a5",
        "block_description" : "Unknown, possible ship dialogue for first world",
        "position" : "0x1df79",
        "method" : 1,
        "start_bytes" : ["10","35"],
        "raw_size" : 31,
        "raw" : ["10","35","a2","ba","b2","c9","d4","cf","b2","ca","20","dc","c0","bc","c9","20","c0","dd","c4","b3","bc","de","ac","03","c5","b2","c9","a1","a3","08","00"],
        "raw_text" : "『こいのやまいは わたしの たんとうじ\nないの゛』<wait>",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x1defc-0x1e0a5",
        "block_description" : "Unknown, possible ship dialogue for first world",
        "position" : "0x1df98",
        "method" : 1,
        "start_bytes" : ["10","36"],
        "raw_size" : 34,
        "raw" : ["10","36","a2","bc","ad","b3","d8","20","b6","dd","d8","ae","b3","a1","03","0d","04","d3","b3","20","ba","dc","bc","c3","20","b8","da","d9","c5","d6","21","a3","08","00"],
        "raw_text" : "『しうり かんりぉう゛\n<pad><04>もう こわして くれるなよ!』<wait>",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x1defc-0x1e0a5",
        "block_description" : "Unknown, possible ship dialogue for first world",
        "position" : "0x1dfba",
        "method" : 1,
        "start_bytes" : ["0c","10"],
        "raw_size" : 38,
        "raw" : ["0c","10","36","a2","5c","d8","cd","df","b1","b7","af","c4","a5","b7","ad","b1","b7","af","c4","5c","a6","20","ce","bc","de","ad","b3","03","bc","c4","b2","c0","be","de","a1","a3","08","00"],
        "raw_text" : "6『リアキット・キュアキットを ほじう\nしといたぜ゛』<wait>",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x1defc-0x1e0a5",
        "block_description" : "Unknown, possible ship dialogue for first world",
        "position" : "0x1dfe0",
        "method" : 1,
        "start_bytes" : ["10","36"],
        "raw_size" : 45,
        "raw" : ["10","36","a2","c4","de","ba","d3","20","ba","dc","da","c3","c5","b2","be","de","a1","03","0d","04","5c","c1","ad","b0","dd","5c","ca","20","5c","b7","de","dd","b7","de","dd","5c","c0","de","be","de","21","21","a3","08","00"],
        "raw_text" : "『どこも こわれてないぜ゛\n<pad><04>チュ‐ンは ンンだぜ!!』<wait>",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x1defc-0x1e0a5",
        "block_description" : "Unknown, possible ship dialogue for first world",
        "position" : "0x1e00d",
        "method" : 1,
        "start_bytes" : ["10","36"],
        "raw_size" : 27,
        "raw" : ["10","36","a2","bf","c9","cf","b4","c6","20","5c","d3","bc","de","ad","b0","d9","5c","a6","20","b7","c3","b8","da","a1","a3","08","00"],
        "raw_text" : "『そのまえに モュ‐ルを きてくれ゛』<wait>",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x1defc-0x1e0a5",
        "block_description" : "Unknown, possible ship dialogue for first world",
        "position" : "0x1e028",
        "method" : 1,
        "start_bytes" : ["20","10"],
        "raw_size" : 7,
        "raw" : ["20","10","ff","09","05","02","00"],
        "raw_text" : "゜<09><05>\n",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x1defc-0x1e0a5",
        "block_description" : "Unknown, possible ship dialogue for first world",
        "position" : "0x1e02f",
        "method" : 1,
        "start_bytes" : ["10","36"],
        "raw_size" : 19,
        "raw" : ["10","36","a2","5c","b8","de","af","c4","de","20","d7","af","b8","21","21","a3","1c","01","00"],
        "raw_text" : "『ッ ラック!!』<1C><01>",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x1defc-0x1e0a5",
        "block_description" : "Unknown, possible ship dialogue for first world",
        "position" : "0x1e042",
        "method" : 1,
        "start_bytes" : ["10","ff"],
        "raw_size" : 5,
        "raw" : ["10","ff","09","04","00"],
        "raw_text" : "<09><04>",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x1defc-0x1e0a5",
        "block_description" : "Unknown, possible ship dialogue for first world",
        "position" : "0x1e047",
        "method" : 1,
        "start_bytes" : ["10","30"],
        "raw_size" : 38,
        "raw" : ["10","30","a2","b5","b6","b4","d8","c5","bb","b2","21","21","03","0d","04","5c","d0","af","bc","ae","dd","5c","20","ba","de","b8","db","b3","bb","cf","c3","de","bc","c0","a1","a3","02","00"],
        "raw_text" : "『おかえりなさい!!\n<pad><04>ミッション ごくろうさまでした゛』\n",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x1defc-0x1e0a5",
        "block_description" : "Unknown, possible ship dialogue for first world",
        "position" : "0x1e06d",
        "method" : 1,
        "start_bytes" : ["10","35"],
        "raw_size" : 35,
        "raw" : ["10","35","a2","cf","c0","bc","dd","c0","de","c9","a1","03","b2","bf","b2","c3","de","20","5c","b8","db","b0","dd","5c","bb","b2","be","b2","20","bd","d9","dc","a1","a3","00"],
        "raw_text" : "『またしんだの゛\nいそいで クロ‐ンさいせい するわ゛』",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x1defc-0x1e0a5",
        "block_description" : "Unknown, possible ship dialogue for first world",
        "position" : "0x1e090",
        "method" : 1,
        "start_bytes" : ["0c","10"],
        "raw_size" : 21,
        "raw" : ["0c","10","35","a2","c2","b7","de","ca","20","b7","a6","c2","b9","c5","bb","b2","d6","a1","a3","08","00"],
        "raw_text" : "5『つぎは きをつけなさいよ゛』<wait>",
        "trans_size" : 0,
        "trans_text" : ""
    }
]

It's valid JSON format, so if you want to import it back into anything, you should be able to in almost any programming language with minimal effort.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/23/2014, 05:23 PM
OK, so here is what my extract script produces for the introductory cinematics:

[
    {
        "block_range" : "0x29eff-0x2a1ad",
        "block_description" : "Introductory cinematics.",
        "position" : "0x29eff",
        "method" : 2,
        "start_bytes" : [],
        "raw_size" : 29,
        "raw" : ["11", "16", "1a", "72", "5c", "c5", "dd", "ca", "de", "b0", "34", "cc", "de", "db", "af", "b8", "5c", "c6", "a4", "5c", "c0", "de", "d2", "b0", "bc", "de", "21", "04", "3c"],
        "raw_text" : "<16><1A>rなんば―4ぶろっくニ、だめ―じ!",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x29eff-0x2a1ad",
        "block_description" : "Introductory cinematics.",
        "position" : "0x29f1c",
        "method" : 2,
        "raw_size" : 19,
        "raw" : ["0c", "5c", "b7", "b1", "c2", "b6", "de", "20", "bb", "b6", "de", "af", "c3", "b2", "cf", "bd", "21", "04", "3c"],
        "raw_text" : "きおつが さがっています!",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x29eff-0x2a1ad",
        "block_description" : "Introductory cinematics.",
        "position" : "0x29f2f",
        "method" : 2,
        "raw_size" : 20,
        "raw" : ["1a", "73", "ba", "c1", "d7", "20", "5c", "d2", "c3", "de", "a8", "b6", "d9", "a5", "d9", "b0", "d1", "21", "04", "3c"],
        "raw_text" : "sコチラ めでぃかる・る―む!",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x29eff-0x2a1ad",
        "block_description" : "Introductory cinematics.",
        "position" : "0x29f43",
        "method" : 2,
        "raw_size" : 21,
        "raw" : ["0c", "bc", "bd", "c3", "d1", "5c", "c6", "20", "45", "4d", "50", "5c", "c0", "de", "d2", "b0", "bc", "de", "21", "04", "3c"],
        "raw_text" : "システムに EMPダメ‐ジ!",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x29eff-0x2a1ad",
        "block_description" : "Introductory cinematics.",
        "position" : "0x29f58",
        "method" : 2,
        "raw_size" : 28,
        "raw" : ["1b", "00", "00", "11", "16", "1a", "73", "5c", "cc", "de", "d8", "af", "bc", "de", "5c", "c6", "20", "b6", "bb", "b2", "20", "ca", "af", "be", "b2", "21", "04", "3c"],
        "raw_text" : "<end><end><11><16><1A>sぶりっじニ カサイ ハッセイ!",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x29eff-0x2a1ad",
        "block_description" : "Introductory cinematics.",
        "position" : "0x29f74",
        "method" : 2,
        "raw_size" : 37,
        "raw" : ["1a", "72", "5c", "ba", "cf", "dd", "c0", "de", "b0", "21", "20", "b7", "ac", "cc", "df", "c3", "dd", "5c", "b6", "de", "3b", "3b", "3b", "3b", "5c", "b7", "ac", "cc", "df", "c3", "dd", "5c", "b6", "de", "21", "04", "3c"],
        "raw_text" : "rこまんだ―! きぷてんガ‥‥‥‥きぷてんガ!",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x29eff-0x2a1ad",
        "block_description" : "Introductory cinematics.",
        "position" : "0x29f99",
        "method" : 2,
        "raw_size" : 38,
        "raw" : ["1b", "00", "00", "11", "16", "1a", "72", "bc", "bc", "ae", "b3", "bc", "ac", "20", "c0", "bd", "b3", "21", "20", "ba", "c9", "cf", "cf", "c3", "de", "ca", "20", "be", "de", "dd", "d2", "c2", "c3", "de", "bd", "21", "04", "3c"],
        "raw_text" : "<end><end><11><16><1A>rシショウシ タスウ! コノママデハ ゼンメツデス!",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x29eff-0x2a1ad",
        "block_description" : "Introductory cinematics.",
        "position" : "0x29fbf",
        "method" : 2,
        "raw_size" : 17,
        "raw" : ["1a", "73", "b8", "bf", "a4", "b6", "b2", "bf", "de", "b8", "c4", "de", "d3", "d2", "21", "04", "3c"],
        "raw_text" : "sクソ、カイゾクドモメ!",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x29eff-0x2a1ad",
        "block_description" : "Introductory cinematics.",
        "position" : "0x29fd0",
        "method" : 2,
        "raw_size" : 24,
        "raw" : ["0c", "b5", "da", "c0", "c1", "a6", "20", "c5", "cc", "de", "d8", "ba", "de", "db", "bc", "c6", "20", "bd", "d9", "b7", "b6", "21", "04", "3c"],
        "raw_text" : "オレタチヲ ナブリゴロシニ スルキカ!",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x29eff-0x2a1ad",
        "block_description" : "Introductory cinematics.",
        "position" : "0x29fe8",
        "method" : 2,
        "raw_size" : 28,
        "raw" : ["1b", "00", "00", "1a", "71", "ba", "b3", "c5", "af", "c3", "ca", "20", "c0", "de", "af", "bc", "ad", "c2", "bd", "d9", "20", "bc", "b6", "c5", "b2", "a1", "04", "3c"],
        "raw_text" : "<end><end><1A>qコウナッテハ ダッシュツスル シカナイ゛",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x29eff-0x2a1ad",
        "block_description" : "Introductory cinematics.",
        "position" : "0x2a004",
        "method" : 2,
        "raw_size" : 24,
        "raw" : ["0c", "5c", "bc", "de", "ac", "dd", "cc", "df", "a5", "c4", "de", "d7", "b2", "cc", "de", "5c", "a6", "20", "c2", "b6", "b3", "21", "04", "3c"],
        "raw_text" : "じんぷ・どらいぶヲ ツカウ!",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x29eff-0x2a1ad",
        "block_description" : "Introductory cinematics.",
        "position" : "0x2a01c",
        "method" : 2,
        "raw_size" : 17,
        "raw" : ["1a", "73", "bc", "b6", "bc", "a4", "5c", "ba", "cf", "dd", "c0", "de", "b0", "5c", "21", "04", "3c"],
        "raw_text" : "sシカシ、こまんだ―!",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x29eff-0x2a1ad",
        "block_description" : "Introductory cinematics.",
        "position" : "0x2a02d",
        "method" : 2,
        "raw_size" : 37,
        "raw" : ["0c", "b2", "cf", "20", "5c", "bc", "de", "ac", "dd", "cc", "df", "5c", "bc", "c0", "d7", "20", "5c", "bc", "de", "ac", "dd", "cc", "df", "a5", "d0", "bd", "5c", "c9", "b7", "b9", "dd", "b6", "de", "3b", "3b", "04", "3c"],
        "raw_text" : "イマ じんぷシタラ じんぷ・みすノキケンガ‥‥",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x29eff-0x2a1ad",
        "block_description" : "Introductory cinematics.",
        "position" : "0x2a052",
        "method" : 2,
        "raw_size" : 33,
        "raw" : ["1a", "72", "bf", "b3", "c3", "de", "bd", "21", "20", "b3", "c1", "ad", "b3", "c9", "ca", "c3", "cf", "c3", "de", "20", "cc", "af", "c4", "dd", "c3", "de", "d5", "b8", "b6", "d3", "3f", "04", "3c"],
        "raw_text" : "rソウデス! ウチュウノハテマデ フットンデユクカモ?",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x29eff-0x2a1ad",
        "block_description" : "Introductory cinematics.",
        "position" : "0x2a073",
        "method" : 2,
        "raw_size" : 25,
        "raw" : ["19", "71", "c0", "de", "b6", "de", "20", "ba", "c9", "cf", "cf", "c3", "de", "ca", "20", "d4", "d7", "da", "c3", "bc", "cf", "b3", "a1", "04", "3c"],
        "raw_text" : "qダガ コノママデハ ヤラレテシマウ゛",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x29eff-0x2a1ad",
        "block_description" : "Introductory cinematics.",
        "position" : "0x2a08c",
        "method" : 2,
        "raw_size" : 20,
        "raw" : ["0c", "d0", "c1", "c9", "20", "b6", "c9", "b3", "be", "b2", "c6", "20", "b6", "b9", "c3", "d0", "d6", "b3", "04", "3c"],
        "raw_text" : "ミチノ カノウセイニ カケテミヨウ",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x29eff-0x2a1ad",
        "block_description" : "Introductory cinematics.",
        "position" : "0x2a0a0",
        "method" : 2,
        "raw_size" : 39,
        "raw" : ["0c", "5c", "ca", "df", "dc", "b0", "5c", "a6", "20", "bd", "cd", "de", "c3", "5c", "bc", "de", "ac", "dd", "cc", "df", "a5", "bc", "de", "aa", "c8", "da", "b0", "c0", "b0", "5c", "c6", "cf", "dc", "be", "21", "05", "1e", "04", "3c"],
        "raw_text" : "ぱわ―ヲ スベテじんぷ・じぇねれ―た―ニマワセ!<05><1E>",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x29eff-0x2a1ad",
        "block_description" : "Introductory cinematics.",
        "position" : "0x2a0c7",
        "method" : 2,
        "raw_size" : 22,
        "raw" : ["19", "73", "5c", "b4", "c8", "d9", "b7", "de", "b0", "a5", "b9", "de", "b2", "dd", "5c", "20", "4d", "41", "58", "21", "04", "3c"],
        "raw_text" : "sえねるぎ―・げいん MAX!",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x29eff-0x2a1ad",
        "block_description" : "Introductory cinematics.",
        "position" : "0x2a0dd",
        "method" : 2,
        "raw_size" : 32,
        "raw" : ["05", "37", "19", "72", "5c", "b5", "b0", "d9", "a5", "bc", "bd", "c3", "d1", "20", "b4", "cf", "bc", "de", "aa", "dd", "bc", "b0", "a5", "d3", "b0", "c4", "de", "5c", "cd", "21", "04", "3c"],
        "raw_text" : "7<19>rお―る・しすてむ えまじぇんし―・も―どヘ!",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x29eff-0x2a1ad",
        "block_description" : "Introductory cinematics.",
        "position" : "0x2a0fd",
        "method" : 2,
        "raw_size" : 37,
        "raw" : ["00", "19", "73", "ce", "de", "b3", "b7", "de", "ae", "5c", "cc", "a8", "b0", "d9", "c4", "de", "5c", "c9", "20", "5c", "ca", "df", "dc", "b0", "5c", "b6", "de", "20", "b5", "c1", "c3", "b2", "cf", "bd", "21", "04", "3c"],
        "raw_text" : "<19>sボウギョふぃ―るどノ ぱわ―ガ オチテイマス!",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x29eff-0x2a1ad",
        "block_description" : "Introductory cinematics.",
        "position" : "0x2a122",
        "method" : 2,
        "raw_size" : 22,
        "raw" : ["19", "71", "b1", "c4", "20", "31", "30", "cb", "de", "ae", "b3", "c0", "de", "b9", "20", "d3", "c0", "be", "db", "21", "04", "3c"],
        "raw_text" : "qアト 10ビョウダケ モタセロ!",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x29eff-0x2a1ad",
        "block_description" : "Introductory cinematics.",
        "position" : "0x2a138",
        "method" : 2,
        "raw_size" : 25,
        "raw" : ["19", "73", "5c", "bc", "bd", "c3", "d1", "a5", "ba", "dd", "c3", "de", "bc", "ae", "dd", "20", "b5", "da", "dd", "bc", "de", "21", "5c", "04", "3c"],
        "raw_text" : "sしすてむ・こんでしぉん おれんじ!",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x29eff-0x2a1ad",
        "block_description" : "Introductory cinematics.",
        "position" : "0x2a151",
        "method" : 2,
        "raw_size" : 30,
        "raw" : ["19", "72", "5c", "bc", "de", "ac", "dd", "cc", "df", "5c", "bb", "de", "cb", "ae", "b3", "a6", "20", "be", "af", "c3", "b2", "c3", "de", "b7", "cf", "be", "dd", "21", "04", "3c"],
        "raw_text" : "rじんぷザヒョウヲ セッテイデキマセン!",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "block_range" : "0x29eff-0x2a1ad",
        "block_description" : "Introductory cinematics.",
        "position" : "0x2a16f",
        "method" : 2,
        "raw_size" : 63,
        "raw" : ["19", "71", "b6", "cf", "dc", "dd", "21", "20", "b2", "b8", "bf", "de", "21", "1a", "74", "3b", "3b", "3b", "3b", "33", "3b", "3b", "3b", "3b", "0c", "3b", "3b", "3b", "3b", "32", "3b", "3b", "3b", "3b", "0c", "3b", "3b", "3b", "3b", "31", "3b", "3b", "3b", "3b", "0c", "3b", "3b", "3b", "3b", "30", "0b", "0e", "05", "5c", "bc", "de", "ac", "dd", "cc", "df", "21", "04", "3c"],
        "raw_text" : "qカマワン! イクゾ!<1A>t‥‥‥‥3‥‥‥‥<0C>‥‥‥‥2‥‥‥‥<0C>‥‥‥‥1‥‥‥‥<0C>‥‥‥‥0<0B><0E><05>じんぷ!",
        "trans_size" : 0,
        "trans_text" : ""
    }
]

It looks reasonable, until you start to examine a few of the strings. Some of them are fine, however, some are in the wrong font set (remember Cyber Knight uses 0x5c to indicate 'swap to alt font set' - Katakana to Hiragana).
The problem being that at least a couple of the strings I've analysed should have a 0x5c byte embedded in order to match the on-screen text, but they don't. Whereas others do and match the on-screen text perfectly.

Depending on whether I choose the initial state to be Hiragana or Katakana, several of the strings break. One state doesn't satisfy all of them. e.g.

Defaulting to switch mode on (Katakana enabled by default) shows the correct output for the 'EMP' string, but causes incorrect characters for the 'MAX' string. Defaulting to Hiragana fixes the 'MAX' output, but switches the characters for 'EMP'.

BTW, code is now on github: https://github.com/megatron-uk/cyberknight-pce

Edit: As a quick workaround, I've enabled dual-output for the affected text - both 'assume katakana' and 'assume hiragana' strings are printed. I'm hoping that this is not also the case elsewhere throughout the game, but this needs more investigation.
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/25/2014, 05:15 PM
Ok, I now have a basic injector script that can take the output form the extractor, pick up those strings that have had a translation entered and reinject them back into the file.

This code is now up on github.

The injector only works with fixed length strings at the moment (eg main menu, hud elements, stat screens etc), but actually, the variable length strings such as dialogue etc, should actually be easier, as they're really only one very long string with the phrases delimited by 0x00 bytes.

The way I have it setup is to load multiple json patch files, so as the main menu text is done, I can forget about that one and move on to the next file - that way I don't have one massive translation file to do.

Here's the result of extractScript and injectScript translating the main menu dialogue strings:

Screenshot-Cyber Knight (E).png Screenshot-Cyber Knight (E)-1.png
Screenshot-Cyber Knight (E)-2.png Screenshot from 2014-02-25 22:07:37.png   
Title: Re: Cyber Knight assets (tiles/text)
Post by: megatron-uk on 02/26/2014, 06:22 PM
Player character name entry screen now done (excluding Kanji symbols), also started translating the list of npc names (located at 0x1cbad-0x1ccac). Here's the first few (your ship mates) re-injected back into the rom.

I haven't yet written an intelligent inserter, so I'm just using the same space I've got from the original strings - in the case of the npc names most of them give extra space when using ascii as they no longer need multiply 0x5c control bytes, that frees up some space for extra characters in the case of 'Ki', this is a 4 character name (Kiri), but I can't currently fit it in as one of those bytes is reserved by 0x00 as a delimiter to the next string. What I need to do now is write some code that given a number of strings (like the names, above) and an address range of a file, shuffle the starting positions of each reinserted string so that they are still in order (important for the seemingly dumb text lookup of Cyber Knight), but gives more space to those strings that need it, and less that don't.
Title: Re: Cyber Knight translation
Post by: TurboXray on 02/26/2014, 07:36 PM
So, you have most of it figured out?
Title: Re: Cyber Knight translation
Post by: megatron-uk on 02/27/2014, 03:56 AM
Pretty much. Your realisation about the dialogue simply being null delimited and the display routine crudely looping over those bytes until it gets to a particular count pretty much broke the back of the thing. There are some exceptions (intro text, some fixed layout like the menu's, player status screen etc - but these are easy to replace and in almost all cases only occur once in the rom), but knowing that, writing an extractor and inserter was pretty simple.

The main things that need doing are identifying all of the text; which will probably only come with a playthrough, and working out how to do longer string insertion, when I come to need to do it for the in-game dialogue.

Oh yeah, the one thing I haven't looked at yet is replacing the double height Kanji characters - there are not many of them (30-40), so it should be possible just to write some new tiles to replace them; I may need help reinserting them however.

I've asked the translator of the SNES version (http://agtp.romhack.net/ (http://agtp.romhack.net/)) if he has the script left over from doing the work, this would *really* speed things up, so I'm hoping he can find it - though it was done almost 10 years ago....

Once that is done, I'd love to do an overhaul of some of the sprites, using SNES ones where appropriate
Title: Re: Cyber Knight translation
Post by: megatron-uk on 02/28/2014, 07:07 PM
Did a short little clip of the translated main menu and some of the introduction - it's not the greatest quality (only my first attempt at using the video record features of Mednafen), but it shows the changes made so far.

Cyber Knight PC-Engine initial translation test (https://youtu.be/j76cYe1r5H0#)
Title: Re: Cyber Knight translation
Post by: peperocket on 03/01/2014, 04:54 AM
Great job!!
Title: Re: Cyber Knight translation
Post by: esteban on 03/01/2014, 06:55 AM
Comrade, good work. I know you will see this through to the end. (https://junk.tg-16.com/images/pcds.png)
Title: Re: Cyber Knight translation
Post by: Keith Courage on 03/01/2014, 04:22 PM
Awesome!!! I love it when pc engine games get translated.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 03/02/2014, 02:07 PM
The introductory cinematics are now fully translated!

http://youtu.be/solMaSfvP1Y

I've also had the help of a couple of people over on Romhacking.net on tracking down the last few unmapped characters (some are only 4-5 pixels high) and now the table is complete (though not all control codes are known yet).

Full files and the scripts to patch your own version are on github, as normal.
Title: Re: Cyber Knight translation
Post by: whisper2053 on 03/03/2014, 01:18 AM
Yesssss :D
Title: Re: Cyber Knight translation
Post by: jeffhlewis on 03/03/2014, 10:44 AM
Great thread - makes me feel horribly inadequate about my programming skills, haha. keep up the good work!
Title: Re: Cyber Knight translation
Post by: megatron-uk on 03/03/2014, 06:13 PM
The scrolling intro text that appears after the intro cinema is now done. Most of what is translated so far is based on the SNES version, but some slight changes where I thought it sounded better. The patches are all on github (not the scrolling text yet however), so you're free to change it yourself.

This text is expanded compared to the japanese original - around 300 bytes more. Fortunately there's a big bank of 0xFF bytes (about 1k of it) that occurs just after the text.

Anyway, patch for this section is not up yet as I have one or two spacing issues to play with - there are a couple of words that run right up to the right hand screen edge.

Other than that, enjoy!
Title: Re: Cyber Knight translation
Post by: SuperPlay on 03/08/2014, 05:02 AM
Great work so far :-) looking forward to this when it is complete.
Title: Re: Cyber Knight translation
Post by: roflmao on 03/09/2014, 12:34 AM
To echo SuperPlay's sentiments, you are an inspiration.  I don't remember if I've posted to this thread yet, but wanted to share my enthusiasm for what you're doing. :D
Title: Re: Cyber Knight translation
Post by: megatron-uk on 03/09/2014, 03:41 AM
Thanks for all the kind words :D

I'm hopeful that when I start the main in-game text it will be all of a similar type to (null delimited strings). The best thing that could happen is I get hold of a dump of the raw and converted SNES translation - I don't have any Japanese language skills - so this would speed up the conversion process immensely.

One thing I've found even with just the intro text is the amount of extra space needed for an English representation of the Japanese text - it's often anywhere up to 50% longer. That could be a sticking point (if there's not room to expand into), and I may need to seek others' help to modify the text loader code as I know virtually zero 6502 assembly.

Things may slow down for the next few weeks - my wife and I have just accepted an offer on our house, so it's all hands to packing and moving duties!
Title: Re: Cyber Knight translation
Post by: megatron-uk on 03/10/2014, 05:57 PM
Hit a bit of stumbling block with the next section of text that appears just before the game actually starts - it's just after the scrolling intro text that summarises what has happened so far.

It's translated, and inserted, but when I run the game it seems as if the previous text block insertion (the scrolling text, as posted earlier) breaks it. So rather than my nicely translated text, I get random hiragana/katakana characters.

The text inserted before this one was expanded into what seems to be an unused section of the rom (0x1bca7 - 0x1bff0, all set to 0xff). When I disable that particular patch, my new translation works correctly. I'm wondering if the game uses some offset from this last text block to jump to the next... in which case, any string expansion is going to fail, like this appears to.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 03/10/2014, 06:06 PM
With expanded scrolling intro text patch applied and without. In both cases the translation is present for the displayed section, but in the case where the intro patch is applied, the output of this particular section is jibberish.
Title: Re: Cyber Knight translation
Post by: OldMan on 03/10/2014, 09:05 PM
QuoteThe text inserted before this one was expanded into what seems to be an unused section of the rom (0x1bca7 - 0x1bff0, all set to 0xff)
Just out of curiosity, did you insetrt the text at the beginning or end of that block? Some of those xFF's may actually be termination codes....
Title: Re: Cyber Knight translation
Post by: megatron-uk on 03/11/2014, 03:17 AM
The beginning. It runs from it's normal start location to around 200-300 bytes into that 0xff section. It's around 850 bytes in normal length, it's now about 1100. The remainder (and end) of the 0xff block is intact.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 03/11/2014, 03:32 PM
Slight OT, but interesting nontheless; it appears as though Cyber Knight may well have a hidden sound/music test facility as I've found this in one of my latest dumps:

    {
        "string_description" : "unknown 1",
        "string_start" : "0x1c0c4",
        "method" : 3,
        "start_bytes" : [],
        "end_bytes" : ["00"],
        "raw_size" : 137,
        "raw" : ["0b", "05", "06", "2a", "2a", "2a", "20", "43", "79", "62", "65", "72", "20", "4b", "6e", "69", "67", "68", "74", "20", "2a", "2a", "2a", "0b", "05", "08", "2a", "2a", "2a", "20", "20", "53", "4f", "55", "4e", "44", "20", "54", "45", "53", "54", "20", "20", "2a", "2a", "2a", "0b", "08", "0c", "41", "75", "64", "69", "6f", "20", "6d", "6f", "64", "65", "20", "3a", "0b", "08", "0e", "42", "47", "4d", "20", "52", "65", "71", "20", "4e", "6f", "2e", "3a", "0b", "08", "10", "42", "47", "4d", "20", "52", "65", "71", "20", "4e", "6f", "2e", "3a", "0b", "08", "12", "45", "46", "43", "20", "52", "65", "71", "20", "4e", "6f", "2e", "3a", "0b", "08", "14", "45", "46", "43", "20", "52", "65", "71", "20", "4e", "6f", "2e", "3a", "0b", "08", "16", "46", "61", "64", "65", "2d", "4f", "75", "74", "20", "20", "20", "3a", "00"],
        "raw_text" : "<0B><05><06>*** Cyber Knight ***<0B><05><wait>***  SOUND TEST  ***<0B><wait><erase>Audio mode :<0B><wait><0E>BGM Req No.:<0B><wait><10>BGM Req No.:<0B><wait><12>EFC Req No.:<0B><wait><14>EFC Req No.:<0B><wait><16>Fade-Out   :",
        "alt_text" : "<0B><05><06>*** Cyber Knight ***<0B><05><wait>***  SOUND TEST  ***<0B><wait><erase>Audio mode :<0B><wait><0E>BGM Req No.:<0B><wait><10>BGM Req No.:<0B><wait><12>EFC Req No.:<0B><wait><14>EFC Req No.:<0B><wait><16>Fade-Out   :",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "string_description" : "unknown 1",
        "string_start" : "0x1c14d",
        "method" : 3,
        "start_bytes" : [],
        "end_bytes" : ["00"],
        "raw_size" : 12,
        "raw" : ["0b", "15", "0c", "4d", "6f", "6e", "61", "75", "72", "61", "6c", "00"],
        "raw_text" : "<0B><15><erase>Monaural",
        "alt_text" : "<0B><15><erase>Monaural",
        "trans_size" : 0,
        "trans_text" : ""
    },

    {
        "string_description" : "unknown 1",
        "string_start" : "0x1c159",
        "method" : 3,
        "start_bytes" : [],
        "end_bytes" : ["00"],
        "raw_size" : 12,
        "raw" : ["0b", "15", "0c", "53", "74", "65", "72", "65", "6f", "20", "20", "00"],
        "raw_text" : "<0B><15><erase>Stereo  ",
        "alt_text" : "<0B><15><erase>Stereo  ",
        "trans_size" : 0,
        "trans_text" : ""
    },
Title: Re: Cyber Knight translation
Post by: megatron-uk on 03/13/2014, 04:47 AM
For now I've backed out the translated scrolling text from the intro. Until I can work out how to get the longer english text in there without affecting later strings I've moved on with the in-game dialogue.

I'm now working on translating the load/save game screens as well as the player character/npc stats screens. The load save game screen is done, except for the Kanji - the player stats screen is also mostly done; including the 4 skill categories (combat, tech/engineering, medic, science) and player roles (commander, soldier, doctor, professor, tech/engineer) and sex.

I'm starting to try and figure out how the Kanji tiles are printed - I thought they may have been double-byte chars, but that doesn't appear to be the case, and I'm not sure yet how the text printing engine figures out if a byte is part of a kanji sequence or a single kana/hira/ascii tile....
Title: Re: Cyber Knight translation
Post by: esteban on 03/13/2014, 07:46 AM
Sound Test must be revealed. (https://junk.tg-16.com/images/pcds.png)
Title: Re: Cyber Knight translation
Post by: megatron-uk on 03/16/2014, 07:55 AM
I have an update with most of the starting menu's and stats screens in the game.

The following are now converted:

Here's what the partially translated player name entry looked like after my initial work:
(https://web.archive.org/web/20170322171327/http://targetearth.dyndns.org/www/gaming/cyber/cyber-partial-translated-player-entry.png)

Here's the fully translated version including Kanji replacement:
(https://web.archive.org/web/20170322171314/http://targetearth.dyndns.org/www/gaming/cyber/cyber-translated-player-entry.png)
Unfortunately I had to lose a character from the 'Del' command, hence 'Rm' (Remove). If it's a deal breaker I may look at it again.



Here's the initial version of my translation for the player skill point assignment screen, including the players teammates with their translated names:
(https://web.archive.org/web/20170322171328/http://targetearth.dyndns.org/www/gaming/cyber/cyber-knight-starting-stats.png)

And here's the final version:
(https://web.archive.org/web/20170322171314/http://targetearth.dyndns.org/www/gaming/cyber/cyber-translated-player-skills.png)



This is the in-game player/teammate stats screen, which included the translated names and skills:
(https://web.archive.org/web/20170322171324/http://targetearth.dyndns.org/www/gaming/cyber/cyber-knight-pc-stats.png)


And this is the final version, with all text and Kanji replaced:
(https://web.archive.org/web/20170322171315/http://targetearth.dyndns.org/www/gaming/cyber/cyber-translated-pc-stats.png)




The load save game screen originally looked like this:
(https://web.archive.org/web/20170322171323/http://targetearth.dyndns.org/www/gaming/cyber/cyber-knight-load-save.png)

Here it is now - some small edits still to do, but it's in 90% English now (all the messages the Doctor says are now translated):
(https://web.archive.org/web/20170322171320/http://targetearth.dyndns.org/www/gaming/cyber/cyber-partial-translated-load-save.png)



So what's next? Finish off the 'load save from password' screen, and then every single initial game menu screen is finished 100%. Next move on to the main in-game screens - already got a good 75% of the status screen that pops up while in the overworld map (list inventory [no items translted yet!], apply repair/medkit etc). And then the in-ship menus, which will make navigating and testing the game easier.
Title: Re: Cyber Knight translation
Post by: Psycho Punch on 03/16/2014, 03:22 PM
Pretty cool, I guess you intend to release a 100% translation patch? Looking forward to it.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 03/16/2014, 04:02 PM
That's the intention. Of course, all of the tools I've written to do this, and the individual patches themselves are all up on my github repository, so anyone can feel free to take the content and do what they want - change the text, grammer or language, if they feel like it. Or simply apply the patches to their own copy of the original Japanese rom file.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 03/16/2014, 06:50 PM
Last update for the weekend.

Most of the in-ship menus are now translated (some better than others - most only have 5 characters of width to play with). This is just a first pass at them, they'll need some work to fit better and some could do with changing: 'Heal' instead of 'Treat', for example.

The 'mecha hangar' screen isn't finished yet as there are loads of sub-menus and sub-options to go through, all to do with reconfiguring your mecha, browsing the parts database etc.

Anyway, here's a picture of the latest in-ship screenshots. This patch isn't up on github yet, but everything else done so far is.
Title: Re: Cyber Knight translation
Post by: spenoza on 03/17/2014, 12:36 PM
I see "search" is a bit long. Would "scan" be an appropriate synonym in this situation?
Title: Re: Cyber Knight translation
Post by: megatron-uk on 03/17/2014, 12:59 PM
Quote from: guest on 03/17/2014, 12:36 PMI see "search" is a bit long. Would "scan" be an appropriate synonym in this situation?
Yep, that's one that needs a rethink. 'Scan' would certainly work in that situation - but I'll have to change 'Scan' in the lab screen to something else... 'Data' perhaps?

They're supposed to be 'Survey' and 'Analyse', respectively.
Title: Re: Cyber Knight translation
Post by: spenoza on 03/18/2014, 05:30 PM
Quote from: megatron-uk on 03/17/2014, 12:59 PM
Quote from: guest on 03/17/2014, 12:36 PMI see "search" is a bit long. Would "scan" be an appropriate synonym in this situation?
Yep, that's one that needs a rethink. 'Scan' would certainly work in that situation - but I'll have to change 'Scan' in the lab screen to something else... 'Data' perhaps?

They're supposed to be 'Survey' and 'Analyse', respectively.
A synonym would certainly be better than an abbreviation. I get the feeling people would be REALLY confused when confronted with SURV as a menu option, or especially if they were trying to figure out what ANAL would do.

What does the search function do? You might be able to keep scan if we look at other alternatives. Hunt? Ping?
Title: Re: Cyber Knight translation
Post by: megatron-uk on 03/18/2014, 05:38 PM
Quote from: guest on 03/18/2014, 05:30 PMA synonym would certainly be better than an abbreviation. I get the feeling people would be REALLY confused when confronted with SURV as a menu option, or especially if they were trying to figure out what ANAL would do.

What does the search function do? You might be able to keep scan if we look at other alternatives. Hunt? Ping?
Yeah, 'ANAL' is an option from a whole other game genre :lol:

The Search/Survey function scans a nearby planet(s) and then pops up a box where the ship AI describes the planet in detail (atmosphere, technology, points of interest etc). Analyse in the Lab screen is the method used to get information from recently defeated enemies to learn new technologies and weapons.

'Scan' and 'Data' is probably sufficient to describe those two functions.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 03/21/2014, 01:18 PM
Quick update!

I now have the following sections identified and extracted:

0x1400c-0x1b8cd - INCOMPLETE - Main game story dialogue.json
0x1b8d6-0x1bfdf - COMPLETE - Intro_scrolling.json
0x1c040-0x1c90d - INCOMPLETE - Game menus and save screens.json
0x1c952-0x1ca2b - COMPLETE - Further game menus.json
0x1cbad-0x1ccac - INCOMPLETE - NPC_names.json
0x1d59f-0x1dc7a - INCOMPLETE - Weapon names.json
0x1dc87-0x1dee3 - INCOMPLETE - Shipboard_menus_1.json
0x1e71c-0x1ea86 - INCOMPLETE - Professor lab screen.json
0x1ea88-0x1ec00 - INCOMPLETE - Mecha overview screen.json
0x1ec08-0x1edfa - INCOMPLETE - Possibly Closing Credits?.json
0x1ee0f-0x1f0df - INCOMPLETE - Game Credits.json
0x1f0f4-0x1fb3f - INCOMPLETE - unknown.json
0x28086-0x29efe - INCOMPLETE - MICA and player dialogue.json
0x29efe-0x2a1af - COMPLETE - Intro_cinema.json
0x2a74c-0x2a930 - COMPLETE - Lights_out_after_intro.json

Virtually all of the game menus are now in english, with just a few (one or two in the hangar/mecha screen and the two 'save game', 'save password' in the med bay screen) left to find.

A good chunk of the weapons/items that are available from the start of the game have had both their name and description translated into english (see images below), but I've yet to find the location of their stats (weapon strength, size, 1/2handed etc).

Translated and retitled most of the main 'display mecha stats' screen.

Translated all of the 'equip your mecha' screen.

Mecha names need translating.

Some NPC names need translating.

Found the main game dialogue and extracted it.

Found the dialogue that is shown when scanning a planet - it's also extracted.

Translated all of the pause-game menu options (heal/repair/show item screen etc), as well as the actions you can perform from there; choose a medic and heal one of your party, choose a tech/engineer and fix one of you party's mecha. etc.

Here's some the images of the new progress:

Screenshot-Cyber Knight (E)-2.png
Title: Re: Cyber Knight translation
Post by: jeffhlewis on 03/21/2014, 04:06 PM
You still need any of this stuff translated, or are these some of those dreaded double-height kanji?

That plasma gun one is reading:

PlasmaGun
Type: Gunshot (Shooting? Firing?)      Hand: 1H         Size: S
Power: Beam            10
Title: Re: Cyber Knight translation
Post by: megatron-uk on 03/21/2014, 07:58 PM
Got them - just need to find the actual occurences in the file for the categories (type, size, etc), the actual data has been converted, but I'm trying to find the position in the rom where 'Type','Size' and 'Power' are stored, at the moment it's eluding me!  :lol:

Here's what it's looking like now, and the difference between a 'shooting' weapon, and an option item (grenade/rocket/missile pack etc), and a field (screen/barrier/jammer) item....
Title: Re: Cyber Knight translation
Post by: megatron-uk on 03/25/2014, 07:05 PM
Just committed a big update to github - this has patches for almost all weapon names and their descriptions, an initial pass at translating enemy names, the battle screen menu interfaces (one or two glitches there), as well as starting the dialogue shown from the scientist when analysing enemy wreckage to earn new parts.

Lots more done, still lots to do!
Title: Re: Cyber Knight translation
Post by: megatron-uk on 03/27/2014, 07:19 PM
Anyone care to help translate the list of NPC names or enemy names:

NPC names:
クレイン Klein
キリ Kiri
シャイン Shine
ヴィンド Vynd
ニジ‐ナ Nejena
MICA MICA
キプテン Captain
パイロット Pilot
プロフェッサ‐ Prof.
ドクタ‐ Dr.
エンジニア Engineer
バックアップメモリ BackupRAM
きぉうじ Ouji ???
だいとうりぉう Daito ???
だいとうりぉうふじん Mrs Daito ???
トレ‐ダ‐ Trader
ロボット Robot
シ‐クラン
ちぉうろう
げんしじん
しうちぉう
いわ Rock
シリコンせいぶつ Silicon Life
クジラ Whale
サソリ Scorpion
かいぞく Pirate
きかい
へいし Soldier
しぉうぐん Shogun
しれいかん Cmmdr
ガライシアン Garaian
スラウレ Suraure
クイ‐ン
メンタ‐ナ Mentana

Enemy names:
ビッグフット Bigfoot
サスカッチ Yeti
ウェンディゴ Wendigo
マンドレイク Mandrake
アルラウネ Arurune
サラマンダ‐ Salamander
タッツェルブルム Tatzelwurm
ナイトガ‐ント Nitegaunt
デュラハン Durahan
ハ‐ピ‐ Harpy
ユ‐ウォ‐キ‐
シムルグ Grifon
グレムリン Gremlin
フィフィネラ Fifinela
スパンジュ‐ル
ジャバ-ウォック Jaberwocky
カトブレパス Catoblepas
ガンカナ‐ Gunner
シ‐オ‐ク Shioku
メロウ Melo
リャナンシ‐ Faery
バンシ‐ Banshee
ザラダン Zaradan
バハム‐ト Bahamut
スキュラ Scyla
テュポ‐ン Typhon
ミナトン Minaton
ゴルゴン Gorgon
ファ‐ゴルタ Fagorut
ファ‐ジャルグ
ガネ‐シャ Ganesha
チャウグナ‐ Chaugnar
ブラウニ‐ Brownie
クルラホ‐ン
レプラホ‐ン Leprechaun
ラクシャサ Rakasha
ジャガナ‐タ Jaganata
クラ‐ケン Kraken
ガタノソア Ghatanothoa
バ‐サ‐カ‐クイ‐ン
ギズモ Gizmo
ウィスプ Wisp
ブル‐ウィプス Blue Wisp
サンドクロ‐ラ‐
ロッククロ‐ラ‐
パイロワ‐ム
ブロッブ Blob
シリコニ‐ Shirikoni
デビルアイ DoubleEye
イ‐ビルアイ OneEye
クアドケラトプス Quadceratops
メガケラトプス Megaceratops
ダイモザウルス Dymo Saurus
アラクニザウルス Ara Kuni Saurus
レプタントスカウト Replicant Scout
レプタントレイバ‐ Replicant Leyvia ???
レプタントロイヤル Replicant Lord ???
パイレ‐ツガンナ‐ Pirate Gun
パイレ‐ツヘッド Pirate Head
パイレ‐ツキング Pirate King
Title: Cyber Knight translation
Post by: jeffhlewis on 03/27/2014, 10:12 PM
Taking a stab here at some...maybe some hints if you know the context of the sentences they're used in.

シ‐クランShi Clan ???
げんしじんGenshi-Jin (Atom Man?)
きかいOpportunity
クイ‐ンQueen

バ‐サ‐カ‐クイ‐ンBerzerker Queen (i think)
サンドクロ‐ラ‐Sandcrawler
ロッククロ‐ラ‐Rockcrawler
パイロワ‐ム Bailo Waamu (Bilo Worm ???)
レプタントレイバ‐Replicant Leyvia ??? (second word might be "labor" or " lever")
レプタントロイヤルReplicant Lord ??? (second word might be "loyal" or "royal")
Title: Re: Cyber Knight translation
Post by: megatron-uk on 03/28/2014, 04:57 AM
Quote from: jeffhlewis on 03/27/2014, 10:12 PMTaking a stab here at some...maybe some hints if you know the context of the sentences they're used in.

シ‐クランShi Clan ???
げんしじんGenshi-Jin (Atom Man?)
きかいOpportunity
クイ‐ンQueen
Thanks for those - I didn't pick any of those up.

Quoteバ‐サ‐カ‐クイ‐ンBerzerker Queen (i think)
That sounds right. The main enemy in the game are the robotic 'Berzerkers'.

Quoteサンドクロ‐ラ‐Sandcrawler
ロッククロ‐ラ‐Rockcrawler
I did pick up the rock/sand thing. I thought they may be variations on the same enemy. Thanks!

Quoteパイロワ‐ム Bailo Waamu (Bilo Worm ???)
Maybe Bile Worm?

Quoteレプタントレイバ‐Replicant Leyvia ??? (second word might be "labor" or " lever")
レプタントロイヤルReplicant Lord ??? (second word might be "loyal" or "royal")
Replicant Laborer / Slave / Worker for the former perhaps?

'Royal' is a bit ambiguous - that's one the reasons I thought 'Lord' might be better - though I don't know the context as I've not played through the game.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 03/28/2014, 06:37 AM
Added in the translations for the enemy names and I'm now over-size for that section  :(  I can handle individual names longer or shorter than original (my inserter script will pad if required, or leave shorter strings as-is to free up additional space for any longer ones), but I've got a hard limit of 555 bytes for the entire section. I'm up at 566 with the current names.  I'm probably going to have to go through and shorten some of them. One of the big culprits is the 'Replicant' name - 'Replicant Scout' is not only long in terms of bytes, it will also be long in on-screen terms.... perhaps 'RepliScout' and 'RepliWorker' ....  :-k
Title: Re: Cyber Knight translation
Post by: jeffhlewis on 03/28/2014, 11:18 AM
Thinking a bit more after reading your responses - I took 4 years of Japanese in High School and that's what I'm working with here, so needless to say I'm very rusty, haha.

シ‐クランShi Clan

The more I think about this - the "Shi-" might translate to "C-" as in "C-Clan." Again - totally depends on the context. The katakana reads "Shii-Ku-La-N"

きかいOpportunity

As I was taught, "kikai" generally means "opportunity" but it can mean a bunch of different things according to some online dictionaries:
-Opportunity
-Chance
-Instrument
-Machine
-Mechanism
-Strange
-Weird
-Etc....

Source: http://www.kanjijapanese.com/en/dictionary-japanese-english/kikai (http://www.kanjijapanese.com/en/dictionary-japanese-english/kikai)
Title: Re: Cyber Knight translation
Post by: megatron-uk on 03/28/2014, 11:52 AM
QuoteThe more I think about this - the "Shi-" might translate to "C-" as in "C-Clan." Again - totally depends on the context. The katakana reads "Shii-Ku-La-N"

きかいOpportunity

As I was taught, "kikai" generally means "opportunity" but it can mean a bunch of different things according to some online dictionaries:
-Opportunity
-Chance
-Instrument
-Machine
-Mechanism
-Strange
-Weird
-Etc....

Source: http://www.kanjijapanese.com/en/dictionary-japanese-english/kikai (http://www.kanjijapanese.com/en/dictionary-japanese-english/kikai)
Machine .... that's interesting. It ties in with the whole Berzerker robot army theme.

I wish I had a copy of the SNES script. As it is, if I want it, I'll either have to play right through the SNES translation, or write another set of tile lookups and extraction scripts for that too. I really don't fancy doing that, just to get the full listing of what they translated. (and anyway, there are apparently differences in the game too).

Edit: There's some interesting background information (that I haven't been able to find elsewhere) on the Japanese wikipedia page (http://ja.wikipedia.org/wiki/%E3%82%B5%E3%82%A4%E3%83%90%E3%83%BC%E3%83%8A%E3%82%A4%E3%83%88)
Title: Re: Cyber Knight translation
Post by: esteban on 03/28/2014, 09:44 PM
Quote from: guest on 03/28/2014, 06:44 PMI have nothing to contribute other than praise. I knew nothing of this game before you made this thread, megatron. And now I'm friggin excited about this project! Looks like such an awesome game... thanks for your efforts on this!
Ditto. Ditto.
Title: Re: Cyber Knight translation
Post by: ccovell on 03/29/2014, 01:05 AM
Quote from: megatron-uk on 03/27/2014, 07:19 PMAnyone care to help translate the list of NPC names or enemy names:

NPC names:
クレイン Klein  <-- Crane
キリ Kiri
シャイン Shine
ヴィンド Vynd
ニジ‐ナ Nejena
MICA MICA
キプテン Captain   <-- "Cptain".  Are you missing small kanas?
パイロット Pilot
プロフェッサ‐ Prof.
ドクタ‐ Dr.
エンジニア Engineer
バックアップメモリ BackupRAM
きぉうじ Ouji ??? <-- Prince
だいとうりぉう Daitoryou  <-- President
だいとうりぉうふじん Mrs Daito <--- 1st Lady
トレ‐ダ‐ Trader
ロボット Robot
シ‐クラン
ちぉうろう <-- Elder (or City Elder)
げんしじん <-- Caveman (maybe)
しうちぉう <-- Chief
いわ Rock
シリコンせいぶつ Silicon Life
クジラ Whale
サソリ Scorpion
かいぞく Pirate
きかい
へいし Soldier
しぉうぐん Shogun
しれいかん Cmmdr
ガライシアン Garaian <-- Garaician
スラウレ Suraure
クイ‐ン   <--- Queen
メンタ‐ナ Mentana

Enemy names:
ビッグフット Bigfoot
サスカッチ Yeti
ウェンディゴ Wendigo
マンドレイク Mandrake
アルラウネ Arurune
サラマンダ‐ Salamander
タッツェルブルム Tatzelwurm
ナイトガ‐ント Nitegaunt
デュラハン Durahan
ハ‐ピ‐ Harpy
ユ‐ウォ‐キ‐
シムルグ Grifon
グレムリン Gremlin
フィフィネラ Fifinela
スパンジュ‐ル
ジャバ-ウォック Jaberwocky
カトブレパス Catoblepas
ガンカナ‐ Gunner  <-- "Guncaner"
シ‐オ‐ク Shioku  <-- Sea Orc
メロウ Melo
リャナンシ‐ Faery
バンシ‐ Banshee
ザラダン Zaradan
バハム‐ト Bahamut
スキュラ Scyla
テュポ‐ン Typhon
ミナトン Minaton
ゴルゴン Gorgon
ファ‐ゴルタ Fagorut
ファ‐ジャルグ
ガネ‐シャ Ganesha
チャウグナ‐ Chaugnar
ブラウニ‐ Brownie
クルラホ‐ン
レプラホ‐ン Leprechaun
ラクシャサ Rakasha
ジャガナ‐タ Jaganata
クラ‐ケン Kraken
ガタノソア Ghatanothoa
バ‐サ‐カ‐クイ‐ン <-- Berserker Queen
ギズモ Gizmo
ウィスプ Wisp
ブル‐ウィプス Blue Wisp
サンドクロ‐ラ‐ <-- Sandcrawler
ロッククロ‐ラ‐ <-- Rock Crawler
パイロワ‐ム    <-- Pyro Worm
ブロッブ Blob
シリコニ‐ Shirikoni   <-- "Siliconi"
デビルアイ DoubleEye   <--- Devil Eye
イ‐ビルアイ OneEye    <-- Evil Eye
クアドケラトプス Quadceratops
メガケラトプス Megaceratops
ダイモザウルス Dymo Saurus
アラクニザウルス Ara Kuni Saurus   <-- Arachnisaurus
レプタントスカウト Replicant Scout
レプタントレイバ‐ Replicant Leyvia ???
レプタントロイヤル Replicant Lord ???
パイレ‐ツガンナ‐ Pirate Gun <-- Pirate Gunner
パイレ‐ツヘッド Pirate Head
パイレ‐ツキング Pirate King
Title: Re: Cyber Knight translation
Post by: BigusSchmuck on 03/29/2014, 01:40 AM
AWESOME! I hope there will be more translations in the future.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 03/29/2014, 04:46 AM
Thanks Chris.

I'll check the Captain text again, but it should be correct.

Yep, sorry, the extraction was messed up on that list. The NPC list is one of my earlier extractions and was missing several characters.

I've just re-run it and it now shows as the following:

0x1cbae クレイン Klein
0x1cbb4 キリ Kiri
0x1cbb8 シャイン Shine
0x1cbbe ヴィンド Vynd
0x1cbc6 ニジ‐ナ Nejena
0x1cbcf MICA
0x1cbd5 キャプテン Captain
0x1cbde パイロット Pilot
0x1cbe7 プロフェッサ‐ Prof.
0x1cbf2 ドクタ‐ Dr.
0x1cbfa エンジニア Engineer
0x1cc07 バックアップメモリ BackupRAM
0x1cc14 きょうじゅ
0x1cc1b だいとうりょう President
0x1cc24 だいとうりょうふじん First Lady
0x1cc31 トレ‐ダ‐ Trader
0x1cc39 ロボット Robot
0x1cc40 シ‐クラン Shi Clan
0x1cc47 ちょうろう
0x1cc4d げんしじん Atom Man
0x1cc55 しゅうちょう
0x1cc5c いわ Rock
0x1cc5f シリコンせいぶつ Silicon Lifeform
0x1cc6b クジラ Whale
0x1cc71 サソリ Scorpion
0x1cc76 かいぞく Pirate
0x1cc7c きかい Machine
0x1cc80 へいし Soldier
0x1cc84 しょうぐん Shogun
0x1cc8b しれいかん Cmmdr
0x1cc91 ガライシアン Garaian
0x1cc9a スラウレ Suraure
0x1cca0 クイ‐ン Queen
0x1cca6 メンタ‐ナ Mentana

The Hiragana きょうじゅ is 'Enjoy' ... but the Kana is キョウジュ which is 'Professor'. All my extractions since the very early ones do both character sets, to help in situations like this. e.g.:

{
        "block_range" : "0x1cbad-0x1ccac - Ouji? - professer/teacher in first world school",
        "block_description" : "NPC character names.",
        "string_start" : "0x1cc14",
        "method" : 3,
        "start_bytes" : [],
        "end_bytes" : ["00"],
        "raw_size" : 7,
        "raw" : ["b7", "ae", "b3", "bc", "de", "ad", "00"],
        "raw_text" : "きょうじゅ",
        "alt_text" : "キョウジュ",
        "trans_size" : 0,
        "trans_text" : "Prof."
    }

The alternate extraction now makes sense as its the character who appears in the school to offer you a star map.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 03/30/2014, 10:22 AM
Demo movie of the translated ship menu structure:
http://youtube.com/xbZdix18ho8 (https://youtu.be/xbZdix18ho8)

Here's another clip - a demo of in-game translation from the first level in the game (Far World).
http://youtu.be/DC6F_XCAc10
Title: Re: Cyber Knight translation
Post by: Arjak on 03/30/2014, 01:09 PM
Wow, this is looking really good! At this rate, you'll be done in no time! It will be great to have another translated PCE game to play.

Keep up the good work!
Title: Re: Cyber Knight translation
Post by: Psycho Punch on 03/30/2014, 02:40 PM
I loved the legend of the Old Earth... this game has great storytelling! :P

Keep it up my friend. It is hard to have translation projects on the PCE, hopefully it will change one day.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 03/30/2014, 03:09 PM
Quote from: guest on 03/30/2014, 02:40 PMI loved the legend of the Old Earth... this game has great storytelling! :P

Keep it up my friend. It is hard to have translation projects on the PCE, hopefully it will change one day.
:lol: you caught that bit, did you? There's a few words in the second part of that phrase I can't quite work out.. so I dropped a placeholder in to remind me to fix it  :D
Title: Re: Cyber Knight translation
Post by: SuperPlay on 04/03/2014, 02:43 PM
Nice to see how you are progressing with this :-) Keep up the great work.
Title: Re: Cyber Knight translation
Post by: esteban on 04/03/2014, 05:02 PM
Quote from: megatron-uk on 03/30/2014, 10:22 AMDemo movie of the translated ship menu structure:
http://youtube.com/xbZdix18ho8 (https://youtu.be/xbZdix18ho8)

Here's another clip - a demo of in-game translation from the first level in the game (Far World).
http://youtu.be/DC6F_XCAc10
Again, I have never paid much attention to Cyber Knight until you started this project.

Now, things are looking pretty darn interesting.  :pcgs:
Title: Re: Cyber Knight translation
Post by: megatron-uk on 04/04/2014, 07:56 PM
Someone I've been talking with over on Romhacking.net has redumped the English script from the SNES version  - I'm currently working on a programme to match the untranslated PCE and SNES text, so that I can automatically pull in the translated SNES text. I'm getting good results initially, and, as there are hundred of strings to translate, this should speed things up somewhat.

It's 'mapScript.py' which is up on my github repository now.
Title: Re: Cyber Knight translation
Post by: NightWolve on 04/06/2014, 06:11 AM
You are the most prolific status updater I have ever seen. Haven't decided if this is a good or a bad thing. ;)
Title: Re: Cyber Knight translation
Post by: esteban on 04/06/2014, 07:20 AM
(https://junk.tg-16.com/images/pcds.png)
Title: Re: Cyber Knight translation
Post by: NightWolve on 04/07/2014, 03:51 AM
That is an impeccable argument, John, I must admit.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 04/07/2014, 10:14 AM
The latest commit (https://github.com/megatron-uk/cyberknight-pce) now adds patches which have gone through the mapping script (mapScript.py) 3 times.

At this point well over 50% of the game dialogue has been matched from PC-Engine to the SNES english dump by my mapScript.py - that's a LOT of work that now doesn't need to be done manually.

I'll have to go back over the matches and manually insert control codes, linebreaks etc, as the SNES version is totally different in that regard. But I'll try and do it world by world, so I can play through each in 100% english before moving on.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 04/10/2014, 04:03 AM
I've re-dumped the main game dialogue - the SNES text doesn't have anything like the <wait for button press at end of dialogue>, so all of the strings dumped for the SNES english version are much, much shorter than the PCE version. To explain, here's a simplified example:

PCE:
StringA: "Hello, I'm the president<0x08>Welcome to farworld, its our home<0x08>Can you get rid of the monsters?<0x08><0x00>"

SNES:
StringA: "Hello, I'm the president"
StringB: "Welcome to farworld, our home"
StringC: "Can you defeat the monsters?"

Instead of breaking on null bytes (0x00), I've dumped the main text again breaking on the <wait> control byte (0x08) - so that the strings are split and more closely match the SNES. I've re-run that new script through my PCE to SNES mapper and have got a good two-thirds to three-quarters match. That's *hell* of a lot better, and means I won't have to go through many hundreds of strings and match them by hand to the SNES text. It does mean, however that I still need to go through and enter control codes, newlines etc by hand, as the SNES text is formatted for a smaller dialogue window. I'm really please d with the way things are going - however things might slow for a few weeks; our house has finally sold and we're in the middle of starting mortgage proceedings on our new one. I'll try and do bits and pieces (including new screenshots and commits to github) when I get the chance though!

The SNES script has been really useful so far, here's the things that it has enabled me to complete:

All NPC names
All planet names
All item/quest objective names
Missing weapon/enemy names

Those are all 100% complete now.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 04/12/2014, 05:54 AM
Got a lot more of the text done for the first world - however an unexpected side affect is that the small expansion of the main text dialogue has pushed back the scrolling intro text and appears to have broken it. So after the cinematic opening the text that appears during the scrolling ("The story of the Swordfish so far...") section is now busted. I think it's been moved past some invisible limit (perhaps an 8k page section?).

This will need looking at, but the in-game text is a priority for me first. I may need some help off an experienced 6502 hacker to try and resolve the loader that displays the scrolling text.
Title: Re: Cyber Knight translation
Post by: NightWolve on 04/12/2014, 09:13 AM
That's good fortune that there's another game to take translated text from. As for capable hackers for text routines, besides malducci->Tomaitheous->Bonknuts-> TurboXray here, there's also EsperKnight who you could contact via SamIAm. Both are typically pretty busy with projects on hand, though.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 04/12/2014, 06:29 PM
Here's the latest state of the battle screen. All the menus options are done - though the main commands are one character too long (I really need to figure out how the dialogue box is drawn). The vast majority of the status text that appears from MICA is also done, but it's difficult to check as it's displayed randomly!

Anyway, to give you a flavour of what it now looks like....

cyber-battle1.png cyber-battle2.png
cyber-battle3.png cyber-battle4.png
Title: Re: Cyber Knight translation
Post by: megatron-uk on 04/12/2014, 06:31 PM
... and here is what is displayed when you find items in the wreckage (the games equivalent of currency or new equipment - there are no shops as such, you research the items you find and it grants new weapons and options to fit to your mecha)...
Title: Re: Cyber Knight translation
Post by: megatron-uk on 04/16/2014, 08:49 AM
I think I've got all of the in-battle text done - including the status effect text, enemy names, level increase messages,  etc. I haven't been able to test all the messages, as some are related to certain game conditions (enemies fleeing etc), but most are now decently enough formatted to allow the battle screens to be played fully.

I'm left with three phrases left from about 90 initial text strings for the battle screens that I cannot either find a decent translation for, or a match in the SNES script. They are:

は ぼうぎょフィールドを はった
I think this is to do with either defense ability, or defense field

5じをまわったので かれらは かえった。
No idea on this one.

『よ―し! かえって ねるぞ!!』
Seems to be a quip from either a teammate or an enemy?

Any ideas?
Title: Re: Cyber Knight translation
Post by: megatron-uk on 04/16/2014, 11:11 AM
Thought it might be useful to show the current status of the translation patches... a lot of work is done, and it's mostly big long sections of text in the main game dialogue that need doing now:

Patch Summary
=============
---------|------------|------------|-------|---------------
  Strings|Translations|SNES matches|Tiny   | Patch Name
---------|------------|------------|-------|---------------
-  717   | 124 / 717  |  466 / 717 |   18  | 0x1400c-0x1bfe0 - INCOMPLETE - Main game story dialogue and scrolling text.json
-  113   |  59 / 113  |    2 / 113 |   11  | 0x1c040-0x1c90d - INCOMPLETE - Game menus and save screens.json
-    4   |   3 /   4  |    0 /   4 |    0  | 0x1c952-0x1ca2b - COMPLETE - Further game menus.json
-   42   |  34 /  42  |    0 /  42 |    8  | 0x1cbad-0x1ccac - INCOMPLETE - NPC_names.json
-   17   |  17 /  17  |    1 /  17 |    0  | 0x1ccbf-0x1cdae - COMPLETE - Items.json
-   41   |  41 /  41  |    3 /  41 |    0  | 0x1cdaf-0x1ceef - COMPLETE - Planet Names.json
-   87   |  41 /  87  |   17 /  87 |   20  | 0x1cef0-0x1d373 - INCOMPLETE - Battle screen.json
-   60   |  60 /  60  |    0 /  60 |    0  | 0x1d374-0x1d59f - COMPLETE - Enemies.json
-  141   |  97 / 141  |    0 / 141 |   33  | 0x1d59f-0x1dc7a - COMPLETE - Weapon names.json
-   44   |  44 /  44  |    0 /  44 |    0  | 0x1dc87-0x1dee3 - COMPLETE - Shipboard_menus_1.json
-  140   |  33 / 140  |   35 / 140 |   56  | 0x1defc-0x1e71c - INCOMPLETE - Some initial in-game dialogue.json
-   15   |   3 /  15  |    0 /  15 |    1  | 0x1e71c-0x1ea86 - INCOMPLETE - Professor lab screen.json
-   54   |  54 /  54  |    0 /  54 |   40  | 0x1ea88-0x1ec00 - COMPLETE - Mecha overview screen.json
-  104   |   0 / 104  |    3 / 104 |   41  | 0x1ec08-0x1edfa - INCOMPLETE - Possibly Closing Credits?.json
-  101   |   0 / 101  |    0 / 101 |   52  | 0x1ee0f-0x1f0df - INCOMPLETE - Game Credits.json
-  128   |  80 / 128  |   57 / 128 |   36  | 0x1f0f4-0x1fb3f - INCOMPLETE - unknown.json
-  159   |  35 / 159  |  112 / 159 |    4  | 0x28086-0x29efe - INCOMPLETE - MICA and player dialogue.json
-   24   |  24 /  24  |    0 /  24 |    0  | 0x29efe-0x2a1af - COMPLETE - Intro_cinema.json
-   10   |  10 /  10  |    0 /  10 |    0  | 0x2a74c-0x2a930 - COMPLETE - Lights_out_after_intro.json
---------|------------|------------|-------|---------------
  Strings|Translations|SNES matches|Tiny   | Patch Name
---------|------------|------------|-------|---------------
- 2001     759 / 2001   696 / 2001    320


Output taken from mapScript.py, my PCE to SNES script matcher.

Strings are extracted 'text' from the game, probably 95% text, but some odd control sequences too.
Translations are fully converted English strings.
SNES matches are text that have found an English match in the SNES script, but have not been added as translations yet.
Tiny strings are 1/2 byte control codes that are probably not real text.
Title: Re: Cyber Knight translation
Post by: NightWolve on 04/16/2014, 11:16 PM
Quote from: megatron-uk on 04/16/2014, 08:49 AMは ぼうぎょフィールドを はった
I think this is to do with either defense ability, or defense field
WWWJDIC (http://www.edrdg.org/cgi-bin/wwwjdic/wwwjdic?9T) can only translate the Katakana out of that and there's only one result: Field.
http://translate.google.com/ (http://translate.google.com/) says: "I put the Defense Field"
Note: For the small stuff, menus, items, inventory, etc. that use mostly Katakana, you can often get away with using online translation sites and not run to a human.

So, this could be "Defense Field On" if it's displayed as a status message in a menu, or if it's a statement in a message box as the character speaks it'd be more like "Defense Field was enabled." Whatever you want. You'll just have to see where, but yeah, "Defense Field" is probably mostly correct or more natural sounding would be to use shield. For lack of spacing, you could use "Force Field" or something.

Well, very efficient progress/status report, megatron. Carry on! ;)
Title: Re: Cyber Knight translation
Post by: megatron-uk on 04/17/2014, 03:42 AM
I think all three strings are messages from the ships AI system (MICA) that scans your enemies before battle (they're found in a section of text with similar messages that I've found matches in the SNES script). These phrases (amongst *many* others) are occasionally added into the normal flow of battle in addition to the normal (player vs enemy 1, attack success, dmg 23). It says things like "enemy strong at melee, engage remotely"  or "enemy weak to beam weapons". This one could well be stating that the enemy has just activated a defense field, which would potentially nullify certain types of weapon attacks. Although I'm not sure whether its telling you that an enemy has done this, or that *you* should.
Title: Re: Cyber Knight translation
Post by: NightWolve on 04/17/2014, 04:21 AM
Quote from: megatron-uk on 04/17/2014, 03:42 AMAlthough I'm not sure whether its telling you that an enemy has done this, or that *you* should.
Yeah, so you'll have to see where it's used in-game to obtain the context. That's why translation work requires a 2nd or 3rd pass to get things done right. 1st pass, you get it all back in the game, play through it all the way to spot such issues, then go back to the script and correct them, etc. I've had stuff corrected years later that was initially missed like in the "Ys I Complete" script...
Title: Re: Cyber Knight translation
Post by: megatron-uk on 04/17/2014, 04:35 AM
Yep, that's what I've been doing a lot of at the moment - I'm fairly sick of world 1 now :lol:
Title: Re: Cyber Knight translation
Post by: NightWolve on 04/17/2014, 04:54 AM
Ah yes, that's the other sacrifice that comes with this kind of work, you'll hate the game by the time you're done with the project. But, you'll gain all these adoring fans and "fans" who'll be playing a polished product for the first time, so... ;)
Title: Re: Cyber Knight translation
Post by: esteban on 04/17/2014, 07:46 AM
Quote from: NightWolve on 04/17/2014, 04:54 AMAh yes, that's the other sacrifice that comes with this kind of work, you'll hate the game by the time you're done with the project. But, you'll gain all these adoring fans and "fans" who'll be playing a polished product for the first time, so... ;)
Fans. Ha!  :pcgs:
Title: Re: Cyber Knight translation
Post by: megatron-uk on 04/17/2014, 09:35 AM
Yes. Get in line, fans.  :mrgreen:
Title: Re: Cyber Knight translation
Post by: TurboXray on 04/17/2014, 01:12 PM
Quote from: NightWolve on 04/17/2014, 04:54 AMAh yes, that's the other sacrifice that comes with this kind of work, you'll hate the game by the time you're done with the project. But, you'll gain all these adoring fans and "fans" who'll be playing a polished product for the first time, so... ;)
Haha - yup!
Title: Re: Cyber Knight translation
Post by: megatron-uk on 04/18/2014, 03:24 AM
Here's an Easter present for you all.... a video of the Cyber Knight gameplay in full English. Enjoy!

http://youtu.be/DVnsNSLQ_Hs

The clip is from the start of the game, loading a save and proceeding from the initial landing site (I'd already talked with the townspeople in an earlier video clip), fighting your way to the crashed ship 'Europa' which is one of the first main quests in the game. This includes several of the in-battle messages that you receive from the AI, MICA, that I've been translating recently - it shows how they're a mix of 'helpful' strategy and, well, just bizarre ramblings!
Title: Re: Cyber Knight translation
Post by: megatron-uk on 04/20/2014, 07:24 AM
Ok, I've got a Mednafen query. I'm debugging some text display errors that appear part way through the exploration of the 'Europa' ship. Up to this point all my in-game text has been ok. But I've got some weird truncation/duplication going on (see attached images):

In the battle scene below I should have a display that reads: "MICA: Engage from a distance" or "MICA: Use ranged attacks" (which are some of the 'helpful' comments that MICA throws at you) , but have instead part of both - all other helpful messages in battle to this point have been ok, as well as some Japanese text - none of which is left in the battle message section of the rom.

In the discovery of a drive coil scene, it should read: "Vynd:Here we go! A Drive Coil!, Ok let's get this back to the ship", and then another window will popup stating "Drive Coil Found!", the same as what happens when you get the Star Map from the University. Instead I get the expected text, then a run-on of untranslated text, then the 'Drive Coil Found' text runs on again from that. Bizarro.

Anyway, what I was wondering is there any way Mednafen can be used to load a state file if the underlying rom file has changed? It's a pain adjusting the rom, reinjecting modified text and then playing right through about 10 or 20 battles to get to the point where Vynd says his text again. I believe the states saved by Mednafen only load when the rom has the same checksum, which it obviously doesn't when I change the data....

Screenshot1.png Screenshot2.png
Title: Re: Cyber Knight translation
Post by: NightWolve on 04/20/2014, 07:53 AM
I would suggest the use of a simple emulator like YAME for what you seem to be describing, wanting to load a state file and get to test new changes within the ROM. It's simple, has a full GUI and is compatible with the game having just checked it.

https://www.pcengine-fx.com/downloads/yame038.zip

Setup Instructions:

1) Launch it.
2) Goto -> "Settings" -> "Advance Settings" -> click "Input Tab".
3) Under "Save" press F1, Under Load, press F5 : Now F1 is a quick key to save a state file, F5 to load it.
4) Configure the rest of the buttons to the keyboard as you prefer.
5) Load the ROM under the File menu "Open" command OR drag'n'drop the ROM file on its window, simple as you'd expect.

Now, here's the thing, when you change the ROM file, but you use the state file to resume the game exactly where you were, you have to go somewhere in the game to force the loading of new ROM data otherwise you may not see your changes. The other way is to load the state file, save your game (if possible) via the game's menu, reset the game, and load it again the same way. That'll force a new loading of ROM data into the system RAM, etc. But, I guess you already knew that and want to avoid it. So yeah, the only other way would be to move around in the game to where a new event is loaded or a new level is, etc.

EDIT: This emulator does not use a checksum system for dealing with state files. The state file will simply be named the same as the ROM but with a ".sta" extension, e.g. "Cyber Knight (J).sta". YAME is perfect for this job, for you, etc. It was very handy when it came to hacking Xak III/Ys IV ages ago for its simplicity and quickness when it comes to insert/test-in-game/insert/test-in-game/etc. You're going through some unneeded headaches/tediousness using a command-line Mednafen, and unless it's time to use its debugger for something advanced, you can do better in the meantime, so I'd stick with YAME for most things (once you get the hang of it, which you should no problem).
Title: Re: Cyber Knight translation
Post by: megatron-uk on 04/20/2014, 11:34 AM
That's great - I managed to find a binary of yame that works on Linux (no recent versions unfortunately - so no sound support, but that's not important). I'll give that a go this evening.

Cheers,
John
Title: Re: Cyber Knight translation
Post by: NightWolve on 04/20/2014, 12:05 PM
Oh, you're a Linux guy, huh ? Hmmmm. Well, I have no idea what the interface is like on a Linux build for it, didn't even know the developer made one. Would've been really convenient for hacking to work on a Windows box for you, but oh well. You've managed fine so far anyhow.
Title: Re: Cyber Knight translation
Post by: cabbage on 04/20/2014, 09:57 PM
Quote from: megatron-uk on 04/20/2014, 07:24 AMAnyway, what I was wondering is there any way Mednafen can be used to load a state file if the underlying rom file has changed? It's a pain adjusting the rom, reinjecting modified text and then playing right through about 10 or 20 battles to get to the point where Vynd says his text again. I believe the states saved by Mednafen only load when the rom has the same checksum, which it obviously doesn't when I change the data....
Go into the save-state directory (mcs) and remove the trailing checksum from the save filename.
e.g. something like CyberKnight.214a55e19aed90a0095ca0d65894b7d3.mc0 can be renamed to simply CyberKnight.mc0 and it should load even without a checksum match
Title: Re: Cyber Knight translation
Post by: megatron-uk on 04/21/2014, 04:10 AM
Thanks to both of you. Both methods work, but I'm still stuck with the problem of refreshing the rom file from disk. Savestates load fine now (thanks for the checksum tip, Cabbage), but at that point, the rom data in memory (checked with mednafen debugger) still reflects the pre-modified file. I've tried loading the status screen, finding a battle to fight, viewing another on-screen text dialogue etc, short of having to exit and leave the main playfield (which is the whole point of using a save state to bypass a dozen or so battles) I can't seem to find anyway past this hurdle.

[Edit] Still can't find a way of refreshing the rom data when using either emulator like this, but on this occaision just using the debugger to to alter one or two bytes was sufficient. There was a sequence of bytes embedded in the text sequence on that screen:

"08", "13", "3a", "6a", "00"

Now 08 is 'pause until button pressed' and 00 is 'end of string'. I don't know what the 13/3a/6a sequence is, but it appeared to jump somewhere else (way, way off, into text for the next world - the 'Trader' quests), rather than printing the 'drive coil found' string which was the next thing to show. This seems to be in the original rom too, so I'm not sure what it's actually doing.

For now, I've replaced those 'jump bytes' (for want of a better term) with my 'no-op' codes - 0x5c (actually, character shift, but they have the effect of padding without printing anything on the screen), and the result is as shown in the image below!

Screenshot from 2014-04-21 09:27:30.png
Title: Re: Cyber Knight translation
Post by: Dicer on 04/22/2014, 11:44 AM
Quote from: megatron-uk on 04/20/2014, 11:34 AMThat's great - I managed to find a binary of yame that works on Linux (no recent versions unfortunately - so no sound support, but that's not important). I'll give that a go this evening.

Cheers,
John
Magic Engine runs under Wine in Linux quite perfectly, just a heads up there.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 04/22/2014, 12:42 PM
As you're probably aware, I'm using the SNES version of Cyber Knight to help in the translation. I have a dump of the English script so that's the basis of the text, it's helped a lot.

I've now played and translated everything up to and including the defeat of the dome on the southern continent and am now back talking with the president (and receiving his deepest thanks for dealing with the robot threat).

In the SNES version the dialogue goes something like this:

"Our deepest thanks, the planet is now free"
"You want to go back to earth, yes?"
"I'm afraid we cannot help with our technology"
"The Traders mentioned a race called the Seeclan with advanced technology, maybe they can help"
"Ask details from the Traders. You can find them on Worldring-3"

At which point I presume you then use the rebuilt jumpdrive and move to the Worldring system.

In our PCE version the President says basically the same thing but his last sentence is different:

"Our deepest thanks, the planet is now free"
"You want to go back to earth, yes?"
"I'm afraid we cannot help with our technology"
"The Traders mentioned a race called the Seeclan with advanced technology, maybe they can help"
"『‥‥‥ただ‥‥‥さいきん れんらくが とだえているそうですが‥‥‥』"

I think the last line is something along the lines of 'we had regular contact with them, but it was suddenly cut-off recently'. - that's the last thing he says - no mention of the Worldring system. Hmm, now how do I know I'm supposed to get to the Worldring system?
Title: Re: Cyber Knight translation
Post by: TurboXray on 04/22/2014, 01:58 PM
In mednafen, if you want to reload the rom, then you'll have to do it through the debugger alt+3 tab/window. Go to that, change the window to 'cpu physical'. Use the L option to load a file into start/end part of external address range. Just make sure the rom is raw (headerless), else it'll load the header as well.

 Be careful, as some games load part of the text string from rom, into ram. So this will mess you up for savestates.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 04/22/2014, 02:37 PM
Ah, so that's how to do it! Nice one Tom. That will be very helpful, thanks!
Title: Re: Cyber Knight translation
Post by: whisper2053 on 06/07/2014, 10:29 PM
I feel incredibly dumb right now.

Python is installed, I have your archive from github, and I acquired the proper rom in hopes of being able to perhaps test this out. For the life of me, I cannot figure out how the heck to get Python to execute the appropriate extract and inject .py's and it's driving me crazy.

Sigh.

Had to vent, sorry :(
Title: Re: Cyber Knight translation
Post by: megatron-uk on 06/08/2014, 04:17 AM
What happens when you run:

python injectScript.py -i "Cyber Knight (J).pce" -o "Cyber Knight (E).pce" -d ./patches/
Sorry for lack of updates recently. House finally sold, everything is packed up or in storage and we're waiting to move into our new home (end of June, most likely) .
Title: Re: Cyber Knight translation
Post by: whisper2053 on 06/08/2014, 09:59 AM
(https://web.archive.org/web/20190510052248/http://ft.trillian.im/0be4fb4624961881e0cd1243974b5435ec208f43/6qb9iDe9bk8DTZpt6aQPzB4sVfixR.jpg)

Granted, I'm almost certain I am doing something wrong, as I have never used Python before.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 06/09/2014, 02:02 PM
Don't type 'python' or double click the python icon - it starts an interactive shell; you're running python code at that point. You just need a Windows command prompt open.

Open a command prompt and paste in the full line that I wrote, above.

Something like (presuming you've created a directory called 'cyber' will all the downloaded files and folders in). As long as Python.exe is in your path, you should be able to do something like this:

c:\> cd cyber
c:\cyber>
c:\cyber> python injectScript.py -i "Cyber Knight (J).pce" -o "Cyber Knight (E).pce" -d ./patches/

I do intend to distribute an ips patch once the translation is complete, but for now, having access to the source of the translation is easier for those with a programming background who may want to help out or try their own script.
Title: Re: Cyber Knight translation
Post by: whisper2053 on 01/28/2015, 04:39 PM
Have there been any further updates on this project?

Also, I finally figured out what my previous issue was (ie, I did *not* have Python.exe in my path, and that is fixed now), but I am running into a new issue currently. When I attempt to run the string you provided above, I receive this error:

(https://web.archive.org/web/20190510052248/http://ft.trillian.im/0be4fb4624961881e0cd1243974b5435ec208f43/6vVBcwUY8dGrDFc7t2c53hcSuA67O.jpg)

Hopefully some light can be shed on this :)
Title: Re: Cyber Knight translation
Post by: megatron-uk on 03/03/2015, 03:19 PM
I've done a little more - I have some changes that take the translation up to the end of the first world, but these are not committed to GIt yet. I haven't made as much progress as I would have liked as I'm constantly hitting free-space issues in the script.... there is some free space in most text segments, but it seems random whether they can be used or not (there's no lookup table as such - each string is simply delimited from the next one and the print routines loop over the delimiters until they get to the correct string).

In terms of the error you're getting, I think you must be using Python3 - the 'print' call changed from Python1/Python2, which used the directive:
print "STRING"

to be a proper function called like this in Python3:
print("STRING")

Fortunately the latter format is backwards compatible with Python2 - so I could go back through the code and change it and it would be compatible with all Python versions.
Title: Re: Cyber Knight translation
Post by: CrackTiger on 03/05/2016, 05:18 PM
Has this project progressed any further?
Title: Re: Cyber Knight translation
Post by: megatron-uk on 04/16/2016, 04:58 AM
Sorry folks, real life has gotten in the way. I haven't had the time to touch this for probably a year now - a promotion at work to managing a team (and all the paperwork that comes with!) and more responsibilities at home mean I just don't have the time to devote to it.
Title: Re: Cyber Knight translation
Post by: esteban on 04/16/2016, 12:40 PM
Quote from: megatron-uk on 04/16/2016, 04:58 AMSorry folks, real life has gotten in the way. I haven't had the time to touch this for probably a year now - a promotion at work to managing a team (and all the paperwork that comes with!) and more responsibilities at home mean I just don't have the time to devote to it.
I hear you :(

:)

Thanks for the update.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 10/23/2016, 04:57 AM
Hello, me again :)

So, I've decided to have a look at Cyber Knight again, after a break of over 2 years. Now's the time to do it, as things are probably going to change quite dramatically in my life in the next 6 months or so and I imagine I'll not have any spare time for the next ... ooh, 18 years or more!

So, where I got to with the translation was a point where the English text simply won't fit in to the space available. I had to start overflowing in to areas of the ROM that appear to be blank, although this gave me some extra space for NPC text for the first world, it broke the scrolling intro and I got random rubbish after finishing the first world, meaning I couldn't progress.

I really need the assistance of someone with more PCE programming knowledge who can help by either changing the text routines to point to blocks outside the main code section, or expand the ROM size, and, again move the text their. There just isn't the space available to replace the bulk of the text without overflowing in to the code sections.

Just as a reminder, Cyber Knight uses a fairly crude text block system, which makes it relatively easy to insert: the text is stored in several positions throughout the ROM, interspersed by code and graphics data. The text strings simply run on from each other and are seperated by a given byte sequence, the text display routine just counts over these text blocks until the right number of skipped sequences have been counted, then displays the next sequence. It makes it easy to modify the text (and its uncompressed, which is a big bonus for me). But expanding beyond the availabvle space, or moving it altogether is out of my league.

So, here's a request for help - would anyone be willing to assist in this particular part of the translation, i.e. changing the text routine, and/or expanding the ROM to include enough space to store the english text?

It's a shame for more of us not to get to play it, it really is a great little hucard rpg/strategy game.
Title: Re: Cyber Knight translation
Post by: NightWolve on 10/23/2016, 10:34 AM
Hey megatron, forgot about ya! You were the best status update provider I had seen, well, before elmer and SamIAm came along, that is. ;) Try talking to Bonknuts for help, elmer's too busy with his projects. Too bad you can't do it yourself, anyway, all the best!
Title: Re: Cyber Knight translation
Post by: elmer on 10/23/2016, 10:25 PM
Quote from: megatron-uk on 10/23/2016, 04:57 AMI really need the assistance of someone with more PCE programming knowledge who can help by either changing the text routines to point to blocks outside the main code section, or expand the ROM size, and, again move the text their. There just isn't the space available to replace the bulk of the text without overflowing in to the code sections.

Just as a reminder, Cyber Knight uses a fairly crude text block system, which makes it relatively easy to insert: the text is stored in several positions throughout the ROM, interspersed by code and graphics data. The text strings simply run on from each other and are seperated by a given byte sequence, the text display routine just counts over these text blocks until the right number of skipped sequences have been counted, then displays the next sequence. It makes it easy to modify the text (and its uncompressed, which is a big bonus for me). But expanding beyond the availabvle space, or moving it altogether is out of my league.
OK, a quick look at the game's scripting system shows that the data in the ROM uses a simple overall structure.

The scripts/data is broken up into 16KB blocks that are mapped into the $4000-$7FFF region.

Bank $0A/$0B, ROM offset $14000-$17FFF
Bank $0C/$0D, ROM offset $18000-$1BFFF
Bank $0E/$0F, ROM offset $1C000-$1FFFF
Bank $14/$15, ROM offset $28000-$2BFFF

Each block starts with 1 byte bank number (presumably just for reference)

Then there's a table of 2-byte pointers to each of the individual asset chunks within the block (a chunk is either script or graphics).

Those pointers are always going to be in the range $4000-$7FFF.

There's a table in bank $01 (mapped into $c000-$dfff) that the code uses to locate each of the script chunks, it's split into 2 parts.

The 1st part gives the bank number to map into $4000-$7fff, and the 2nd part gives the low byte of the address of the table of asset chunk pointers to load the actual pointer from to get to one of your "text blocks".

A number of those text blocks are contiguous in memory, so if you're just looking at the raw data in the ROM, then you're probably losing track of what the game thinks of as the actual start-end of the different individual asset blocks.

With this info, you should be able to expand the game's ROM and relocate any of the text blocks into the extra memory and so avoid the problems that you're having.

The 1st table of bank numbers is at $C939 (ROM $02939), and the 2nd table of asset pointer offsets is at $C95A (ROM $0295A).

The contents of those tables are ...

; rom $02939 : script bank

$c939 0e 0e 0e 0e 0e 0e 0c 0e
$c941 0e 00 00 00 00 00 00 00
$c949 14 0a 0a 0a 0a 0a 0c 0c
$c951 0c 0c 0c 0c 0c 0c 0c 0c
$c959 0c

; rom $0295a : script offset

$c95a 01 03 05 07 09 0b 17 0d
$c962 0f 00 00 00 00 00 00 00
$c96a 83 01 03 05 07 09 01 03
$c972 05 07 09 0b 0d 0f 11 13
$c97a 15


Does that make any sense?
Title: Re: Cyber Knight translation
Post by: megatron-uk on 10/24/2016, 05:31 AM
That does indeed explain why some of the expanded text seems to have broken other, unrelated parts of the game.

I think I've got my head around most of what you've explained (and thank you very much for taking the time to do so!). Those 4 text banks you've identified overlap with the text sections I have extracted and am injecting back in, so that makes sense why the bulk of what I'm doing works, but also why it starts to go awry later one.

I have a couple of clarifications though:

- In the 16kb text block 'banks' (0x0a/0x0b, 0x14/0x15 etc), In the table of 2-byte asset pointers immediately following the text bank number, how do I know how many of those such pointers there are before the assets begin?

- In the main bank 0x01, in the second table, what are those asset pointer offsets doing? I can see that when bank 0x14 is loaded for example, the corresponding offset is 0x83. But what does that actually result in?

Many thanks for the information so far, I think this will be an incredible step forward. I was at the point of contemplating going back to fixed-length string insertions, which, while it would have worked, would have resulted in some awful dialogue compared to the original Japanese.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 10/24/2016, 08:16 AM
Okay, I think I can answer the first question myself now, having looked at one of the asset banks in more detail.

Here's the start of 0x0C bank:

ROM position 0x18000-0x1BFFF

0C 19 40 62 44 61 4C 47 51 27 59 16 5C 4B 63 17 6E A2 6E D5 71 DB 75 CE 78 00 .....

So, if I follow your advice, the first byte, 0x0C is the text bank number. Then there is a single byte 0x19 which is 25 in decimal. Count 25 bytes, take off 1 for the 'size' and then that's the number of pointers in the table. That gives the following:

Bank: 0x0C
Size: 0x19
Pointers: 0x4062, 0x4461, 0x4C47, 0x5127, 0x5916, 0x5C4B, 0x6317, 0x6EA2, 0x6ED5, 0x71DB, 0x75CE, 0x7800

That seems to work for the other asset banks, too, with asset bank 0x14 the biggest, with a 147 byte table.

So, (now the gears are turning in my mind...) does that mean that when bank 0x0E is loaded from the first entry in the main bank 0x01, it jumps to pointer 1 (offset 0x01 from the offset table) in the table? Likewise, the last load entry for table 0x0C jumps to position 0x15 in the header table?

Have I got that right?

I guess that once it jumps to pointer position 0x15 (which will point to a block of text at 0x5FFF, say), it will then follow the logic from before, of looping over the script fragments in that block at 0x5FFF, skipping end-of-string bytes until it finds the 93rd one (for example), then displays it?
Title: Re: Cyber Knight translation
Post by: megatron-uk on 10/24/2016, 08:20 AM
To clarify the number of pointers at the start of an asset bank:

Second byte = 0x19 = 25 = 24/2 = 12x 2-byte pointers

I think that's correct.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 10/24/2016, 10:43 AM
Okay, I've written up a summary of the asset banks so far.

==============================================

Master Bank $01, always loaded at 0xC000-0xDFFF

Table A, ROM offset 0x02939, type = Script Text Bank pointers
This is a table of which Text Bank to load at any point.

Start byte position = 10553

Data:
0e 0e 0e 0e 0e 0e 0c 0e
0e 00 00 00 00 00 00 00
14 0a 0a 0a 0a 0a 0c 0c
0c 0c 0c 0c 0c 0c 0c 0c
0c

There are 33 bank load addresses, many of which, as shown, are duplicates.

---

Table B, ROM offset 0x0295a, type = Script Offset pointers.
This is a table of offsets into the pointer tables at the start of each text bank.

Start byte position = 10586

Data:
01 03 05 07 09 0b 17 0d
0f 00 00 00 00 00 00 00
83 01 03 05 07 09 01 03
05 07 09 0b 0d 0f 11 13
15

There are 33 asset chunk pointer offset addresses, one for each of the bank load entries in Table A.

=============================================

The below text banks are always loaded at 0x4000-0x7FFF

=============================================

Text Bank $0A/$0B, ROM offset $14000-$17FFF
First byte = Bank number = 0x0A
Second byte = 0x0B = 11 = 10/2 = 5x 2-byte pointers

Pointer Data
-------------
0A 0B 40 FD 54 6F 6A 29
6C C1 75 00

Pointer Table
----------------
Position / Value
0x00 = 0A = Bank ID
0x01 = 0B = Table size
0x02 = 40 FD - 0x4000 + 0x14000 = 0x140FD
0x03 = 54 6F - 0x4000 + 0x14000 = 0x1546F
0x04 = 6A 29 - 0x4000 + 0x14000 = 0x16A29
0x05 = 6C C1 - 0x4000 + 0x14000 = 0x16CC1
0x06 = 75 00 - 0x4000 + 0x14000 = 0x17500

=============================================

Text Bank $0C/$0D, ROM offset $18000-$1BFFF
First byte = Bank number = 0x0C
Second byte = 0x19 = 25 = 24/2 = 12x 2-byte pointers

Pointer Data
-------------
0C 19 40 62 44 61 4C 47
51 27 59 16 5C 4B 63 17
6E A2 6E D5 71 DB 75 CE
78 00

Pointer Table
----------------
Position / Value
0x00 = 0C = Bank ID
0x01 = 19 = Table size
0x02 = 40 62 - 0x4000 + 0x18000 = 0x18062
0x03 = 44 61 - 0x4000 + 0x18000 = 0x18461
0x04 = 4C 47 - 0x4000 + 0x18000 = 0x18C47
0x05 = 51 27 - 0x4000 + 0x18000 = 0x19127
0x06 = 59 16 - 0x4000 + 0x18000 = 0x19916
0x07 = 5C 4B - 0x4000 + 0x18000 = 0x19C4B
0x08 = 63 17 - 0x4000 + 0x18000 = 0x1A317
0x09 = 6E A2 - 0x4000 + 0x18000 = 0x1AEA2
0x0A = 6E D5 - 0x4000 + 0x18000 = 0x1AED5
0x0B = 71 DB - 0x4000 + 0x18000 = 0x1B1DB
0x0C = 75 CE - 0x4000 + 0x18000 = 0x1B5CE
0x0D = 78 00 - 0x4000 + 0x18000 = 0x1B800

=============================================

Text Bank $0E/$0F, ROM offset $1C000-$1FFFF
First byte = Bank number = 0x0E
Second byte = 0x11 = 17 bytes = 16/2 = 8x 2-byte pointers

Pointer Data
-------------
0E 11 40 4C 4B ED 4E 73
53 B2 55 86 5C FF 6B EC
70 00

Pointer Table
----------------
Position / Value
0x01 = 0E = Bank ID
0x02 = 11 = Table size
0x03 = 40 4C - 0x4000 + 0x1C000 = 0x1C04C
0x04 = 4B ED - 0x4000 + 0x1C000 = 0x1CBED
0x05 = 4E 73 - 0x4000 + 0x1C000 = 0x1CE73
0x06 = 53 B2 - 0x4000 + 0x1C000 = 0x1D3B2
0x07 = 55 86 - 0x4000 + 0x1C000 = 0x1D586
0x08 = 5C FF - 0x4000 + 0x1C000 = 0x1DCFF
0x09 = 6B EC - 0x4000 + 0x1C000 = 0x1EBEC
0x0A = 70 00 - 0x4000 + 0x1C000 = 0x1F000

=============================================

Text Bank $14/$15, ROM offset $28000-$2BFFF
First byte = Bank number = 0x14
Second byte = 0x93 = 147 bytes = 146/2 = 73x 2-byte pointers <- INCORRECT, only 66 pointers in this table.

Pointer Data
-------------
14 93 6D 95 6D A4 6D 36
6E 65 6E B7 6E 95 6D 0E
6F 95 6D 3D 6F 5E 70 95
6D 84 70 95 6D 93 6D 45
71 71 71 02 72 37 72 B2
72 95 6D 0E 73 DF 73 95
6D 20 74 81 74 AA 74 BD
74 D4 74 93 6D 93 6D 42
75 95 6D 74 75 B4 75 95
6D F7 75 49 76 95 6D 6F
76 F4 76 B4 77 CF 77 CF
77 95 6D 93 6D 93 6D 93
6D 93 6D 93 6D 93 6D 93
6D 93 6D 93 6D 93 6D DA
77 95 6D FA 77 6C 78 C1
78 E4 78 0C 79 93 6D 3D
79 15 6A 85 40 00

Pointer Table
----------------
Position / Value
0x01 = 14 = Bank ID
0x02 = 93 = Table size <- INCORRECT

TODO: The table size byte (0x93 == 147) doesn't appear to correlate to the actual number of entries (133 - 1 / 2 = 66) in the table for bank 0x0E, as there are not 73 entries in the table.

One problem I've found is that the second byte == pointer table size assumption doesn't hold true for bank 0x14/0x15 - it should have a 147 byte pointer table == 76 pointers, but it only appears to have 66 pointers if we assume they all have to be within 0x4000-0x7FFF. Something odd happening with that one.
Title: Re: Cyber Knight translation
Post by: elmer on 10/24/2016, 12:11 PM
Quote from: megatron-uk on 10/24/2016, 10:43 AMOkay, I've written up a summary of the asset banks so far.

==============================================

Text Bank $14/$15, ROM offset $28000-$2BFFF
First byte = Bank number = 0x14
Second byte = 0x93 = 147 bytes = 146/2 = 73x 2-byte pointers <- INCORRECT, only 66 pointers in this table.

Pointer Data
-------------
14 93 6D 95 6D A4 6D 36
6E 65 6E B7 6E 95 6D 0E
6F 95 6D 3D 6F 5E 70 95
6D 84 70 95 6D 93 6D 45
71 71 71 02 72 37 72 B2
72 95 6D 0E 73 DF 73 95
6D 20 74 81 74 AA 74 BD
74 D4 74 93 6D 93 6D 42
75 95 6D 74 75 B4 75 95
6D F7 75 49 76 95 6D 6F
76 F4 76 B4 77 CF 77 CF
77 95 6D 93 6D 93 6D 93
6D 93 6D 93 6D 93 6D 93
6D 93 6D 93 6D 93 6D DA
77 95 6D FA 77 6C 78 C1
78 E4 78 0C 79 93 6D 3D
79 15 6A 85 40 00

One problem I've found is that the second byte == pointer table size assumption doesn't hold true for bank 0x14/0x15 - it should have a 147 byte pointer table == 76 pointers, but it only appears to have 66 pointers if we assume they all have to be within 0x4000-0x7FFF. Something odd happening with that one.
You've made a logical-deduction that makes sense from a programmer's point-of-view, but it doesn't actually match reality.  :wink:

You're making the mistake of assuming that the 1st asset chunk in the 16KB block is actually always going to be the 1st entry in the table (I made that same mistake when I was doing the Zeroigar translation).

Since the table is a list of pointers, and the game will just refer to assets by block number & index number, then it's OK for a data block to have things stored out-of-order.

That's the case with bank $14 ... if you look, the last pointer in the table (at $4083) is the pointer to that script chunk in there that starts at $4085.

So the script chunk is located 1st in the 16KB block, but it's asset-index is $83, and all the other graphics chunks have lower asset-index values.

So to actually find out how many assets are in a 16KB block, you've got to read all of the 2-byte pointers, as 2-byte pointers, from the start of the table onwards, keeping track of the lowest value that you see, and then stop reading when you hit the point that the pointer to the table is the same as the lowest asset pointer.

Or you can just do it by-eye since there are so few chunks that you'll need to modify.
Title: Re: Cyber Knight translation
Post by: elmer on 10/24/2016, 12:27 PM
Quote from: megatron-uk on 10/24/2016, 08:16 AMSo, (now the gears are turning in my mind...) does that mean that when bank 0x0E is loaded from the first entry in the main bank 0x01, it jumps to pointer 1 (offset 0x01 from the offset table) in the table? Likewise, the last load entry for table 0x0C jumps to position 0x15 in the header table?

Have I got that right?

I guess that once it jumps to pointer position 0x15 (which will point to a block of text at 0x5FFF, say), it will then follow the logic from before, of looping over the script fragments in that block at 0x5FFF, skipping end-of-string bytes until it finds the 93rd one (for example), then displays it?
It's all just a way for the original game programmers to be able to move individual asset-chunks around inside a sort of pseudo file-system in the ROM so that they can pack each ROM bank as-full-as-possible and just refer to assets as a 2-byte bank+index.

Remember, ROM is (was) expensive, and the challenge of packing things into cartridge space was one of the biggest problems of that era of game programming.

The tables at $02939 & $0295a for the script bank and offset look like they're just for the text blocks.

There could easily be some other tables somewhere for graphics blocks or other assets, or maybe the original developers just used a list of equates/#defines for those in their source code.

The big thing is ... there's now nothing stopping you from creating new 16KB blocks of text assets in an expanded ROM for the translation, and then moving some/all of the English text asset-chunks into the new blocks.

You'd only need to change the tables at $02939 & $0295a to point to your new asset chunks.

You can do the same when it comes to changing any of the graphics that you want to (like the lower-case descenders on that font).
Title: Re: Cyber Knight translation
Post by: megatron-uk on 10/24/2016, 12:43 PM
I think I've got it, but I want to run through the pointer system a few times so that I'm clear. I think I've got where asset index $83 is coming from, and what it relates to, but want to be sure before asking more inane questions  #-o

I'll play around with a few examples this evening first.
Title: Re: Cyber Knight translation
Post by: elmer on 10/24/2016, 01:14 PM
Quote from: megatron-uk on 10/24/2016, 12:43 PMI think I've got it, but I want to run through the pointer system a few times so that I'm clear. I think I've got where asset index $83 is coming from, and what it relates to, but want to be sure before asking more inane questions  #-o
Things become clearer if you just reformat the data so that it's more like what a programmer would write ...

ROM     Idx   Pointer
----------------------
$4000         db  $14
$4001   $01   dw  $6D93
$4003   $03   dw  $6D95
$4005   $05   dw  $6DA4
$4007   $07   dw  $6E36
$4009   $09   dw  $6E65
$400b   $0b   dw  $6EB7
$400d   $0d   dw  $6D95
$400f   $0f   dw  $6F0E
$4011   $11   dw  $6D95
$4013   $13   dw  $6F3D
$4015   $15   dw  $705E
$4017   $17   dw  $6D95
$4019   $19   dw  $7084
$401b   $1b   dw  $6D95
$401d   $1d   dw  $6D93
$401f   $1f   dw  $7145
$4021   $21   dw  $7171
$4023   $23   dw  $7202
$4025   $25   dw  $7237
$4027   $27   dw  $72B2
$4029   $29   dw  $6D95
$402b   $2b   dw  $730E
$402d   $2d   dw  $73DF
$402f   $2f   dw  $6D95
$4031   $31   dw  $7420
$4033   $33   dw  $7481
$4035   $35   dw  $74AA
$4037   $37   dw  $74BD
$4039   $39   dw  $74D4
$403b   $3b   dw  $6D93
$403d   $3d   dw  $6D93
$403f   $3f   dw  $7542
$4041   $41   dw  $6D95
$4043   $43   dw  $7574
$4045   $45   dw  $75B4
$4047   $47   dw  $6D95
$4049   $49   dw  $75F7
$404b   $4b   dw  $7649
$404d   $4d   dw  $6D95
$404f   $4f   dw  $766F
$4051   $51   dw  $76F4
$4053   $53   dw  $77B4
$4055   $55   dw  $77CF
$4057   $57   dw  $77CF
$4059   $59   dw  $6D95
$405b   $5b   dw  $6D93
$405d   $5d   dw  $6D93
$405f   $5f   dw  $6D93
$4061   $61   dw  $6D93
$4063   $63   dw  $6D93
$4065   $65   dw  $6D93
$4067   $67   dw  $6D93
$4069   $69   dw  $6D93
$406b   $6b   dw  $6D93
$406d   $6d   dw  $6D93
$406f   $6f   dw  $77DA
$4071   $71   dw  $6D95
$4073   $73   dw  $77FA
$4075   $75   dw  $786C
$4077   $77   dw  $78C1
$4079   $79   dw  $78E4
$407b   $7b   dw  $790C
$407d   $7d   dw  $6D93
$407f   $7f   dw  $793D
$4081   $81   dw  $6A15
$4083   $83   dw  $4085


In particular, note the multiple table entries that refer to the same asset-chunk at $6D93 & $6D95.

That's some programming-trick going on, possibly for level-loading or something like that where you want to use the same graphics for the main character in the game.
Title: Re: Cyber Knight translation
Post by: elmer on 10/24/2016, 01:41 PM
BTW ... there's no substitute for running the game in Mednafen when it comes to understanding what's going on (if you understand assembly language).

I'd recommend using my custom-built version that's linked in one of the other threads because the new larger fonts make it so much easier to read.

If you run the game in there, then here are the locations of some useful routines (all logical game addresses, and not ROM offsets) ...

$f1e0 - code that maps a 16KB data block into $4000-$7fff.
$f1ce - code that takes a Bank & Asset-Index and maps in the bank and returns a pointer to the asset.

$cd32 - code that takes a script number, and uses the tables at $c939 & $c95a to map in the correct data bank and then load the pointer to the script asset into $30/$31.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 10/24/2016, 02:10 PM
(https://web.archive.org/web/20210728225634im_/https://i.stack.imgur.com/jiFfM.jpg)

I know why I'm confused.

The pointer index table is little endian. I was reading it as big endian.

I couldn't work out where you were getting the pointer $4085 from, since my last pointer was $4000. But of course it's not, and the second byte isn't a counter at all.

SO where I thought the 0x0A table (which is 0A 0B 40 FD 54 6F 6A 29 6C C1 75 00)...

0x0A (asset bank id)
0x0B (table pointer size)
40 FD (pointer)
54 6F (pointer)
... and so on.

it's actually (yes, you can slap me now):
0x0A (asset bank id)
40 0B (pointer)
54 FD (pointer)
... etc.

I'll go sit in the corner now.  :oops:
Title: Re: Cyber Knight translation
Post by: elmer on 10/24/2016, 03:49 PM
Quote from: megatron-uk on 10/24/2016, 02:10 PMI'll go sit in the corner now.  :oops:
Hahaha ... nope, now you'll be too busy working on it all to take time off and sit in the corner!  :wink:

It's a very easy mistake to make, don't worry about it.

Once you've got the translation fixed up, it'll all be about the presentation.

Please, please, please consider customizing the font a bit.  [-o<

Lower-case descenders that appear completely above the baseline just make my eyes bleed!  ](*,)

If you can't afford the extra lines to drop those descenders, then you might want to consider changing the lower-case part of the font into small-caps instead (which also gives you the excuse to put crossbars on the "i" and "l" so that they don't look so thin and out-of-place next to the other fixed-width letters).

Or not ... just an IMHO.  8-[
Title: Re: Cyber Knight translation
Post by: elmer on 10/24/2016, 06:05 PM
Quote from: elmer on 10/24/2016, 03:49 PMOnce you've got the translation fixed up, it'll all be about the presentation.

Please, please, please consider customizing the font a bit.  [-o<

Lower-case descenders that appear completely above the baseline just make my eyes bleed!  ](*,)
OK, a quick look at the game (I'd never seen if before yesterday), shows that all the text is done as 8x8 background tiles, and not dynamically drawn into sprites.

That means no VWF, but, on the positive side, every piece of text, and box, that I've seen is spaced to allow for 16-pixel-high Kanji.

So once the game is 100% translated, and the Japanese font is no longer needed, it should be possible to modify both the font and the display code so that all of the english text is drawn as 2 8x8 tiles ... allowing for real descenders, and a slightly taller font, all without screwing up the overall look.

Since there's also the whole Kanji font bank in there that's permanently loaded as well, that space could potentially be used to make custom bitmaps for some of the game's options, making those look even prettier.

Oh, and just for giggles, I checked, and it looks like there is a permanent "black" in the text palette, so it would be possible to add a drop-shadow to the text, even though it doesn't strictly need one!

Finally ... there's a ton of unused space in the permanent gamecode banks (bank $00 and $01) to actually implement any code hacks.
Title: Re: Cyber Knight translation
Post by: ccovell on 10/24/2016, 07:13 PM
Elmer: programmer by day, typeface fetishist by night.  :)
Title: Re: Cyber Knight translation
Post by: CrackTiger on 10/24/2016, 07:26 PM
Programming isn't the only thing he's a "pro" at. :)
Title: Re: Cyber Knight translation
Post by: esteban on 10/24/2016, 08:53 PM
Quote from: ccovell on 10/24/2016, 07:13 PMElmer: programmer by day, typeface fetishist by night.  :)
Hahahhhahah.

It is so true, and it is so awesome.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 10/25/2016, 11:46 AM
Right, so it looks like all of the assets in banks 0x0A, 0x0C and 0x0E are script. They also coincide with my extracted text regions, so that's great. I can use these new accurate locations to extract the sections in more detail.

Here are the sections so far:

Bank 0x0C
Bank: 0xc
---> Contains 12 asset pointers
---> Region 0x18000 - 0x1bfff
---> Starting asset chunk (0x1) located at: 0x18019
---> Finding next asset sequence
-----> 0x1: 0x18019 - 0x18462 [1097 bytes]
-----> 0x3: 0x18462 - 0x18c61 [2047 bytes]
-----> 0x5: 0x18c61 - 0x19147 [1254 bytes]
-----> 0x7: 0x19147 - 0x19927 [2016 bytes]
-----> 0x9: 0x19927 - 0x19c16 [751 bytes]
-----> 0xb: 0x19c16 - 0x1a34b [1845 bytes]
-----> 0xd: 0x1a34b - 0x1ae17 [2764 bytes]
-----> 0xf: 0x1ae17 - 0x1aea2 [139 bytes]
-----> 0x11: 0x1aea2 - 0x1b1d5 [819 bytes]
-----> 0x13: 0x1b1d5 - 0x1b5d8 [1027 bytes]
-----> 0x15: 0x1b5d8 - 0x1b8ce [758 bytes]
-----> 0x17: 0x1b8ce - 0x1bfff [1841 bytes]

Bank 0x0A
Bank: 0xa
---> Contains 5 asset pointers
---> Region 0x14000 - 0x17fff
---> Starting asset chunk (0x1) located at: 0x1400b
---> Finding next asset sequence
-----> 0x1: 0x1400b - 0x154fd [5362 bytes]
-----> 0x3: 0x154fd - 0x16a6f [5490 bytes]
-----> 0x9: 0x175c1 - 0x17fff [2622 bytes]
-----> 0x5: 0x16a6f - 0x16c29 [442 bytes]
-----> 0x7: 0x16c29 - 0x175c1 [2456 bytes]

Bank 0x0E
Bank: 0xe
---> Contains 8 asset pointers
---> Region 0x1c000 - 0x1ffff
---> Starting asset chunk (0x1) located at: 0x1c011
---> Finding next asset sequence
-----> 0x1: 0x1c011 - 0x1cb4c [2875 bytes]
-----> 0x3: 0x1cb4c - 0x1ceed [929 bytes]
-----> 0x5: 0x1ceed - 0x1d373 [1158 bytes]
-----> 0x7: 0x1d373 - 0x1d5b2 [575 bytes]
-----> 0x9: 0x1d5b2 - 0x1dc86 [1748 bytes]
-----> 0xb: 0x1dc86 - 0x1ebff [3961 bytes]
-----> 0xd: 0x1ebff - 0x1f0ec [1261 bytes]
-----> 0xf: 0x1f0ec - 0x1ffff [3859 bytes]


With bank 0x14 there only appear to be two script assets. Identified by asset index 0x79 and 0x83. Though only 0x83 is referenced by the main asset table in bank 0x01.

0x83: 0x28085 - 0x288e4 [2143 bytes]
0x79: 0x288e4 - 0x2aa15 [8497 bytes]

I've taken the end address of each chunk to be the start address of the next chunk in ram, so these are the maximum sizes of those asset chunks. You can see that 0x79 runs on from 0x83, so one thing I need to find out is how the script in that section is addressed, when asset 0x79 is not directly referenced in the main bank 0x01 offset pointer table, only 0x83 is for bank 0x14.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 10/25/2016, 11:53 AM
So to clarify, every single asset chunk in bank 0x0A, 0x0C and 0x0E is directly accessed by the offset pointers in the bank 0x01 table:

Asset bank load table
Quote0e 0e 0e 0e 0e 0e 0c 0e
0e 00 00 00 00 00 00 00
14 0a 0a 0a 0a 0a 0c 0c
0c 0c 0c 0c 0c 0c 0c 0c
0c
Asset index chunk offset table
Quote01 03 05 07 09 0b 17 0d
0f 00 00 00 00 00 00 00
83 01 03 05 07 09 01 03
05 07 09 0b 0d 0f 11 13
15
Each asset bank has a corresponding index offset in the second table. No asset index is without an entry, apart from asset bank 0x14, which only has one index offset: 0x83.

It looks like from index 0x83 (0x28085) until the end of the chunk 0x79 (0x2aa15) is one contiguous script chunk. This looks to be remarkably similar to my existing extractions, which have grabbed text in that region from 0x28086 up to 0x2a930.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 10/25/2016, 12:24 PM
Argh, ignore my 0x79 asset chunk comments. It's transcription error - I wrote 0x288E4 instead of 0x2B8E4 as the physical ROM address.

That makes things simpler; just one script location (pointed to by 0x83) in that bank now. Albeit with 10 KBytes of text data....
Title: Re: Cyber Knight translation
Post by: megatron-uk on 10/25/2016, 12:30 PM
Since the asset pointed at by 0x83 is the story main script for the game, it should be possible to relocate that one asset to a new bank, to give me another 60% of space to expand it (from ~10KB of Japanese dialogue to 16KB of English to fill the new bank).

That's going to be a massive help.
Title: Re: Cyber Knight translation
Post by: elmer on 10/25/2016, 01:14 PM
Quote from: ccovell on 10/24/2016, 07:13 PMElmer: programmer by day, typeface fetishist by night.  :)
A man's got to have his passions!  :wink:

Do I get a superhero name to go with it?  :-k


Quote from: megatron-uk on 10/25/2016, 12:30 PMSince the asset pointed at by 0x83 is the story main script for the game, it should be possible to relocate that one asset to a new bank, to give me another 60% of space to expand it (from ~10KB of Japanese dialogue to 16KB of English to fill the new bank).

That's going to be a massive help.
You've got it beaten now!  :D

<EDIT>

You don't even need to expand the ROM at all ... I just can't believe how much unused space there is in there!!!  :shock:

You've got a completely free 16KB region in the ROM at offset $50000-$53fff (bank $28).

You've also got over 8KB free in the 16KB asset-block at bank $08 where you could rewrite the block to add some new asset chunks into it.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 10/25/2016, 04:24 PM
Just made a new commit to github with a Python based extractor tool (extractAssets.py) that outputs one data file per asset chunk. It saves to ./assets/raw by default.

Files are named as ASSETBANK.ASSETCHUNK.dat.

e.g.

0x14.0x83.dat

Now need to write something to load each data file, split the script into strings and then pass back through my translation table that I used earlier. Hopefully have something working for that tomorrow.

There are 26 script asset chunks, allocating a new 16KB bank for each would increase the size of the ROM by 420KB, which, although wasteful, would mean each block of text would have more than enough space to be fully expanded, without worrying whether each of the assets would fit back in to a single bank.

The main story script asset chunk will probably need the most space for expansion, but I had to make savings here and there in a lot of other places, so this would help with those sections, too.
Title: Re: Cyber Knight translation
Post by: ccovell on 10/25/2016, 06:46 PM
Quote from: elmer on 10/25/2016, 01:14 PM
Quote from: ccovell on 10/24/2016, 07:13 PMElmer: programmer by day, typeface fetishist by night.  :)
A man's got to have his passions!  :wink:

Do I get a superhero name to go with it?  :-k
"The Masked Descender!"
Title: Re: Cyber Knight translation
Post by: elmer on 10/26/2016, 10:57 AM
Quote from: ccovell on 10/25/2016, 06:46 PM"The Masked Descender!"
Hahaha ... I like it!  :lol:

I'll have to think up a Halloween costume to go with it.  :-k
Title: Re: Cyber Knight translation
Post by: NecroPhile on 10/26/2016, 11:31 AM
"Wait... that's not a serif.  Dammit, elmer, zip up!"
Title: Re: Cyber Knight translation
Post by: ccovell on 10/26/2016, 07:15 PM
Quote from: guest on 10/26/2016, 11:31 AM"Wait... that's not a serif.  Dammit, elmer, zip up!"
And you would be his sidekick, "Iota".
Title: Cyber Knight translation
Post by: esteban on 10/26/2016, 07:42 PM
Quote from: ccovell on 10/26/2016, 07:15 PM
Quote from: guest on 10/26/2016, 11:31 AM"Wait... that's not a serif.  Dammit, elmer, zip up!"
And you would be his sidekick, "Iota".
And your nemesis, Signor SanSerif, would go by the nickname "Eunuch".
Title: Re: Cyber Knight translation
Post by: elmer on 10/26/2016, 09:18 PM
Quote from: guest on 10/26/2016, 11:31 AM"Wait... that's not a serif.  Dammit, elmer, zip up!"
You mean that I couldn't walk around and say "Don't you want to touch my dangling serif?"  #-o

But it would still be better to go, in costume, as the intrepid Financial Times reporters (with name badges), Harry Chad, and his wife Penny. After Harry's investigation of the Clinton Foundation, they're now just know to the police as "Hanging Chad" and "Pregnant Chad".

Ba...da...boooom!  :wink: ...  :roll:
Title: Re: Cyber Knight translation
Post by: NightWolve on 10/26/2016, 09:58 PM
Wow, didn't expect all that help by elmer! Looks like megatron is on his way!
Title: Re: Cyber Knight translation
Post by: spenoza on 10/27/2016, 12:39 PM
Quote from: megatron-uk on 10/25/2016, 04:24 PMFiles are named as ASSETBANK.ASSETCHUNK.dat.
Don't be like me and misread that as ASSBANK.ASSCHUNK

Unless you like to giggle like a child.

Totally understand the reason you'll be out of the game for 18 years or so. I just killed my free time for the same reason this month.
Title: Re: Cyber Knight translation
Post by: johnnykonami on 10/27/2016, 02:50 PM
Quote from: guest on 10/27/2016, 12:39 PMDon't be like me and misread that as ASSBANK.ASSCHUNK

Unless you like to giggle like a child.

Totally understand the reason you'll be out of the game for 18 years or so. I just killed my free time for the same reason this month.
I'm learning some programming now, and I'm officially moving learning about using ASSBANK.ASSCHUNK to the top of my list.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 10/27/2016, 05:15 PM
QuoteTotally understand the reason you'll be out of the game for 18 years or so. I just killed my free time for the same reason this month.
Commiserations  :wink:
Title: Re: Cyber Knight translation
Post by: megatron-uk on 10/28/2016, 12:27 PM
New commits to Github (https://github.com/megatron-uk/cyberknight-pce) - replaced the old script extractor with extractAssets.py and a translator/splitter named splitAssets.py. If you want to give it a go, run the former, then the latter, just make sure you have your "Cyber Knight (J).pce" rom in the same directory.

This should result in a directory of JSON files (under assets/raw and assets/split), split by asset bank and chunk (no, I refuse to stoop to calling them Assbank and Asschunk you bunch of lowlifes!) with translations using my old lookup tables. I've also been able to embed a few more special byte sequences; some missing NPC names and the like.

It's a much cleaner data set to work with now.

Next step is to run through the SNES translation and re-insert the strings again where they match.

Then write a patcher to expand the ROM, update the main bank asset map with the new ROM bank numbers to suit.

Perhaps a week until I can inject the output in to the ROM again and do a playtest? Hopefully, at that point we'll find those bits that were broken are no longer broken!
Title: Re: Cyber Knight translation
Post by: elmer on 10/28/2016, 06:52 PM
Quote from: megatron-uk on 10/28/2016, 12:27 PMPerhaps a week until I can inject the output in to the ROM again and do a playtest? Hopefully, at that point we'll find those bits that were broken are no longer broken!
Great news! I hope it all works properly now.  :D

You're doing excellent work getting everything refactored so quickly.

FYI ... It looks like it's possible to move the English font up by a pixel so that the lower-case descenders actually drop below the baseline (even if only by 1 pixel).  :-"
Title: Re: Cyber Knight translation
Post by: megatron-uk on 10/29/2016, 04:56 AM
So, as part of cleaning things up and trying to get to 100% on working out the remaining control bytes I've decided to try and work out the dialogue window control codes.

0x1a is the control code to open a dialogue window, it is then followed by a single byte that controls the size and placement of the window.

Recording the location of the dialogue windows and altering the second byte I get the following:

(http://target-earth.dyndns.org/www/misc/Cyber%20Knight%20(E)-0000.png)

(http://target-earth.dyndns.org/www/misc/Cyber%20Knight%20(E)-0002.png)

(http://target-earth.dyndns.org/www/misc/Cyber%20Knight%20(E)-0004.png)

(http://target-earth.dyndns.org/www/misc/Cyber%20Knight%20(E)-0007.png)

(http://target-earth.dyndns.org/www/misc/Cyber%20Knight%20(E)-0008.png)

(http://target-earth.dyndns.org/www/misc/Cyber%20Knight%20(E)-0009.png)

(http://target-earth.dyndns.org/www/misc/Cyber%20Knight%20(E)-0011.png)

(http://target-earth.dyndns.org/www/misc/Cyber%20Knight%20(E)-0012.png)

The control codes that start with 0x5 seem to be 2-line windows, and those that start with 0x7 are 1-line. I can't figure out how the length of the window is set, as the control code 0x74 is a tiny, 1-line box used for the jumpdrive countdown timer in the intro cinema, so it's not as if the lower 4 bits of this byte are width.

I can't work out how the second byte can encode the window position and size - I'm leaning more towards the fact that there may be a list of window position and sizes stored somewhere in the ROM and this byte is perhaps a lookup code.

If we want to control the size and location of the dialogue boxes for the expanded English translation, then I need to get a much more extensive lookup table of the boxes that these control codes produce. The intro and initial world get me some of them, but there will be lots more.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 10/29/2016, 04:59 AM
Looks like the images are not coming through. Must be a problem with my router.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 10/29/2016, 09:04 AM
Okay, no height/width link to the value of the second byte, that I can determine. I'm fairly sure that the value of this second byte is a lookup to a table of pre-set dialogue box dimensions.

I've been through about 50 or 60 variations of the second byte and although there is some similarity between some values and their nearest neighbours, ultimately it just seems to be a series of variations.

I'll upload a screen grab of all the working second byte values I've found so far a little later today. At least that will give me a manual look-up of whether I want to change the dialogue box for a given script segment or not.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 10/29/2016, 09:25 AM
Here are all the 0x1a 0x?? dialogue box control codes that I've been able to find in the game.

(http://target-earth.dyndns.org/www/misc/CyberKnight_DialogueBoxes.jpg)

I've tried variations on the second byte for codes that don't appear in the game, but I get weird results, so it would appear that this is the limit of what is used in the normal dialogue code. There are quite a few other instances that this doesn't cover:

- In-and-out of battle menus
- Planet names
- Discussions with crew

The above don't appear to use the same display code; the script for much of the above doesn't begin with the same control codes as elsewhere in the game. Much of this isn't a problem, but there are a few areas (ship locations, dialogue snippets with the crew) that would benefit from alternative text box location and/or sizes.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 10/29/2016, 10:50 AM
Small commit to github. The splitAssets.py script now embeds dialogue box information into the extracted Japanese text.

Each of the text strings in a given asset chunk now looks something like this:

{
      "string_number" : 2,
      "string_size" : 59,
      "start_pos" : 0x28086,
      "bytes" : ["1a", "4f", "10", "30", "a2", "cb", "ae", "b3", "d2", "dd", "b5", "dd", "c4", "de", "20", "31", "30", "30", "30", "20", "df", "43", "20", "b2", "bc", "de", "ae", "b3", "a1", "03", "d6", "b3", "b6", "de", "dd", "c6", "20", "b5", "b5", "dc", "da", "c0", "20", "c1", "b2", "bb", "c5", "20", "dc", "b8", "be", "b2", "03", "c3", "de", "bd", "a1", "a3", "08"],
      "PCE_japanese" : "<newbox><24x3@bottom_centred><NPC_Mica>『ひょうめんおんど 1000゜C いじょう。\nようがんに おおわれた ちいさな わくせい\nです。』<wait>",
      "SNES_english" : "",
      "SNES_accuracy" : 0.0,
      "PCE_english" : "",
      "notes" : "",
},
Title: Re: Cyber Knight translation
Post by: elmer on 10/29/2016, 12:46 PM
Quote from: megatron-uk on 10/29/2016, 09:04 AMOkay, no height/width link to the value of the second byte, that I can determine. I'm fairly sure that the value of this second byte is a lookup to a table of pre-set dialogue box dimensions.

I've been through about 50 or 60 variations of the second byte and although there is some similarity between some values and their nearest neighbours, ultimately it just seems to be a series of variations.
Quote from: megatron-uk on 10/29/2016, 09:25 AMHere are all the 0x1a 0x?? dialogue box control codes that I've been able to find in the game.
Excellent work!  :D

It's great to see your posts and follow your investigations.

You and I approach the investigation process in different ways, and I find it fascinating to see how you're so successfully coming up with all this critically-important information.

My approach is more of "dive into the code and see what it's doing", which takes a lot longer to get any results, and can easily lead to lot of unnecessary disassembly.



Quote from: megatron-uk on 10/29/2016, 10:50 AMSmall commit to github. The splitAssets.py script now embeds dialogue box information into the extracted Japanese text.
I absolutely love your JSON-format extraction with all of its extra meta-data.

It presents the information in a way that's easy to read and modify (especially by automated tools).
Title: Re: Cyber Knight translation
Post by: megatron-uk on 10/29/2016, 01:31 PM
Cheers!

I've modified the old mapScript.py code to work on the new JSON. I've run it through a few asset files and it's working the same as before, so it looks like I can get pretty much all of the automatic translations done again in no time.

The time sink will be the manual patches I did by hand in the old files.

I do prefer JSON to XML, it's much less noisy and XML always reminds me of LISP; matching opening and closing braces. I hated that at University.

JSON is a much lighter transfer format and it maps neatly to Python/Ruby/Javascript dictionaries and lists. Normally I wouldn't manually write JSON like I am doing now with file.write(), but because we've got a mix of ASCII and various Unicode fonts (shift-JIS, UTF-8, etc) all in the same file, it's a bit of pain to serialize it all just using a single json.dumps() call. Do as I say, not as I do, folks  :lol:
Title: Re: Cyber Knight translation
Post by: megatron-uk on 10/31/2016, 01:22 PM
Update from a couple of passes through the mapAssets.py script, using only the first couple of 'quality' levels of the fuzzy matching against the SNES script. Each 'quality level' represents a pass through the code where I use a different level at which I consider a script to be matched. I start of at level 1 with only strings that are >80% as possible matches, then each level after that drops by 5-10%. Once you get to level 6 or so you really need to start comparing by eyeball.

There's a lot more automatic matches still to get:

mapAssets.py - Map untranslated patches from the PC-Engine CyberKnight to the SNES script
----------------

Configuration
=============
Verbose: False
Over-write: False
Pass Type: 1
SNES Script File: CyberKnightSNES.csv <- OK
Input Directory: assets/split <- OK
Output Directory: assets/converted <- OK
Patches Found: 26 <- OK

Patch Summary
=============
|--------|------------|------------|----------|----------|---------|--------|---------------
| Strings|Translations|SNES matches|SNES best |SNES worst|SNES avg.| Tiny   | Patch Name
|--------|------------|------------|----------|----------|---------|--------|---------------
|  198   |   7 / 198  |   48 / 198 |      100 |        0 |      56 |   114  | 0x14.0x83.dat
|  136   |   0 / 136  |   39 / 136 |       93 |        0 |      52 |    80  | 0xa.0x1.dat
|  131   |   0 / 131  |   41 / 131 |      100 |        0 |      56 |    69  | 0xa.0x3.dat
|   21   |   0 /  21  |    7 /  21 |      100 |        0 |      68 |    13  | 0xa.0x5.dat
|   42   |   0 /  42  |   12 /  42 |       92 |        0 |      62 |    22  | 0xa.0x7.dat
|   82   |   0 /  82  |   31 /  82 |      100 |        0 |      74 |    45  | 0xa.0x9.dat
|   24   |   0 /  24  |    2 /  24 |        0 |        0 |       0 |    13  | 0xc.0x1.dat
|   25   |   0 /  25  |    9 /  25 |       92 |        0 |      59 |    13  | 0xc.0x11.dat
|    9   |   0 /   9  |    3 /   9 |       90 |        0 |      57 |     6  | 0xc.0x13.dat
|   29   |  22 /  29  |   12 /  29 |       89 |        0 |      70 |    16  | 0xc.0x15.dat
|  105   | 105 / 105  |    1 / 105 |        0 |        0 |       0 |    65  | 0xc.0x17.dat
|   74   |   0 /  74  |   33 /  74 |      100 |        0 |      73 |    38  | 0xc.0x3.dat
|   50   |   0 /  50  |   20 /  50 |      100 |        0 |      82 |    27  | 0xc.0x5.dat
|   53   |   0 /  53  |   22 /  53 |       90 |        0 |      61 |    27  | 0xc.0x7.dat
|   42   |   0 /  42  |   14 /  42 |      100 |        0 |      63 |    25  | 0xc.0x9.dat
|   29   |   0 /  29  |   10 /  29 |       90 |        0 |      17 |    16  | 0xc.0xb.dat
|   93   |   0 /  93  |   37 /  93 |       92 |        0 |      46 |    47  | 0xc.0xd.dat
|    5   |   3 /   5  |    2 /   5 |       92 |       72 |      82 |     3  | 0xc.0xf.dat
|  277   |   0 / 277  |   17 / 277 |      100 |        0 |      62 |   151  | 0xe.0x1.dat
|  287   |   0 / 287  |   66 / 287 |      100 |        0 |      49 |   185  | 0xe.0x3.dat
|  161   |   0 / 161  |   20 / 161 |      100 |        0 |      69 |    93  | 0xe.0x5.dat
|  149   | 148 / 149  |   56 / 149 |       92 |        0 |      48 |    89  | 0xe.0x7.dat
|  260   |   0 / 260  |   77 / 260 |      100 |        0 |      71 |   153  | 0xe.0x9.dat
|  385   |   0 / 385  |   77 / 385 |      100 |        0 |      68 |   227  | 0xe.0xb.dat
|  221   |   0 / 221  |    0 / 221 |        0 |      100 |       0 |   152  | 0xe.0xd.dat
|  228   |   0 / 228  |   75 / 228 |      100 |        0 |      81 |   134  | 0xe.0xf.dat
|--------|------------|------------|----------|----------|---------|--------|---------------
| Strings|Translations|SNES matches|SNES best |SNES worst|SNES avg.| Tiny   | Patch Name
|--------|------------|------------|----------|----------|---------|--------|---------------
| 3116   | 285 / 3116 | 731 / 3116 |      100 |        0 |      14 |   1823  |

Patch Summary Key
=================
Strings      : Total number of text strings in the patch file
Translations : How many strings already have we added a full english translation for?
SNES Matches : How many strings have matching SNES english text that canbe used as a basis for an english translation?
SNES Best    : The most accurate SNES match in this patch file.
SNES Worst   : The least accurate SNES match in this patch file.
SNES average : The average accuracy of SNES matches in this patch file.
Tiny         : How many strings are sub-2 characters (ie not text)?

The key figures are the SNES matches and Translations columns. The former is the number of automatic matches my code made against the SNES script, by comparing the decoded PCE byte sequences, and the latter is the number of by-hand entries I've made, or cut-and-paste from the SNES matches. I don't automatically use the SNES script matches, as there's a lot of improvements that can be made by eye (available length of text boxes, some grammar, as well as some words/phrases that I think can be improved).

On that note I'm thinking about having all of the items, monsters and names parameterised,  liked I'm using <newbox> or <wait> to represent different byte sequences. I'm using <NPC_name> at present, to ease name replacement if I desire, but I could extend it to items and other things. One thing that I want to replace for a more 'science fiction' feel is the use of the phrase 'monopole coil' (which is an in-game widget that you need to find to fix your ship FTL drive). I think a far more appropriate phrase would be a 'singularity' coil or cell. I think there is quite a bit of that kind of stuff in the game though  :wink:
Title: Re: Cyber Knight translation
Post by: megatron-uk on 11/03/2016, 06:10 PM
Found a couple more text control codes:

0x19 0x71 - Move cursor to the box that was active 2 windows ago
0x19 0x72 - Move cursor to the box that was active 3 windows ago
0x19 0x73 - Move cursor to the box that was active before this one

These take effect during the intro cinema and there are matching control codes used throughout in-game dialogue (both prefixed with 0x19, but the second byte differs). It doesn't appear the second byte can be swapped between the values used in-game and the cinema section (random text appears instead).

It explains how the active text window changes, especially when multiple characters have open dialogue on screen at any one point. Also gives me a bit more flexibility when 'enhancing' the more in-depth segments.

I've updated the translation table to include these and will run the script through it again.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 11/04/2016, 09:16 PM
More story and in-game text done. Some bits I've expanded substantially; reworking simple one-line , one-way dialogue into more developed conversations between the player party and NPC's. Nothing earth shattering. For example, later in the game you run into the Garaians, and advanced but quite staid and pacifist race. They are also extremely boring and dislike violence.

One of the conversations originally went something like this:

NPC: "X warring race is awful."
NPC: "So are the killer machines."
NPC: "In fact, so is anyone with weapons."

I added in a dialogue box from one of the players party after the second line to say:

Player Party: "That sounds like they're talking about the <bleep> there..."

Just to give you an indication that these alien dudes are talking about the same nasties that you start to encounter yourself :)

What I would like to do, and haven't found a way to do it yet, is any control codes that adjust font colour. It would be nice to have 'inner thought' type messages like that in a different colour. Same goes for 'You acquired the widget!' in game messages. Just to differentiate against spoken dialogue.

Here's the stats so far:

mapAssets.py - Map untranslated patches from the PC-Engine CyberKnight to the SNES script
----------------

Configuration
=============
Verbose: False
Over-write: False
Pass Type: 1
SNES Script File: CyberKnightSNES.csv <- OK
Input Directory: assets/split <- OK
Output Directory: assets/converted <- OK
Patches Found: 26 <- OK

Patch Summary
=============
|--------|------------|------------|----------|----------|---------|--------|---------------
| Strings|Translations|SNES matches|SNES best |SNES worst|SNES avg.| Tiny   | Patch Name
|--------|------------|------------|----------|----------|---------|--------|---------------
|  198   |   7 / 198  |   48 / 198 |      100 |        0 |      56 |   114  | 0x14.0x83.dat
|  136   |   0 / 136  |   39 / 136 |       93 |        0 |      52 |    80  | 0xa.0x1.dat
|  131   |   0 / 131  |   41 / 131 |      100 |        0 |      56 |    69  | 0xa.0x3.dat
|   21   |  21 /  21  |    7 /  21 |      100 |        0 |      53 |    13  | 0xa.0x5.dat COMPLETE
|   42   |   0 /  42  |   12 /  42 |       92 |        0 |      62 |    22  | 0xa.0x7.dat
|   82   |   0 /  82  |   31 /  82 |      100 |        0 |      74 |    45  | 0xa.0x9.dat
|   24   |  15 /  24  |    2 /  24 |        0 |        0 |       0 |    13  | 0xc.0x1.dat
|   25   |  25 /  25  |    9 /  25 |       92 |        0 |      59 |    13  | 0xc.0x11.dat COMPLETE
|    9   |   9 /   9  |    3 /   9 |       90 |        0 |      57 |     6  | 0xc.0x13.dat COMPLETE
|   29   |  29 /  29  |   12 /  29 |       89 |        0 |      70 |    16  | 0xc.0x15.dat COMPLETE
|  105   | 105 / 105  |    1 / 105 |        0 |        0 |       0 |    65  | 0xc.0x17.dat COMPLETE
|   74   |   0 /  74  |   33 /  74 |      100 |        0 |      73 |    38  | 0xc.0x3.dat
|   50   |   0 /  50  |   20 /  50 |      100 |        0 |      82 |    27  | 0xc.0x5.dat
|   53   |   0 /  53  |   22 /  53 |       90 |        0 |      61 |    27  | 0xc.0x7.dat
|   42   |   0 /  42  |   14 /  42 |      100 |        0 |      63 |    25  | 0xc.0x9.dat
|   29   |  28 /  29  |   11 /  29 |       90 |        0 |      15 |    16  | 0xc.0xb.dat
|   93   |   0 /  93  |   37 /  93 |       92 |        0 |      46 |    47  | 0xc.0xd.dat
|    5   |   5 /   5  |    2 /   5 |       92 |       72 |      82 |     3  | 0xc.0xf.dat COMPLETE
|  274   | 149 / 274  |   13 / 274 |       93 |        0 |      51 |   149  | 0xe.0x1.dat
|  287   |   0 / 287  |   66 / 287 |      100 |        0 |      49 |   185  | 0xe.0x3.dat
|  161   |   0 / 161  |   20 / 161 |      100 |        0 |      69 |    93  | 0xe.0x5.dat
|  149   | 148 / 149  |   56 / 149 |       92 |        0 |      48 |    89  | 0xe.0x7.dat
|  260   |   0 / 260  |   77 / 260 |      100 |        0 |      71 |   153  | 0xe.0x9.dat
|  385   |   0 / 385  |   77 / 385 |      100 |        0 |      68 |   227  | 0xe.0xb.dat
|  221   |   0 / 221  |    0 / 221 |        0 |      100 |       0 |   152  | 0xe.0xd.dat
|  228   |   0 / 228  |   75 / 228 |      100 |        0 |      81 |   134  | 0xe.0xf.dat
|--------|------------|------------|----------|----------|---------|--------|---------------
| Strings|Translations|SNES matches|SNES best |SNES worst|SNES avg.| Tiny   | Patch Name
|--------|------------|------------|----------|----------|---------|--------|---------------
| 3113   | 541 / 3113 | 728 / 3113 |      100 |        0 |      14 |   1821  |


Also found another control code: 0x07 - sets the printing speed of the text string that follows. It's controlled by the immediately following byte. Experimentation shows that setting 0x07 0x01 is very slow, about 1 character every 2 seconds, whereas 0x07 0x2F is very fast, perhaps 10 characters a second.

The expanded text isn't getting inserted just yet. I'd like to try and track down as much of the control codes as possible before then.
Title: Re: Cyber Knight translation
Post by: elmer on 11/04/2016, 11:07 PM
Quote from: megatron-uk on 11/03/2016, 06:10 PMFound a couple more text control codes:

0x19 0x71 - Move cursor to the box that was active 2 windows ago
0x19 0x72 - Move cursor to the box that was active 3 windows ago
0x19 0x73 - Move cursor to the box that was active before this one
Excellent detective work!  :D

You're finding and understanding things through alteration/experimentation that I'd have found by looking at the source-code itself.

The method isn't important ... for something like this, it's all about the result, and your persistence is paying off!

Congratulations!  :wink:


Quote from: megatron-uk on 11/04/2016, 09:16 PMI added in a dialogue box from one of the players party after the second line to say:

Player Party: "That sounds like they're talking about the <bleep> there..."
And this is why I'm taking some of my time to hopefully provide a little help.

You care enough about the game that you want it to sound good to an English-speaking audience, and you're not just satisfied with making it understandable.

Some passions deserve to be supported, even if they're not initially understood or shared.

That's how I felt when I saw SamIAm's work on Zeroigar, and you can see where his passion and enthusiasm lead in our continuing work on Xanadu 1 and Xanadu 2.

At the end of the day, it all (and IMHO always) relies upon someone who has a belief and passion in something informing/educating everyone else into seeing why they love that thing (game/sport/food/etc), and becoming "believers" themselves.

SamIAm has managed to get me to "believe" in a couple of games now that I didn't even know existed a few years ago.

And after the introduction, and spending months working on them, I understand why he loves those games, and I've come to share that love, and to trust his opinion.

What megatron-uk is doing here with Cyber Knight seems (to me) to show that same kind of passion.

He's doing (as esteban would say) "God's Work!".


Speaking of which ...

... here's the data table that Cyber Knight uses to load up the ASCII font into VRAM.

They're grouping a number of individual graphical "assets" into different VRAM "layout" arrangements that get set up at different times.

The ASCII font itself is compressed in bank $16, asset $01 ... but as you can see, the reference to it will need to be modified in a lot of places if that font is going to be changed.

ASSET BANK $10 ASSET CHUNK $0d

                .org  $4875

                .dw  group_4807 ; idx $00 - $4785 (1st font data loaded @ $1000-$1fff)
                .dw  group_4843 ; idx $01 - $4787
                .dw  group_487f ; idx $02 - $4789
                .dw  group_4893 ; idx $03 - $478b
                .dw  group_4893 ; idx $04 - $478d
                .dw  group_48d9 ; idx $05 - $478f
                .dw  group_48ed ; idx $06 - $4791
                .dw  group_4901 ; idx $07 - $4793
                .dw  group_4915 ; idx $08 - $4795
                .dw  group_4929 ; idx $09 - $4797
                .dw  group_493d ; idx $0a - $4799
                .dw  group_4951 ; idx $0b - $479b
                .dw  group_495b ; idx $0c - $479d
                .dw  group_496f ; idx $0d - $479f
                .dw  group_49c9 ; idx $0e - $47a1
                .dw  group_49d3 ; idx $0f - $47a3
                .dw  group_49dd ; idx $10 - $47a5
                .dw  group_49dd ; idx $11 - $47a7
                .dw  group_4a41 ; idx $12 - $47a9
                .dw  group_4aa5 ; idx $13 - $47ab
                .dw  group_4b13 ; idx $14 - $47ad
                .dw  group_4b77 ; idx $15 - $47af
                .dw  group_4bef ; idx $16 - $47b1
                .dw  group_4c67 ; idx $17 - $47b3
                .dw  group_4c67 ; idx $18 - $47b5
                .dw  group_4cdf ; idx $19 - $47b7
                .dw  group_4cfd ; idx $1a - $47b9
                .dw  group_4d1b ; idx $1b - $47bb
                .dw  group_4d43 ; idx $1c - $47bd
                .dw  group_4da7 ; idx $1d - $47bf
                .dw  group_4dc5 ; idx $1e - $47c1
                .dw  group_4ded ; idx $1f - $47c3
                .dw  group_4ded ; idx $20 - $47c5
                .dw  group_4ded ; idx $21 - $47c7
                .dw  group_4ded ; idx $22 - $47c9
                .dw  group_4ded ; idx $23 - $47cb
                .dw  group_4ded ; idx $24 - $47cd
                .dw  group_4ded ; idx $25 - $47cf
                .dw  group_4ded ; idx $26 - $47d1
                .dw  group_4ded ; idx $27 - $47d3
                .dw  group_4ded ; idx $28 - $47d5
                .dw  group_4ded ; idx $29 - $47d7
                .dw  group_4ded ; idx $2a - $47d9
                .dw  group_4ded ; idx $2b - $47db
                .dw  group_4ded ; idx $2c - $47dd
                .dw  group_4ded ; idx $2d - $47df
                .dw  group_4ded ; idx $2e - $47e1
                .dw  group_4ded ; idx $2f - $47e3
                .dw  group_4ded ; idx $30 - $47e5
                .dw  group_4df7 ; idx $31 - $47e7
                .dw  group_4e01 ; idx $32 - $47e9
                .dw  group_4e1f ; idx $33 - $47eb
                .dw  group_4e29 ; idx $34 - $47ed
                .dw  group_4e47 ; idx $35 - $47ef
                .dw  group_4e51 ; idx $36 - $47f1
                .dw  group_4e5b ; idx $37 - $47f3
                .dw  group_4e5b ; idx $38 - $47f5
                .dw  group_4e5b ; idx $39 - $47f7
                .dw  group_4e97 ; idx $3a - $47f9
                .dw  group_4edd ; idx $3b - $47fb
                .dw  group_4f0f ; idx $3c - $47fd
                .dw  group_4f41 ; idx $3d - $47ff
                .dw  group_4f73 ; idx $3e - $4801
                .dw  group_4faf ; idx $3f - $4803
                .dw  group_4faf ; idx $40 - $4805

        ; idx $00 : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_4807:     .db $00,$11,  $6b,  $0c,  $80,$00,$80,$80,  $16,$01
group_4811:     .db $00,$18,  $80,  $0c,  $80,$00,$80,$80,  $16,$03
group_481b:     .db $b0,$17,  $04,  $24,  $00,$08,$10,$18,  $16,$05
group_4825:     .db $00,$11,  $01,  $08,  $c0,$80,$80,$80,  $16,$01
group_482f:     .db $00,$10,  $08,  $20,  $00,$08,$10,$18,  $16,$0d
group_4839:     .db $00,$20,  $00,  $0c,  $80,$00,$80,$80,  $16,$07

        ; idx $01 : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_4843:     .db $00,$11,  $6b,  $0c,  $40,$00,$40,$40,  $16,$01
group_484d:     .db $00,$18,  $80,  $0c,  $40,$00,$40,$40,  $16,$03
group_4857:     .db $b0,$17,  $04,  $24,  $00,$08,$10,$18,  $16,$05
group_4861:     .db $00,$11,  $01,  $08,  $c0,$80,$80,$80,  $16,$01
group_486b:     .db $00,$10,  $08,  $20,  $00,$08,$10,$18,  $16,$0d
group_4875:     .db $00,$20,  $00,  $0c,  $40,$00,$40,$40,  $16,$07

        ; idx $02 : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_487f:     .db $00,$30,  $0c,  $1c,  $00,$08,$10,$80,  $1c,$11
group_4889:     .db $c0,$30,  $a7,  $1c,  $00,$08,$10,$c0,  $1c,$13

        ; idx $03 : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_4893:     .db $00,$11,  $6b,  $0c,  $40,$00,$40,$40,  $16,$01
group_489d:     .db $00,$18,  $80,  $0c,  $40,$00,$40,$40,  $16,$03
group_48a7:     .db $b0,$17,  $04,  $24,  $00,$08,$10,$18,  $16,$05
group_48b1:     .db $00,$11,  $01,  $08,  $c0,$80,$80,$80,  $16,$01
group_48bb:     .db $00,$10,  $07,  $20,  $00,$08,$10,$18,  $16,$0d
group_48c5:     .db $00,$21,  $20,  $0c,  $80,$00,$80,$80,  $16,$01
group_48cf:     .db $00,$21,  $01,  $08,  $c0,$80,$80,$80,  $16,$01

        ; idx $05 : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_48d9:     .db $80,$24,  $08,  $0c,  $40,$00,$40,$40,  $16,$09
group_48e3:     .db $80,$26,  $08,  $0c,  $40,$00,$40,$40,  $16,$0b

        ; idx $06 : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_48ed:     .db $00,$70,  $0d,  $24,  $00,$08,$10,$18,  $36,$05
group_48f7:     .db $d0,$70,  $30,  $1c,  $00,$08,$10,$80,  $36,$0f

        ; idx $07 : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_4901:     .db $00,$70,  $0a,  $24,  $00,$08,$10,$18,  $36,$07
group_490b:     .db $a0,$70,  $2a,  $1c,  $00,$08,$10,$80,  $36,$11

        ; idx $08 : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_4915:     .db $00,$70,  $0b,  $24,  $00,$08,$10,$18,  $36,$09
group_491f:     .db $b0,$70,  $3c,  $1c,  $00,$08,$10,$80,  $36,$13

        ; idx $09 : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_4929:     .db $00,$70,  $03,  $24,  $00,$08,$10,$18,  $36,$0b
group_4933:     .db $30,$70,  $46,  $1c,  $00,$08,$10,$80,  $36,$15

        ; idx $0a : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_493d:     .db $00,$70,  $0b,  $24,  $00,$08,$10,$18,  $36,$0d
group_4947:     .db $b0,$70,  $3f,  $1c,  $00,$08,$10,$80,  $36,$17

        ; idx $0b : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_4951:     .db $00,$20,  $64,  $0c,  $80,$00,$80,$80,  $24,$13

        ; idx $0c : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_495b:     .db $00,$20,  $98,  $0c,  $80,$80,$80,$80,  $24,$15
group_4965:     .db $00,$30,  $98,  $0c,  $80,$00,$80,$80,  $24,$15

        ; idx $0d : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_496f:     .db $00,$10,  $01,  $20,  $00,$08,$10,$18,  $16,$0d
group_4979:     .db $00,$11,  $6b,  $0c,  $00,$80,$80,$80,  $16,$01
group_4983:     .db $00,$18,  $80,  $0c,  $00,$80,$80,$80,  $16,$03
group_498d:     .db $00,$21,  $6b,  $0c,  $80,$00,$80,$80,  $16,$01
group_4997:     .db $00,$28,  $80,  $0c,  $80,$00,$80,$80,  $16,$03
group_49a1:     .db $00,$31,  $6b,  $0c,  $00,$00,$80,$80,  $16,$01
group_49ab:     .db $00,$38,  $80,  $0c,  $00,$00,$80,$80,  $16,$03
group_49b5:     .db $00,$41,  $6b,  $0c,  $80,$80,$00,$80,  $16,$01
group_49bf:     .db $00,$48,  $80,  $0c,  $80,$80,$00,$80,  $16,$03

        ; idx $0e : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_49c9:     .db $00,$30,  $40,  $1c,  $00,$08,$10,$80,  $24,$11

        ; idx $0f : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_49d3:     .db $00,$30,  $f0,  $24,  $00,$08,$10,$18,  $24,$01

        ; idx $10 : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_49dd:     .db $00,$40,  $36,  $1c,  $00,$08,$10,$80,  $2c,$03
group_49e7:     .db $60,$43,  $04,  $1c,  $00,$08,$10,$80,  $2c,$05
group_49f1:     .db $a0,$43,  $20,  $1c,  $00,$08,$10,$80,  $2c,$07
group_49fb:     .db $a0,$45,  $31,  $1c,  $00,$08,$10,$80,  $2e,$0d
group_4a05:     .db $b0,$48,  $0c,  $1c,  $00,$08,$10,$80,  $2c,$0b
group_4a0f:     .db $70,$49,  $01,  $1c,  $00,$08,$10,$80,  $2c,$0d
group_4a19:     .db $80,$49,  $43,  $1c,  $00,$08,$10,$80,  $2c,$0f
group_4a23:     .db $b0,$4d,  $05,  $1c,  $00,$08,$10,$80,  $2c,$11
group_4a2d:     .db $00,$4e,  $19,  $24,  $00,$08,$10,$18,  $2c,$13
group_4a37:     .db $90,$4f,  $04,  $1c,  $00,$08,$80,$10,  $2e,$19

        ; idx $12 : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_4a41:     .db $00,$40,  $36,  $1c,  $00,$08,$10,$80,  $2c,$03
group_4a4b:     .db $60,$43,  $04,  $1c,  $00,$08,$10,$80,  $2c,$05
group_4a55:     .db $a0,$43,  $02,  $1c,  $00,$08,$10,$80,  $2c,$15
group_4a5f:     .db $c0,$43,  $2f,  $1c,  $00,$08,$10,$80,  $2c,$09
group_4a69:     .db $b0,$46,  $0c,  $1c,  $00,$08,$10,$80,  $2c,$1d
group_4a73:     .db $70,$47,  $01,  $1c,  $00,$08,$10,$80,  $2c,$33
group_4a7d:     .db $80,$47,  $10,  $1c,  $00,$08,$10,$80,  $2c,$35
group_4a87:     .db $80,$48,  $09,  $1c,  $00,$08,$10,$80,  $2c,$21
group_4a91:     .db $10,$49,  $22,  $1c,  $00,$08,$10,$10,  $2c,$25
group_4a9b:     .db $b0,$4b,  $44,  $1c,  $00,$08,$10,$c0,  $2e,$11

        ; idx $13 : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_4aa5:     .db $00,$40,  $36,  $1c,  $00,$08,$10,$80,  $2c,$03
group_4aaf:     .db $60,$43,  $04,  $1c,  $00,$08,$10,$80,  $2c,$05
group_4ab9:     .db $a0,$43,  $31,  $1c,  $00,$08,$10,$80,  $2e,$0d
group_4ac3:     .db $b0,$46,  $0c,  $1c,  $00,$08,$10,$80,  $2c,$17
group_4acd:     .db $70,$47,  $01,  $1c,  $00,$08,$10,$80,  $2c,$19
group_4ad7:     .db $80,$47,  $22,  $1c,  $00,$08,$10,$80,  $2c,$1b
group_4ae1:     .db $a0,$49,  $02,  $1c,  $00,$08,$10,$80,  $2c,$15
group_4aeb:     .db $c0,$49,  $2f,  $1c,  $00,$08,$10,$80,  $2c,$09
group_4af5:     .db $b0,$4c,  $0c,  $1c,  $00,$08,$10,$80,  $2c,$1d
group_4aff:     .db $70,$4d,  $01,  $1c,  $00,$08,$10,$80,  $2c,$1f
group_4b09:     .db $00,$4e,  $20,  $1c,  $00,$08,$10,$80,  $2c,$03

        ; idx $14 : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_4b13:     .db $00,$40,  $36,  $1c,  $00,$08,$10,$80,  $2c,$03
group_4b1d:     .db $60,$43,  $04,  $1c,  $00,$08,$10,$80,  $2c,$05
group_4b27:     .db $a0,$43,  $02,  $1c,  $00,$08,$10,$80,  $2c,$15
group_4b31:     .db $c0,$43,  $2f,  $1c,  $00,$08,$10,$80,  $2c,$09
group_4b3b:     .db $b0,$46,  $0c,  $1c,  $00,$08,$10,$80,  $2c,$1d
group_4b45:     .db $70,$47,  $01,  $1c,  $00,$08,$10,$80,  $2c,$2b
group_4b4f:     .db $80,$47,  $10,  $1c,  $00,$08,$10,$80,  $2c,$2d
group_4b59:     .db $80,$48,  $09,  $1c,  $00,$08,$10,$80,  $2c,$21
group_4b63:     .db $10,$49,  $22,  $1c,  $00,$08,$10,$10,  $2c,$25
group_4b6d:     .db $30,$4b,  $3b,  $1c,  $00,$08,$80,$10,  $2c,$23

        ; idx $15 : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_4b77:     .db $00,$40,  $36,  $1c,  $00,$08,$10,$80,  $2c,$03
group_4b81:     .db $60,$43,  $04,  $1c,  $00,$08,$10,$80,  $2c,$05
group_4b8b:     .db $a0,$43,  $02,  $1c,  $00,$08,$10,$80,  $2e,$21
group_4b95:     .db $c0,$43,  $2f,  $1c,  $00,$08,$10,$80,  $2c,$09
group_4b9f:     .db $b0,$46,  $0c,  $1c,  $00,$08,$10,$80,  $2c,$1d
group_4ba9:     .db $70,$47,  $01,  $1c,  $00,$08,$10,$80,  $2c,$2b
group_4bb3:     .db $80,$47,  $10,  $1c,  $00,$08,$10,$80,  $2c,$2d
group_4bbd:     .db $80,$48,  $09,  $1c,  $00,$08,$10,$80,  $2c,$21
group_4bc7:     .db $10,$49,  $22,  $24,  $00,$08,$10,$18,  $2e,$0f
group_4bd1:     .db $30,$4b,  $0b,  $1c,  $00,$08,$80,$10,  $2c,$27
group_4bdb:     .db $e0,$4b,  $3c,  $1c,  $00,$08,$80,$10,  $2c,$29
group_4be5:     .db $00,$4f,  $10,  $1c,  $00,$08,$80,$10,  $2c,$01

        ; idx $16 : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_4bef:     .db $00,$40,  $36,  $1c,  $00,$08,$10,$80,  $2c,$03
group_4bf9:     .db $60,$43,  $04,  $1c,  $00,$08,$10,$80,  $2c,$05
group_4c03:     .db $a0,$43,  $02,  $1c,  $00,$08,$10,$80,  $2c,$2f
group_4c0d:     .db $c0,$43,  $2f,  $1c,  $00,$08,$10,$80,  $2c,$09
group_4c17:     .db $b0,$46,  $0c,  $1c,  $00,$08,$10,$80,  $2c,$1d
group_4c21:     .db $70,$47,  $01,  $1c,  $00,$08,$10,$80,  $2c,$2b
group_4c2b:     .db $80,$47,  $10,  $1c,  $00,$08,$10,$80,  $2c,$2d
group_4c35:     .db $80,$48,  $09,  $1c,  $00,$08,$10,$80,  $2c,$21
group_4c3f:     .db $10,$49,  $22,  $24,  $00,$08,$10,$18,  $2c,$25
group_4c49:     .db $30,$4b,  $0b,  $1c,  $00,$08,$80,$10,  $2c,$27
group_4c53:     .db $e0,$4b,  $30,  $1c,  $00,$08,$80,$10,  $2c,$31
group_4c5d:     .db $00,$4f,  $10,  $1c,  $00,$08,$80,$10,  $2c,$01

        ; idx $17 : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_4c67:     .db $00,$40,  $36,  $1c,  $00,$08,$10,$80,  $2c,$03
group_4c71:     .db $60,$43,  $04,  $1c,  $00,$08,$10,$80,  $2c,$05
group_4c7b:     .db $a0,$43,  $02,  $1c,  $00,$08,$10,$80,  $2c,$15
group_4c85:     .db $c0,$43,  $2f,  $1c,  $00,$08,$10,$80,  $2c,$09
group_4c8f:     .db $b0,$46,  $0c,  $1c,  $00,$08,$10,$80,  $2c,$1d
group_4c99:     .db $70,$47,  $01,  $1c,  $00,$08,$10,$80,  $2c,$33
group_4ca3:     .db $80,$47,  $10,  $1c,  $00,$08,$10,$80,  $2c,$35
group_4cad:     .db $80,$48,  $09,  $1c,  $00,$08,$10,$80,  $2c,$21
group_4cb7:     .db $10,$49,  $22,  $24,  $00,$08,$10,$18,  $2c,$25
group_4cc1:     .db $30,$4b,  $0b,  $1c,  $00,$08,$80,$10,  $2c,$27
group_4ccb:     .db $e0,$4b,  $30,  $1c,  $00,$08,$80,$10,  $2c,$29
group_4cd5:     .db $00,$4f,  $10,  $1c,  $00,$08,$80,$10,  $2c,$01

        ; idx $19 : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_4cdf:     .db $00,$40,  $36,  $1c,  $00,$08,$10,$80,  $2c,$03
group_4ce9:     .db $60,$43,  $08,  $1c,  $00,$08,$10,$80,  $2e,$13
group_4cf3:     .db $e0,$43,  $92,  $1c,  $00,$08,$10,$c0,  $2e,$15

        ; idx $1a : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_4cfd:     .db $00,$40,  $16,  $1c,  $00,$08,$10,$80,  $2e,$17
group_4d07:     .db $60,$41,  $1e,  $1c,  $00,$08,$80,$10,  $2e,$1f
group_4d11:     .db $40,$43,  $cc,  $1c,  $00,$08,$10,$c0,  $2e,$25

        ; idx $1b : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_4d1b:     .db $00,$40,  $0b,  $1c,  $00,$08,$10,$80,  $2c,$37
group_4d25:     .db $b0,$40,  $0c,  $1c,  $00,$08,$10,$80,  $2c,$39
group_4d2f:     .db $70,$41,  $98,  $1c,  $00,$08,$10,$c0,  $2c,$3b
group_4d39:     .db $f0,$4a,  $17,  $1c,  $00,$08,$10,$c0,  $2e,$23

        ; idx $1c : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_4d43:     .db $00,$40,  $36,  $1c,  $00,$08,$10,$80,  $2c,$03
group_4d4d:     .db $60,$43,  $04,  $1c,  $00,$08,$10,$80,  $2c,$05
group_4d57:     .db $a0,$43,  $02,  $1c,  $00,$08,$10,$80,  $2c,$15
group_4d61:     .db $c0,$43,  $2f,  $1c,  $00,$08,$10,$80,  $2c,$09
group_4d6b:     .db $b0,$46,  $0c,  $1c,  $00,$08,$10,$80,  $2c,$1d
group_4d75:     .db $70,$47,  $01,  $1c,  $00,$08,$10,$80,  $2c,$1f
group_4d7f:     .db $80,$47,  $10,  $1c,  $00,$08,$10,$80,  $2e,$01
group_4d89:     .db $80,$48,  $09,  $1c,  $00,$08,$10,$80,  $2c,$21
group_4d93:     .db $30,$4a,  $1a,  $1c,  $00,$08,$10,$c0,  $2e,$03
group_4d9d:     .db $b0,$4a,  $08,  $24,  $00,$08,$10,$18,  $2e,$1b

        ; idx $1d : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_4da7:     .db $40,$40,  $13,  $1c,  $00,$08,$10,$c0,  $2e,$05
group_4db1:     .db $70,$41,  $98,  $1c,  $00,$08,$10,$c0,  $2c,$3b
group_4dbb:     .db $f0,$4a,  $49,  $1c,  $00,$08,$10,$80,  $2e,$07

        ; idx $1e : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_4dc5:     .db $00,$40,  $36,  $1c,  $00,$08,$10,$80,  $2c,$03
group_4dcf:     .db $60,$43,  $11,  $1c,  $00,$08,$10,$80,  $2e,$1d
group_4dd9:     .db $70,$44,  $0c,  $1c,  $00,$08,$80,$10,  $2e,$09
group_4de3:     .db $30,$45,  $96,  $1c,  $00,$08,$10,$c0,  $2e,$0b

        ; idx $1f : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_4ded:     .db $00,$60,  $fb,  $1c,  $00,$08,$10,$80,  $22,$01

        ; idx $31 : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_4df7:     .db $00,$70,  $32,  $1c,  $00,$08,$10,$80,  $22,$03

        ; idx $32 : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_4e01:     .db $00,$70,  $b3,  $0c,  $00,$00,$00,$80,  $22,$07
group_4e0b:     .db $30,$7b,  $25,  $1c,  $00,$08,$10,$80,  $22,$09
group_4e15:     .db $80,$7d,  $09,  $1d,  $00,$08,$10,$c0,  $22,$0b

        ; idx $33 : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_4e1f:     .db $00,$70,  $a1,  $1c,  $00,$08,$10,$80,  $22,$05

        ; idx $34 : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_4e29:     .db $00,$70,  $b3,  $8c,  $00,$00,$00,$80,  $22,$07
group_4e33:     .db $30,$7b,  $25,  $9c,  $00,$08,$10,$80,  $22,$09
group_4e3d:     .db $80,$7d,  $03,  $9d,  $00,$08,$10,$c0,  $22,$0b

        ; idx $35 : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_4e47:     .db $00,$0e,  $10,  $0c,  $80,$00,$80,$80,  $16,$01

        ; idx $36 : $32 $33   $34   $35   $36 $37 $38 $39   BNK IDX

group_4e51:     .db $10,$70,  $7d,  $1c,  $00,$08,$10,$80,  $22,$0d

...

group_4faf:     .ds $42af - *
Title: Re: Cyber Knight translation
Post by: esteban on 11/05/2016, 05:31 AM
Elmer: there is a typo in line 3d2d: it should read "B$NK", not "BNK"

:)

I agree that megatron-UK's passion for Cyber Knight is going to awaken an appreciation for something that 99.6% of us were either completely ignorant of, or had barely any genuine experiences with.

:)
Title: Re: Cyber Knight translation
Post by: megatron-uk on 11/05/2016, 05:57 AM
Many thanks, I do appreciate it.

In respect to the font, one thing I have been thinking about is if I can truly get every bit of Japanese text replaced then it would mean I would I would have over 100 free 8x8 tiles. It should be possible to get several variations on the ASCII font in those remaining tiles; italic, bold, etc. That would be one way of doing it and not have to mess with the font loader - it could all remain the same and simply load  italic ascii tiles, rather than the katakana/hiragana tiles.

Back to the dialogue, I do love this game. It's a fab SRPG, the graphics could be better - especially the overworld, but the music and sound fx are really great and there are some nice mecha designs and, judging from the amount of dialogue (I still haven't actually played the game beyond the first world) it seems to be fairly big and varied in terms of maps, characters and enemies.

Having the SNES english text has made translating our PCE version significantly easier (google translate is great, but it really doesn't beat someone fluent in Japanese) although there are a few small differences between the two that meant I had to write the mapper script to do fuzzy matching between the two. The big thing is, like you say, the script is understandable, but, it does leave a little to be desired. With a bit of tweaking (and judicious use of those extra control codes!) it can be made much more interactive. Hopefully people will enjoy it when it is eventually done!
Title: Re: Cyber Knight translation
Post by: megatron-uk on 11/08/2016, 06:29 PM
Just about finished writing the updated insert tool. It can currently expand the rom by the correct number of extra banks and also does a double conversion of the english text - once to hex bytecodes and then back again, just to be doubly sure that my conversion table is working properly. Tracking down dodgy bytes once inserted would be a pain!

Hopefully actually start to insert the text again before the end of the week and then test whether the expanded bank scheme works. I'll have to mess around with the bank loader table by hand first, but should be able to automate it once done.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 11/09/2016, 11:49 AM
Quickie - There are 27 text assets in the game, meaning 27 extra 8k banks. But that's an odd number so I've decided to up it to 32. That rounds the size up nicely to 768KB (64x8KB banks in the original ROM, plus 32 extra banks to hold the expanded English script).

There's only one asset that is larger than a single bank - it's already 10KB in Japanese (and thus in two consecutive banks already in the original game) and will probably expand to 14KB or more once fully converted. Everything else is well within the 8KB bank limit.

It's pretty wasteful though - the script is roughly 55-60KB in size so far, that's with about 20% English and 80% Japanese. That means we're wasting about 200KB of that extra ROM space... oh well, good job it's not a problem with SD card space!
Title: Re: Cyber Knight translation
Post by: elmer on 11/10/2016, 12:26 AM
Quote from: megatron-uk on 11/09/2016, 11:49 AMIt's pretty wasteful though - the script is roughly 55-60KB in size so far, that's with about 20% English and 80% Japanese. That means we're wasting about 200KB of that extra ROM space... oh well, good job it's not a problem with SD card space!
Space doesn't really matter in terms of ROM expansion these days, so you're doing the sensible and sane thing IMHO.  =D>

Now ... programmer "bragging rights" ... well that's a different kettle-of-fish!  :wink:
Title: Re: Cyber Knight translation
Post by: megatron-uk on 11/10/2016, 12:16 PM
Well, I've got the text inserted into the expanded ROM banks, also altered the asset bank table and asset offset table and.... well... it still loads! Some of translated script displays, but it seems to be off by a few strings as the main menu and player creation screen is showing garbage for a few areas and a few sections are wrong ('Del' in place of the 'End' button for the name entry... that means just a couple of delimeters are out of place).

The original script is left as-is. It still occupies banks 0x0a to 0x15.

The intro cinematic plays nicely and most of that text is okay, which is another bank, so it, I must have that one pretty much spot on. I've got some work to do with the text box creation control codes - I must have counted the size wrong as I have a couple of characters spilling over what I thought was a 22 character box with 20 characters in it!  :roll:

So, it's looking promising; the relocation to the additional banks does seem to have worked, I now just need to track down the 'usual' problems of missing control bytes, delimiters, etc.  :)
Title: Re: Cyber Knight translation
Post by: megatron-uk on 11/13/2016, 04:32 AM
Starting to make real progress now. The intro cinema is back working again, the expanded scrolling story after the cinema works perfectly (unlike my original translation which bumped up against rom size limits!) and the 'power is out' dialogue that happens immediately after is 100%.

Shipboard menu entries are next - the move options are done, now working on the Kanji text. I did this by hand last time around, so unlike most dialogue in the game, they're not all seperated by 0x00 bytes (lots of Kanji which uses that byte as well!) so I'm having to do them by eyeball. But there's only a couple of dozen individual entries to do, so it's not like it's going to take forever.

Also added a automatic way to get the Git revision number into the game, so I now embed the revision into the main menu so that I know what version it is that I (or anyone else) is playing.

Latest script and patched assets up on Github. If you're savvy with Python on Linux, just clone the repo and drop a 'Cyber Knight (J).pce' rom in the folder then run 'expandRom.py'. You shouldn't need to do anything else.

If you want to play around with it, then the following sequence is how to get all the assets from the game:

extractAssets.py - Extract all text asset banks to ./assets/raw... e.g. ./assets/raw/0x0a.dat
splitAssets.py - Split asset banks by asset chunks e.g. ./assets/split/0x0a.0x01.dat
mapAssets.py - Programatically match SNES translations to the split assets, outputs to ./assets/converted
expandRom.py - Inserts assets from ./assets/converted back in to an expanded (768KB) ROM file.

At the moment I'm tweaking asset files and then re-running expandRom.py each time, then firing up Mednafen and either loading BRAM saves, or save states to test each time.
Title: Re: Cyber Knight translation
Post by: elmer on 11/14/2016, 11:44 AM
Quote from: megatron-uk on 11/13/2016, 04:32 AMStarting to make real progress now. The intro cinema is back working again, the expanded scrolling story after the cinema works perfectly (unlike my original translation which bumped up against rom size limits!) and the 'power is out' dialogue that happens immediately after is 100%.
Great progress!  :D


QuoteLatest script and patched assets up on Github. If you're savvy with Python on Linux, just clone the repo and drop a 'Cyber Knight (J).pce' rom in the folder then run 'expandRom.py'. You shouldn't need to do anything else.
I just gave it a try ... it's looking good!  8)

It was nice to see the intro in English.


QuoteAt the moment I'm tweaking asset files and then re-running expandRom.py each time, then firing up Mednafen and either loading BRAM saves, or save states to test each time.
That's what we're doing with the Xanadu games, too.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 11/19/2016, 08:56 AM
Okay, here is your chance to contribute, folks.

We have 5 playable mecha in the game, they are:

レックス,Rex
ウイナー,Winner
シェリフ,Sheriff
タイタン,Titan
サウルス,Saurus

Catchy, eh?  ](*,)

We have a restriction in that their displayed name can be no more than 4 characters in length - this is decided by the width of the status window in combat, the fact that all 5 are displayed horizontally, in a list in the hangar bay to select from, etc. Not a lot I can do about it without some serious hacking, or a variable width font (not at the moment, thank you!).

So, let's have your ideas, ladies and gentlemen. My first thought was to use something like Gundam model numbers; RX78 and similar, as there's not a lot of 'cool' names to fit in just four characters!
Title: Re: Cyber Knight translation
Post by: elmer on 11/19/2016, 11:15 AM
Quote from: megatron-uk on 11/19/2016, 08:56 AMWe have a restriction in that their displayed name can be no more than 4 characters in length - this is decided by the width of the status window in combat, the fact that all 5 are displayed horizontally, in a list in the hangar bay to select from, etc. Not a lot I can do about it without some serious hacking, or a variable width font (not at the moment, thank you!).
You're never going to get a VWF in that game with any kind of reasonable-to-do hack ... the whole thing is too tile-based.

But ... when you get rid of the need for all of those Japanese characters, you can predraw certain strings in a VWF.

For instance, in this case you've got 5 mecha using 4 tiles each ... so if you have 20 tiles free in memory, then you just give them names like "abcd", "efgh", "ijkl", etc, and then draw each name into their unique set of 4 tiles.

It looks like you'll have 128 tiles free to do this trick in the main font in VRAM, and even more in the alternate font.

Perhaps that will be an option for you at some point?
Title: Re: Cyber Knight translation
Post by: TurboXray on 11/19/2016, 04:14 PM
elmer beat me to it. That's a common trick: hardcoded vwf stuff in tiles no longer needed.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 11/20/2016, 02:55 PM
There's an interesting string hidden amongst those that are used for the main menu and player creation screen.

Hidden at 0x1c086 there is this sequence of bytes:
["20", "20", "be", "b2", "cd", "de", "c2", "3a", "02", "20", "b5", "dd", "c5", "20", "20", "b5", "c4", "ba"]

Translated, this is:
"  せいべつ:\n おんな  おとこ"

In English, this seems to be:
" Sex: Male Female"

It would appear that at some point, Cyber Knight perhaps was planned to allow the player to choose the sex of the character they played, but that string is never shown and there isn't any option to select a gender at character creation time. It's odd, as it's between strings that construct the player creation screen, so I would have expected it to be shown on that screen.

There are a few other bits and pieces like this that I've spotted, a sound menu is the other interesting one, which, if we could figure out how to access it, would be fantastic.

On translation news, I've got most of the battle menu text back in, and having figured out more of the control codes along the way, it's shaping up to be nicer looking that the previous version. In pure numbers I've got about 50% of the ~3000 strings translated so far (many are just control codes, so it's not all printable). The big job is jumping back and forth between savegames to make sure that inserted text doesn't break things... it often does!
Title: Cyber Knight translation
Post by: esteban on 11/20/2016, 07:08 PM
I don't care what gender I adopt when I play Cyber Knight—it's gonna be great!

:)

Also, I have suggestions for the names, should we limit them to only four characters (I know a solution was already found, humor me):

Leif
Pewk
Slam
Torn
Dr.B
Kawk
Tawk
Bono
Edge
Lynk
Lugi
Luge
Title: Re: Cyber Knight translation
Post by: megatron-uk on 11/21/2016, 03:23 AM
There are actually only a few cases where NPC's having names longer than 4 characters makes a difference:

Team menu - selecting the two active members of your squad. Only 4 tiles of space is present between the 'selector' buttons, so anything longer gets cropped.

Battle menu - in the right hand stats column there is a space of 6 tiles between the border of the column and the current HP of the character. When you first start the game you only have a double-digit amount of HP, so 5 characters in your squadmates names will fit early on, with a space between the values, but later in the game they will overlap the displayed HP.

Mecha-equip menu - 4 characters width in the box to select the mecha to view. If longer, then the team mates name overflows the selection box.

At the moment I'm using 'Rx78', 'Wasp', 'Seta', 'Ogre' and 'Soru' as the mecha/combat suit names to simplify display issues.
There's nothing that 'breaks' the game as such, but it's a bit messy.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 11/21/2016, 04:18 AM
One thing I've been thinking about is trying to work out how the portraits of the various characters are displayed. There is also some code somewhere that displays a graphic when you find an 'item', whatever that widget is - for instance in the first world, when the Professor gives you his Star Map, it pops up a little window with a start chart inside it; there are lots of items in the game to acquire, so I would think this will be consistent throughout the code.

It would be great to be able to do that throughout the game - pop up a window in the corner of the screen for whoever is speaking, for example. As well as reference any items when they are discussed on other worlds.

I thought there may have been some control codes embedded in the script that did this, but it doesn't appear to be the case.
Title: Re: Cyber Knight translation
Post by: elmer on 11/21/2016, 04:18 AM
Quote from: megatron-uk on 11/21/2016, 03:23 AMAt the moment I'm using 'Rx78', 'Wasp', 'Seta', 'Ogre' and 'Soru' as the mecha/combat suit names to simplify display issues.
Here are some others ... can you guess the theme?  :wink:

ErNy
KrMt
MsPg
KuKi
GrVr
BgBd

Too much? :roll:
Title: Re: Cyber Knight translation
Post by: megatron-uk on 11/21/2016, 04:22 AM
I should try to extract these graphic assets... I haven't seen any images of NPC's you talk to, and getting them in to the game, alongside the portraits of your team would be a fantastic achievement.

This is probably going to be a side-project alongside the translation. Need to track the existing graphic assets down first!
Title: Re: Cyber Knight translation
Post by: NecroPhile on 11/21/2016, 09:18 AM
Quote from: elmer on 11/21/2016, 04:18 AMHere are some others ... can you guess the theme?  :wink:

ErNy
KrMt
MsPg
KuKi
GrVr
BgBd

Too much? :roll:
No OsCr?
Title: Re: Cyber Knight translation
Post by: elmer on 11/21/2016, 01:02 PM
Quote from: guest on 11/21/2016, 09:18 AMNo OsCr?
#-o How did I miss him?  :wink:
Title: Re: Cyber Knight translation
Post by: Psycho Punch on 11/21/2016, 02:31 PM
レックス,REX
ウイナー,ACE
シェリフ,STAR/CAPT
タイタン,HULK
サウルス,DINO

I tried.

Title: Re: Cyber Knight translation
Post by: megatron-uk on 11/26/2016, 05:14 AM
I keep coming across 'weird' behaviour in the new expanded ROM. There are certain bits of dialogue that I know are in one bank (the NPC text for the first world - Farworld, for example), that when triggered, display a partial string from a completely different bank.

Here's an example:

The first NPC triggers a string that was originally in bank 0x0a, asset ID 01 located at 0x1403e, and is now in bank 0x4c, asset ID 01 at ROM location 0x9805a:
(http://targetearth.dyndns.org/www/misc/c-0001.png)

But then this second NPC is supposed to trigger a string that was in the same bank (it was the next string in fact - seperated only by a single 0x00 delimeter [and still is] and started a dialogue about robots going around kidnapping people), but instead triggers one that is in fact located in another bank completely now. It triggers a string from bank 0x41, asset ID 01 (originally bank 0x14, asset ID 0x83) way, way off at 0x94004:
(http://targetearth.dyndns.org/www/misc/c-0002.png)

There's also the fact that whenever the save/load game screen is shown it gets corrupted with some of 'test screen' text that is normally never shown in the game:
(http://targetearth.dyndns.org/www/misc/c-0003.png)

I've even tried getting rid of every single byte of translated text and simply re-inserting the original japanese script (as raw hex bytes) as-is, in the new asset banks. I get the same result in this case, as well as a weird 'off by one string' error as shown here in the original, unaltered Japanese ROM:
(http://targetearth.dyndns.org/www/misc/j-0000.png)

... and here with the same data, but relocated to the new banks:
(http://targetearth.dyndns.org/www/misc/c-0000.png)

I'm starting to think that there may be some other logic to loading the script assets going on here. But I'm out of my depth when it comes to assembler.

I'm starting to tear my hair out with this now, actually.
Title: Re: Cyber Knight translation
Post by: elmer on 11/26/2016, 12:00 PM
Quote from: megatron-uk on 11/26/2016, 05:14 AMI've even tried getting rid of every single byte of translated text and simply re-inserting the original japanese script (as raw hex bytes) as-is, in the new asset banks. I get the same result in this case, as well as a weird 'off by one string' error as shown here in the original, unaltered Japanese ROM:
(http://targetearth.dyndns.org/www/misc/j-0000.png)

... and here with the same data, but relocated to the new banks:
(http://targetearth.dyndns.org/www/misc/c-0000.png)

I'm starting to think that there may be some other logic to loading the script assets going on here. But I'm out of my depth when it comes to assembler.

I'm starting to tear my hair out with this now, actually.
This is the test to concentrate on for the moment ... insert all of the original Japanese text chunks (as binary blobs) into the new banks.

If you do that and get "off-by-one" errors like that, then I suspect that you could be constructing the new banks incorrectly.

Thinks to look at would be not putting in the beyond-the-last-chunk offset correctly (that's needed to calculate the length of a chunk), getting an off-by-one in the actual length of the text chunk (i.e. missing the last string, or the text-chunk-terminator).

The off-by-one of the text in those screens suggests that the game is looking for the strings correctly, but that it's missing one string in the count and so displaying stuff one string too soon.

That could easily be a missing terminator on the previous chunk, or the structure of the bank not pointing to the beginning of the chunk properly.


Quote from: megatron-uk on 11/26/2016, 05:14 AMBut then this second NPC is supposed to trigger a string that was in the same bank (it was the next string in fact - seperated only by a single 0x00 delimeter [and still is] and started a dialogue about robots going around kidnapping people), but instead triggers one that is in fact located in another bank completely now. It triggers a string from bank 0x41, asset ID 01 (originally bank 0x14, asset ID 0x83) way, way off at 0x94004:
(http://targetearth.dyndns.org/www/misc/c-0002.png)
Hold on ... ...  :-k

It could be as simple as a misunderstanding of the bank structure.

The game uses pairs of banks per 16KB block, starting at an even bank number ... so originally banks $0A+$0B, $0C+$0D, $0E+$0F, $14+$15.

So ... how come you've got a string in bank $41? Are you not putting things in 16KB blocks?

<EDIT>

Hmmm ... a quick look at my disassembly suggests that the code *should* handle data blocks on odd banks, but the game never does it, so I'd be worried about something in the code that I've not seen yet.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 11/26/2016, 07:29 PM
I'm remapping to 8kb banks for all assets apart from the one originally located in bank 0x14, asset ID 0x83, which is ~10KByte in size, so spans two banks (16KB).

So, for example, I'll have assets in bank 0x49, then one in 0x49, then my 16kb bank starting at 0x4a, then the next 8kb bank is 0x4c. The asset table at 0x2939 is modified to point to the correct one - i.e. anything originally referencing 0x14 is modified to reference 0x4a. All offset pointers in the pointer table are set to 0x1, as there is only one asset in each bank now (in fact, the header of each asset table in each bank is as simple as <bank_id> 0x03 0x40... as it always only has one asset entry that immediately follows at offset +3).

The asset bank table used to look like this:
0002930: aa7a 6020 46c7 4cb0 c50e 0e0e 0e0e 0e0c  .z` F.L....
0002940: 0e0e 0000 0000 0000 0014 0a0a 0a0a 0a0c  ...........
0002950: 0c0c 0c0c 0c0c 0c0c 0c0c 0103 0507 090b  ...........
0002960: 170d 0f00 0000 0000 0000 8301 0305 0709  ...........
0002970: 0103 0507 090b 0d0f 1113 1511 1111 1111  ...........

It now looks like this:
0002930: aa7a 6020 46c7 4cb0 c541 4243 4445 4647  .z` F.L..AB
0002940: 4849 0000 0000 0000 004a 4c4d 4e4f 5051  HI.......JL
0002950: 5253 5455 5657 5859 5a5b 0101 0101 0101  RSTUVWXYZ[.
0002960: 0101 0100 0000 0000 0000 0101 0101 0101  ...........
0002970: 0101 0101 0101 0101 0101 0111 1111 1111  ...........
Title: Re: Cyber Knight translation
Post by: megatron-uk on 11/26/2016, 07:36 PM
I could go to a 16kb pair of banks per asset scheme, though that would take me straight to a 1MB ROM, as there are 27 individual assets that need to go in the new banks, using 16kb for each that's 54 x 8kb, rounding up to 64 to make it 512KBytes.

Potentially, that might be relatively easy to try.

EDIT: Nope, using 2 banks per asset and keeping it aligned on even bank numbers didn't make a difference.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 11/27/2016, 12:24 PM
Okay, some good news, some bad news.

The good news is that I've figured out what is causing the 'jump to other asset string' behaviour that I've been seeing. Here's that second NPC that you talk to, he's the one who expressed shock at your 'robot costume'. Remember it was showing this dialogue, in error:
(http://targetearth.dyndns.org/www/misc/c-0002.png)

Here's what he should be saying (and now does!):
(http://targetearth.dyndns.org/www/misc/c-0004.png)

The bad news come from the realisation that the dialogue box display codes now appear to be game-area dependent. So I had changed the dialogue box from his Japanese text:

<1a><22>よくできた ぬいぐるみだなお。<pause><1E> え?<pause><1E> ほんもの?<wait>

Where the control codes "1a" "22" generated a 25x1 text box in the middle of the screen.

To my desired english translation:
<1a><70>Man: This robot really\nlooks real!<pause><1E>\nEh?<pause><1E> It IS real!!!<wait>

Where "1a" "70" displayed a 26x3 text box at the top of the screen. Only in this case, it didn't, it caused the game to jump to another bank and load an asset from the ship-based dialogue (i.e. the bit about a lava planet).

I just tried change the dialogue box codes to "1a" "1e" (24x2 box, top centred), thus:

<1a><1e>Man: This robot really\nlooks real!<pause><1E>\nEh?<pause><1E> It IS real!!!<wait>

...and that works, as seen in the image above.

So, the bad news is that I'll need to go through each dialogue box display, in turn, and work out which ones actually work in the overworld, in town, in the ship, during cinemas, etc and change the english translation to suit. What a goddamn pain.

It isn't the problem affecting the savescreen, that seems to be something else. At least this solves one problem, though generates quite a bit of work to go back through all the asset data and check if I'm using dialogue box codes that don't work in that particular section (and bearing in mind I've not played through the bulk of the game, a lot of that I'm simply not going to know, yet).
Title: Re: Cyber Knight translation
Post by: megatron-uk on 11/27/2016, 01:17 PM
Right, so through some judicious testing of the control codes it seems that of the 44 unique dialogue box control combinations, only 20 actually work correctly in the overworld and in NPC dialogue.

[0x1a, 0x1a] through to [0x1a, 0x2e] work correctly, but from [0x1a, 0x3a] to [0x1a, 0x74] cause the symptoms as described earlier - jumping into other banks. There must be some code specifically doing this, I can't imagine it's just a coincidence.

Now that I know what causes it, I'll have to go back through all of the text converted so far and see where I'm using any of those broken control codes, then try and pack the text into one of the dialogue boxes made available by the lower range of control codes instead. :roll:
Title: Re: Cyber Knight translation
Post by: elmer on 11/27/2016, 02:30 PM
Quote from: megatron-uk on 11/27/2016, 12:24 PMThe good news is that I've figured out what is causing the 'jump to other asset string' behaviour that I've been seeing.
...
The bad news come from the realisation that the dialogue box display codes now appear to be game-area dependent. So I had changed the dialogue box from his Japanese text:
Nice sleuthing!  =D>

Yep, it's not horribly uncommon to change the meaning of script/command codes/parameters in assembly language.

It's not a problem when you're original developer with the source code, but it's a huge PITA when you're trying to do a post-mortem translation like this.

Sorry to hear that you've hit this problem.


Quote from: megatron-uk on 11/27/2016, 01:17 PMNow that I know what causes it, I'll have to go back through all of the text converted so far and see where I'm using any of those broken control codes, then try and pack the text into one of the dialogue boxes made available by the lower range of control codes instead. :roll:
That's what makes translations so much fun to work on!  ](*,)
Title: Re: Cyber Knight translation
Post by: megatron-uk on 11/27/2016, 05:06 PM
Okay, all NPC text for the first world now in place again. All displaying correctly now, too. That's all 80 or so strings for 'Farworld' in English and showing as intended. There's only one place that I'm not sure what to do with - I think I mentioned it early in the translation; the SNES text has the president giving you information about where to find the next people to help you. This is not in the PCE, so I need to make a decision about whether to put that in, or leave it up to the player to go exploring.

Perhaps the decision was made when coding the later SNES release to make it a little easier on the player?

I'll go back to the ship menu and battle messages now. Since it's the same asset used throughout the game, if I get the bugs worked out now, then it should be fine for the rest of the game.
Title: Re: Cyber Knight translation
Post by: esteban on 11/27/2016, 06:34 PM
I am so happy to see the NPC text progress... things went from crap to crystal in a relatively brief period of time (I'm not suggesting it wasn't frustrating as hell, mind you).

:)

I'm just happy things are moving forward.
Title: Re: Cyber Knight translation
Post by: megatron-uk on 11/28/2016, 12:15 PM
Got pretty much all of the battle text done now. Two little fragments left to translate, and some overflow/corruption to track down, but nothing major.

One thing I am trying to do is to slow down the display of the 'player attacks badguy', 'player causes 5pts damage'  type messages that flash up after the battle animations. They fly past incredibly fast and the normal <pause> and <slow> control codes are not obeyed during battle. I'll try and figure something out.

The SNES and PCE script differ again, here in the battle screen - the SNES has a heap of extra strings that the onboard compute flashes up when an enemy has a particularly strong shield ('makes your laser weaker', 'cools your heat attacks', etc), these are not present in the PCE. Though instead we get not one, but two messages from the ships cook about the quality of tonights dinner onboard the SS-Swordfish when we return from battle! There's always a silver lining!  :lol:
Title: Re: Cyber Knight translation
Post by: elmer on 11/28/2016, 05:24 PM
Quote from: megatron-uk on 11/27/2016, 05:06 PMThere's only one place that I'm not sure what to do with - I think I mentioned it early in the translation; the SNES text has the president giving you information about where to find the next people to help you. This is not in the PCE, so I need to make a decision about whether to put that in, or leave it up to the player to go exploring.

Perhaps the decision was made when coding the later SNES release to make it a little easier on the player?
If the alternative is having the player walk randomly around the map with no idea of what to do, then I'd definitely favor giving them some kind of hint ... but it doesn't have to be hand-holdingly blatant.


Quote from: megatron-uk on 11/28/2016, 12:15 PMThe SNES and PCE script differ again, here in the battle screen - the SNES has a heap of extra strings that the onboard compute flashes up when an enemy has a particularly strong shield ('makes your laser weaker', 'cools your heat attacks', etc), these are not present in the PCE. Though instead we get not one, but two messages from the ships cook about the quality of tonights dinner onboard the SS-Swordfish when we return from battle! There's always a silver lining!  :lol:
:wink: