TurboRip v1.42: The PC Engine/TG-16 CD ISO/WAV/CUE Ripper [2019]!

Started by NightWolve, 04/19/2006, 06:49 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

PCEngineFX

I'm not only posting, but I'm going to try this out on some FX games!
// Aaron Nanto | The Ultimate Resource for NEC Console Information!
Papa PCEFX 1997-2020 [Retired]
IMG

NightWolve

Quote from: Pcenginefx on 08/11/2015, 09:16 PMI'm not only posting, but I'm going to try this out on some FX games!
As a Microsoft guy (I consider myself as one as well, in spirit, brother! Heh-heh!), you'll be pleased to know that it is likely certified/approved for use all the way back to Windows 95 (official testing though was limited to Windows 98SE, 2000 Pro, Vista Home, and Windows 10 Home on my Winbook tablet with a USB connected DVD drive!!!!) as well as all the way to the present with Windows 10!! :) But I'm a meticulous coder and I always look up my WinAPI calls to make sure the function I'm using exists in the kernel32.dll or user32.dll of Windows 95 via the Microsoft Platform SDK WinAPI documentation system (which is very well-made BTW!) so I'm leaning on it being a safe assumption.

I really DO gotta give it the GUI that it deserves some day though! Just a lightweight version, no cheating by using Visual Basic or other RAD development packages and then forcing the need for a bunch of .NET runtimes and all the rest of it!! :) I wanna do it limited to just using WinAPI calls to kernel32, user32, shell32, comdlg32, etc. as is the case with something like TocFixer.

Quote from: TurboXray on 08/11/2015, 09:06 PMI just used this today (new version). Big thanks NW for this app! Much appreciated.
Thanks Tom! Lemme know if you got any ideas/suggestions for it!!

NightWolve

Alright guys, another silent update. Download TurboRip again if you've been using it and plan to in the future.

ysutopia.net/get.php?id=TurboRip

If the build date is anywhere between 8-7-2015  to 8-19-2015, it's an old 1.40 version. The latest best build will show August 24, 2015. The changes/improvements are reflected via editing in the same 1.40 section which I am listing below. One thing not listed is that (J) and (U) region codes are now added back to the automatic naming system via the TOC, they stopped working for older builds as I was making tweaks.
Quote from: TurboRip 1.40 build 8-24-2015 wrap-up+ Bug fix: Fixed another minor bug with the /name parameter where all letters
   were forcibly lowercased. Casing is now properly preserved.

 + Changed options /pcep and /hugox to /psp and /xbox respectively. Shorter and
   easier to go by the gaming platform of those emulators I figure.

 + Added a shortcut /turbo option for /speed=max - Adding /turbo at the prompt
   for parameters is a quicker way to set the drive's reading speed to maximum.

 + Other options that changed: /rs to /mrs, /br to /mbr, /mbr to /mmbr, /vbr to
   /mvbr. For help, it now can either be /?, /h, or /help.
 
 + When using /?, /h, or /help for a parameter list, TurboRip no longer exits
   and forces you to restart it - it will list the parameters, then let you
   enter what you want to use and resume! The nice thing is you'll still be
   able to see many parameters as you decide what to use. You shouldn't avoid
   reading the ReadMe to understand everything, but it's a nice shortcut!

 + TurboRip sets the TOP_MOST flag 'on' of the Command Prompt window so it can
   never be hidden behind other windows until it's closed or minimized. If it's
   minimized while ripping, it'll restore and flash itself when it's finished.

 + To reduce the size of TurboRip, all third-party components for MP3 (LAME)
   and APE are now zipped within TurboRip and extracted/unzipped on demand!
   TurboRip is now 4096 aligned to the preferences of Win98® as is the APE DLL.
   4K alignment normally makes an EXE/DLL bigger, but with some reduction in
   the way CD TOC naming data was stored, the EXE wound up smaller than ever!


So, I have more upcoming updates I'm working on right now and I will make them overt with the versioning system.

1) TurboRip 1.41 will add OGG support just as soon as I compile the vorbis "C" files into a usable DLL to my tastes. After talking with a) Vinny who made this TurboRip Guide here for WiiMednafen and b) Mednafen's creator, I finally decided to greenlight the idea. Gotta also figure what the /option will be besides /ogg for general music CDs, probably /wii or something.

2) TurboRip 1.42 - Seeing the patching process of Zeroigar led to deciding I wanna add the BinChunker ability directly into TurboRip as it's not a lot of code. Just a matter of parsing the CUE file of a BIN set and reading/writing out the sectors. The wave files are straightforward and you can detect the data mode for a data track as far as differentiating mode2 forms like PSX and so forth, etc. So I think it might be useful to just add that, pass TurboRip a CUE file as a parameter, and if it's of a BIN, spit out the tracks per file by parsing the CUE's lines, etc. I might as well make it more useful instead of how I would have people in the past mount a CUE file, point TurboRip to the virtual CD drive, in order to convert it to ISO/WAV/CUE that way.

3) TurboRip 1.50 - That's the version # I really wanted to release before I was contacted about adding PCFX TOCs given/for the Zeroigar project. By this point, I wanna add the Q subchannel analysis to properly detect 00-99 indexes for any tracks... This will make TurboRip a good/respectable CD rip app finally! If you want to defeat copy protection, then you still gotta go for CloneCD or Alcohol instead; those were made for all the tricks that are used, and I don't need nor can't ever get that far... TurboRip will be straight for Sega/Neo/PCE/PC-FX, all the retro CDs that never used copy protection but did make use of nonstandard indexing (via pregaps), etc.

There is the matter of CD+G discs which is data stored on R-W subchannels and to handle those, you must in fact become CloneCD and rip a separate .sub file for all 96 bytes of subchannel data and do it in a format that burning software recognizes, so that means no BIN/CUE, but IMG/SUB/CCD as in CloneCD... I have to consider that down the road when/if I get good enough... :/



Well, that concludes this status update and future plans for now. Yeah, please delete any old copies and get the upgrade if you like the app and plan on using it in the future!

Expect the 1.41, 1.42 versions soon, within this week or next, while 1.50 is the big upgrade I was planning all along that I never got around to which will happen eventually. That would be months away if I continue with my motivation to upgrade this app!

P. S. Thanks to David Shadoff, I will have a real copy of the "Super PCEngine Fan Deluxe - Special CD-ROM Vol.1" PC-FX CD in a couple of days which uses multiple indexing in one of the data tracks, something that TurboRip can't detect at present. He donated it to me in the name of science, that is advancement in the science of CD analysis. :) So yeah, I really wanted a copy of the disc so I can study it and help perfect TurboRip with it; it'll be cool to finally get that done!

IMG

This is what I'm talking about, check out a portion of its CUE file produced by CloneCCD:
QuoteFILE "IMAGE.img" BINARY
   TRACK 1 AUDIO
     INDEX 1 00:00:00

   TRACK 2 MODE1/2352
     INDEX 0 00:55:45
     INDEX 1 00:58:45

   TRACK 3 MODE1/2352
     INDEX 1 11:33:74

   TRACK 4 MODE1/2352
     INDEX 1 13:34:51

   TRACK 5 MODE1/2352
     INDEX 1 13:51:56
     INDEX 2 13:58:45
     INDEX 3 14:01:12
     INDEX 4 16:12:59
     INDEX 5 22:43:38
     INDEX 6 28:16:20
     INDEX 7 29:53:41
     INDEX 8 34:04:26
     INDEX 9 34:29:34

   TRACK 6 MODE1/2352
     INDEX 1 34:51:19
If your ripping software can produce a CUE file like that with the detection of those 9 indexes in total for track 5, it separates the men from the boys basically in terms of proper, expert track analysis and makes it top notch!

johnnykonami

Thanks for your work on this NightWolve, it's a really great tool!

NightWolve

Quote from: johnnykonami on 08/24/2015, 04:45 PMThanks for your work on this NightWolve, it's a really great tool!
You're welcome! Feel free to list the games you used it with and ideas of how else it could be improved.



Random Thoughts: A part of me feels maybe I should've cheated and used Visual Basic 6 to develop a Windows GUI version when I got started. It's slower to do it in "C" but more rewarding that the executable is lightweight by the end of it and you don't have to direct the user to download a bunch of runtime installers which only keep getting bigger such as with the later .NET upgrades to the Microsoft IDEs... An example where I did get it done in "C" was with my other PC Engine/TG-16 TocFixer app, very simple, very basic GUI that I started off from a default Visual C++ 6.0 GUI project.

But yeah, it's nice to be able to quickly slap a GUI together with RAD IDEs, just that the penalty sucks of then needing all these runtime dependencies... That's kinda why I started with a console/command-line version initially for TurboRip and it was a bit later that I learned how to do something like TocFixer with VC++ 6.0... In a professional IT setting, software developers go with stuff like Visual Basic because time matters far more than efficiency and neat/readable/organized code - it's about finishing something in a reasonable amount of time that works and has a good interface!

Since TurboRip is a fan project, I worked on it whenever and didn't apply a deadline/business thought process of course. I personally never liked software developers that could slap stuff together quickly because it almost always equals shitty code and just one bug meant days/weeks of debugging it for those left with it to pick up the pieces after they were long gone...



Anyway, David's shipment of the "Super PC Engine Fan Deluxe - Special CD-ROM Vol.1" PC-FX CD arrived today! I now have what I need to help perfect TurboRip! He sent me the whole magazine that it came in since it was a duplicate. I decided to scan it for the hell of it. So, the magazine page is cardboard and the back side of it included another CD, the "Develo Magazine Vol. 1" which is a PC Engine Super CD that contains demos for Popful Mail, Fray CD Xak Gaiden and Assembly/Basic-developed games, along with development tools, etc.

IMG


NightWolve



NightWolve

Per Dicer's problem with my server not hosting files properly when they're not hard links but instead are accomplished with a PHP script plus custom headers, could people test these links for me ??

www.ysutopia.net/get.php?id=TurboRip

www.ysutopia.net/downloads/TurboRip.zip

After the last couple of days, I optimized the PHP code every which way Sunday and did learn that redirect links are better off with the FULL url, hostname included, but I still couldn't solve the problem for Dicer. And then! I used the best possible solution beyond a hard link, a symbolically linked file, but he reports back that EVEN THAT doesn't work, giving him a connection interrupted error...

The only thing that works for him through his network and 2 browsers is a public direct link to a real file, this link:

www.ysutopia.net/software/TurboRip.zip

Now I examined the HTTP response headers sent with a real file versus a symbolic link, they're exactly the same! I spent quite a bit of time trying to figure this out but I'm out of ideas.

If anyone with web experience has any ideas, lemme know. In the meantime, if people could test the first 2 links and lemme know if they don't work, I'd appreciate it.

TurboRip Status: johnnykonami caught a major bug in Windows 7 that renders it useless. Turned out that was one Windows version I couldn't boast compatibility with. While I finished all the code enhancements/research to fix TurboRip for Windows 7, I've procrastinated and got distracted with other things, so haven't pushed the update out yet. But anyway, for anybody with Windows 7, there'll be an update as soon as I get a chance if not this week, the next. I mean, the work is basically done, but I'm optimizing other stuff and then I have to document everything in the ReadMe, so it's just a matter of when I get a chance. Afraid I used up too much time and the OGG support will have to wait.

Gentlegamer

IMG

NightWolve

Some time ago, I was emailed a question asking me for help to rip Arkhan's Insanity game so that it could be shared on ROM sites. Said individual noticed that it wasn't available and he wanted to be the first to get to upload it. I politely declined and informed him that it's my friend Arkhan's homebrew game and I won't help as I doubt he sold out of his pressed CDs.

Well, turns out that unbeknownst to Arkhan and Oldman, they actually accomplished a form of copy protection by unintentionally creating a large postgap of unburned sectors at the end of the first data track!

According to Yellow book rules, in designing the layout of a mixed mode CD, you're supposed to specify a 2 second postgap when transitioning from a data track to an audio track. There have been 2 ways to reflect this and you've seen it all the time:

QuoteFILE "02 Akumajou Dracula X - Chi no Rondo (J).iso" BINARY
  TRACK 02 MODE1/2048
    PREGAP 00:03:00
    INDEX 01 00:00:00
FILE "03 Akumajou Dracula X - Chi no Rondo (J).wav" WAVE
  TRACK 03 AUDIO
    PREGAP 00:02:00
    INDEX 01 00:00:00
Or, like this:
QuoteFILE "02 Akumajou Dracula X - Chi no Rondo (J).iso" BINARY
  TRACK 02 MODE1/2048
    PREGAP 00:03:00
    INDEX 01 00:00:00
    POSTGAP 00:02:00
FILE "03 Akumajou Dracula X - Chi no Rondo (J).wav" WAVE
  TRACK 03 AUDIO
    INDEX 01 00:00:00
The second way is technically more proper, but it's mostly done the first way. Burning software seems to correct the first way while burning the CD-R and treat it as a postgap anyway which will be reflected as 2 seconds of skipped/unburned sectors.

Now, TurboRip subtracts this 2 seconds off by default, but because they specified a much bigger postgap for Insanity, the true final sector is unknown, and TurboRip continues to read into the unburned postgap area resulting in a Layered-Error uncorrectable error, meaning, either the area of the CD was really scratched up and can't be read, or as I later realized, it's totally unburned, so naturally there is no correct EDC/ECC data to speak of resulting in total erroring out and failure.

Alcohol 52% also fails to continue reading after encountering this, so in effect, a basic copy-protection was created by inadvertently specifying a larger than standard postgap transition area. 

Long story short, I contacted Arkhan about the matter and offered to block any ripping attempts for Pyramid Plunder and Insanity. I've added the blocks, just have to write a message in the form of please respect our homebrew guys and buy the game until it sells out, then redirect to their website. Pyramid Plunder does need the block as it's just a 2 meg data track and it easily is ripped by TurboRip or any other program. But anyhow, it was interesting how Insanity actually already achieved a decent copy protection.

IMG

elmer

Haha ... that's an interesting little copy-protection "trick", I'd be curious to know if it was deliberate.

Would the PCE's CD drive be able to find any subsequent audio tracks OK after a gap like that?

NightWolve

Nope, I was talking to OldMan about it, it was just something that inadvertently happened given how they wrote their CUE file! Totally unintentional!

At first, I was unsure, so I asked them if they did something on purpose because I was wondering if I got a badly pressed CD or something! Arkhan sent me 2 copies of their games rather than giving me the TOCs so I could block them in TurboRip, so while doing that, I encountered this issue. This is why some random dude contacted me asking me for help all along, his ripping attempt was getting blocked by encountering that LEC error! Heh.

The game works fine since it boots the first data track it finds and the game code points to proper LBA offsets when it comes time to play redbook audio tracks. So yeah, it works!

You're right to question this though, messing around like that could create trouble! For example, when it comes to Ys IV, if you don't burn 3 seconds of postgap after the final audio track if you exclude the final data track, that final audio track won't get played properly! I used to exclude the final duplicate backup data track (track #32) and didn't know a postgap was somehow needed and that resulted in the final boss music not getting played properly on real NEC hardware! It was not an error you could catch with emulators! That's what you get with tinkering!

TurboXray

If a PCE CD game uses CDPLAY with the track argument, instead of start and end LBA sectors, you need an end track for it to work. This is why some games wouldn't play the audio last track, since the end track didn't exist for that request (cue needs to be fixed). I did this for the Megaman CD audio routines (used the first data track as the last track as well). It doesn't matter if the last track is audio or not - just that it exists.

 Pre-gaps and post-gaps (less so) need to be exact for games that use LBA sector mode for music play routine. The alignment will be off otherwise.   

 There's something else people should be aware of; the data track repeated as the last data track. There is a function/setup in the bios that holds the LBA offset of a redundant track - in case the primary track is have issues with data reads. The game can theoretically use the alt LBA offset to read from the redundant track instead. Whether any game actually does this or not, I don't know. But the software is there and setup for it (for those games that use that layout). Good to know of you're hacking games.

NightWolve

Quote from: TurboXray on 11/07/2015, 06:02 PMIf a PCE CD game uses CDPLAY with the track argument, instead of start and end LBA sectors, you need an end track for it to work. This is why some games wouldn't play the audio last track, since the end track didn't exist for that request (cue needs to be fixed). I did this for the Megaman CD audio routines (used the first data track as the last track as well). It doesn't matter if the last track is audio or not - just that it exists.
Ah, I see. So CDPLAY, the dynamic track play method (since it'll fetch the start LBA from the TOC), needs an ending marker, which yeah, you get by using the start of the next track and subtract off pregap sector markings if need be.

Interesting thing, I did solve the problem with the last audio track by adding a POSTGAP 03:00 to the CUE file when I used to share Ys IV images without the backup tracks to compress them better. That also served as a solution somehow instead of putting data track #32 back with its 03:00 PREGAP command.

Nowadays though, I wouldn't do that, there was a smarter way. Just copy a translated/patched data track #2 to data track #32, and then resize it to what the original size is, case closed. I still get a more compressed archive of one data track and all its audio tracks. And when the batch file unpacks all the APE zipped waves, it will copy the data track to track #32 and resize it proper.

If you think about it, it ought to work properly without an extra track, it should be smart enough to use the Leadout LBA for the final track. Think about audio CDs. It plays them correctly on the other hand. I guess CD player code does advanced analysis on the TOC but that doesn't happen with general game code or something.

Quotethe data track repeated as the last data track. There is a function/setup in the bios that holds the LBA offset of a redundant track - in case the primary track is have issues with data reads. The game can theoretically use the alt LBA offset to read from the redundant track instead. Whether any game actually does this or not, I don't know. But the software is there and setup for it (for those games that use that layout). Good to know of you're hacking games.
Yeah, I learned that long ago from David Michel who assured me it's a backup track and the design is there to read sectors from it if the first data track #2 starts to encounter read errors.

It's a fine idea I guess, but not really needed if discs are properly maintained. I suppose if you have the space, it doesn't hurt.

Incidentally, they ran out of space for Ys IV and fully maxed out the CD! Every last byte that you can burn back then was used up! Most of the ADPCM voice-acting is clipped out when it comes to track #32, so it really wouldn't do any good if track #2 went bad and the system switched to using track #32 throughout the rest of the game. I would presume it'd eventually crash when it came time to play the voice-acting, or you just won't hear anything when it tries.

NecroPhile

Any flags for Implode, Meteor Blaster DX, Hypernova Blast, Mysterious Song, and Revival Chase too?
Ultimate Forum Bully/Thief/Saboteur/Clone Warrior! BURN IN HELL NECROPHUCK!!!

NightWolve

Funny you mentioned that. I was chatting with Bernie the other day, and he pointed out that Implode and Meteor Blaster DX were in Squaresoft74's TOC database already, what TurboRip relies on for the titling magic. Since I got the block going for Arkhan's games now, I can do it for BT's on those.

However, the rest of the games you mentioned, I don't have TOCs for. Somebody would have to ask Old Rover or somebody with an original copy of the game to get them to me if they want/care for such a block.

If you run "TurboRip /cue /1" with the CD in your first drive, it'll just dump a TOC and CUE then exit. That'd be one way to get it to me. It's also printed in the DOS emulated window, so if you know how to mark text/copy it, that's another way.

Bernie


ccovell

Hi, I just downloaded v 1.40 (Aug 24) and it aborts ("Progress 100% LBA: 013229 to 013255 / Read Error: Unknown read failure") after track 2 on Road Spirits (the first disc I tried), where the old version of TurboRip (1.00) I was using rips the whole disc just fine.

I'm using a WinXP Toshiba laptop with built-in DVD/CD-R drive.

NightWolve

Hi Chris, thanks for the feedback.

Try this copy, the next release candidate, which fixed many problems, johnnykonami's Windows 7 total failure and crashing on tbone's Windows 10 machine.

https://www.ysutopia.net/software/TurboRipTest.zip

Hopefully the error trapping improvements show something more with your CD if it still fails.

ccovell

Quote from: NightWolve on 11/21/2015, 09:11 PMHopefully the error trapping improvements show something more with your CD if it still fails.
It did!  It did!  It changed the failure text from white to red.   :-\

NightWolve

Hah! Yeah, figured if I'm still gonna be stuck as a command-line app, might as well use all possible features and bloody red text for errors was a nice touch I thought that was available all along. :mrgreen:

Well, that sucks... I do know what the problem is though since it's happening right at the end of the data track 2: the 2 second pregap transition area is right there for audio track 3 and determining the true final sector of any track on a CD is a problem because of this concept of gaps/index 0 and varying behavior among drives...

If the CD format was designed proper, the TOC would also tell you the true end LBA, not just the start, or the LBA of the next track - 1 would always work which is what you use now but also have to take gaps into consideration which normally will be added if transitioning to different track types (data to audio and vice versa).

So anyway, when you read a few sectors near that gap area between track 2 and 3 on a PCE CD, some drives error out 1 sector away or 3 sectors away from where these 2 or 3 second areas begin... It actually varies, so I predict this and zero out those sectors before writing them to disk in the data track 2 file. This is how I ensure the track file sizes are universal. Right now the prediction is at 5 or 7 sectors, but I guess in your case I'd have to increase that value to trap the error, zero out the sectors and continue instead of giving up. Old version of TurboRip you say works maybe had a bigger value.

The other basic problem is I need to implement "Abort, Retry, Ignore" so you could manually ignore your way out of a few unreadable sectors at the end of a data track that would be zero anyway instead of TurboRip trying to automatically do it... It's on my To-Do list, but I never got around to it.

Brings up another idea, the /track command. Another workaround to this is you'd use /track=3, then /track=4 to rip the CD the rest of the way each time. But I should implement support for greater than '>' so a user could do "/track>2" as a parameter to rip the remainder of the tracks. Ah, the fun of figuring out textual input commands for cmd apps when they should just be a GUI. :)

Well, I'm tied up for all of this week and on, but maybe I'll try to compile another version with that constant increased for you to test with next weekend I hope. Maybe 9 sectors is the right amount to be willing to subtract off if read errors are encountered at the end of a track with a gap complication. There's some rule in Redbook/Yellowbook docs about at least 9 out of every 10 sectors need to be encoded with indexing in the Q subchannel data, so my choice of 5 was too low.

Psycho Punch

/Turbo_Rip_Home_Brew_Block.png

I understand why you did that but I'm someone who likes to have backups of his games available. Not everyone is going to upload homebrew to the internet just because they can dump their discs. I should have done this sooner ](*,)
This Toxic Turbo Troll & Clone Warrior calls himself "Burning Fight!!" on Neo-Geo.com
For a good time, reach out to: aleffrenan94@gmail.com or punchballmariobros@gmail.com
Like DildoKobold, dildos are provided free of charge, no need to bring your own! :lol:
He also ran scripts to steal/clone this forum which blew up the error logs!
I had to delete THOUSANDS of error log entries cause of this nutcase!
how_to_spell_ys_sign_origin_ver.webp

NightWolve

Like I said, it was just a courtesy on their behalf. I'm aware 100% would not upload their rip, but a % will or have and this is simply a polite message asking them to show mercy to our homebrew guys, if you will, let them know CDs are still for sale, etc. Of course it's technically a futile gesture, but why not until their CDs fully sell out.

BTW, if you look back further in this thread, TurboRip was going to fail anyway because of the larger-than-usual pregap OldMan assigned in the CD layout. All TurboRip versions will fail, ImgBurn crashes on track analysis and Alcohol raises errors too.

IMG

I need to implement Q-subchannel analysis to detect the true end LBA for data track 2, a smart track mode analysis in other words to be able to skip those gap sectors and reflect the size in the CUE file... It's somewhat complicated, but I got a long way to go before I could rip a CD like Insanity... So, long story short, there was already an unintended copy protection. ;)

OldMan

QuoteAll versions will fail, ImgBurn crashes on track analysis and Alcohol raises errors too.
Yeah, but he used clone cd (I recognized the ccd format) and uploaded it anyway.

I almost downloaded it to see if it worked. Trying to use ccd images was a pain, iirc. Though maybe you could extract the tracks from it.... :\

NightWolve


OldMan

QuoteI didn't wanna mention what works...
Have you ever used clone-cd on a pce cd? It took me several (ie >10. I went to bed leaving it running) hours to backup a few of my games :)

Gotta admire the guys perseverence, though. Maybe next time, we will use oddball gaps for all the tracks ;)

--edit--
You really can't stop it. Pirates gonna pirate....

NightWolve

Update: Another RC/test build of TurboRip to solve Chris' problem. For anyone else, it's here:

ysutopia.net/software/TurboRipTest.zip

What I did is this: On the final 1 - 27 (+10) sectors that are left to read of a data track that is preceded by an audio track and will have at least 2 seconds of pregap (150 sectors), if a read error is trapped, I switch to reading every sector one by one instead of burst mode (reading 27*2352 sectors at a time=64KB) writing them individually. If I hit a read error, I write a null/zeroed out sector (2048 bytes for cooked, 2352 for raw). The only logical solution, really, as well as informing the user.

I think that should do it for this issue and it's the more proper way to have behaved around these transition areas, I just haven't gotten around to dealing with this more thoroughly and I still have much work ahead to truly detect gap areas instead of assuming PCE CD rules as I currently do!

I learned a while back in testing/debugging that when you do CD burst reads 2-27 (>1) sectors at a time and you hit a gap area where one sector is marked data while the next one is marked audio, that throws an error, whereas reading raw one sector at a time could avoid it. I really don't understand though why errors are caused in reading *near* the gap area as PCE CDs do follow strict size rules which are subtracted off proper. Track #3's LBA - 150 sectors - 1 is the true stop LBA for data track #2, and you should be able to read that final sector at 13279 in Road Spirit's case, but not always with certain drives/OS conditions/etc.

Anyway, hopefully it works a lot better now! :)

Quote from: TheOldMan on 11/23/2015, 09:51 PMHave you ever used clone-cd on a pce cd? It took me several (ie >10. I went to bed leaving it running) hours to backup a few of my games :)
Eh, once or twice. Not much. That is the format David originally sent me that weird PC-FX CD given its very unusual indexing, but now I have an original to play with thanks to him.

It's really powerful and one of the few formats that can store/copy CD+Graphics CDs, Karaoke CDs, etc. given the R-W-Q subchannel (.sub) dumping file, but it's usually excessive/not needed for most gaming CDs of the retro console era.

MNKyDeth

Should I be using TurboRip 1.40 or TurboRiptest currently?

I have been backing up all my game discs to my server over the past day or two. I have been using the TurboRipTest but started wondering if it even really mattered.

Also, I have the Dracula X repro that I got from Bernie with the English translation. Does turborip leave everything intact or does it put the disc back to JP format? I would assume it just rips it as it is but I have no real idea so figured I would ask.

NightWolve

You're fine - the latest TurboRipTest is the best. It's fully working, has all the latest improvements thanks to johnnykonami and my recent interaction with Chris Covell which restored it to full speed for older OSes going back to WinXP.

It's pretty much an RC, I'm ready to recompile with a new version, but I gotta write up stuff in the ReadMe which I haven't gotten around to, that's the hold up. I'm gonna hold off on binchunking and OGG support till later.

On Dracula X, it would rip anything in tact of course. No changes like that would ever be planned.

MNKyDeth

I actually just tried Dungeon Master: Theron's Quest and TurboRip says this cd-rom does not match any TOC in the internal videogame database.

This is the original disc, is there something I can do to help resolve this issue?

NightWolve

Oh awesome, guess it's another pressing or something. So it's not a really good bootleg - it's a 100% original ? Yeah, if so, PM or paste the TOC file. It's printed both on screen and written to file, you can grab it from either.

The data I have for this game is as follows:

US:

Track (01)   audio   00:02:00   LBA=000000
Track (02)   data   00:45:09   LBA=003234
Track (03)   audio   01:30:05   LBA=006605
Track (04)   audio   04:20:66   LBA=019416
Track (05)   audio   05:50:74   LBA=026174
Track (06)   audio   07:04:14   LBA=031664
Track (07)   audio   07:39:64   LBA=034339
Track (08)   audio   10:42:56   LBA=048056
Track (09)   audio   13:33:70   LBA=060895
Track (10)   audio   15:06:27   LBA=067827
Track (11)   audio   17:00:74   LBA=076424
Track (12)   audio   19:30:61   LBA=087661
Track (13)   audio   22:34:54   LBA=101454
Track (14)   audio   24:17:13   LBA=109138
Track (15)   audio   24:50:39   LBA=111639
Track (16)   audio   27:56:70   LBA=125620
Track (17)   audio   30:16:40   LBA=136090
Track (18)   audio   37:30:02   LBA=168602
Track (19)   data   38:50:37   LBA=174637
Leadout: 39:31:34   LBA 177709

Japan

Track (01)   audio   00:02:00   LBA=000000
Track (02)   data   00:49:65   LBA=003590
Track (03)   audio   01:34:61   LBA=006961
Track (04)   audio   04:23:37   LBA=019612
Track (05)   audio   05:53:17   LBA=026342
Track (06)   audio   07:06:32   LBA=031832
Track (07)   audio   07:42:07   LBA=034507
Track (08)   audio   10:44:74   LBA=048224
Track (09)   audio   13:36:13   LBA=061063
Track (10)   audio   15:08:45   LBA=067995
Track (11)   audio   17:03:17   LBA=076592
Track (12)   audio   19:33:04   LBA=087829
Track (13)   audio   22:36:72   LBA=101622
Track (14)   audio   24:19:31   LBA=109306
Track (15)   audio   24:52:57   LBA=111807
Track (16)   audio   27:59:13   LBA=125788
Track (17)   audio   30:18:58   LBA=136258
Track (18)   audio   37:37:39   LBA=169164
Track (19)   data   38:57:74   LBA=175199
Leadout: 39:38:71   LBA 178271

MNKyDeth

This is a 100% original. My dad bought this game when I was a teenager. It is one of the original games I got with my TurboDuo for xmas one year. It was purchased from Electronics Boutique.

Some pics.
IMG IMG IMG IMG IMG

---------Table of Contents--------

Track 01 Audio 00:02:00 LBA=000000
Track 02 Data  00:45:09 LBA=003234
Track 03 Audio 01:30:05 LBA=006605
Track 04 Audio 04:20:66 LBA=019416
Track 05 Audio 05:50:74 LBA=026174
Track 06 Audio 07:04:14 LBA=031664
Track 07 Audio 07:39:64 LBA=034339
Track 08 Audio 10:42:56 LBA=048056
Track 09 Audio 13:33:70 LBA=060895
Track 10 Audio 15:06:27 LBA=067827
Track 11 Audio 17:00:74 LBA=076424
Track 12 Audio 19:30:61 LBA=087661
Track 13 Audio 22:34:54 LBA=101454
Track 14 Audio 24:17:13 LBA=109138
Track 15 Audio 24:50:39 LBA=111639
Track 16 Audio 27:56:70 LBA=125620
Track 17 Audio 30:16:40 LBA=136090
Track 18 Audio 37:30:01 LBA=168601
Track 19 Data  38:50:36 LBA=174636

Leadout: 39:31:33       LBA 177708

----------------------------------


I copied from the terminal, if you prefer the entire .toc file I will get that for you.

NightWolve

Thanks man, and good anticipation, as I would've had to ask you for photos of the CD so I could get the catalog/reference# 'TGXCD1041' and use it for this alternate pressing's naming!

Next time you run TurboRip, your CD should be identified as:

"Dungeon Master - Theron's Quest {TGXCD1041} (U)"

https://www.ysutopia.net/software/TurboRipTest.zip

Give it a try, it's been recompiled in, should work if all went right.

MNKyDeth

I sent a PM as a reply but it all seems to work just fine now. Glad I could help.

brizio

Hi! I own all homebrew games released. If you are interested, I can provide you all missing TOCs in order to better protect all releases still not correclty recognized. I sent you a PM a few days ago with no answer. Let me know.

NightWolve

Hm, I looked in my PM inbox, it doesn't exist; if you sent a PM, it wasn't to me.

I remembered Bernie gave me Mysterious Song, so I'm adding that now. Whatever the case, feel free to dump the all the homebrews you say you have, and I'll take a look if I don't have them. Duplication would just serve for verification against what I have, so no harm done.

It's about time I push the RC release to the main download link after I finish the ReadMe. Gonna work on that now since I'm free.

brizio

Quote from: NightWolve on 02/24/2016, 07:19 PMHm, I looked in my PM inbox, it doesn't exist; if you sent a PM, it wasn't to me.
PM resent: now I can see it in my outbox.

brizio

Looks like you still have not received my personal message.

Anyway, please find below the data for the first few games, more will follow later.

Meteor Blaster DX Signature Edition

Track 01 Data  00:05:00 LBA=000225
Track 02 Audio 00:34:73 LBA=002473
Track 03 Audio 03:59:48 LBA=017823
Track 04 Audio 07:47:31 LBA=034906
Track 05 Audio 10:49:00 LBA=048525
Track 06 Audio 15:37:01 LBA=070126
Track 07 Audio 18:57:01 LBA=085126
Track 08 Audio 21:35:55 LBA=097030
Track 09 Audio 25:35:61 LBA=115036
Track 10 Audio 28:45:61 LBA=129286
Track 11 Audio 31:49:53 LBA=143078
Track 12 Audio 35:44:44 LBA=160694
Track 13 Audio 39:30:44 LBA=177644
Track 14 Audio 44:03:56 LBA=198131
Track 15 Audio 47:53:56 LBA=215381
Track 16 Audio 51:43:42 LBA=232617
Track 17 Audio 55:43:42 LBA=250617
Track 18 Audio 59:18:67 LBA=266767
Track 19 Audio 62:56:67 LBA=283117
Track 20 Audio 66:38:24 LBA=299724
Track 21 Audio 70:22:24 LBA=316524

Leadout: 72:58:27   LBA 328227

Mysterious Song - Second edition (pressed)

Track 01 Audio 00:02:00 LBA=000000
Track 02 Data  01:21:01 LBA=005926
Track 03 Audio 02:05:08 LBA=009233
Track 04 Audio 04:15:68 LBA=019043
Track 05 Audio 06:37:73 LBA=029698
Track 06 Audio 07:42:33 LBA=034533
Track 07 Audio 09:06:40 LBA=040840
Track 08 Audio 10:55:49 LBA=049024
Track 09 Audio 14:15:33 LBA=064008
Track 10 Audio 19:00:20 LBA=085370
Track 11 Audio 23:11:20 LBA=104195
Track 12 Audio 25:55:18 LBA=116493
Track 13 Audio 28:06:23 LBA=126323
Track 14 Audio 30:10:65 LBA=135665
Track 15 Audio 32:52:50 LBA=147800
Track 16 Audio 33:00:34 LBA=148384
Track 17 Audio 34:42:38 LBA=156038
Track 18 Audio 35:20:73 LBA=158923
Track 19 Audio 36:37:65 LBA=164690
Track 20 Audio 38:55:69 LBA=175044
Track 21 Audio 40:00:71 LBA=179921
Track 22 Audio 41:15:05 LBA=185480
Track 23 Audio 42:08:62 LBA=189512
Track 24 Audio 42:47:23 LBA=192398
Track 25 Audio 43:43:72 LBA=196647
Track 26 Audio 45:54:42 LBA=206442
Track 27 Audio 49:06:15 LBA=220815
Track 28 Audio 49:37:28 LBA=223153
Track 29 Audio 50:50:31 LBA=228631

Leadout: 54:44:30   LBA 246180

Revival Chase (please note needs to be confirmed as the copy I checked had read errors on the disc)

Track 01 Data  00:02:00 LBA=000000
Track 02 Audio 00:34:16 LBA=002416
Track 03 Audio 02:53:16 LBA=012841
Track 04 Audio 04:03:16 LBA=018091
Track 05 Audio 04:25:16 LBA=019741
Track 06 Audio 05:31:16 LBA=024691
Track 07 Audio 06:55:16 LBA=030991
Track 08 Audio 09:22:16 LBA=042016
Track 09 Audio 11:34:16 LBA=051916
Track 10 Audio 11:42:16 LBA=052516
Track 11 Audio 14:52:16 LBA=066766
Track 12 Audio 17:28:16 LBA=078466

Leadout: 20:04:05   LBA 090155

Ys 1&2 Special edition (Dark City Production, 2001)

Track 01 Audio 00:02:00 LBA=000000
Track 02 Data  00:09:00 LBA=000525
Track 03 Audio 02:04:06 LBA=009156
Track 04 Audio 05:12:66 LBA=023316
Track 05 Audio 07:52:04 LBA=035254
Track 06 Audio 09:58:26 LBA=044726
Track 07 Audio 13:45:51 LBA=061776
Track 08 Audio 18:26:50 LBA=082850
Track 09 Audio 20:11:09 LBA=090684
Track 10 Audio 21:50:09 LBA=098109
Track 11 Audio 24:47:42 LBA=111417
Track 12 Audio 27:25:22 LBA=123247
Track 13 Audio 29:24:05 LBA=132155
Track 14 Audio 31:24:32 LBA=141182
Track 15 Audio 34:25:06 LBA=154731
Track 16 Audio 37:22:56 LBA=168056
Track 17 Audio 39:33:51 LBA=177876
Track 18 Audio 41:23:40 LBA=186115
Track 19 Audio 42:17:19 LBA=190144
Track 20 Audio 43:56:07 LBA=197557
Track 21 Audio 47:50:15 LBA=215115
Track 22 Audio 50:35:13 LBA=227488
Track 23 Audio 52:29:46 LBA=236071
Track 24 Audio 54:26:62 LBA=244862
Track 25 Audio 55:29:72 LBA=249597
Track 26 Audio 58:02:10 LBA=261010
Track 27 Audio 60:47:67 LBA=273442
Track 28 Audio 64:00:56 LBA=287906
Track 29 Audio 64:59:37 LBA=292312
Track 30 Audio 65:18:13 LBA=293713
Track 31 Audio 66:18:16 LBA=298216
Track 32 Audio 66:58:25 LBA=301225
Track 33 Audio 67:20:48 LBA=302898
Track 34 Audio 67:39:65 LBA=304340
Track 35 Audio 68:13:27 LBA=306852
Track 36 Audio 68:24:18 LBA=307668
Track 37 Audio 69:29:47 LBA=312572
Track 38 Audio 70:40:65 LBA=317915
Track 39 Audio 71:46:44 LBA=322844
Track 40 Audio 72:01:58 LBA=323983
Track 41 Audio 72:49:31 LBA=327556
Track 42 Audio 74:04:69 LBA=333219
Track 43 Audio 74:44:18 LBA=336168

Leadout: 75:48:37   LBA 340987

I'll post here other TOCs as soon as possible.

NightWolve

Thanks brizio, will do.

Quick Turbo update. Per an idea I got from SamIAm's Amp thread and my experiences with discovering/understanding the really low preamp level practice of the industry when it came to redbook audio tracks in CDs in the 80's when I was adding an Ys MP3 player to my website, I added a basic auto-normalizing/max-amplifying feature to TurboRip which is now fully tested and working! Works very fast too, helps to use the OS's memory map features over traditional ReadFile/WriteFile APIs! Most especially compared to when the code was written in Visual Basic 6 and I had to wait 3 minutes per wave file for completion, heh.

Shared some of "C" source code below for the curious after converting the algorithm from VB6. Man, it was VERY hard to find useful code for this, I looked at Audacity's source and it exemplifies what I hate about C++, everything had been 'classed' up, dependencies all over the friggin' place, would be extremely hard to extract just the algorithm out of the damn thing for easy drop'n'use for your own project... :/ Granted, their algorithm is more complex than the basic one I found and used here, but I honestly just hate excessive use of C++... Added complexity AND plus doggedly much slower execution of code too... When it comes to that, you just want their compiled DLL and a competent API/interface to make use of the functionality, that is, if you're willing to bloat your software just for one little feature...

#include <global.h>
#include <NumbersASM.h>
#include "WAVE.h"

BOOLEAN WAVE_Normalize(LPSTR lpFileName) {
// VB6 norm/amplify generic algorithm by piuskerala
short LowestDataValue, HighestDataValue, *lp16BitSample, SampleValue;
WAVE_FILE_HEADER *WaveHeader;
MappedFile mFile;
long NewSampleValue;
float VolumeLevel;
DWORD Count;
// 1) Open wave file memory mapping style
if ( !MapFile(lpFileName, &mFile, FALSE, FALSE) )
return FALSE;
WaveHeader = (WAVE_FILE_HEADER*)mFile.lpMapAddress;
if ( WaveHeader->riffFORMAT != 'EVAW' ) // Verify it's a wave file
return FALSE;
// 2) Find highest and lowest value in wave data
LowestDataValue = HighestDataValue = 0;
lp16BitSample = MakePtr(short*, mFile.lpMapAddress, 44);
for ( Count = WaveHeader->dataSIZE / 2; Count; Count-- ) {
SampleValue = *lp16BitSample;
if ( SampleValue > HighestDataValue )
HighestDataValue = SampleValue;
if ( SampleValue < LowestDataValue )
LowestDataValue = SampleValue;
lp16BitSample++;
}
// 3) Normalize wave a sample at a time
VolumeLevel = ((HighestDataValue + -LowestDataValue) / 2) / (float)32767;
VolumeLevel = 1 / VolumeLevel; // For maximum amplitude/normalizing
lp16BitSample = MakePtr(short*, mFile.lpMapAddress, 44);
// 32767 is the highest integer(signed) value
for ( Count = WaveHeader->dataSIZE / 2; Count; Count-- ) {
NewSampleValue = RoundValue(*lp16BitSample * VolumeLevel);
if ( NewSampleValue > 32767 ) NewSampleValue = 32767;
if ( NewSampleValue < -32768 ) NewSampleValue = -32768;
*lp16BitSample = (short)NewSampleValue;
lp16BitSample++;
}
 // 4) Close wave file, we're done!
CloseMap(mFile);
return TRUE;
}

// Library stuff/functions copied/pasted for reference

#define CloseMap(hMap) {UnmapViewOfFile((hMap).lpMapAddress); CloseHandle((hMap).hFileMapping); CloseHandle((hMap).hFile);}


// Quickly takes a float value and returns it as a rounded 32-bit integer
// Use Agner Fog's quicker way with 2 FPU instructions

DWORD __fastcall RoundValue(float ftNumber) {
DWORD dwIntValue;
__asm {
LEA  EBX, dwIntValue      ;// EBX gets the address of our local variable
FLD  DWORD PTR [ftNumber] ;// Load floating point input on FPU stack
FISTP DWORD PTR [EBX]     ;// Store INT result at dwIntValue's address
XOR  EAX, EAX             ;// Intentional delay between FPU write and read
NOP
NOP
MOV  EAX, [EBX]  ;// Read rounded integer from @dwIntValue, return in EAX
}
}

BOOLEAN MapFile(LPSTR lpFileName, MappedFile *mFile, BOOLEAN ReadOnly, BOOLEAN SequentialScanOnRead) {
DWORD dwFlagsAndAttributes, CreateFileAccess, CreateFileMappingAccess, MapViewOfFileAccess;

dwFlagsAndAttributes = FILE_ATTRIBUTE_NORMAL;
if ( ReadOnly ) {
CreateFileAccess = GENERIC_READ;
CreateFileMappingAccess = PAGE_READONLY;
MapViewOfFileAccess = FILE_MAP_READ;
if ( SequentialScanOnRead )
dwFlagsAndAttributes |= FILE_FLAG_SEQUENTIAL_SCAN;
} else {
CreateFileAccess = GENERIC_READ | GENERIC_WRITE;
CreateFileMappingAccess = PAGE_READWRITE;
MapViewOfFileAccess = FILE_MAP_ALL_ACCESS;
}
mFile->hFile = CreateFileA(lpFileName, CreateFileAccess, 0, NULL,
OPEN_EXISTING, dwFlagsAndAttributes, NULL);
if ( mFile->hFile == INVALID_HANDLE_VALUE ) {
PrintLastError("Error: Cannot open '%s'", lpFileName);
return FALSE;
}
//mFile->qwFileSize.LowPart = GetFileSize(mFile->hFile, (LPDWORD)&mFile->qwFileSize.HighPart);
mFile->dwFileSize = GetFileSize(mFile->hFile, NULL);
// Create a file-mapping object for the file.
mFile->hFileMapping = CreateFileMappingA(mFile->hFile, // current file handle
                                NULL, // default security
                                CreateFileMappingAccess, // read/write permission
                                0,  // size of mapping object, high
                                0, // size of mapping object, low
                                NULL); // name of mapping object
if ( mFile->hFileMapping == NULL ) {
PrintLastError("Error: File mapping object creation failed");
return FALSE;
}
// Map the view and test the results.
mFile->lpMapAddress = (LPSTR)MapViewOfFile(mFile->hFileMapping, // handle to mapping object
                               MapViewOfFileAccess, // read/write permission
                               0, // high-order 32 bits of file offset
                               0, // low-order 32 bits of file offset
                               0); // number of bytes to map
if ( mFile->lpMapAddress == NULL ) {
PrintLastError("Error: File mapping view failed");
return FALSE;
}
return TRUE;
}


So, simply adding /normalize to your TurboRip extraction command now does the trick:

IMG

IMG

And now here is a sample result of auto-normalization/amplify, which is basically an automatic amplifying of the wave file based on picking the largest and smallest values and computing an amplification factor to minimize clipping.

IMG

The above is track 3 of "Ys IV: Dawn of Ys," the composition known as "Field." Ys IV is a prime example of the REALLY low preamp volume level found on CDs of that era which makes it REALLY hard to amplify it on your CD sound system without normalization processing. Those 2 red lines show abruptness or technical clipping, and with only 2 cases it's really minimal so you'll never notice or should care. Technical clipping will vary with every track, but it should be negligible to the human ear.

Now, it might be worth allowing your own amplification factor so that all tracks of a CD get amplified the same exact rate and show consistency in volume level though instead of only an automatic determination per track per the norm. I could adjust the parameter by allowing an equal sign and a value, so "/normalize=2" would simply double the volume level of all tracks without regard to clipping if you wanna be aggressive like SamIAm suggests in his thread and go heavy-metal loud preamp levels... But anyway, auto-normalization/amplify gets the ball rolling for something that should be pretty useful for now.

As I'm still behind on updates to the ReadMe, I'm still not doing a formal version update release, but in case anything happens between then and now, here is the best version of TurboRip so far with this new feature along with all the bug fixes/enhancements thanks to johnnykonami/Sam, and Chris Covell:

ysutopia.net/software/TurboRipTest.zip

:D

SuperPlay

Just had a play with the new version and the /normalize switch. Nice addition to an already top piece of software

Thanks as always for the continued work.

poponon

Anyone else try TurboRip with mounted image of Exile 2 -Wicked Phenomenon (US) ?   Mine keeps crashing - tried 2 different images from usually very reliable sources.

NightWolve

Hi poponon, download this link and try again:

ysutopia.net/software/TurboRipTest.zip

FYI, I just pushed the latest build up a second ago (still rebuilding my new Win7 system), it's the most stable version yet with all the fixes, and menu enhancements, etc. I'm sure you have one of the old builds that were crashing on tbone and others.

And f*ck it, about time I make it live since a few bad builds got loose:

ysutopia.net/software/TurboRip.zip (same one)

(Just not doing a good job juggling a release deemed BETA, RC, versus ready-to-go-can-stand-behind official new version with completed docs, etc.)

poponon

thank you so much! Worked like a charm! new colored text is pretty snazzy  8)

also don't fret man - I appreciate all your work on pce related stuff so much!

pceslayer

Thanks for the updated tool! Ive been using TurboRip for a long time now... heck my username was created just to join RIGG back in the day.

I recently started ripping my original discs to use on my XBOX with MednafenX_PCE and I noticed the latest version of TurboRip creates an .cue with incorrect file names. The issue only seems to be present when using the /XBOX flag.

NightWolve

/xbox switch will change letters to comply with the XBOX file system.

Unless that somehow broke, it's on purpose. There are more prohibited characters than Windows on XBOX and max length is 64 or something, so I crop length as well.

pceslayer

Quote from: NightWolve on 11/16/2016, 11:40 PM/xbox switch will change letters to comply with the XBOX file system.

Unless that somehow broke, it's on purpose. There are more prohibited characters than Windows on XBOX and max length is 64 or something, so I crop length as well.
What I meant was the binary filenames in the cue do not match the actual filename so the cue is bad.

Something like this...

Filename:
01-Exile (U).mp3
02-Exile (U).iso

CUE Filename:
Exile (U)-01.mp3
Exile (U)-02.iso

NightWolve

Aaaah, thanks, a minor bug, I'll fix it when I get some free time again. I had to rebuild my Win 7 box after buying a new quad-core cpu and ssd internal drive, I have lots of reinstalling to do to make it a development workstation again...

Random ProTip: Upgrading a CPU with Windows 7 appears to be dangerous, it's better to wipe the OS and do a clean install with the CPU in place!!!

At first it seemed to work, it detected the triple core CPU was changed, it installed the quad-core drivers, things worked fine for a few minutes, next thing I know, weird slowdowns start occurring, and destructive sector writes to the HDD also.... Every boot would launch chkdsk, and it marked a cluster or two bad, and some files were recovered... I dunno what got damaged, but it was scary as hell... So far, a fresh install though, everything is fine, but I'm a little nervous about it all going forward, whenever that'll be.

Anyway, thanks for the feedback.

esadajr

Most of those snafus are now history thanks to Windows 10. I moved a SSD (W10 Pro) through 3 laptops and everything works super.
Gaming since 1985

pceslayer

 I went to check for the latest version and the download link is gone... You had to pull the download?