Shenmue 2 Mod Topic Swapping "Unswappable" Music Tracks (DC)

(Modding and other series related projects)

Shenmue 2 Mod Topic Swapping "Unswappable" Music Tracks (DC)

Postby shengoro86 » Wed Dec 09, 2015 1:15 pm

Hi all,

So a little while ago Ziming came to me with a problem that couldn't be solved.

There are two tracks for Shenmue 2, Dreamcast, that need to be recorded.

The intro tracks for Kowloon and Guilin.

However, those tracks can not be swapped with the main menu music for some odd reason.

Until now...

Here is my brain dump on this matter.

****************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************

What we want to do
Be able to record these two music tracks:
0002_023.SND - Guilin
0003_001.SND - Kowloon

How do we do it?
We swap these tracks with the main menu music: X:/MISC/0017_012.SND
1. Rename the target song to 0017_012.SND
2. Overwrite the 0017_012.SND file in X:/MISC/
3. Test

What was the problem?
So every track (not 100% tested) in Shenmue 2 seems to work fine for this method, except for the two requested to me.

How do we solve this?
OK So I actually managed to get the Guilin track to play! :D

Here is the technique I employed to do this.
So when examining the files, I noticed that each track has its own header data.

Here is a comparison:
Main Menu Music:
Image

Guilin Intro Music:
Image


Now you see the difference in the first 16 hex block?

So all I did was take the first 16 characters from the main menu track and overwrote the first 16 blocks of the guilin track with it.

Then I renamed the Guilin track to 0017_012.SND and put it in the X:/MISC/ folder and voila.

Now there is a problem:
The track only plays up to 30 seconds on the main menu and then resets on real hardware.
I tested it on NullDC and it keeps playing after the 30 seconds but the song is distorted.

So I would like to make this hypothesis right now:
The way the track is played could be determined in the first 16 bytes of data.

NOTE: If you copy the second set of 16 bytes, the entire track breaks in the menu. Which shows me even more how these two weird tracks work.

So right now it works half and half. I am still looking for a more full solution.

Thoughts?
Last edited by shengoro86 on Wed Dec 09, 2015 3:53 pm, edited 1 time in total.

shengoro86 has received a thanks from: Giorgio
User avatar
shengoro86
Admin - Shenmue500K
Shenmue 500K Staff
 
Joined: August 2004
Location: New Jersey, USA
PSN: Shengoro86
XBL: Rock Is Sponge
Favorite title: Shenmue IIx
Currently playing: Shenmue III (PC)

Re: Shenmue 2 Tutorial: Swapping "Unswappable" Music Tracks

Postby BlueMue » Wed Dec 09, 2015 3:39 pm

Those two tracks are really stubborn. I remember messing around with them quite a bit and I never got them to play at the title screen. There are a few unused tracks in Shenmue aswell that just won't play no matter where I put them and what I tried.

The results from changing the entire first line of the header are a great step forward! Too bad it doesn't play it right though. I tired it with the Kowloon opening too (wich is 0003_001) and it sounds glitchy right from the start.
The fact that it crashes the Dreamcast makes me believe even more in my old theory that these tracks are too much for the sound RAM. The tracks are definately the largest in file size (both around 750 Kb) and while that isn't close to the 2 Mb sound RAM that the DC has it definately takes a lot of it.

What I think is that they somehow have a slightly different sound driver for those two cutscenes to fit those large music files into the RAM. One way to finally get these tracks to play as long as desired would be to lock up the cutscenes. This is however no easy thing to do either. I've tried it, as it does work with some cutscenes, but in these cases they lock up before the music is loaded. Best way would be to somehow bring over the sound driver from the cutscene to the title screen. I haven't looked deep into the code for that but there must be something in the files of those cutscenes that makes a difference.

By the way, I find the thread title slightly misleading with it being called a "tutorial". As long as we are looking for a way to get this done it's really rather just brainstorming and discussions LOL.

BlueMue has received a thanks from: shengoro86
User avatar
BlueMue
Machine Gun Fist
Machine Gun Fist
 
Joined: August 2008
Location: Germany
Favorite title: Shenmue II

Re: Shenmue 2 Mod Topic Swapping "Unswappable" Music Tracks

Postby shengoro86 » Wed Dec 09, 2015 3:54 pm

I changed the thread title.

Yeah i think you are right about the RAM issues. Since it works on an emulator.

I am looking into locking cutscenes at the moment.
User avatar
shengoro86
Admin - Shenmue500K
Shenmue 500K Staff
 
Joined: August 2004
Location: New Jersey, USA
PSN: Shengoro86
XBL: Rock Is Sponge
Favorite title: Shenmue IIx
Currently playing: Shenmue III (PC)

Re: Shenmue 2 Mod Topic Swapping "Unswappable" Music Tracks

Postby NeoShredder » Wed Dec 09, 2015 8:03 pm

I don't know what the state of modding (and certainly PC emulation) is with the Xbox version, but if the theory is lack of RAM would it be possible to try it on that version of the game?

Also, it looks like the difference of that line you've highlighted between the two files is just 3 different characters (08 09 and 0A). If my maths is correct (disregarding the two original sets, both which don't work), that means 6 possible combinations of the two sets (of three characters) you could try to swap, see if that does anything....?

Finally, just an observation, looks like the 0A byte in that offset doesn't change the character on the ASCII representation on the right between the two files whereas for 08 and 09 it does. Don't know if that's significant or not? (I don't know much at all about HEX)
User avatar
NeoShredder
Machine Gun Fist
Machine Gun Fist
 
Joined: April 2005
Location: England, UK
PSN: NeoShredder5
Favorite title: Shenmue II

Re: Shenmue 2 Mod Topic Swapping "Unswappable" Music Tracks

Postby Sappharad » Wed Dec 09, 2015 8:48 pm

shengoro86 wrote:So I would like to make this hypothesis right now:
The way the track is played could be determined in the first 16 bytes of data.

Thoughts?

The value at 0x8 is just a 32-bit Little Endian value which contains the size of the file.

Examples:
0000_000.SND is 9,160 bytes in size. Value is set to 0x23C8, which is 9160.
0001_002.SND is 382,848 bytes in size. Value is set to 0x5D780 which is 382848.
0003_001.SND is 769,184 bytes in size. Value is set to 0xBBCA0, which is 769184.
0017_012.SND is 536,032 bytes in size. Value is set to 0x082DD4, which is 536,020... which happens to be 12 bytes smaller than the actual file size, this seems to be the only file with this problem out of the random ones I picked to look at.

You're just making it load less data, which is probably why it won't play. I don't think the files are too big, but rather it's probably loaded to a hard-coded location which doesn't have room for everything you want to load.
Sappharad
Man Mo Acolyte
Man Mo Acolyte
 
Joined: October 2015
Favorite title: Shenmue II

Re: Shenmue 2 Mod Topic Swapping "Unswappable" Music Tracks

Postby shengoro86 » Thu Dec 10, 2015 11:07 am

Sappharad wrote:
shengoro86 wrote:So I would like to make this hypothesis right now:
The way the track is played could be determined in the first 16 bytes of data.

Thoughts?

The value at 0x8 is just a 32-bit Little Endian value which contains the size of the file.

Examples:
0000_000.SND is 9,160 bytes in size. Value is set to 0x23C8, which is 9160.
0001_002.SND is 382,848 bytes in size. Value is set to 0x5D780 which is 382848.
0003_001.SND is 769,184 bytes in size. Value is set to 0xBBCA0, which is 769184.
0017_012.SND is 536,032 bytes in size. Value is set to 0x082DD4, which is 536,020... which happens to be 12 bytes smaller than the actual file size, this seems to be the only file with this problem out of the random ones I picked to look at.

You're just making it load less data, which is probably why it won't play. I don't think the files are too big, but rather it's probably loaded to a hard-coded location which doesn't have room for everything you want to load.



Yup! I found this out while testing and not updating this thread.

I might have a better solution so we can record these tracks.

I may have a solution to lock the cutscene so we can just record that way.
User avatar
shengoro86
Admin - Shenmue500K
Shenmue 500K Staff
 
Joined: August 2004
Location: New Jersey, USA
PSN: Shengoro86
XBL: Rock Is Sponge
Favorite title: Shenmue IIx
Currently playing: Shenmue III (PC)

Re: Shenmue 2 Mod Topic Swapping "Unswappable" Music Tracks

Postby Sappharad » Thu Dec 10, 2015 9:59 pm

shengoro86 wrote:Yup! I found this out while testing and not updating this thread.

I might have a better solution so we can record these tracks.

I may have a solution to lock the cutscene so we can just record that way.

I haven't tested this, so it might be a bad suggestion, but... Would it be easier if you just used Shenmue Passport?

The audio driver is AICADRV.BIN in the MISC directory on both games. Copy the newer driver from Shenmue 2 into Shenmue Passport's MISC folder, replacing the old one, and see if passport is willing to play the newer files then?

Otherwise your solution is probably faster than the remaining options.
Sappharad
Man Mo Acolyte
Man Mo Acolyte
 
Joined: October 2015
Favorite title: Shenmue II

Re: Shenmue 2 Mod Topic Swapping "Unswappable" Music Tracks

Postby shengoro86 » Fri Dec 11, 2015 12:20 pm

Sappharad wrote:
shengoro86 wrote:Yup! I found this out while testing and not updating this thread.

I might have a better solution so we can record these tracks.

I may have a solution to lock the cutscene so we can just record that way.

I haven't tested this, so it might be a bad suggestion, but... Would it be easier if you just used Shenmue Passport?

The audio driver is AICADRV.BIN in the MISC directory on both games. Copy the newer driver from Shenmue 2 into Shenmue Passport's MISC folder, replacing the old one, and see if passport is willing to play the newer files then?

Otherwise your solution is probably faster than the remaining options.


Before I read this, I stumbled upon this file. Ill be testing it later today.

Sorry for my late responses, I am currently moving out of my apartment. Right now I have no internet at home :X
User avatar
shengoro86
Admin - Shenmue500K
Shenmue 500K Staff
 
Joined: August 2004
Location: New Jersey, USA
PSN: Shengoro86
XBL: Rock Is Sponge
Favorite title: Shenmue IIx
Currently playing: Shenmue III (PC)

Re: Shenmue 2 Mod Topic Swapping "Unswappable" Music Tracks

Postby shengoro86 » Fri Dec 11, 2015 12:31 pm

So it seems that disc 4 has 3 AICADRV.BINs
\MISC\
\OUTRUN\
\SCENE\04\MOVIE\

I know the outrun theme is really long so that is testable but ill try the movie driver first.

BTW, these files are NOT identical at hex level, despite file size.

There seems to be different versions of this driver.

Image
User avatar
shengoro86
Admin - Shenmue500K
Shenmue 500K Staff
 
Joined: August 2004
Location: New Jersey, USA
PSN: Shengoro86
XBL: Rock Is Sponge
Favorite title: Shenmue IIx
Currently playing: Shenmue III (PC)

Re: Shenmue 2 Mod Topic Swapping "Unswappable" Music Tracks

Postby shengoro86 » Fri Dec 11, 2015 12:41 pm

No good.

Using the other AICADRIV.bin files gives the same result with and without hex editing the headers of the tracks.
User avatar
shengoro86
Admin - Shenmue500K
Shenmue 500K Staff
 
Joined: August 2004
Location: New Jersey, USA
PSN: Shengoro86
XBL: Rock Is Sponge
Favorite title: Shenmue IIx
Currently playing: Shenmue III (PC)

Re: Shenmue 2 Mod Topic Swapping "Unswappable" Music Tracks

Postby IlDucci » Sat Dec 12, 2015 7:11 pm

I don't think the AICADRV from the Movie folder will do any good. The only thing on that folder that should play is SFD audio from the Digest Movie/Saturn Unreleased.
IlDucci
Master of the Three Blades
Master of the Three Blades
 
Joined: February 2012
Location: Spain
Favorite title: Shenmue

Re: Shenmue 2 Mod Topic Swapping "Unswappable" Music Tracks

Postby Giorgio » Sat Dec 12, 2015 7:53 pm

Some observations. The file TIT_SND.BIN in the MISC folder is about which sounds and music to play in the title screen. (For example, you can copy the file 0002_023.snd to the MISC folder, and then change the text 0017_012 to 0002_023 in the .bin file, without having to rename and overwrite any file. This, of course, will make a different music to play in the title screen.)

I tried to copy the SMAP data bytes section from the MAPINFO.BIN of the KRF1 area folder (from SMAP's offset plus 194 bytes) and overwrite the one in the TIT_SND.BIN (that is overwrite the whole data). However, nothing noticeable happened.

It is still a mystery to me why changing the 4 bytes at offset 0x08 (that is the value of the bytes size) of the 0002_023.snd makes it being played. Why changing the size to a lower value makes the music track playable? Not entirely playable, though; why? And why is it playable in the cinematic and not in the title screen? It does not seem to use a different AICA driver version in the cinematic. Maybe somewhere a check is being made for the size of the music track to be played in the title screen? Trying to find coherency, but with so little evidence and ignorance on the cause.

Interestingly, there is a reference of TIT_SND.BIN in the 1ST_READ.BIN:
Code: Select all
....TITLE.GZ....0002S000.sbt....tit_snd.bin.0002S000.smp....base: %03d %03d %03d %03d...ofst:     %03d %03d %03d....>.......


UPDATE:
----------
Success!

Managed to play the entire track in the title screen, without interruptions! And it's looping. The latest observation was the most crucial (in combination with the second paragraph above).

Will soon post a tutorial on how to do it.

Giorgio has received a thanks from: Ziming
User avatar
Giorgio
"After Burner...Great!"
"After Burner...Great!"
 
Joined: February 2009
Favorite title: Shenmue
Currently playing: Yakuza series

Tutorial how to play any music track in the title screen

Postby Giorgio » Sat Dec 12, 2015 8:36 pm

Tutorial how to play any music track
in the title
(more precisely, the menu) screen of Shenmue II.


How to play "0003_001.SND" (Kowloon intro)
STEPS:
Code: Select all
1) In Shenmue II's root folder, find and open the file "1ST_READ.BIN" with a hex editor (and leave it open).
2) Navigate to \SCENE\03\SOUND; find and copy (CTRL+C) the file "0003_001.SND".
3) Navigate (go up) to MISC and paste (CTRL+V) (the file "0003_001.SND").
4) In the MISC folder, find and open the file "TIT_SND.BIN" with a hex editor (and leave it open).
5) Navigate to \SCENE\03\0200; find and open the file "MAPINFO.BIN" with a hex editor.
5.1) Find (CTRL+F) the text-string reference of SMAP (in the file "MAPINFO.BIN").
5.2) Place the cursor before the character S of the text-string SMAP.
5.3) Press the combination of keys CTRL+E (to select a block of bytes).
5.4) Select the radio-button of the "Length:" legend.
5.5) In the text-area, replace the highlighted text with 194.
5.6) Make sure the radio-button of the "hex" legend is selected (below).
5.7) Click on the "OK" button.
5.8) Copy (CTRL+C) the selected/highlighted block of bytes (in your clipboard).
6) In the hex editor, select the TIT_SND.BIN tab.
6.1) Select all (CTRL+A) the bytes.
6.2) Paste (CTRL+V) the content of your clipboard.
6.3) Select/highlight the text-string 0200A000 and copy (CTRL+C) it.
7) In the hex editor, select the 1ST_READ.BIN tab.
7.1) Find (CTRL+F) the text-string reference of "tit_snd.bin".
7.2) Look (with your eyes) for the text-string "0002S000.sbt" before the "tit_snd.bin" one, select/highlight the "0002S000" bit of it and replace it with "0200A000".
7.3) Look (with your eyes) for the text-string "0002S000.smp" after the "tit_snd.bin" one, select/highlight the "0002S000" bit of it and replace it with "0200A000".


How to play "0002_023.SND" (Guilin intro)
STEPS:
Code: Select all
1) In Shenmue II's root folder, find and open the file "1ST_READ.BIN" with a hex editor (and leave it open).
2) Navigate to \SCENE\04\SOUND; find and copy (CTRL+C) the file "0002_023.SND".
3) Navigate (go up) to MISC and paste (CTRL+V) (the file "0002_023.SND").
4) In the MISC folder, find and open the file "TIT_SND.BIN" with a hex editor (and leave it open).
5) Navigate to \SCENE\04\KRF1; find and open the file "MAPINFO.BIN" with a hex editor.
5.1) Find (CTRL+F) the text-string reference of SMAP (in the file "MAPINFO.BIN").
5.2) Place the cursor before the character S of the text-string SMAP.
5.3) Press the combination of keys CTRL+E (to select a block of bytes).
5.4) Select the radio-button of the "Length:" legend.
5.5) In the text-area, replace the highlighted text with 194.
5.6) Make sure the radio-button of the "hex" legend is selected (below).
5.7) Click on the "OK" button.
5.8) Copy (CTRL+C) the selected/highlighted block of bytes (in your clipboard).
6) In the hex editor, select the TIT_SND.BIN tab.
6.1) Select all (CTRL+A) the bytes.
6.2) Paste (CTRL+V) the content of your clipboard.
6.3) Select/highlight the text-string 0500A000 and copy (CTRL+C) it.
7) In the hex editor, select the 1ST_READ.BIN tab.
7.1) Find (CTRL+F) the text-string reference of "tit_snd.bin".
7.2) Look (with your eyes) for the text-string "0002S000.sbt" before the "tit_snd.bin" one, select/highlight the "0002S000" bit of it and replace it with "0500A000".
7.3) Look (with your eyes) for the text-string "0002S000.smp" after the "tit_snd.bin" one, select/highlight the "0002S000" bit of it and replace it with "0500A000".


Note (valid for both STEPS sections above): the step 6.3 is potentially unusable, however, it was left just to emphasize where the text-string "0200A000"/"0500A000" comes from when it used later on (steps 7.2-7.3). Also, in this tutorial the hex editor named HxD was used.

How to play any other music track
STEPS:
Code: Select all
1) Navigate to \SCENE\0X\SOUND (where X is a number); find and copy (CTRL+C) the .SND file (of the music track you want to play).
2) Navigate (go up) to MISC and paste (CTRL+V) (the .SND file).
3) In the MISC folder, find and open the file "TIT_SND.BIN" with a hex editor.
3.1) Select/highlight the text-string 0017_012.
3.2) Replace the selected/highlighted text-string with the prefix of the .SND file.


This is my first mod/hack for Shenmue 2!

Giorgio has received 5 thanks from: BlueMue, NeoShredder, shengoro86, Yokosuka, Ziming
User avatar
Giorgio
"After Burner...Great!"
"After Burner...Great!"
 
Joined: February 2009
Favorite title: Shenmue
Currently playing: Yakuza series

Re: Shenmue 2 Mod Topic Swapping "Unswappable" Music Tracks

Postby NeoShredder » Mon Dec 14, 2015 9:13 am

Incredible work guys! :)
So let's hear it then! :P
User avatar
NeoShredder
Machine Gun Fist
Machine Gun Fist
 
Joined: April 2005
Location: England, UK
PSN: NeoShredder5
Favorite title: Shenmue II

Re: Shenmue 2 Mod Topic Swapping "Unswappable" Music Tracks

Postby Sappharad » Mon Dec 14, 2015 9:22 pm

Giorgio wrote:It is still a mystery to me why changing the 4 bytes at offset 0x08 (that is the value of the bytes size) of the 0002_023.snd makes it being played. Why changing the size to a lower value makes the music track playable? Not entirely playable, though; why? And why is it playable in the cinematic and not in the title screen? It does not seem to use a different AICA driver version in the cinematic. Maybe somewhere a check is being made for the size of the music track to be played in the title screen?

Based on your two posts, I can provide another theoretical answer to consider.

The .snd files with a letter in the middle of the numbers instead of an underscore are the packs of background sounds loaded for that area. (You probably know that, but if not, just mentioning it. They're standard DTPK files with an extra header at the beginning that defines information about the sound effects. If you delete the extra data you can convert them to DSF using the SND to DSF tools.)

Perhaps those two cutscene tracks have references to the sound effects used during those scenes, and the reason they start to play back partially loaded is because those references can't be resolved to the corresponding sound effects? Just a theory.

Congratulations on figuring that stuff out.
Sappharad
Man Mo Acolyte
Man Mo Acolyte
 
Joined: October 2015
Favorite title: Shenmue II

Next

Return to Community Projects

Who is online

Users browsing this forum: No registered users and 1 guest

Powered by phpBB © 2000-
ShenmueDojo.net