News About Shenmue 1 Modding

[dotsona]

Well I have some good news today, I now know exactly why Shenmue One produces broken models during the extraction process using 3d Ripper DX. I have hypothesized that Shenmue One's core engine hides polygons to help performance and today I can now confirm that it is true after speaking with an actual game developer. The concept is known as Back-face culling, and it simply means that the engine determines whether or not a polygon of a graphical object is visible, if the polygon is not visible to the user the engine does not draw the polygon. This is the technique that is being utilized in the Shenmue One Engine.

The Shenmue Two engine on the other hand does not use Back-face culling it uses a different culling technique. Which is why we are able to produce full models in Shenmue Two. After speaking with the game developer that I am in contact with, we talked about solutions on how to fix this issue. It turns out that there may in fact be a solution. Here is what he said.

Developer:
"I think that ripping from Shenmue 1 may remain difficult (as I think the issue is that Shenmue 1 has better culling of backfaces, so you'll always encounter the issue of faces not visible to the camera not being captured), but likely there is a way to hack this out of the emulator directly (the emulator relies on DX for the backface culling, most-likely, and normally you can force-override DX settings for an application and disable it). That's just a guesstimation from similar experiences on other (non-DC) emulator based model ripping"

I then go on to ask him what exactly we need to modify in order for this to work.

Developer:
"Well, Direct X has configuration settings per instance, so the changes would need to be effected on NullDC (3D Ripper DX merely talks to the instance of DirectX as set-up by NullDC and scraps the vertex / texture information from there).

So the two choices are really either hack the DX PVR plug-in from NullDC to disable backface culling, or use a tool to override the setting once the application is already running (The DirectX SDK used to ship with such tools, but I'm not sure if it still does)."

We are hoping that we can use the DirectX SDK to achieve this rather than have to modify the PVR plugin. Lastly he did mention we could run into one final problem, which we hope does not happen.

Developer:
"The bigger issue may be if the Shenmue code itself performs a pre-render calculation of visible faces before it sends vertex data to DirectX (in that case, 3D Ripper will never be able to rip it, as DirectX never get's sent the relevant data). Overriding that behavior would be quite out of scope of my abilities (I've dabbled in hacking DC games before, but never something so complex)."

I will keep you guys posted on the issue, if we can disable the culling feature we will be able to ripp full Shenmue One models with ease

[LawXiu]

Are the problems with the characters only? How about buildings and objects?

[dotsona]

Nope, the problem is universal in Shenmue One, we can rip characters and buildings, but they are not full models, so if I were going to rip the dojo for example, you would only get the front part.

Shenmue Two on the other hand can rip everything, characters, buildings and they are the full model. Right now we just need to fix the issue in Shenmue One so the models come out as a full mesh so they are not missing parts. Hopefully we can pull it off because Shenmue one has some of the best characters and environments.

[wude]

i hope so man , shenmue 1 is full of babes!

btw, maybe you should make a topic where we can exchange ripped models , and renders.

[LawXiu]

If SEGA had used the same methods in Shenmue 2 would we have seen less slow down and pop up characters?

[dotsona]

@LawXiu - I don't think so, I think the technique they are using now (in Shenmue 2) are probably more effective than the techniques used in Shenmue 1, I am not sure on the exact culling technique they are using in Shenmue 2, but I believe it has allowed them to pack alot more into Shenmue 2, considering the environments are more detailed and the whole game itself is on a much larger scale.

@finendandy - Yeah we will have that soon I am going to create a sticky that will contain a list of characters that will include the corresponding 3ds file and textures. We will also most likely have a renders section as well.

[Master Kyodai]

The DirectX SDKs are all still available for download from microsoft back to stone age. However, i was unable to identify the tool he mentioned. I think it might be PIX, but i have not found any option i could identify to be related to the backface culling issue. about the nullDC plugin - as i see it they come as dlls and i don't think they could be decompiled. So unless DarkRaziel gives us the source this is imho definitely a dead-end.

[dotsona]

Here is another update from the Developer, we disscused working with Demul utilizing opengl but that idea is now out. However he does have a new idea

The Developer:
I think Demul is out, as it provides really poor quality rips (but not due to the Emulator itself, the fact is it is OpenGL based and the tools for ripping geometry straight from OpenGL are of varying quality, none of which really meet our needs).

My new idea is to write a short D3D9 device hook that will force the cull mode to be none. This will severly hamper performance, but ensures that we are getting DirectX totally out of the way before we capture geometry. It may take some time to get it working though, since hooking and dll injection can be a bit of a pain

[LawXiu]

Here is another update from the Developer, we disscused working with Demul utilizing opengl but that idea is now out. However he does have a new idea

The Developer:


My new idea is to write a short D3D9 device hook

I was going to suggest an HB and a smooth textured flat form of w00d, but I spose he is the expert.

[dotsona]

Here is another update from the Developer, we disscused working with Demul utilizing opengl but that idea is now out. However he does have a new idea

The Developer:


My new idea is to write a short D3D9 device hook

I was going to suggest an HB and a smooth textured flat form of w00d, but I spose he is the expert.

[LawXiu]

Come on then, explain. What is it he's writing? And please dont say " a program".

[dotsona]

Actually to be completely honest I am not entirely sure, I don't know much about dll injection, I am pretty sure he is hacking into nulldc and forcing the engine to turn off culling by changing the boolean value to false. However I am not sure on the process that he is taking since my experience in that area is null

[LawXiu]

Ok I'm partially with you. I'm the kind of person that likes to break things open to see how they work. They never work again though. I studied electronic engineering at college. We programed simple programs but nothing on this level. I understand binary and hexadecimal etc but I assume they don't come into play here.

The new forum's keeping me glued to the dojo that's for sure.

[dotsona]

Yeah the forum has kept me glued haha, Shenmue keeps distracting me from my Senior thesis in College (which is due Jan 30th). It has almost become a slight obsession. I suppose it's the drive to see Shenmue still exist through the fans, my goal is to release all the characters (main characters) and major scenes from the game to the fans, once that is complete I can finally rest (After a few comics are made of course) I just love modding, it's fun, been doing it since I was a kid.

[gothguy5969]

I was having a conversation about this with my friend and he told me this and thought everyone would love to see it.

(7:50:39 PM) Snowy: someone should tell him/her to play the games
(7:50:44 PM) Snowy: and have it render it onto a computer
(7:50:48 PM) Snowy: or a dreamcast emulator
(7:50:59 PM) Snowy: that as it's compiling the files
(7:51:04 PM) Snowy: it automatically dumps said files
(7:51:23 PM) Snowy: so it would be able to extract the 3d files as well as the sounds and textures
(7:51:34 PM) Snowy: getting them to render via other means
(7:51:41 PM) Snowy: is what is going to be the annoying part
(7:53:21 PM) Snowy: u would also need to see how the files actually react in same engine
(7:53:34 PM) Snowy: but if you can get the actual 3d models extracted via any means
(7:53:39 PM) Snowy: it makes the process a shit ton easier
(7:53:46 PM) Snowy: and if said models are done the same in other games
(7:53:57 PM) Snowy: then that opens up modding to all of the games compatible with it
(7:54:44 PM) Snowy: so you might be able to extract other game models
(7:54:54 PM) Snowy: (jet set radio) : D

Hope our convo was of help.