[REL|Tool] RW Analyze

[steve-m]

Edit 07/22/04: I've updated my RW Analyze tool to version 0.4 beta.

 

 

[ Download RW Analyze ]

 

An excerpt from the description:

 

RW Analyze may be used to obtain a detailed visualization and analyzation of the contents of a RenderWare binary stream file (like .rws, .anm, .bsp, .dff, .dma or .txd), including the contents of individual components and the hierarchical structure of the stream. Furthermore the tool offers various modification possibilities.

 

Features:

• visualizes the hierarchical structured contents of a RW file
• contains a sophisticated content parsing engine with several error tracking functions
• enables to move, add, copy, change, clear, delete, import and export sections
• offers various data analyzation installments (hex view, data table, edit in (external) hex editor, and more...)

New in version 0.4 Beta:

• DFF to RW 3.4 Conversion (single and batch)
• TXD to XBOX Conversion (single and batch)
• Copy/Import/Export complex sections
• Append File
• Texture List
• icons for the popup menu
• improved section path box
• various internal changes and bug fixes

This tool is also used in combination with my tutorial on how to create character models for VC.

 

 

[TRN4L]

ALL HAIL KING STEVE OF TOOL MAKING! but seriously you're the man steve. i'm gonna start workin on a ped soon as i release my myriad lot, which is gonna be in about 5mins.

 

PS: its like 8 days to your bday right

[solo]

Very nice steve. You should start your own company

[Demarest]

Again, Just wanted to be the first to recommend the inevitable pin here

[ModelingMan]

You are right Dem this should be a definite pin. It is a very handy tool, and it is progressing very well. I don't think you have any more room for more stars SteveM

 

I was wondering Steve, is future releases of RWAnalyze gonna support XBOX DFFs and TXDs?

Edited March 17, 2004 by ModelingMan

[steve-m]

I was wondering Steve, is future releases of RWAnalyze gonna support XBOX DFFs and TXDs?

RW Analyze works with all RW file formats as long as they are in the standard format.

Since those XBOX files are compressed (or at least encoded) and I don't know how to read them - nope...

[jcab42]

Nice, I was able to use this tool to repare a dff that could not be opened by zmod. Mind you it wasn't locked, it was truelly damaged. After using the tool I still lost the texture names but not the mappings so it wasn't too hard to get the right textures back in. Cheers Steve, your program just saved me a lot of work.

[DexX]

i cant take it anymore. wtf does the "PLG" stand for at the end of everything???

 

btw, did you ever think of a way to fix the max 3 bump / reflection mapped material issues? ive been exporting vehicles directly from max 3, to rw analyze, to GTA, that work flawlessly, all they need is reflections. zmod wont open them.

 

anyway, great tool, its VERY handy

[steve-m]

 

i cant take it anymore. wtf does the "PLG" stand for at the end of everything???

 

btw, did you ever think of a way to fix the max 3 bump / reflection mapped material issues? ive been exporting vehicles directly from max 3, to rw analyze, to GTA, that work flawlessly, all they need is reflections. zmod wont open them.

 

anyway, great tool, its VERY handy

PLG stands for plugin, because these functions are optional extensions of the RenderWare engine. I used and modified the original names Criterion used. Take a look at the RW_Secs.ini file, the first set of names is used by default, the second are the unmodified original names and the third are those from RWA 0.2 beta x. If you want to switch, just change the Set parameter in the [Main] section. You can also change any of the names or add your own section types (e.g. when you found a new unknown section).

This file is loaded on startup of RWA, but you can reload it using Tools|Expert Settings|Reload Section Types.

 

Don't miss reading the readme.txt, I've explained every single function!

 

I haven't looked at the bump/reflection stuff very closely yet. Will do soon.

 

@jcab42: Yeah, one of it's many practical uses...

(You can even remove the ZMod lock easily, but I better don't mention that )

 

Thanks for all the s, now I'm fat...

[Demarest]

(You can even remove the ZMod lock easily, but I better don't mention that )

Lest your stars begin to resemble those of us mortals again

[JernejL]

Again, Just wanted to be the first to recommend the inevitable pin here

done

 

[markadams]

hi

 

thanks for a great tool.

 

i don't know if it's just me, but the addresses given in the tool are ahead by a value of 0x0C. ie. the address for the start of the clump is given as 0x0c instead of 0.

 

i realise that 12 is the length of the section header (4 bytes each chunkid/size/verNum), so the offset is actually giving the start of the next section instead of the section it's on.

 

maybe it's because i'm using win98?

 

thanks

mark

[steve-m]

The offset always indicates the start of the section's data, which comes after the 12 byte header. Also when you export a section, only its data is exported, not including the header. As you know each header includes the section size - also without the 12 byte of the header. It's logical, isn't it?

[DexX]

Steve, manhunt has hit the pc, if your interested, since it also runs on renderware.

The players in Manhunt have a new unknown section type in them, and dont appear to be openable using any of the current methods (deleting sections, or extracting the data).

The level .dff files are multi clump, could you add the option to export a single clump node, to an individual dff file? naturally an import function goes with that.

i can post some samples if you'd like. ciao!

dexx

[ghost of delete key]

<edit> you already knew that...

 

Does the game have separate animation data files, like GTA's .ifp files?

 

Because if not, then my edumacated guess is that the unknown types would have to do with animation data.

 

Otherwise, ???

Edited May 17, 2004 by ghost of delete key

[ModelingMan]

Hex data of an UNKNOWN section of cash_pc.dff:

 

0001A1A1  00 00 80 3F 00 00 80 3F  ..€?..€?0001A1A9  00 00 80 3F 00 00 80 3F  ..€?..€?0001A1B1  00 00 80 3F 00 00 00 00  ..€?....

 

 

The Manhunt DFFs also have an Extension under the Frame List with no data.

 

TXDs are easily openable.

[DexX]

 

Does the game have separate animation data files, like GTA's .ifp files?

 

Because if not, then my edumacated guess is that the unknown types would have to do with animation data.

 

Otherwise, ???

Yea, it actually has a master .ifp file for each level, appropriately named "allanims.ifp"

ill upload a couple samples of each later on, for the masses to examine.

[ghost of delete key]

Hex data of an UNKNOWN section of cash_pc.dff:

 

0001A1A1  00 00 80 3F 00 00 80 3F  ..€?..€?0001A1A9  00 00 80 3F 00 00 80 3F  ..€?..€?0001A1B1  00 00 80 3F 00 00 00 00  ..€?....

 

 

The Manhunt DFFs also have an Extension under the Frame List with no data.

 

 

Frame List = animation data

 

00 00 80 3F = 0.000 in decimal (unsigned long, right?)

 

so that looks like a portion of a transform matrix, starting with the root of something...

 

(0.000,0.000,0.000),(.,.,.),(.,.,.)etc.

 

...would be interesting to see exactly what it applies to.

[Nemesis1207]

This program is awesome, but I've got a few questions/requests. First of all, I'm not using this program for GTA3 media. I'm trying to work with a few different games, spread over a few different platforms that all use different versions of Renderware. So far I've encountered the following versions:

 

0x0C02FFFF 3.3.0.2

0x1003FFFF 3.4.0.3

0x1004FFFF 3.4.0.4

0x1005FFFF 3.4.0.5

0x1400FFFF 3.5.0.0

 

I'm trying to convert all these files up to version 3.7.0.2, which requires identifying all the changes that have been made between each version. This program is proving to be extremely useful in doing that.

 

Anyway, first of all, I just want to know what version(s) of renderware this program was made to support. I know it will display the overall structure of any valid renderware file on any recognised chunk, but what about the internal viewer in the righthand column? It'll just be useful to know what version it supports so I know when I can and can't trust what it reports.

 

 

Also, a few requests.

-First of all, a small one. If you could assign a hotkey to the dialog for changing the file version to a user-specified value, it would be a lot more convenient.

-I'm also dealing with some media from a GC game. The GC is a big-endian system, so you can see the problem I'm having. I know it'd be a reasonable amount of work, but if you could add the option to switch between big-endian and little-endian, both when viewing a file, and when saving it back, it would be extremely useful. That of course requires knowing the exact structure of each chunk in the file, which leads onto my next request.

-If you could expand this program to support a whole slew of renderware versions, and give the user the option of manually selecting one if they wanted to, it would be a lot more useful to people outside the GTA3 community.

-And, if you go as far as the one above, this one isn't much further. Allowing the user to convert the file to any of the known versions would make it a very poweful tool. The guys from Renderware themselves might get quite interested. Thier own stream viewer sucks ass.

[steve-m]

 

Anyway, first of all, I just want to know what version(s) of renderware this program was made to support. I know it will display the overall structure of any valid renderware file on any recognised chunk, but what about the internal viewer in the righthand column? It'll just be useful to know what version it supports so I know when I can and can't trust what it reports.

Well, I've only been experimenting with PC's GTA 3 and VC dff and txd files (RW 3.3 and 3.4) yet. Differences I have seen in these files:

- Geometry Structs (0x0F/0x01) of versions less than 3.4 have 12 byte ambient, diffuse and specular color after the header. 3.4 and above (also 3.7) don't have the colors anymore.

- The skin section in versions less than 3.3 is different. Furthermore I've discovered new values in some of the skin sections, I'll try to support them with the next RWA.

- I've also started looking at the Manhunt character model, the geometry data is in different order, which is unsupported by RWA (yet). I've also noticed some strange stuff going on in there, I hope to get that sorted out.

- Geometry and also the material split (0x50E) changes heavily between consoles, RWA only supports PC.

 

Basically you should notice if values aren't read correctly. And you can always check values by looking at the hex data.

 

 

First of all, a small one. If you could assign a hotkey to the dialog for changing the file version to a user-specified value, it would be a lot more convenient.

Already done, will be in the next version. The inofficial beta already has it, use this meanwhile (it's Ctrl+R).

 

 

If you could expand this program to support a whole slew of renderware versions, and give the user the option of manually selecting one if they wanted to, it would be a lot more useful to people outside the GTA3 community.

I don't have enough versions and files to compare and find out new stuff. If you discover new data and identify it's meaning, you can always tell me and I'll implement the interpreter (if possible).

Switching between versions is not necessary, since the version is included in every file, so the parser can always switch automatically (of course only if the differences are known).

 

 

And, if you go as far as the one above, this one isn't much further. Allowing the user to convert the file to any of the known versions would make it a very poweful tool. The guys from Renderware themselves might get quite interested. Thier own stream viewer sucks ass.

Like above, if you tell me the differences, I can implement a conversion function. I just don't have the time to find all differences myself. And since we can't know all possible sections and differences between version, a converter would never be perfect. It would require detailed documents from Criterion, ask them if they give you some.

 

About GC's big-endian: no way! This tool is made for the PC and for PC's RW versions, I don't want to imagine what work it would be to code multiple parsing routines just to support big-endian. Sorry, not with me.

 

Why not make your own tool?

[Nemesis1207]

Heh, pretty much all the answers I was expecting. Thanks for the beta. I'll give it a spin. It sounds like you already know about all the changes I've identified at this point. I'm still having problems converting all but the most simple models though, so I'll let you know what those differences are when I track them down. That damn first unknown in the geometry struct is also causing some problems. Some simple data only loads at all if I change that from 1 to 0, and I don't know why at this point. I'll probaby invest some more time hammering away at that value trying to figure out what it does just so I don't have to worry about it anymore. I tried that though and didn't get very far. At this point, I'm seriously considering disassembling the whole renderware library just to see how it uses that value.

 

If only the folks at renderware believed in backwards compatibility. Or error feedback. Hell, for that matter, if only they believe in open source. Meh, whatever. This is just a bit annoying though because this is the first time I've had a full SDK for the assets relating to any game I've worked on, and I'm finding it virtually useless.

 

 

Why not make your own tool?

Because I'm lazy, and you're doing such a good job of making it for me. =P

[steve-m]

I've no idea what this value is used for. The Manhunt player model even has a 2 there!

 

Well, somehow it is backwards compatible, VC works with 3.4, 3.3 and also 3.1 files. Even 3.7 files work, as long as you change the version to 3.4.

And if you are a registered developer you have access to a big support database and probably get all help you need.

 

BTW, what are you actually doing (or trying to do)?

[Nemesis1207]

My plan is to write a converter that'll bring all the different files I have up to version 3.7, so I can use the renderware tools that are available in the 3.7 SDK, which I have a copy of. That way I'll be able to view all these files without writing a mass of importers for them all. When it comes to editing, I can write importers for one single format (3.7) which I can do testing on, rather than targeting a whole slew of versions. I can also then use the offical renderware exporters to export the data once it has been edited, and then downgrade the file format back to versions the games I'm working with can load.

 

All of that will be damn easy, once I identify the changes that have been made between all these different file versions. The changes are minor, so it shouldn't take too much time to get a fair amount loading. Right now I don't have much time to work on this though, which is annoying.

 

Anyway, once I successfully convert a complex model manually, I'll write a proper converter to do it for me, and I'll see how many models I can load. When I get that perfect, I'll move onto the TXD files (which won't take much time at all), and then, the BSP files.

[DexX]

oo, an importer would be nice, but for what program? ill assume you mean 3ds Max. Just for the record, i know your doing this for more than vice city files, but vice itself doesnt load bsp's.

 

If you write a convertor for the RW files, what do you code in? and games are you working with anyway?

 

good luck

[Nemesis1207]

Yeah, I'm talking about 3DSMax. I don't know much about the VC files, or any of the GTA files for that matter, as I don't have the games. I code in C++, and the games I'm working on are the Sonic Heroes E3 demo, as well as the final versions for all three consoles. I'm also interested in working on the Sonic Heroes alpha, however the only guy who has it is being a bastard.

[Sodisna]

I have Sonic Heroes models

[steve-m]

I've just released the updated version 0.4 beta. An overview of the new functions:

 

 

• DFF to RW 3.4 Conversion (single and batch)

  applies the "UserData to Frame Convertion" and changes the file version to 3.4.0.3

  (only useful for versions higher than 3.4)

   

• TXD to XBOX Conversion (single and batch)

  converts a PC .txd file to XBOX format according to Ashdexx' guide

  Note: all textures must be in DXT1 or DXT3 compressed format!

   

• Copy/Import/Export complex sections

  What was only possible with empty and data sections, can now also be applied to sections containing child sections. The "Don't parse geometry section" option has been removed, since it isn't needed anymore.

   

• Append File

  allows you to append a valid RW file at the end of the current file. The file version is adjusted automatically.

   

• Texture List

  shows a list of textures either needed by or included in the current file and allows to save it as plain text list or as texture list for use with TXD Builder (both .txt)

  Hint: You can use the above mentioned "Append File" feature to temporarily append a .txd file to a .dff file and then check via "Texture List", if the texture dictionary meets the requirements of the model file.

   

• Improved section path box

  You can now scroll and select/copy the text. The word "Path:" was removed and the "/" delimiter replaced by ">".

[DexX]

neat. id been wondering when a new version would come along. The batch conversion for txd's is good, and i love the improved section import / export. Excellent work as always

[jcab42]

That's great that you found the code that gives the model prelight. I don't suppose you could make it switchable so we can actually use RW Anylize to turn the prelight on and off?

 

And if you could do the same for single sided/ two sided faces then there'd never be a need to export DFF's with 3ds Max ever again. I know I don't have to tell you how buggy the Max DFF exporter is.

[steve-m]

You got something wrong. Prelighting isn't just a switch, you need a 3d program to calculate a color for every vertex, based on how much light it gets (distance from a lamp).

Run RWA, press F8, deactivate "skip detailed data", then find the geometry section of a dff, and select it's struct section. If there are vertex colors in the model, scroll down the table on the right and you will see a listing of all the colors it has, as many colors as there are vertices.

 

I could code a function to remove all the vertex colors of a model, but that wouldn't make sense. What would make more sense is one to remove all the normals, since normals are only used for cars and peds, not buildings. They just waste space. If anyone wants that feature, I'll make it.