Quantcast

Jump to content

» «
Photo

[REL|Tool] RW Analyze

72 replies to this topic
steve-m
  • steve-m

  • Feroci
  • Joined: 26 Jul 2002

#1

Posted 14 March 2004 - 11:18 PM

Edit 07/22/04: I've updated my RW Analyze tool to version 0.4 beta.

[ Download RW Analyze ]


An excerpt from the description:

RW Analyze may be used to obtain a detailed visualization and analyzation of the contents of a RenderWare binary stream file (like .rws, .anm, .bsp, .dff, .dma or .txd), including the contents of individual components and the hierarchical structure of the stream. Furthermore the tool offers various modification possibilities.

Features:
  • visualizes the hierarchical structured contents of a RW file
  • contains a sophisticated content parsing engine with several error tracking functions
  • enables to move, add, copy, change, clear, delete, import and export sections
  • offers various data analyzation installments (hex view, data table, edit in (external) hex editor, and more...)
New in version 0.4 Beta:
  • DFF to RW 3.4 Conversion (single and batch)
  • TXD to XBOX Conversion (single and batch)
  • Copy/Import/Export complex sections
  • Append File
  • Texture List
  • icons for the popup menu
  • improved section path box
  • various internal changes and bug fixes
This tool is also used in combination with my tutorial on how to create character models for VC.

user posted image

TRN4L
  • TRN4L

    Novus Ordo Seclorum

  • Members
  • Joined: 10 Jul 2003

#2

Posted 15 March 2004 - 03:55 AM

colgate.gif ALL HAIL KING STEVE OF TOOL MAKING! but seriously you're the man steve. i'm gonna start workin on a ped soon as i release my myriad lot, which is gonna be in about 5mins. colgate.gif

PS: its like 8 days to your bday right smile.gif

solo
  • solo

    Homie

  • Members
  • Joined: 25 Jul 2002

#3

Posted 15 March 2004 - 07:26 AM

Very nice steve. You should start your own company smile.gif

Demarest
  • Demarest

    what could be

  • BUSTED!
  • Joined: 12 Jul 2003

#4

Posted 15 March 2004 - 11:47 AM

Again, Just wanted to be the first to recommend the inevitable pin here smile.gif

ModelingMan
  • ModelingMan

    Crackalacking!

  • Feroci
  • Joined: 23 Jan 2004

#5

Posted 17 March 2004 - 12:24 AM Edited by ModelingMan, 17 March 2004 - 12:30 AM.

You are right Dem this should be a definite pin. It is a very handy tool, and it is progressing very well. I don't think you have any more room for more stars SteveM smile.gif

I was wondering Steve, is future releases of RWAnalyze gonna support XBOX DFFs and TXDs?

steve-m
  • steve-m

  • Feroci
  • Joined: 26 Jul 2002

#6

Posted 17 March 2004 - 03:50 PM

QUOTE (ModelingMan @ Mar 17 2004, 01:24)
I was wondering Steve, is future releases of RWAnalyze gonna support XBOX DFFs and TXDs?

RW Analyze works with all RW file formats as long as they are in the standard format.
Since those XBOX files are compressed (or at least encoded) and I don't know how to read them - nope... confused.gif

jcab42
  • jcab42

    Custom Planet Builder

  • Members
  • Joined: 21 Sep 2002

#7

Posted 17 March 2004 - 11:48 PM

Nice, I was able to use this tool to repare a dff that could not be opened by zmod. Mind you it wasn't locked, it was truelly damaged. After using the tool I still lost the texture names but not the mappings so it wasn't too hard to get the right textures back in. Cheers Steve, your program just saved me a lot of work. cookie.gif cookie.gif cookie.gif cookie.gif cookie.gif

DexX
  • DexX

    Black Hat

  • Feroci
  • Joined: 16 May 2002

#8

Posted 18 March 2004 - 05:22 AM

i cant take it anymore. wtf does the "PLG" stand for at the end of everything???

btw, did you ever think of a way to fix the max 3 bump / reflection mapped material issues? ive been exporting vehicles directly from max 3, to rw analyze, to GTA, that work flawlessly, all they need is reflections. zmod wont open them.

anyway, great tool, its VERY handy cookie.gif

steve-m
  • steve-m

  • Feroci
  • Joined: 26 Jul 2002

#9

Posted 18 March 2004 - 04:46 PM

QUOTE (ashdexx @ Mar 18 2004, 06:22)
i cant take it anymore. wtf does the "PLG" stand for at the end of everything???

btw, did you ever think of a way to fix the max 3 bump / reflection mapped material issues? ive been exporting vehicles directly from max 3, to rw analyze, to GTA, that work flawlessly, all they need is reflections. zmod wont open them.

anyway, great tool, its VERY handy cookie.gif

PLG stands for plugin, because these functions are optional extensions of the RenderWare engine. I used and modified the original names Criterion used. Take a look at the RW_Secs.ini file, the first set of names is used by default, the second are the unmodified original names and the third are those from RWA 0.2 beta x. If you want to switch, just change the Set parameter in the [Main] section. You can also change any of the names or add your own section types (e.g. when you found a new unknown section).
This file is loaded on startup of RWA, but you can reload it using Tools|Expert Settings|Reload Section Types.

Don't miss reading the readme.txt, I've explained every single function!

I haven't looked at the bump/reflection stuff very closely yet. Will do soon.

@jcab42: Yeah, one of it's many practical uses... wink.gif
(You can even remove the ZMod lock easily, but I better don't mention that biggrin.gif )

Thanks for all the cookie.gif s, now I'm fat... tounge2.gif

Demarest
  • Demarest

    what could be

  • BUSTED!
  • Joined: 12 Jul 2003

#10

Posted 18 March 2004 - 05:04 PM

QUOTE (ST.MU @ Mar 18 2004, 11:46)
(You can even remove the ZMod lock easily, but I better don't mention that biggrin.gif )

Lest your stars begin to resemble those of us mortals again wink.gif

JernejL
  • JernejL

    Big Homie

  • Feroci
  • Joined: 11 Mar 2002

#11

Posted 19 March 2004 - 11:44 AM

QUOTE (Demarest @ Mar 15 2004, 12:47)
Again, Just wanted to be the first to recommend the inevitable pin here smile.gif

done smile.gif

markadams
  • markadams

    Player Hater

  • Members
  • Joined: 22 Apr 2004

#12

Posted 03 May 2004 - 10:20 AM

hi

thanks for a great tool.

i don't know if it's just me, but the addresses given in the tool are ahead by a value of 0x0C. ie. the address for the start of the clump is given as 0x0c instead of 0.

i realise that 12 is the length of the section header (4 bytes each chunkid/size/verNum), so the offset is actually giving the start of the next section instead of the section it's on.

maybe it's because i'm using win98?

thanks
mark

steve-m
  • steve-m

  • Feroci
  • Joined: 26 Jul 2002

#13

Posted 04 May 2004 - 01:15 AM

The offset always indicates the start of the section's data, which comes after the 12 byte header. Also when you export a section, only its data is exported, not including the header. As you know each header includes the section size - also without the 12 byte of the header. It's logical, isn't it?

DexX
  • DexX

    Black Hat

  • Feroci
  • Joined: 16 May 2002

#14

Posted 17 May 2004 - 05:10 PM

Steve, manhunt has hit the pc, if your interested, since it also runs on renderware.
The players in Manhunt have a new unknown section type in them, and dont appear to be openable using any of the current methods (deleting sections, or extracting the data).
The level .dff files are multi clump, could you add the option to export a single clump node, to an individual dff file? naturally an import function goes with that.
i can post some samples if you'd like. ciao!
dexx

ghost of delete key
  • ghost of delete key

    aggronormical

  • The Connection
  • Joined: 27 Dec 2003
  • None

#15

Posted 17 May 2004 - 07:57 PM Edited by ghost of delete key, 17 May 2004 - 08:02 PM.

<edit> you already knew that...

Does the game have separate animation data files, like GTA's .ifp files?

Because if not, then my edumacated guess is that the unknown types would have to do with animation data.

Otherwise, ???

ModelingMan
  • ModelingMan

    Crackalacking!

  • Feroci
  • Joined: 23 Jan 2004

#16

Posted 17 May 2004 - 10:01 PM

Hex data of an UNKNOWN section of cash_pc.dff:
CODE
0001A1A1  00 00 80 3F 00 00 80 3F  ..?..?
0001A1A9  00 00 80 3F 00 00 80 3F  ..?..?
0001A1B1  00 00 80 3F 00 00 00 00  ..?....


The Manhunt DFFs also have an Extension under the Frame List with no data.

TXDs are easily openable.

DexX
  • DexX

    Black Hat

  • Feroci
  • Joined: 16 May 2002

#17

Posted 18 May 2004 - 01:04 AM

QUOTE
Does the game have separate animation data files, like GTA's .ifp files?

Because if not, then my edumacated guess is that the unknown types would have to do with animation data.

Otherwise, ???

Yea, it actually has a master .ifp file for each level, appropriately named "allanims.ifp" tounge.gif
ill upload a couple samples of each later on, for the masses to examine.

ghost of delete key
  • ghost of delete key

    aggronormical

  • The Connection
  • Joined: 27 Dec 2003
  • None

#18

Posted 18 May 2004 - 06:38 AM

QUOTE (ModelingMan @ May 17 2004, 18:01)
Hex data of an UNKNOWN section of cash_pc.dff:
CODE
0001A1A1  00 00 80 3F 00 00 80 3F  ..?..?
0001A1A9  00 00 80 3F 00 00 80 3F  ..?..?
0001A1B1  00 00 80 3F 00 00 00 00  ..?....


The Manhunt DFFs also have an Extension under the Frame List with no data.


Frame List = animation data

00 00 80 3F = 0.000 in decimal (unsigned long, right?)

so that looks like a portion of a transform matrix, starting with the root of something...

(0.000,0.000,0.000),(.,.,.),(.,.,.)etc.

...would be interesting to see exactly what it applies to.

Nemesis1207
  • Nemesis1207

    Player Hater

  • Members
  • Joined: 10 Oct 2003

#19

Posted 09 June 2004 - 05:42 AM

This program is awesome, but I've got a few questions/requests. First of all, I'm not using this program for GTA3 media. I'm trying to work with a few different games, spread over a few different platforms that all use different versions of Renderware. So far I've encountered the following versions:

0x0C02FFFF 3.3.0.2
0x1003FFFF 3.4.0.3
0x1004FFFF 3.4.0.4
0x1005FFFF 3.4.0.5
0x1400FFFF 3.5.0.0

I'm trying to convert all these files up to version 3.7.0.2, which requires identifying all the changes that have been made between each version. This program is proving to be extremely useful in doing that.

Anyway, first of all, I just want to know what version(s) of renderware this program was made to support. I know it will display the overall structure of any valid renderware file on any recognised chunk, but what about the internal viewer in the righthand column? It'll just be useful to know what version it supports so I know when I can and can't trust what it reports.


Also, a few requests.
-First of all, a small one. If you could assign a hotkey to the dialog for changing the file version to a user-specified value, it would be a lot more convenient.
-I'm also dealing with some media from a GC game. The GC is a big-endian system, so you can see the problem I'm having. I know it'd be a reasonable amount of work, but if you could add the option to switch between big-endian and little-endian, both when viewing a file, and when saving it back, it would be extremely useful. That of course requires knowing the exact structure of each chunk in the file, which leads onto my next request.
-If you could expand this program to support a whole slew of renderware versions, and give the user the option of manually selecting one if they wanted to, it would be a lot more useful to people outside the GTA3 community.
-And, if you go as far as the one above, this one isn't much further. Allowing the user to convert the file to any of the known versions would make it a very poweful tool. The guys from Renderware themselves might get quite interested. Thier own stream viewer sucks ass.

steve-m
  • steve-m

  • Feroci
  • Joined: 26 Jul 2002

#20

Posted 10 June 2004 - 04:15 PM

QUOTE (Nemesis1207)
Anyway, first of all, I just want to know what version(s) of renderware this program was made to support. I know it will display the overall structure of any valid renderware file on any recognised chunk, but what about the internal viewer in the righthand column? It'll just be useful to know what version it supports so I know when I can and can't trust what it reports.

Well, I've only been experimenting with PC's GTA 3 and VC dff and txd files (RW 3.3 and 3.4) yet. Differences I have seen in these files:
- Geometry Structs (0x0F/0x01) of versions less than 3.4 have 12 byte ambient, diffuse and specular color after the header. 3.4 and above (also 3.7) don't have the colors anymore.
- The skin section in versions less than 3.3 is different. Furthermore I've discovered new values in some of the skin sections, I'll try to support them with the next RWA.
- I've also started looking at the Manhunt character model, the geometry data is in different order, which is unsupported by RWA (yet). I've also noticed some strange stuff going on in there, I hope to get that sorted out.
- Geometry and also the material split (0x50E) changes heavily between consoles, RWA only supports PC.

Basically you should notice if values aren't read correctly. And you can always check values by looking at the hex data.

QUOTE (Nemesis1207)
First of all, a small one. If you could assign a hotkey to the dialog for changing the file version to a user-specified value, it would be a lot more convenient.

Already done, will be in the next version. The inofficial beta already has it, use this meanwhile (it's Ctrl+R).

QUOTE (Nemesis1207)
If you could expand this program to support a whole slew of renderware versions, and give the user the option of manually selecting one if they wanted to, it would be a lot more useful to people outside the GTA3 community.

I don't have enough versions and files to compare and find out new stuff. If you discover new data and identify it's meaning, you can always tell me and I'll implement the interpreter (if possible).
Switching between versions is not necessary, since the version is included in every file, so the parser can always switch automatically (of course only if the differences are known).

QUOTE (Nemesis1207)
And, if you go as far as the one above, this one isn't much further. Allowing the user to convert the file to any of the known versions would make it a very poweful tool. The guys from Renderware themselves might get quite interested. Thier own stream viewer sucks ass.

Like above, if you tell me the differences, I can implement a conversion function. I just don't have the time to find all differences myself. And since we can't know all possible sections and differences between version, a converter would never be perfect. It would require detailed documents from Criterion, ask them if they give you some. sarcasm.gif

About GC's big-endian: no way! This tool is made for the PC and for PC's RW versions, I don't want to imagine what work it would be to code multiple parsing routines just to support big-endian. Sorry, not with me. suicidal.gif

Why not make your own tool? tounge.gif

Nemesis1207
  • Nemesis1207

    Player Hater

  • Members
  • Joined: 10 Oct 2003

#21

Posted 11 June 2004 - 03:30 AM

Heh, pretty much all the answers I was expecting. Thanks for the beta. I'll give it a spin. It sounds like you already know about all the changes I've identified at this point. I'm still having problems converting all but the most simple models though, so I'll let you know what those differences are when I track them down. That damn first unknown in the geometry struct is also causing some problems. Some simple data only loads at all if I change that from 1 to 0, and I don't know why at this point. I'll probaby invest some more time hammering away at that value trying to figure out what it does just so I don't have to worry about it anymore. I tried that though and didn't get very far. At this point, I'm seriously considering disassembling the whole renderware library just to see how it uses that value.

If only the folks at renderware believed in backwards compatibility. Or error feedback. Hell, for that matter, if only they believe in open source. Meh, whatever. This is just a bit annoying though because this is the first time I've had a full SDK for the assets relating to any game I've worked on, and I'm finding it virtually useless.

QUOTE
Why not make your own tool?

Because I'm lazy, and you're doing such a good job of making it for me. =P

steve-m
  • steve-m

  • Feroci
  • Joined: 26 Jul 2002

#22

Posted 11 June 2004 - 01:12 PM

I've no idea what this value is used for. The Manhunt player model even has a 2 there!

Well, somehow it is backwards compatible, VC works with 3.4, 3.3 and also 3.1 files. Even 3.7 files work, as long as you change the version to 3.4.
And if you are a registered developer you have access to a big support database and probably get all help you need.

BTW, what are you actually doing (or trying to do)?

Nemesis1207
  • Nemesis1207

    Player Hater

  • Members
  • Joined: 10 Oct 2003

#23

Posted 15 June 2004 - 03:06 AM

My plan is to write a converter that'll bring all the different files I have up to version 3.7, so I can use the renderware tools that are available in the 3.7 SDK, which I have a copy of. That way I'll be able to view all these files without writing a mass of importers for them all. When it comes to editing, I can write importers for one single format (3.7) which I can do testing on, rather than targeting a whole slew of versions. I can also then use the offical renderware exporters to export the data once it has been edited, and then downgrade the file format back to versions the games I'm working with can load.

All of that will be damn easy, once I identify the changes that have been made between all these different file versions. The changes are minor, so it shouldn't take too much time to get a fair amount loading. Right now I don't have much time to work on this though, which is annoying.

Anyway, once I successfully convert a complex model manually, I'll write a proper converter to do it for me, and I'll see how many models I can load. When I get that perfect, I'll move onto the TXD files (which won't take much time at all), and then, the BSP files.

DexX
  • DexX

    Black Hat

  • Feroci
  • Joined: 16 May 2002

#24

Posted 15 June 2004 - 10:18 AM

oo, an importer would be nice, but for what program? ill assume you mean 3ds Max. Just for the record, i know your doing this for more than vice city files, but vice itself doesnt load bsp's.

If you write a convertor for the RW files, what do you code in? and games are you working with anyway?

good luck smile.gif

Nemesis1207
  • Nemesis1207

    Player Hater

  • Members
  • Joined: 10 Oct 2003

#25

Posted 15 June 2004 - 01:52 PM

Yeah, I'm talking about 3DSMax. I don't know much about the VC files, or any of the GTA files for that matter, as I don't have the games. I code in C++, and the games I'm working on are the Sonic Heroes E3 demo, as well as the final versions for all three consoles. I'm also interested in working on the Sonic Heroes alpha, however the only guy who has it is being a bastard.

Sodisna
  • Sodisna

    Punk-ass Bitch

  • BUSTED!
  • Joined: 08 Jul 2004

#26

Posted 08 July 2004 - 01:49 PM

I have Sonic Heroes models

steve-m
  • steve-m

  • Feroci
  • Joined: 26 Jul 2002

#27

Posted 22 July 2004 - 09:02 PM

I've just released the updated version 0.4 beta. An overview of the new functions:

QUOTE (readme.txt)
  • DFF to RW 3.4 Conversion (single and batch)
    applies the "UserData to Frame Convertion" and changes the file version to 3.4.0.3
    (only useful for versions higher than 3.4)
  • TXD to XBOX Conversion (single and batch)
    converts a PC .txd file to XBOX format according to Ashdexx' guide
    Note: all textures must be in DXT1 or DXT3 compressed format!
  • Copy/Import/Export complex sections
    What was only possible with empty and data sections, can now also be applied to sections containing child sections. The "Don't parse geometry section" option has been removed, since it isn't needed anymore.
  • Append File
    allows you to append a valid RW file at the end of the current file. The file version is adjusted automatically.
  • Texture List
    shows a list of textures either needed by or included in the current file and allows to save it as plain text list or as texture list for use with TXD Builder (both .txt)
    Hint: You can use the above mentioned "Append File" feature to temporarily append a .txd file to a .dff file and then check via "Texture List", if the texture dictionary meets the requirements of the model file.
  • Improved section path box
    You can now scroll and select/copy the text. The word "Path:" was removed and the "/" delimiter replaced by ">".

DexX
  • DexX

    Black Hat

  • Feroci
  • Joined: 16 May 2002

#28

Posted 23 July 2004 - 02:05 PM

neat. id been wondering when a new version would come along. The batch conversion for txd's is good, and i love the improved section import / export. Excellent work as always

jcab42
  • jcab42

    Custom Planet Builder

  • Members
  • Joined: 21 Sep 2002

#29

Posted 07 August 2004 - 07:35 AM

That's great that you found the code that gives the model prelight. I don't suppose you could make it switchable so we can actually use RW Anylize to turn the prelight on and off?

And if you could do the same for single sided/ two sided faces then there'd never be a need to export DFF's with 3ds Max ever again. I know I don't have to tell you how buggy the Max DFF exporter is.

steve-m
  • steve-m

  • Feroci
  • Joined: 26 Jul 2002

#30

Posted 07 August 2004 - 01:06 PM

You got something wrong. Prelighting isn't just a switch, you need a 3d program to calculate a color for every vertex, based on how much light it gets (distance from a lamp).
Run RWA, press F8, deactivate "skip detailed data", then find the geometry section of a dff, and select it's struct section. If there are vertex colors in the model, scroll down the table on the right and you will see a listing of all the colors it has, as many colors as there are vertices.

I could code a function to remove all the vertex colors of a model, but that wouldn't make sense. What would make more sense is one to remove all the normals, since normals are only used for cars and peds, not buildings. They just waste space. If anyone wants that feature, I'll make it.




1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users