Quantcast

Jump to content

» «
Photo

Compiled Path Nodes

70 replies to this topic
steve-m
  • steve-m

  • Members
  • Joined: 26 Jul 2002

#1

Posted 10 September 2005 - 12:04 AM Edited by steve-m, 14 November 2006 - 10:08 PM.

Anyone up for a new riddle?

Been looking at the nodes*.dat files, which I guess are the compiled path nodes, so the game doesn't have to process them and can load faster. There are 64 files, each one for an area of 750 * 750 units, starting with the SW corner. Also, these node files are streamed by the game, only the active area and those surrounding it are loaded, not more.

Now the problem is, these files have a very ugly and hard to decode format. There is a line in the stream.ini, "dontbuildpaths", which might have disabled the internal path compiler, but removing it doesn't seem to have any effect, maybe someone could confirm that. So unless someone is able to re-activate it, we have to try to decode the format and write our own path compiler.

And this is where the fun starts. There are tons of parameters and flags in those files, so I assume there is even some advanced stuff stored there, such as traffic lights and such. What would be a nightmare of course.

- - - - -

Update: file format specification moved to the wiki

- - - - -

Download empty path nodes (745 Bytes)

Use these to get rid of all paths, they contain an empty header and the section 4 data. See here on how to use them.

- - - - -

And btw, SA only uses the nodes*.dat files inside an img, not those from the data\paths\ directory.

Hammer83
  • Hammer83

    Hustler

  • Members
  • Joined: 11 Jan 2003

#2

Posted 14 September 2005 - 05:44 AM

Can't say I can contribute much to what you already discovered. Basically I can cofirm that what you describe here is correct. I was able to create a parser for node*.dat files according to your structure and was able to plot the paths:

http://suchy63.rhlmo...es/nodesdat.gif (WARNING: 1 MB image)

So, since the game uses these files for paths, are we to assume it ignores the entries in the ipl files? These paths seem complete to me, even interiors (like rinks) are there, so I guess ipl paths are ignored.

JernejL
  • JernejL

    Big Homie

  • Feroci Racing
  • Joined: 11 Mar 2002

#3

Posted 14 September 2005 - 10:17 AM

so the dontbuildpaths maybe says the game not to use ipls but nodes.dat files?

steve-m
  • steve-m

  • Members
  • Joined: 26 Jul 2002

#4

Posted 14 September 2005 - 11:27 AM

Thanks Hammer. No, ipl paths aren't used, that's for sure. But they were used to create those node files by R*'s compiler, which is either an extra program, or built into the game but not included in the retail version, or just disabled. But as I said, the "dontbuildpaths" doesn't seem to have any effect.

The problem atm is that I don't know all sections, so I can't create my own files for testing. I wasn't even able to create empty files, which would be needed, since the original files always try to connect to each other (unless I remove all external links).

Edit: But the graphic doesn't include ped paths, does it? Or no car paths? I'm confused...

Edit 2: Yes, peds are missing. Compare to this map (1.32 MB).

steve-m
  • steve-m

  • Members
  • Joined: 26 Jul 2002

#5

Posted 14 September 2005 - 03:27 PM

Made my own plot map, and it's getting interesting:

SA-nodes-plot.png (1.22 MB)

I tried to visualize the value I called 'node type'. Red for car nodes (1), Blue for boat nodes (2), and yellow for other nodes from section 1a. All ped nodes from section 1b are green. For green and yellow nodes, the type is printed next do the dot. As you can see, yellow tracks are used either for special vehicles, or mission specific routes (like races), and are identified by an ID > 2. I assume cars can be attached to paths with a certain ID, so in races they don't leave their circuit and continue driving on normal roads. Same for ped paths, they have their own "circuits"; each city, village and separate path have their own IDs, not sure what that's used for.

Edit: The value before type seems to be the path width, multiplied by 8 (or something), with 0 being default.

Edit 2: The value after type contains 2 things. The lower 4 bits are the number of nodes it is connected to (aka link count), the higher 4 bits are unknown. Quite high for boat paths, probably defines what vehicle types are allowed (as flags).

Edit 3: Second value after type - unknown flags. Third value after type - more flags, 15 default, 47 for parking, 1 used quite often north of vegas, 7 used in vegas as well.
Last value is always zero, so I assume the last 4 byte can be seen as DWORD, full of flags.

Edit 4: Most flags in section 1 discovered, first post updated.

steve-m
  • steve-m

  • Members
  • Joined: 26 Jul 2002

#6

Posted 16 September 2005 - 07:49 PM

Section 2: Still no idea, very confusing. Seems to be extended information for links of vehicle nodes, such as traffic light stops, lanes and directions.

Section 5: lower 10 bits - index into section 2 items; upper 6 bits - area ID; only used for vehicle nodes

Section 6: Link lengths (distance between two linked nodes) in full units. Example (double size):

user posted image

We definitely need a new path editor, this is way too complex for MaxScript. And just converting the old ipl paths won't work either, since they don't have all the information these files have. I already have lots of ideas, but I think I lack the time and motivation to do it. But technically I think it'd be easier than an editor for ipl paths, due to the used data structures.

(and yes, I enjoy talking to myself)

Jevon
  • Jevon

    Advocate

  • Inactive Staff
  • Joined: 21 Nov 2001
  • None

#7

Posted 16 September 2005 - 10:10 PM

QUOTE (steve-m @ Sep 16 2005, 20:49)
(and yes, I enjoy talking to myself)

And there was us thinking you were going crazy!

JernejL
  • JernejL

    Big Homie

  • Feroci Racing
  • Joined: 11 Mar 2002

#8

Posted 19 September 2005 - 11:54 AM

QUOTE (steve-m @ Sep 16 2005, 20:49)
We definitely need a new path editor, this is way too complex for MaxScript. And just converting the old ipl paths won't work either, since they don't have all the information these files have. I already have lots of ideas, but I think I lack the time and motivation to do it. But technically I think it'd be easier than an editor for ipl paths, due to the used data structures (undirected graphs).

i'm too lame for anything map related right now. try and make a editor and a converter ..


Gforce
  • Gforce

    "Segregate and Rearrange" "The Hills"

  • Feroci Racing
  • Joined: 17 Sep 2003
  • United-Kingdom

#9

Posted 19 September 2005 - 12:26 PM

QUOTE (Delfi @ Sep 19 2005, 11:54)
QUOTE (steve-m @ Sep 16 2005, 20:49)
We definitely need a new path editor, this is way too complex for MaxScript.

try and make a editor and a converter ..

i second that request.

you're not talking to yourself steve, it's just that some of us are not clever enough to add anything constructive to your finding's, so i guess all i can really say is .... well done so far and keep going smile.gif

steve-m
  • steve-m

  • Members
  • Joined: 26 Jul 2002

#10

Posted 19 September 2005 - 02:16 PM

QUOTE (Gforce @ Sep 19 2005, 14:26)
QUOTE (Delfi @ Sep 19 2005, 11:54)
QUOTE (steve-m @ Sep 16 2005, 20:49)
We definitely need a new path editor, this is way too complex for MaxScript.

try and make a editor and a converter ..

i second that request.

But I don't want to... cry.gif

Edit: bumped and moved

mickarrow
  • mickarrow

    ---

  • Members
  • Joined: 04 Feb 2004

#11

Posted 17 October 2005 - 04:42 PM

I don't know if this is helpfull, but the guys that are working on the BLUE platform were making a race editor. Wouldn't they need access to the path nodes to do this? Maybe they can help.

steve-m
  • steve-m

  • Members
  • Joined: 26 Jul 2002

#12

Posted 23 October 2005 - 10:50 AM

I thought their in-game editor was only for placing objects and to define race track checkpoints for human competitors, no?

I was thinking, maybe it would be easier to explore the node data when displayed in a proper graph. I could write a small tool that prints everything into a text .gdl file, which is rendered by software such as aiSee, for example. Will look into that next week, when I hopefully have some more time.

aad
  • aad

    3d artist

  • Members
  • Joined: 15 Mar 2004

#13

Posted 23 October 2005 - 05:58 PM

So its true that the game doesnt uses the paths folder in the data map but you cant remove it eather because the game crashes but the weird thing is you can just clean out every file to 0KB in size even the carrec.img dont know where that is for but what i was saying its still searching for the map if it isnt there the game wont load and will just crash so it has to do something. Also in the gta3.img are a lot of ipl files that are called stream1 stream2 etc although those are not the same as normal ipl files that can be edited with notepad or something similair so i hope someone can find out how it all works the flags are known but the making of is not confused.gif


JernejL
  • JernejL

    Big Homie

  • Feroci Racing
  • Joined: 11 Mar 2002

#14

Posted 23 October 2005 - 07:38 PM

Steve, so is this the only way to edit paths?

steve-m
  • steve-m

  • Members
  • Joined: 26 Jul 2002

#15

Posted 23 October 2005 - 10:40 PM

@aad: It uses the paths folder, just not the nodes*.dat files there (they are just a copy of those in the gta3.img). The files in carrec.img are special paths that are used within missions for car chases and such; tracks*.dat and train*.dat are used for the trains. Dunno about roadblox.dat, but I think it isn't used since such info is included in the node files as well.
The IPLs in gta3.img are in binary format - that's offtopic and explained everywhere, just search.

@Delfi: Yes. Unless R* hid their path compiler somewhere.

NathanHenry
  • NathanHenry

    Homeboy

  • Members
  • Joined: 18 Apr 2005

#16

Posted 24 October 2005 - 07:39 AM

So does this mean that the tool at the top can edit the San Andreas paths nodes because it would really help the Liberty City in San Andreas engine that I am on with xrk and AdTec_224 and dastardly.

grovespaz
  • grovespaz

    Group: Morons

  • Members
  • Joined: 22 Feb 2004

#17

Posted 12 November 2005 - 01:31 PM

I don"t know if this helps you or not, but:
I tryed to delete the nodes from the img, and run, it didn"t work(duhuh)
Then i removed teh dont build paths line from the streams.ini
didnt work
then i added the line buildpaths, didnt work, then i added the line dobuildpaths, didnt work.
So i decided to try sumthing else that might help you guys in developing a paths editor. I created a new IMG in the paths directory called paths.img(i dont really think the name matters) and added it to the gta.dat. In the img i put the nodes0-63. It worked.

This might help you if you are developing a editor so it you dont have to rebuild the gta3.img over and over again.

steve-m
  • steve-m

  • Members
  • Joined: 26 Jul 2002

#18

Posted 12 November 2005 - 07:03 PM

Yea, I already checked that stream.ini stuff back then. Didn't try moving to another img though, apparently we can put absolutely everything in external imgs now, and double entries are loaded from the file mentioned first.

I didn't work on this anymore, did someone else?

commando
  • commando

    Hustler

  • Feroci Racing
  • Joined: 18 Nov 2003
  • None

#19

Posted 12 November 2005 - 10:15 PM

So i guess for building new paths in SA the old path editor code for VC wouldnt be much useful and a new program must be builded to create the node paths and max script could be used if coded properly.

Am i missing something?

JGuntherS@NL
  • JGuntherS@NL

    Being a being

  • Members
  • Joined: 15 Nov 2005

#20

Posted 26 November 2005 - 08:31 PM

I made empty nodes*.dat and the game doesn't even crash!
I put them in a new img and deleted the old ones in gta3.img and I tested them in two environments, normal SA and GTI, and they both work.

Here they are: Download (contains a nodes__.dat and an img)

steve-m
  • steve-m

  • Members
  • Joined: 26 Jul 2002

#21

Posted 27 November 2005 - 10:58 AM

I've tried exactly those before, and using yours the game still crashes for me. sad.gif

JGuntherS@NL
  • JGuntherS@NL

    Being a being

  • Members
  • Joined: 15 Nov 2005

#22

Posted 27 November 2005 - 12:43 PM Edited by JGuntherS@NL, 04 December 2005 - 06:54 PM.

QUOTE (steve-m @ Nov 27 2005, 10:58)
I've tried exactly those before, and using yours the game still crashes for me. sad.gif

Strange... I wil try to make a simple path on my SA (peds walking in a square or something).

EDIT: Could section 4 mark the end of section 3 and the start of section 5? Sounds a bit strange, but why should that FF FF 00 00 be put in the nodes**.dat and not just in the executable?

EDIT2: (Steve, could you give me some more information about the links, it seems to me with your explanation that a link is done between node ID and area ID... A link must be between two nodes, isn't it? Or must it be this way to link node 00 to 01 (in area 36):
24 00 00 00 24 00 01 00) Sorry, I didn't read it well

EDIT3: My SA seems to be extremely tolerant. I've corrupted my section 6 (just deleted half of the entries), and the game still works. The only way to crash it is to delete a file or to write nonsense in a file. (That file was corrupted anyway, I made it myself, so I don't know what would happen if I corrupted an original)

EDIT4: Disabling "dontbuildpaths" does absoluty nothing. I've disabled it while having empty paths, and nothing happened.

EDIT5: (I made a simple nodes36.dat, but it doesn't work, can anyone help?) It was wrong anyway, I'll try to make a new one.

EDIT6: I finally got my game to crash... When I approach the path it crashes

EDIT7: (What is the link ID used for? If you have a node using link ID 2 and the second link is area ID 36 node ID 3, is the node then linked to node 3 (in area 36)? If it is that way, I understand it) Sorry I didn't read well

EDIT8: In section 2, the fifth argument is the area ID. The last two argument are (nearly) always zero. The tenth argument is for a whole bunch of lines the same. I can't find any coordinate-like data in it, so one of the arguments must be a node ID or link ID. The ninth argument is (nearly) always zero) (For information, I'm analyzing nodes36.dat, because it's from (0.0, 0.0) to (750.0, 750.0) and I like positive coordinates)

EDIT9: Are you sure the section 6 data is in full units? It seems to me a bit high (for example: 54, 31, 31, the first sec. 6 entries in nodes36). I think it's in 1/8's of the size, I'll check it later.

EDIT10: I think the link ID refers to the first link of that node. For example, in nodes36 (again tounge.gif ) node 0 has link ID 0, but no node has link ID 1. These are the first few entries:

CODE

#0000:     44   106
#0001:     36     1
#0002:     36     0
#0003:     36     3
#0004:     36    38
#0005:     36     3
#0006:     36     1


As you can see, node 0 links to an external node AND to an internal node, namely 1. And node 1 is linking back in line 0002, so that's another proof for Steve-m's double-link-theory.

CODE

#0213:     44   245
#0214:     44   109
#0215:     36     0
#0216:     44    16
#0217:     44    78


In line 0215, the external node (area 44, node 106) links back to area 36, node 0. (I see you already discovered this)

EDIT11: My decompiler reads all the stuff in structure-arrays, and it's pretty fast (about 2 sec. for a file). Check this topic on gtaforum.nl: SA Path Editor. Progress going faster than I thought, a compile function is already available (no edit function!).

(Maybe this post can go into the Guinness Book of Records for the most edited post tounge.gif )

steve-m
  • steve-m

  • Members
  • Joined: 26 Jul 2002

#23

Posted 04 December 2005 - 11:41 PM

QUOTE (JGuntherS@NL @ Nov 27 2005, 13:43)
(Maybe this post can go into the Guinness Book of Records for the most edited post  tounge.gif )

Hehe, sorry for ignoring you. Maybe you should've bumped the topic a couple of times to get attention. tounge.gif

QUOTE (JGuntherS@NL @ Nov 27 2005, 13:43)
EDIT: Could section 4 mark the end of section 3 and the start of section 5? Sounds a bit strange, but why should that FF FF 00 00 be put in the nodes**.dat and not just in the executable?

Dunno, I think it's just a placeholder for data that's put there at runtime.

QUOTE (JGuntherS@NL @ Nov 27 2005, 13:43)
EDIT9: Are you sure the section 6 data is in full units? It seems to me a bit high (for example: 54, 31, 31, the first sec. 6 entries in nodes36). I think it's in 1/8's of the size, I'll check it later.

Yes, I'm quite sure. I checked it on the map I plotted, with 1 pixel = 1 unit, and there it was true. And IIRC I even calculated some examples by hand and compared.

QUOTE (JGuntherS@NL @ Nov 27 2005, 13:43)
EDIT10: I think the link ID refers to the first link of that node. For example, in nodes36 (again  tounge.gif ) node 0 has link ID 0, but no node has link ID 1.

Yes, that's what it does. Just the ID of the first one is given, but the number of links tells you how many of them belong to that node.

NathanHenry
  • NathanHenry

    Homeboy

  • Members
  • Joined: 18 Apr 2005

#24

Posted 05 December 2005 - 02:38 AM

So is this any closer to having a SA Path Editor Program.

JGuntherS@NL
  • JGuntherS@NL

    Being a being

  • Members
  • Joined: 15 Nov 2005

#25

Posted 05 December 2005 - 07:14 AM

QUOTE ([email protected] @ Dec 5 2005, 02:38)
So is this any closer to having a SA Path Editor Program.

I'm writing one right now.Check this topic on GTAForum.nl: SA Path Editor

NathanHenry
  • NathanHenry

    Homeboy

  • Members
  • Joined: 18 Apr 2005

#26

Posted 05 December 2005 - 08:25 AM

So there is gonna be a SA Path Editor soon really that is cool.

JGuntherS@NL
  • JGuntherS@NL

    Being a being

  • Members
  • Joined: 15 Nov 2005

#27

Posted 05 December 2005 - 06:00 PM

Progress on my editor is going pretty fast. I decompiled and recompiled every file, and the game still works, so that means that my (de)compile-subroutines are good. (Unusable fact: my editor is multithreading tounge.gif )

JGuntherS@NL
  • JGuntherS@NL

    Being a being

  • Members
  • Joined: 15 Nov 2005

#28

Posted 11 December 2005 - 01:58 PM

Woohoo

My path compiler can now edit section 1a entries (finally...)

Topic on GTAForum.nl: SA Path Editor

NathanHenry
  • NathanHenry

    Homeboy

  • Members
  • Joined: 18 Apr 2005

#29

Posted 11 December 2005 - 08:42 PM

So now paths are editable???

BenMillard
  • BenMillard

    aka Cerbera

  • Members
  • Joined: 22 Jun 2002

#30

Posted 11 December 2005 - 09:58 PM

Please run an English thread about your paths editor in this area of GTAF! You could even develop it in this topic.

When you are able to reliably construct paths for GTASA using your program, you could contact TonyWob. He is making a GTA3/VC/SA Map Editor (MEd) and a 3D paths builder would be a great addition to it, once the core is made stable. smile.gif




1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users