BenMillard Posted June 6, 2006 Share Posted June 6, 2006 (edited) All of the GTA3/VC/SA text files seem to share a set of rules, some using more of its possibilities than others. For example, some files use commas and whitespace to seperate columns while others just use whitespace. Some files have named sections while others use a special character at the start. All the text data files seem to allow hash (#) and semicolon (;) commenting. If we figure out exactly what all the rules and allowed syntax is we could create a 'Content Sniffing' technique. This could determine what format a file is by looking at: the rules it uses; the names of its sections; the number of columns per line in each section; and the formats (numerical, string, boolean) used by specific columns. CFG Studio has a very rough sort of Content Sniffing which detects which edition of handling.cfg is being opened. It also detects when a file is too different from handling.cfg to be recovered and goes into a Format: Unknown mode which is a bit like the 'Quirks Mode' some browsers have. Parsing of files should be line-by-line to avoid lots of resources being used at once. So far, these are the steps I think would work: Remove any leading whitespace (some files have whitespace at start of some lines). Check for "#" or ";" at start. Discard line if either is present since it is a comment. You could store it somewhere internally to restore it when saving data, although good apps make commenting data redundant. Check for "!" or "%" or "^" or "$" at start. These indicate it is a special handling.cfg line and that means file must be a GTAVC or GTASA handling.cfg format, since no other file uses these. Start at the end of the line and work back, searching for "#" and ";". These are comments at the ends of lines and can be discarded (or stored internally). Comments cannot be relied on to determine file format. Remove any trailing whitespace. Do this even if no comment has been found because some R* generated files have whitespace trash at ends of lines. If resulting line contains no whitespace and is not empty, it is a single string of characters:Assume this is a data section title and use this string to entitle a new grid or set of controls to store the data it contains. You can sniff the value of section titles to narrow down the format being used, but unrecognised titles should be allowed. If the value of the string is "end" then do not load any more data into the grid or set of controls for that section. If any data follows after "end" but before a new section title, load it into a grid or set of controls for "Unknown" data lines. If resulting line contains any whitespace, treat it as a line of data:Collapse each string of whitespace to be one whitespace character so that a Split() or Explode type of command can be used. (If you collapse whitepace strings to each be one "tab", some grid controls will automatically split the data into columns for you.) If no section titles have been found so far, add it into a grid or set of controls for "Standard" or "General" data. You can count the number of columns to narrow down the possible format, but count all lines in every section and allow for the longest one. This prevents any data being lost while loading and prevents any lines with missing data from causing a different format to be assumed. A big benefit with Content Sniffing is that it will find the correct format for any filename. This lets people can use names like: handling.cfg; Copy of handling.cfg; Copy (5) of handling(2006-06-06@06h06m06s).cfg; misa-setups.bak; or any other name. Does anyone know of a better method, or knows of any rules unique to particular text data files? Edited June 11, 2006 by Cerbera Link to comment https://gtaforums.com/topic/246484-content-sniffing-for-gta-text-files/ Share on other sites More sharing options...
Gforce Posted June 9, 2006 Share Posted June 9, 2006 i know exactly what you mean. i think the gta modding world needs something that looks a little bit like this -------> born from frustration.......... i still have a sh!tload of extra interface functions to add and a lot more code to learn and implement but i think you get the idea of where this could be going ................. Link to comment https://gtaforums.com/topic/246484-content-sniffing-for-gta-text-files/#findComment-3730861 Share on other sites More sharing options...
fuckindumass Posted June 9, 2006 Share Posted June 9, 2006 I think you are taking the correct approach at this Ben, but since the number of file variations amongst the 3 games is so limited, I would take the easy way out and just have it look for known header text. Of course my method would fail as soon as someone makes a change to the header, your method would not be fooled that way. But I would guess most headers remain unchanged no matter how much the content is modified. Good luck, I hope you do it your way. Link to comment https://gtaforums.com/topic/246484-content-sniffing-for-gta-text-files/#findComment-3732282 Share on other sites More sharing options...
BenMillard Posted June 9, 2006 Author Share Posted June 9, 2006 (edited) Interesting project, GForce. There have been attempts at programs which edit everything in the past (such as Delfi's Data Tool) but I think all their interfaces have been too fiddly for modders to use. I think a simple, gridded interface for GTA text data makes more sense than a big text boxes or complex tabbed arrangements of various controls. GTA text data files are saved in rows and columns, sometimes with sections. A grid control for each section keeps each column aligned, lets you supply column names and keeps each section of the file nicely seperated. (EDIT) FDA, only just noticed your message! A lot of editors put custom comments in the data files or remove them completely, so you can't judge file formats using them. Thanks for the suggestion, though. Edited June 11, 2006 by Cerbera Link to comment https://gtaforums.com/topic/246484-content-sniffing-for-gta-text-files/#findComment-3732296 Share on other sites More sharing options...
sAdIsTiCmAcHiNe Posted June 12, 2006 Share Posted June 12, 2006 The following is a perl Regular Expression that should complete steps 1-5 of Cerbera's parsing. ^\s*([#;]?.*$)?([!%^$]?)(.*?)\s*([#;].*?)\s*$ It will return 4 variables in this order: •First variable will contain a comment if the entire line is a comment. If so all other variables will be null. •Second variable will be one of !, %, ^, or $ depending on whether the line begins with one of these. •Third variable will be the relevant contents of the line. •Fourth variable will be any end-of-line comment that may have occured. Can't really debug this, or give implementation instructions without the specifics of the language's implementation of perl regex. Link to comment https://gtaforums.com/topic/246484-content-sniffing-for-gta-text-files/#findComment-3736923 Share on other sites More sharing options...
JernejL Posted June 13, 2006 Share Posted June 13, 2006 Interesting project, GForce. There have been attempts at programs which edit everything in the past (such as Delfi's Data Tool) but I think all their interfaces have been too fiddly for modders to use. thanks for reminding me.. i need to get data tool for san andreas out Link to comment https://gtaforums.com/topic/246484-content-sniffing-for-gta-text-files/#findComment-3737506 Share on other sites More sharing options...
BenMillard Posted June 16, 2006 Author Share Posted June 16, 2006 I've got regular expressions set up and working in VB6 now. SadMac taught me some useful things you can do with RegEx which I didn't know before. We made a little regex for finding ordinal numbers (1st, 4th, 10th, 31st, etc). It was fun learning how to refine the things it would allow; reminded me of how powerful RegEx is. At the moment I'm tidying up my code (which is very messy) so that CFG Studio can be made a lot more flexible. Link to comment https://gtaforums.com/topic/246484-content-sniffing-for-gta-text-files/#findComment-3741344 Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now