Quantcast

Jump to content

» «
Photo

San Andreas Save File Companion

91 replies to this topic
Nick007J
  • Nick007J

    Mark Chump

  • Members
  • Joined: 17 Jan 2010
  • Russia

#31

Posted 11 August 2017 - 09:42 PM Edited by Nick007J, 11 August 2017 - 09:42 PM.

I thought we discuss resource manager structures. I suggest changing what follows line
tThread             threads[nRunningThreads] <optimize=false>;
with:
    if (isMobile == 1) {
        WORD                resourcesCount;    // nRecords

        struct tResourceManager
        {
            struct tResource
            {
                WORD                index;        // minIndex
                WORD                scriptIndex;        // maxIndex
                int                resourceID;           // modelID
                WORD               type;          // type?
                if (type == 2 && resourceID >= 290 && 299 >= resourceID) {
                    int length;
                    char specialModelName[length];
                }
            } Resource[resourcesCount] <optimize=false>;
        } ResourceManager;
        int IsOddJob;
        if (isMissionThread == 1 && isOddJob != 1) {
            char missionText[8];
            int missionReplaySetting;
        }
    }
} Script;
I mean I think it corresponds to what I see in mobile version code, and I see successful usage of it on the save you posted earlier (one with 75 resources).

OrionSR
  • OrionSR

    Chain Game Development Team

  • Feroci
  • Joined: 23 May 2007
  • None
  • Helpfulness Award [GTA & Modding]

#32

Posted 12 August 2017 - 05:32 AM Edited by OrionSR, 12 August 2017 - 06:32 AM.

Clarification: 

        struct tMobileStruct

This structure forces the the resource records to group (nest) within it's own container. This usually happens automatically for arrays but <optimize=false> ruins that process. Structures with variable length elements get corrupted by the optimization routines which require fixed length elements. I'm still not sure how I want to resolve this issue in the final template. I'm still trying to bash it into shape for all known saves except pre-v1.06 mobile saves. It's been a rather brutal process. I'm somewhat embarrassed to share the code.
 
Yes. We talked at length about the resource records but in the end I was unable to find enough consistency to name my fields confidently. The structure I posted works for all known saves. There is nothing I can do to force the save into another structure. I'll try to clean things up as best I can and look for a way to display data across multiple records so we can get a better idea of the patterns. 
 
As far as the mission saves are concerned, resource records can be ignored for now. There are none in the saves I'm examining. I don't know how to add them but perhaps I can borrow some from my sample saves. Update: I found some resources in my Burning Desire mission save, the data in MobileA and MobileB rule out the reuse of these fields for an isOddJob variable.
 
One thing is clear though, some of these variables are serving a dual purpose depending on the situation.
 
Working Example: Mobile2, which usually holds a value appropriate for an index, is used for the string length only for special actors. The template properly parses 75 resources with multiple special actors mixed in. Before I narrowed things to special actor model I was using Mobile3 = 0 to flag the presences of the character field and trailing integer. Mobile3 usually holds an value appropriate for an index that tends to be consistent between records. I have a vague memory of this breaking down once. [confirmed] Or maybe I switched to modelID because I understood the process better. Can't remember now.
 
Hm... You're talking integer length for string length. Could Mobile3 be cleared by writing an int to the previous word?


Nick007J
  • Nick007J

    Mark Chump

  • Members
  • Joined: 17 Jan 2010
  • Russia

#33

Posted 12 August 2017 - 08:13 AM

Here is the thing: your structure is supposed to work on all saves, because it's size is always identical to size of real structure. But logically it's not correct. I still recommend changing it to one with my suggestion and checking that it works for all known saves as well.

It can be checked quite simply. What is the size of your structure: 3 words before structure (6 bytes), size of constant fields of the structure (4 + 2*3 = 10 bytes), and if value at global offset 6 + (sizes of previous elements) equals 2 and at global offset 6 + 4 + (sizes of previous elements) is between 290 and 299, it's 14 + value at global offset 6 + 6 + (sizes of previous elements).

Now in my structure. 1 word before structure + 1 int after structure (6 bytes). Size of constant fields is 10 bytes as well, and if value at global offset 2 + 4 + (sizes of previous elements) equals 2 [same as in your!], value at global offset 2 + 8 + (sizes of previous elements) is between 290 and 299 [same!], it's 14 + value at global offset 2 + 10 + (sizes of previous elements).

So success rate can't change with switching to my structure. But I am certain my one is the correct one. The example where your structure may fail is mission oddjob checkpoint save, where you would have to check value of isOddJob, which you simply can't have in your current template.
  • OrionSR likes this

OrionSR
  • OrionSR

    Chain Game Development Team

  • Feroci
  • Joined: 23 May 2007
  • None
  • Helpfulness Award [GTA & Modding]

#34

Posted 12 August 2017 - 09:59 AM Edited by OrionSR, 12 August 2017 - 11:06 AM.

Ah, okay. Got it. It crashed on a mission thread until I got rid of the caps on isOddJob. I don't know why now that it all make sense, but for some reason I couldn't get the repeating structure to... fit. It was the wrong size. I must have lost track of a word somewhere. Thanks for continuing to push back.

 

I've still got garbage data in an unused missionText field (intro1) but that's not unexpected in gaps.

 

I lost track of where I was but wanted to bring up something else that you mentioned.

 

 

SaveGameStateType.

 
Possible values are:
0 - done by CHECKPOINT_SAVE (0A6F)
1 - done by AUTO_SAVE (0A72)
2 - done by TERMINATE_THIS_SCRIPT (004E)
3 - done by OS_ApplicationEvent (possible when 'close app' event is triggered?)
4 - done by CHECKPOINT_SAVE_ODDJOB (0A77) and function CPad::CheckPointSave() [hard to say what latter one is, but it seems to be triggered when player is in a vehicle, and there are checks for model ids 537, 538 and 570)
5 - done by SCRIPT_NAME (03A4)

 

So I don't need a train struct for the tram or carriages, just the engines? I wonder if I could save a train ride.

 

I don't have 0A72: auto-save in my opcode list for SA Mobile. What is it's form, or where can I get a better sascm.ini for mobile?

 

0A6F: checkpoint_save 1 seems to save on the spot. But I lost track of this early save. It should be easy enough to reproduce by entering the building during Architectural Espionage. 

 

0A6F: checkpoint_save 99 - I can't tell that these do anything. Maybe they prevent quick-saves, because usually it's tough to quick-save during a mission. The trick to glitching objects is to fail the mission and reload the the initial checkpoint save from the start of the mission. Then, for some reason, the game will quick-save when paused (usually - or the first time, but maybe not again for a while).

 

0A77: CHECKPOINT_SAVE_ODDJOB 1 - I haven't been having any luck getting oddjobs to save either, but I think I may have cheated on this one and triggered it with cleo. Any you mentioned somewhere that OM0 might be a factor for saves. I need a mission save with isOddJob =1 to make a final test of the structure...

 

.. so far. How much more can mobile throw at me?

 

Looking through examples:

 

Found an EOL save with SaveGameStateType 4, and isOddJob 2. Could be I edited it, or it's glitchy as I was hunting for unusual saves. An isOddJob value of 2 doesn't fit with an is_something variable name.

 

Mostly I'm seeing type 2 and 3 saves. One of these should be a normal save screen save (03D8:). And the mission threads I've been saving by failing, retrying and pausing are type 5. No sign of an auto-save yet. I'll see what I can do to produce a few normal saves for reference. I'm having trouble with the playing part but I can sometimes teleport my way through a mission.


Nick007J
  • Nick007J

    Mark Chump

  • Members
  • Joined: 17 Jan 2010
  • Russia

#35

Posted 12 August 2017 - 11:00 AM

I don't have 0A72: auto-save in my opcode list for SA Mobile. What is it's form, or where can I get a better sascm.ini for mobile?


For names, I use this dump from mobile version that contains original command names from R*.
http://pastebin.com/raw/kCwS6rdG
This list doesn't contain parameters though.

0A72: auto_save doesn't have parameters. It saves game with setting SaveGameStateType to 1.
0A6F: checkpoint_save 99 does nothing. For some reason, this opcode checks its only parameter, if it's 99, nothing is done, otherwise, a game is saved with SaveGameStateType == 0 (and setting missionReplaySetting to 1).

I'll try to understand meaning of the values of SaveGameStateType. It doesn't look like the value is used when loading, but it affects saving.

Looks like types 3, 4 and 5 always make the game save to slot 10. By the way, game generates debug message "Saving Checkpoint" for a save to slot 10, and "Saving Autosave" for a save to slot 9. Other types make game save to slot 10 if any mission is running, and to slot 9 otherwise. Type 4 also produces a message on the screen ("checkpoint save" I guess).

Type 0 also sets variable IsMissionSave to 1 (so does IsOddJob). It affects a lot of stuff in save routines.

0A77 checks OM0 but I would expect it to prevent saving if current script has name "crane2":
 
 case CHECKPOINT_SAVE_ODDJOB:
      name = this->scriptName;
      CRunningScript::CollectParameters(this, 1);
      if ( !strcmp(v43, "crane2") && CTheScripts::IsPlayerOnAMission(0) )
        return 0;
      missionReplaySetting = 0;
      IsOddJob = ScriptParams[0];
      SaveGameForPause(4, name);
      return 0;

So I don't need a train struct for the tram or carriages, just the engines? I wonder if I could save a train ride.


I'm not sure what that check actually does. I think it doesn't prevent saving, but there are some minor differences with this check?
 

.. so far. How much more can mobile throw at me?


It looks like quite a while. If IsMissionSave is set to 1, there is drastic addition of stuff saved into save file, most notably in blocks 2 (CPools::Save()), where I would expect tasks, ped groups, decision makers, timers, task records, audio engine state and some more minor stuff to be added. Looks like radar data might be affected, also peds and vehicles structures depend on whether save is a mission save.

OrionSR
  • OrionSR

    Chain Game Development Team

  • Feroci
  • Joined: 23 May 2007
  • None
  • Helpfulness Award [GTA & Modding]

#36

Posted 13 August 2017 - 05:51 AM Edited by OrionSR, 13 August 2017 - 09:05 AM.

IsMissionSave is not defined in my template. I have a poorly named local int isMissionThread that is set by the template to 1 if the thead.index is negative - because I haven't worked out the proper bitwise evaluation for 0x8000. Should I rename this variable to isMissionSave. It seems like a more appropriate description. No. that bit is part of the mission thread.

 

I'm worried about IsOddJob = 2 as I have a Not = 1 evaluation for the missionText field I need to test.

 

I'm going to try again to make a few saves using different saving strategies.

 

Tracked variables for each save are SaveGameStateType, isOddJob and missionReplaySetting if available:

 

Alley.Quicksave.GTASAsf10.b, 3, 0 (195,000 bytes)

Started a new game and walked towards Jefferson until the mini-map stopped flashing.

 

Alley.SaveAnywhere.GTASAsf1.b, 3, 0 (195,000 bytes)

Used a cleo mod to show the save screen.

 

Jefferson.Safehouse.GTASAsf3.b, 3, 0 (195,000 bytes)

Edited the previous save with some cash, fireproof, infinite sprint, cities unlocked and,

Bought the Jefferson safehouse to save.

 

ArchEsp.Checkpoint.GTASAsf10.b 3, 0, 1 - (390,000 bytes) broke my template (dropbox)

Modified save, cleo mission select, proceeded with Architectural Espionage until CJ enters the building. "Saving Checkpoint"

 

ArchEsp.Autosave.GTASAsf9.b, 2, 0, 0 (195,000 bytes)

Auto-save generated after completing Architectural Espionage without failing or reloading. Didn't see the message, may have missed it.

 

ArchEsp.RetrySave.GTASAsf10.b, 5, 0, 1 (260,000 bytes)

Save created in slot 10 when a mission begins. Used to retry a failed mission.

 

ArchEsp.ObjectSave.GTASAsf10.b, 3, 0, 1 (260,000 bytes)

Save created when pausing during a mission retry (tends to glitch objects)

if (FileSize() == 390000) isCheckpointSave = 1, isMobile = 1;;

When isMobile was enabled for all files of this size, the checkpoint save managed to parse until it hit the nPlayerCars field at offset +576 in the players struct of the PlayerObjects block. Earlier in the players struct is a set of 12 unknownmobile1 bytes in what looks like an int and 8 flag bytes of an unknown grouping, My memory suggests that the first int is usually -1, as it is in the ObjectSave. In the Checkpoint save it's 147.

 

Crap. The int(?) at +408 has a value of 18 in the Jefferson.Safehouse. 


Nick007J
  • Nick007J

    Mark Chump

  • Members
  • Joined: 17 Jan 2010
  • Russia

#37

Posted 13 August 2017 - 09:05 AM

IsMissionSave is not defined in my template. I have a poorly named local int isMissionThread that is set by the template to 1 if the thead.index is negative - because I haven't worked out the proper bitwise evaluation for 0x8000. Should I rename this variable to isMissionSave. It seems like a more appropriate description. No. that bit is part of the mission thread.


IsMissionSave is not a variable saved to a save file, it's a memory variable used to select what should be saved. If I understand correctly, it is set to 1 is there is a mission thread and SaveGameStateType is 0, or if IsOddJob equals to 1. isMissionThread in the template serves a different purpose.

As for bitwise and: there is no difference in current template whether a sign is checked or a butwise and with 0x8000 is done because the checks are semantically the same. The only difference that could matter is that if it was required to get actual index array from the save. In this case, negative value of a saved index is not a correct value, bitwise and with 7FFF would be required to get it (this is what game does when it loads the game). But I don't think it matters too much in a template.

Also for regular saves SaveGameStateType doesn't matter, it holds data remaining from previous pause save. It can be any value, and it is not used by game during loading (although if SaveGameStateType is 5 in a save, some variable called skipMissionStartCount is set to 10 but I don't see it being used later by game).

OrionSR
  • OrionSR

    Chain Game Development Team

  • Feroci
  • Joined: 23 May 2007
  • None
  • Helpfulness Award [GTA & Modding]

#38

Posted 13 August 2017 - 09:44 AM

Eventually I will need to read and set individual bit flags within sets of bytes. Bitwise is probably a related topic. I'll look it up if I need to.

 

I'd better not dig into this new checkpoint format tonight. If I make any progress I'm liable to be at it all night. What I'm looking for now is some sort of flag within the player structure to indicate an optional structure. 


Nick007J
  • Nick007J

    Mark Chump

  • Members
  • Joined: 17 Jan 2010
  • Russia

#39

Posted 13 August 2017 - 10:00 AM

Not sure if this is a source of an error, but vehicles array doesn't lie within players array. Game consequentially saves peds pool, vehicles pool * and object pool. [* there might be stuff between vehicles and objects, if IsMissionSave is 1]. I understand so that usually number of peds is 1, as the only ped to be saved is player, but if there are more, it can lead to errors.

OrionSR
  • OrionSR

    Chain Game Development Team

  • Feroci
  • Joined: 23 May 2007
  • None
  • Helpfulness Award [GTA & Modding]

#40

Posted 13 August 2017 - 10:52 AM

Yeah. I was just starting to figure that out but I'm too tired to try tonight. It looks like this checkpoint save has 28 players.


Nick007J
  • Nick007J

    Mark Chump

  • Members
  • Joined: 17 Jan 2010
  • Russia

#41

Posted 13 August 2017 - 11:21 AM

I see, it looks like mission saves (IsMissionSave == 1) save every ped that exists during saving. In your template everything that starts with "_size2" is only saved for players (for other peds previous field is the last one saved). PC version would have been doing the same (it has functions to generate data for all ped types) but save routine only asks to generate data for players (pedtype 0).

OrionSR
  • OrionSR

    Chain Game Development Team

  • Feroci
  • Joined: 23 May 2007
  • None
  • Helpfulness Award [GTA & Modding]

#42

Posted 14 August 2017 - 02:22 AM Edited by OrionSR, 14 August 2017 - 03:13 AM.



Not sure if this is a source of an error, but vehicles array doesn't lie within players array. Game consequentially saves peds pool, vehicles pool * and object pool. [* there might be stuff between vehicles and objects, if IsMissionSave is 1]. I understand so that usually number of peds is 1, as the only ped to be saved is player, but if there are more, it can lead to errors.

 

This has me worried. The issue of breaking out so many peds and a player is daunting enough, especially when there is so much evidence that more trouble will immediately follow. Suspicion that structs may not be nested properly has been a growing concern. But the task has been getting easier as the unknown mobile variables have been filled in with useful descriptions.

 

I think what I need to do before I continue is break out my smaller structs and move them to the top of the template, like Seeman did in the original version. This should make it easier to shuffle defined structures around.

            } pedHandle;
        }playerCar[nPlayerCars];
        int mobileJunk[4];  //mobile
    }
};

Can you offer any predictions as to the purpose of mobileJunk[4] at the end of the tPlayer struct? I don't remember any data observed; the ints might be a logical grouping for unknown bytes. Of primary concern is, if the PlayerCars structures are outside of the of the player struct then I'll need to move these too. As new structs are discovered where will these bytes fall?

 

Damn... Where to begin?

    DWORD       __gap4;
    if (isMobile==1) {
        float   unMobileFloat;
        int     nPlayerCars;
        struct  tPlayerCars {

Is __gap4 the true end of the player section that follows  __gap2 in the ped section?  [Nope, unMobileFloat end the struct on mobile versions.]


OrionSR
  • OrionSR

    Chain Game Development Team

  • Feroci
  • Joined: 23 May 2007
  • None
  • Helpfulness Award [GTA & Modding]

#43

Posted 14 August 2017 - 05:23 AM Edited by OrionSR, 14 August 2017 - 07:43 AM.

Woohoo! After reworking my player struct for multiple peds and players the template managed to parse through 28 peds and 4 cars before finally crashing on unknown data. 

 

For this save (I'll work in the checkpoint test later), I inserted a struct of 68580 bytes after the mobileJunk to align my objects and found another struct of unknown data before the end of the block.

 

The struct after objects appear to have a form of a short, and 154 record of 4 bytes and a short. Active data appears to be filled into the end of the structure so I'm hoping this is a fixed array.

 

Note: This segment assumes a checkpoint save while I'm hacking at the structure. All offsets are very rough estimates to give form to the data.

// Block 2: Players & Objects
struct tBlock02
{
    tblockHeader header;
    int         nPlayers;
    tPlayer     players[nPlayers] <optimize=false>;
    if (isMobile==1) {
        int     nPlayerCars;
        tPlayerCars  playerCar[nPlayerCars];
        int mobileJunk[4];  //mobile
    }
    byte        unCheckpointA1[5505];
    struct      GiantCheckArray {
        byte    unElement[60];
    } elementG[820];
    byte    unCheckpointA41[56];
    int     unCheckCountUp[74]; 
    byte    unCheckpointA43[32];
    struct      LargeCheckArray {
        byte    unElement[60];
    } elementL[205];

    byte    unCheckpointA6[1191];

    int         nObjects;
    tObject     objects[nObjects] <optimize=true>;
    short       unShort;
    struct      unCheckpointB {
        DWORD   unFlags <format=hex>;
        short   unShort;
    }WhoKnows[154];
} PlayerObjects;

I found a large block of data that looks like 820 records, 60 bytes in length.  GiantCheckArray

And shortly after that, another array with a similar format, has  205 records. LargeCheckArray

 

Any idea what this string is associated with? "COMMAND_TASK_CHAT_WITH_CHAR"


Nick007J
  • Nick007J

    Mark Chump

  • Members
  • Joined: 17 Jan 2010
  • Russia

#44

Posted 14 August 2017 - 08:17 AM Edited by Nick007J, 14 August 2017 - 01:03 PM.

    DWORD       __gap4;
    if (isMobile==1) {
        float   unMobileFloat;
        int     nPlayerCars;
        struct  tPlayerCars {

Is __gap4 the true end of the player section that follows  __gap2 in the ped section?  [Nope, unMobileFloat end the struct on mobile versions.]


__gap4 is actually 1 byte of something related to current weapon followed by 3 bytes of padding. float "unMobileFloat" looks like 'fMoveBlendRatio', a part of ped structure specific to player.

Now for second part of a question. CPools::Save() looks like this in mobile version:
 
if (CPools::SavePedPool() && CPools::SaveVehiclePool()){
  if (IsMissionSave){
    CPools::SavePedTasks();
    CPedGroups::Save();
    CDecisionMakerTypes::Save();
    COnscreenTimer::Save(CUserDisplay::OnscnTimer);
    CPedScriptedTaskRecord::Save();
    CAudioEngine::Save(AudioEngine);
    SaveSomeStaticSettings();
  }
  char r = CPools::SaveObjectPool();
  if (IsMissionSave){
    CGenericGameStorage::_SaveDataToWorkBuffer(&CPopulation::m_bDontCreateRandomGangMembers, 1);
    CGenericGameStorage::_SaveDataToWorkBuffer(&CPopulation::m_bOnlyCreateRandomGangMembers, 1);
    CTheScripts::SaveAfter();
  }
  return r;
}
return 0;
It should hint the order of data in pools block. I'll try to add some information about it here in edits.

For mobileJunk: it looks like that following array of vehicle, there are 4 additional ints:
+0: vehicle handle related to player?
+4: type of camera related entity (2 is vehicle, 3 is ped)
+8: handle of said entity
+12: -1 or handle of vehicle related to player ped?
 
Updated pools block.

Spoiler

// Block 2: Pools
struct tBlock02
{
    tblockHeader header;
    int         nPeds;
    tPed     peds[nPeds] <optimize=false>;
    if (isMobile==1) {
        int     nVehicles;
        tVehicle  vehicles[nVehicles]<optimize=false>;
        int playerVehicleHandle1;
        int cameraTargetEntityType;
        int cameraTargetHandle;
        int playerVehicleHandle2;
    }
    if (IsMissionSave == 1 ){
        //CPools::SavePedTasks
        tScriptSequenceTask sSequenceTasks[64];    
        byte bIsSequenceTaskOpen[64];
        tTaskSequence nSequenceTasks[64]<optimize=false>;
        int nPedInts;
        tPedIntelligence pedint[nPedInts]<optimize=false>;

        //CPedGroups::Save
        tPedGroups pedGroups;

        //CDecisionMakerTypes::Save
        char decisionMakerIsActive[20];
        short decisionMakerReferenceIndex[20];
        char decisionMakerTypes[20];
        CDecisionMakerTypes decisionMaker;

        //COnscreenTimer::Save
        COnscreenTimerEntry timer;
        COnscreenCounterEntry counters[4];
        
        //CPedScriptedTaskRecord::Save
        tPedScriptedTaskRecord scriptedTasks[128]<optimize=false>;

        //CAudioEngine::Save
        CAEMissionAudio missionAudio[4];

        //SaveSomeStaticSettings
        float carDensityMultiplier;
        char bAllowEmergencyServicesToBeCreated;
        float pedDensityMultiplier;
    }

    int         nObjects;
    tObject     objects[nObjects] <optimize=true>;
    if (IsMissionSave == 1){
        char bDontCreateRandomGangMembers;
        char bOnlyCreateRandomGangMembers;
        tEntityWaitingForBrain entitiesWaitingForBrain[150]<optimize=false>;
    }
} Pools;
The save file I used for testing had only 3 types of tasks, which allowed me to be somewhat successful in creating structures. Unfortunately, there are a lot more task types and each has its own structure.
  • Seemann and OrionSR like this

OrionSR
  • OrionSR

    Chain Game Development Team

  • Feroci
  • Joined: 23 May 2007
  • None
  • Helpfulness Award [GTA & Modding]

#45

Posted 14 August 2017 - 04:36 PM Edited by OrionSR, 14 August 2017 - 04:37 PM.

Wow. You made remarkable progress after I went to bed frustrated. Would you mind sharing you sub-structs? tPedGroups; and whatnot. I won't be able to work on this until tonight though.

 

This template is rapidly diverging from the SA Save Wiki documentation. Eventually that page will need to be updated, but I don't see a point to starting that process now. I think the best strategy in the long run is to create a script that can produce the wiki for us. But then again, replacing the entire document would certainly mess with the history.


Nick007J
  • Nick007J

    Mark Chump

  • Members
  • Joined: 17 Jan 2010
  • Russia

#46

Posted 14 August 2017 - 04:38 PM

Wow. You made remarkable progress after I went to bed frustrated. Would you mind sharing you sub-structs? tPedGroups; and whatnot. I won't be able to work on this until tonight though.


They are above main ones under spoiler, I hid them as they took way too much space.

OrionSR
  • OrionSR

    Chain Game Development Team

  • Feroci
  • Joined: 23 May 2007
  • None
  • Helpfulness Award [GTA & Modding]

#47

Posted 14 August 2017 - 04:46 PM Edited by OrionSR, 15 August 2017 - 05:23 AM.

Ah, there we go. I clicked on the spoiler but the sub-structs looked the same and I thought it was empty.

 

Added: Ah, my topics have merged and been renamed. I'm going to rework the title page before testing out the new checkpoint struct.


Seemann
  • Seemann

    Ruhe

  • GTA Mods Staff
  • Joined: 03 Sep 2004
  • Russia
  • Best Tool 2016 [OpenIV]
    Best Tool 2013 [Sanny Builder]
    Contribution Award [Mods]
    Helpfulness Awards [Mods]

#48

Posted 15 August 2017 - 09:05 AM

I suggest using GitHub as a platform to track your progress on the templates and allow several people to work together. I could help you if you wish.


OrionSR
  • OrionSR

    Chain Game Development Team

  • Feroci
  • Joined: 23 May 2007
  • None
  • Helpfulness Award [GTA & Modding]

#49

Posted 15 August 2017 - 07:07 PM Edited by OrionSR, 16 August 2017 - 02:39 AM.

Yes, but when? Publishing documentation for PS2 and mobile save structures was a primary motivation for starting this project. And right now the SAS.multi template is the only documentation available, and only to a few. But everything keeps changing so quickly. I can't find anything close to a stable version. Also, much of my coding is embarrassingly poor. I was hoping to clean things up before sharing openly on the web. But if you are willing to help follow through to something we can all be proud of we can start anytime.

 

A stable version: I was working towards something that was self contained; one file that managed all SA saves. But I'm anxious to start breaking out some parts of the template. Specific modules include the checksum fix script, and maybe the SA save enums, since I want to use those in other scripts.

 

I want to break out the Save (type) Analysis section into an installed template that runs whenever a .B file is loaded that can detect any GTA3 era save and launch the appropriate multi platform templates for GTA III, Vice City or San Andreas. The save type analysis needs to generate it's report even if the template fails. It does not read or set any template variables.

 

And soon I will need a strategy to apply custom global variable names for specific versions. The original goal was to develop a script to remove glitchy objects, so I need control of all standard objects. Naming those variables should allow me to assign values based on version. What would be the best strategy for a cloud based database for custom variable names for different versions that can easily produce a file for use in my 010 templates and scripts, and also produce customvariable.ini files for use in Sanny? I'm thinking about a file with a format that can be parsed with a template rather than produce actual code. The idea is that I could update Sanny variable names if I wanted and have more compatibility in my cleo scripts, but it wouldn't be required.

 

Please don't get overwhelmed. I'm mostly trying to look ahead to potential problems with file management. Anyway, I need to check in with my doctor (shoulder problems) and finish what I started on the opening post before I fell asleep at the keyboard. And I need to produce more checkpoint saves for Nick to play with.

 

 If these maintain a constant file size I'll talk to Samutz about hosting on SnP.

 

ArchEsp.Checkpoint.GTASAsf10.b - Checkpoint after entering Planning Department.

ArchEsp.Checkpoint2.GTASAsf10.b - Checkpoint after talking photo of plans.

BigSmoke.Checkpoint.GTASAsf10.b - Checkpoint after meeting Sweet under the Mulholland Intersection.

Ryder.Checkpoint.GTASAsf10.b - After getting a haircut.

CleanHood.Checkpoint.GTASAsf10.b - with "Go beat up the crack dealer."

NinesAKsG.Checkpoint.GTASAsf10.b - When CJ gets to shoot bottles.

DriveBy.Checkpoint.GTASAsf10.b - 2 Stars for Pay'n'Spray


Nick007J
  • Nick007J

    Mark Chump

  • Members
  • Joined: 17 Jan 2010
  • Russia

#50

Posted 15 August 2017 - 09:49 PM Edited by Nick007J, 15 August 2017 - 10:09 PM.

Updated task type for first of saves to work (adding necessary task types).
Spoiler


Second one has something weird going on with peds. Interesting to check that out.

Edit 1:
tPed (or, as originally, tPlayer) needs to be updated like this:
struct tPed
{
    DWORD       handle;
    int         modelID;
    int         pedType;
    if (modelID >= 290 && modelID <= 299) char modelName[9];
...
.

I will update tasks shortly to let this save be read properly. Done.

OrionSR
  • OrionSR

    Chain Game Development Team

  • Feroci
  • Joined: 23 May 2007
  • None
  • Helpfulness Award [GTA & Modding]

#51

Posted 16 August 2017 - 12:45 AM Edited by OrionSR, 16 August 2017 - 04:42 AM.

Feed the Beast! - A local expression about a sports hero meaning, give him the ball and let him score. I'll see about producing more checkpoint saves.

 

Nick, feel free to rename variables as you see fit. I'll work on creating saves and you can manage the template updates this round and pass it back with the checkpoint upgrade. Actually, it's more like, please rename every variable where you have strong confidence of a match. But I'll be happy with anything convenient. 

 

Don't worry, I'll eventually get curious and work in your updates, but we clearly have different strengths for this structure. Nice work.

 

Added: Can we predict which tasks will be active based on mission scripts? Currently I'm just scanning for 0A6F: checkpoint_save 1 and choosing what looks easy to complete. I'll try to frighten some peds to see if that does anything interesting for you. Do you have any suggestions on how I might trigger unique tasks?

 

Note: Checkpointsave 1s from the stadium sniffer scripts produce a mission save but not a full checkpoint save.

 

New, but also added to the checkpoint list above:

Ryder.Checkpoint.GTASAsf10.b - After getting a haircut.

CleanHood.Checkpoint.GTASAsf10.b - with "Go beat up the crack dealer."

NinesAKsG.Checkpoint.GTASAsf10.b - When CJ gets to shoot bottles.

DriveBy.Checkpoint.GTASAsf10.b - 2 Stars for Pay'n'Spray


Nick007J
  • Nick007J

    Mark Chump

  • Members
  • Joined: 17 Jan 2010
  • Russia

#52

Posted 16 August 2017 - 09:16 AM

Updated tTask for all saves above.

Spoiler


By the way, it looks like all mobile saves size is a multiple of 65000, because game uses buffer of this size. PC version uses buffer of 51200 bytes but it doesn't correspond to size of file like mobile version.

OrionSR
  • OrionSR

    Chain Game Development Team

  • Feroci
  • Joined: 23 May 2007
  • None
  • Helpfulness Award [GTA & Modding]

#53

Posted 16 August 2017 - 02:57 PM Edited by OrionSR, 16 August 2017 - 03:14 PM.

Thanks again. I will continue working my way through the main.scm and attempting trigger each checkpoint save in sequence. For the record, Big Smoke was a natural save, and several of early game saves above were based on a save that loaded that checkpoint. I'm going to buff CJ a little more in the ITBEG save I modified to unlock all cities for Architectural Espionage and start using that as a base to trigger missions with a cleo tool.

 

Is there a correspondence between the tasks found in the saves and tasks loaded by the mission associated with that save? Am I hunting a specific set of mission tasks or are ambient tasks included? Can we predict which missions will hold new tasks? I think for now there are still plenty of common tasks to be discovered, and a list of checkpoint 1 saves is useful on it's own.

 

Speculation:

 

A prime limitation of save editing over cleo is that ending and launching threads is easy with cleo, but I don't even know how to kill an active thread with an editor. I can put it to sleep, but can't force it to shut down on it's own. I've never understood thread structure well enough to manipulate it beyond timer fixes. I don't even know how to manually convert between v1 and v2, a basic function of save editing since before I got into the field. So the question remains, how to launch and kill threads with an editor?

 

But we have these new mission saves now that include the full mission in scm format. When I pasted the block into a hex file named mission.cm Sanny had not trouble decompiling it as a custom mission. I've been looking for a good test of reversing the process and pasting a custom mission into the mission code block. Launching and ending threads would be a prime candidate for an application of this strategy. It would open up a lot more modding strategies to iOS and Remastered players.


Nick007J
  • Nick007J

    Mark Chump

  • Members
  • Joined: 17 Jan 2010
  • Russia

#54

Posted 16 August 2017 - 03:06 PM

Is there a correspondence between the tasks found in the saves and tasks loaded by the mission associated with that save? Am I hunting a specific set of mission tasks or are ambient tasks included? Can we predict which missions will hold new tasks? I think for now there are still plenty of common tasks to be discovered, and a list of checkpoint 1 saves is useful on it's own.


One of structs with task data is array of sequence tasks. They are opened by 0615 (OPEN_SEQUENCE_TASK) and it's usually followed by commands that add tasks to sequence (such command originally have name "TASK" in it, and older documentation, if I remember correctly, has "AS" in name of such opcodes). I believe GTAG SASCM.ini with newer names can be generated here with selected V2 option:
https://gtagmodding....base/generator/

It shouldn't be used for modding, as said, to avoid losing compatiability with old scripts, but it might be used for reading original script.

OrionSR
  • OrionSR

    Chain Game Development Team

  • Feroci
  • Joined: 23 May 2007
  • None
  • Helpfulness Award [GTA & Modding]

#55

Posted 16 August 2017 - 04:00 PM Edited by OrionSR, 16 August 2017 - 05:04 PM.

Be sure to check my speculation notes above - they were added after your post. You don't need to take action but I'd like you to be aware of the challenges in case some interesting ideas come to mind.

 

Ah, so I could work with a PC script from Sanny's GTA SA SCR mode to isolate tasks until I learn to recognize them in SA Mobile mode. I don't have an SA Mobile mode with the new format, but I could probably build one. I've had some luck with custom Sanny modes. I didn't link it on the opening post but probably should as the strategies vastly increase the functionality of the custom modes for non-PC applications. The upgrades include customvariable.ini associated with each version, IDE support, and GXT strings, iirc.

 

A byproduct of this project was my GTASA Global Translation Table. It's a google sheet with an expanded customvariable.ini that has references for PC, Android (mobile), PS2v1 and PS2v2. This file could use some work. I was in the process of condensing arrays to reduce the number of variables that needed to be named but... It's been too long and I've lost track of where I was so I'd better start over.

 

I plan to update this post with a few ideas for scripts.

 

Regarding an expanded customvariable.ini and renaming common global variables names: I spoke with Seemann about this and if I understood correctly, if two variable names are assigned to the same global variable number then Sanny will compile both as expect, and will decompile to the first definition encountered in the file. This should allow players to continue using existing scripts with a new set of custom variable names. 

 

Ideas for scripts that can use this template:

  • Remove garbage objects glitched by mission saves.
  • Remove stray, duplicate, and other unwanted blips.
  • Remove duplicate non-standard cargens, pickups, objects and blips from bad cleo scripts.
  • Cleanup expired pickups that clutter areas where CJ seldom revisits. (more of a refresh than a fix)

_________________________________________________________

 

Update:

 

Hurray! Samutz has added enhanced support for Checkpoint and Mission saves, and it looks like everything is working as expected for the original ArchEsp checkpoint save. I can see the extra armor added by my edits, and the rural safehouses were unlocked as expected when the cities unlocked stat was adjusted.

 

Whoops. There seems to be issues with the mission saves, but it shouldn't be hard to fix since the more complex checkpoint saves look great. Fixed!


Nick007J
  • Nick007J

    Mark Chump

  • Members
  • Joined: 17 Jan 2010
  • Russia

#56

Posted 16 August 2017 - 05:24 PM Edited by Nick007J, 16 August 2017 - 06:33 PM.

Cancelling the thread... The obvious solution would be to decrease number of threads, remove its structure entirely and recalculate checksum. The only way to do it without doing so drastic change would be to look up any address that executes command 'terminate_this_script' and set current address to it. This should make game terminate it immediately after loading.

Adding the thread certainly requires drastic changes, by adding to thread total and inserting thread structure into save. It's quite difficult as it requires finding necessary index, setting up right pointers, including those of different threads.

As for mission code, maybe option 'IGNORE_SCM_HEADER' can help? I think its intention is to decompile external scripts but probably it works with separate blocks of mission.

Edit: pointer stuff within structure should be checked. Quite likely they are not actually used during loading.

OrionSR
  • OrionSR

    Chain Game Development Team

  • Feroci
  • Joined: 23 May 2007
  • None
  • Helpfulness Award [GTA & Modding]

#57

Posted 16 August 2017 - 06:49 PM Edited by OrionSR, 16 August 2017 - 07:12 PM.

I think naming the copied block as .cm forced the custom mission directive which ignored the header. The block decompiled without issue. It is my hope that a compiled custom mission can be overwritten to the block, but I'm worried about things like stacks or pointer or next instruction - I'm lost here. How would I reset the thread to start the mission properly?

 

Impractical or impossible? Either way, other strategies for manipulating threads might be preferable. If I start with a mission save so I don't need to fiddle with other threads or adjust the overall file size it brings the scope of the project into the difficult range - and worth a shot.

 

Okay, new plan for the tasks. Rather than struggle with laggy missions and awkward controls to scrounge up an incomplete set of tasks, lets take control of the situation and modify a mission to include the specific tasks we need to test in the order we want to test them with known parameters so we can make a good guess at naming the variables. First step would be a modified main.scm. Modified files found in a specific user folder will be used instead of the standard files. so it's an easy mod strategy that should work on any mobile device. I'll try to keep a restart save with the modified code. Maybe we can adjust that for the next batch of tasks rather then recompile another full main.scm.

 

I like this plan, the main problem -- I've never worked with tasks before so I've barely a clue what to do. And I've little experience working with the actors or other entities required to carry out the tasks. So it's liable to take a while before I can actually produce a save worth testing. However, the basic mission can be tested on PC, we just can't take a snapshot of the tasks... Or can we?

 

Have you tried applying your task structure to PC memory? I think I have an example of this for CPeds - from III probably. I'll check.

 

Yeah. Found it. The spoiler contains an example of applying a single structure from my III save template to a section of process memory. So wrong game and a bit too large for the required example but I think you'll get the basic idea.

Spoiler

Nick007J
  • Nick007J

    Mark Chump

  • Members
  • Joined: 17 Jan 2010
  • Russia

#58

Posted 16 August 2017 - 07:24 PM

Regarding tasks from memory: they do not correspond. I have almost all their structures (at least class hierarchy + basic types of fields) but far from everything is saved. It could be possible for me to get structures for all of the task types, but it's harder without actual saves - an error is much more costly. What I do is get a structure based on function Serialize() [there is own version of it for most of tasks] for a type of given ID. It doesn't exist on PC because tasks are never saved on PC.

Regarding killing threads. I checked changing IP strategy and it worked. E.g. these two saves:
http://gtasnp.com/SpKXuD
http://gtasnp.com/I4s5nM
only have one field changed: int relativeIP in threads[36] (0xDB26 within save). I changed original value to 61825, an address which contains command 'terminate_this_script'. In first save Misappropriation is available, in second it is not, because 'VCRASH' thread has beed killed.

Note that for mission threads (those that have baseIP other than 0) their baseIP either needs to be reset to 0, otherwise game would still crash. [on an related topic: while searching for some crazy strategy to run unintended code for speedrunning, we considered using saves with active mission. Didn't work out but for Vice City, for example, it is possible to manipulate some values into running mission code from waterpro.dat, if I remember correctly). Later we found a method to run arbitrary code, using a different trick. The point of this is that game doesn't always crash if saved during mission, but it crashes in almost any curcumstances.]

Adding new one is a lot more difficult. You would have to set up the stack, stack pointer, current address etc. I believe this should be possible but quite time consuming.

OrionSR
  • OrionSR

    Chain Game Development Team

  • Feroci
  • Joined: 23 May 2007
  • None
  • Helpfulness Award [GTA & Modding]

#59

Posted 16 August 2017 - 08:30 PM

Oh, nice work on the thread death. I'd need a specific address for each mission but it's possible. It might be tricky on Android though. Pointers tend to use two sets of base addresses. I'm hoping this quirk will provide a fingerprint for Android and iOS saves.

 

Okay, back on task, unless you think natural saves would be more productive, I'm going to try to tweak an existing mission - Nine's and AKs since the checkpoint is so easy. Hopefully I can just stack up a set of tasks without actually applying it.


Nick007J
  • Nick007J

    Mark Chump

  • Members
  • Joined: 17 Jan 2010
  • Russia

#60

Posted 16 August 2017 - 08:41 PM

Oh, nice work on the thread death. I'd need a specific address for each mission but it's possible. It might be tricky on Android though. Pointers tend to use two sets of base addresses. I'm hoping this quirk will provide a fingerprint for Android and iOS saves.


I think it's possible to use the same address for all missions (as long as version of script is the same). I think combination of setting baseIP to 0 and relativeIP to 61825 should do the trick for all V1 saves.
  • OrionSR likes this




1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users