Quantcast
Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
    1. Welcome to GTAForums!

    1. Red Dead Redemption 2

      1. PC
      2. Gameplay
      3. Missions
      4. Help & Support
    2. Red Dead Online

      1. Gameplay
      2. Find Lobbies & Outlaws
      3. Help & Support
      4. Frontier Pursuits
    1. Crews & Posses

      1. Recruitment
    2. Events

    1. GTA Online

      1. DLC
      2. Find Lobbies & Players
      3. Guides & Strategies
      4. Vehicles
      5. Content Creator
      6. Help & Support
      7. The Diamond Casino Heist
    2. Grand Theft Auto Series

    3. GTA 6

    4. GTA V

      1. PC
      2. Guides & Strategies
      3. Help & Support
    5. GTA IV

      1. Episodes from Liberty City
      2. Multiplayer
      3. Guides & Strategies
      4. Help & Support
      5. GTA IV Mods
    6. GTA Chinatown Wars

    7. GTA Vice City Stories

    8. GTA Liberty City Stories

    9. GTA San Andreas

      1. Guides & Strategies
      2. Help & Support
      3. GTA SA Mods
    10. GTA Vice City

      1. Guides & Strategies
      2. Help & Support
      3. GTA VC Mods
    11. GTA III

      1. Guides & Strategies
      2. Help & Support
      3. GTA III Mods
    12. Top Down Games

      1. GTA Advance
      2. GTA 2
      3. GTA
    13. Wiki

      1. Merchandising
    1. GTA Modding

      1. GTA V
      2. GTA IV
      3. GTA III, VC & SA
      4. Tutorials
    2. Mod Showroom

      1. Scripts & Plugins
      2. Maps
      3. Total Conversions
      4. Vehicles
      5. Textures
      6. Characters
      7. Tools
      8. Other
      9. Workshop
    3. Featured Mods

      1. DYOM
      2. OpenIV
      3. GTA: Underground
      4. GTA: Liberty City
      5. GTA: State of Liberty
    1. Red Dead Redemption

    2. Rockstar Games

    1. Off-Topic

      1. General Chat
      2. Gaming
      3. Technology
      4. Programming
      5. Movies & TV
      6. Music
      7. Sports
      8. Vehicles
    2. Expression

      1. Graphics / Visual Arts
      2. GFX Requests & Tutorials
      3. Writers' Discussion
      4. Debates & Discussion
    1. News

    2. Forum Support

    3. Site Suggestions

demonj0e

Creating More Space In The SCM File

Recommended Posts

Demarest

 

Eh? If the game can run it, but other tools can't read it, that would mean the "other tools" are buggy.

Never suggested otherwise. I do still believe it to be a bad idea to arm somebody with the ability to make a release that can only be used and not deconstructed for the sake of tweaking, expanding upon, learning from, etc. But that's just one opinion on the matter. The fact that Point is largely welcomed should indicate that I heavily have the minority opinion wink.gif I just feel the need to educate others and I find that point to be of particular interest. Others may not. I historically have a reputation for being an MB enthusiast whether that's a good idea or not.

Share this post


Link to post
Share on other sites
Un3462
Never suggested otherwise.

you may not directly have suggested otherwise, but it is still very much the tone of your posts (though perhaps unintentionally).

 

 

I do still believe it to be a bad idea to arm somebody with the ability to make a release that can only be used and not deconstructed for the sake of tweaking, expanding upon, learning from, etc.

you have it backwards. point isn't doing anything; its output can be disassembled just fine, if the bugs in those tools are fixed. in fact, it was already possible to crash/confuse the disassemblers & the builder with valid code, before point ever existed.

Share this post


Link to post
Share on other sites
demonj0e
so point can automaticly slim down the scm when it dcompiles it?

Share this post


Link to post
Share on other sites
tomworld10

Hi

 

When I sent my slimmed down I didn't relly explainde very well what i've done. I do so now. I've actually only done what Demarest explained back a page, which was :

- cutscene model check

- conditional lumping

I'll repeat it here :

In many places through the main.scm you can find :

:INTRO1_12800D6: if  086B9:   NOT   cutscene_data_loaded004D: jump_if_false ��INTRO1_1330001: wait  0 ms0002: jump ��INTRO1_128:INTRO1_133

 

which from where you can see by turning around the check you can free up 7 bytes by deleting the jump which look like that :

 

 

:INTRO1_1280001: wait  0 ms00D6: if  006B9:   cutscene_data_loaded004D: jump_if_false ��INTRO1_128:INTRO1_133

 

 

So what I've done was to find them "

86B9:   NOT   cutscene_data_loaded

" check if it the same configuration (it's always it expect once or twice) and paste

0001: wait  0 ms00D6: if  006B9:   cutscene_data_loaded004D: jump_if_false ��

instead of

00D6: if  086B9:   NOT   cutscene_data_loaded004D: jump_if_false ��INTRO1_1330001: wait  0 ms0002: jump ��

 

 

And that all the way to the end. But if you look at the picture this only freed 88 bytes in the main part (everything before --------mission 0-------------) and i think about 300, 400 in the whole main.scm. I was very happy after that because I wanted room in the main part, and that's where jonc was pretty right i got insane and i wanted room up there !!!!!!!!!!!!!

 

So i've started to do the conditional lumping :

In even more many many places you can find code with this shape, see how they send to the same place 004D: jump_if_false ��LA1FIN_51

 

00D6: if  00256:   player $PLAYER_CHAR defined004D: jump_if_false ��[color=red]LA1FIN_51[/color]00D6: if  00038:   $ON_MISSION ==  0;; integer values004D: jump_if_false ��[color=red]LA1FIN_51[/color]00D6: if  000FF:   actor $PLAYER_ACTOR  0 ()near_point_on_foot $462 $463 $464 radius  1.0  1.0  2.0004D: jump_if_false ��[color=red]LA1FIN_51[/color]00D6: if  00038:   $SWEETS_MISSIONS_PASSED ==  9;; integer values004D: jump_if_false ��[color=red]LA1FIN_51[/color]00D6: if  00038:   $SMOKES_MISSIONS_PASSED ==  4;; integer values004D: jump_if_false ��[color=red]LA1FIN_51[/color]00D6: if  00038:   $STRAP_MISSIONS_PASSED ==  5;; integer values004D: jump_if_false ��[color=red]LA1FIN_51[/color]00D6: if  00038:   $RYDERS_MISSIONS_PASSED ==  3;; integer values004D: jump_if_false ��[color=red]LA1FIN_51[/color]00D6: if  00038:   $CRASH_MISSIONS_PASSED ==  2;; integer values004D: jump_if_false ��[color=red]LA1FIN_51[/color]00D6: if  00038:   $CESAR_1_MISSIONS_PASSED ==  1;; integer values004D: jump_if_false ��[color=red]LA1FIN_51[/color]00D6: if  003EE:   player $PLAYER_CHAR controllable004D: jump_if_false ��[color=red]LA1FIN_51[/color]

 

Well all this stuff can be much more skinnier, lets slimmed down this fat ass code !!!!!!!!!!!!!!!!! die.gif fat code angry.gifdie.gif HAHAHAHA............ devil.gifangry.gif !!!!!!!!!!!!!! rampage_ani.gif

 

00D6: if  40256:   player $PLAYER_CHAR defined0038:   $ON_MISSION ==  0;; integer values0038:   $SWEETS_MISSIONS_PASSED ==  9;; integer values0038:   $SMOKES_MISSIONS_PASSED ==  4;; integer values0038:   $STRAP_MISSIONS_PASSED ==  5;; integer values004D: jump_if_false ��LA1FIN_5100D6: if  40038:   $RYDERS_MISSIONS_PASSED ==  3;; integer values0038:   $CRASH_MISSIONS_PASSED ==  2;; integer values0038:   $CESAR_1_MISSIONS_PASSED ==  1;; integer values00FF:   actor $PLAYER_ACTOR  0 ()near_point_on_foot $462 $463 $464 radius  1.0  1.0  2.003EE:   player $PLAYER_CHAR controllable004D: jump_if_false ��LA1FIN_51

 

 

Look how they been split up because you can only cheks 8 thing in a if ( if 0-1 check- to if 7-8 checks-) Well that's like the best exemple we can get the others you only merge 2 or 3 checks and one often too, but it's always some bytes rescued rahkstar2.gif, So I've done that all the way down to --------mission 0------------- notify.gif , and I stopped cause I was barf8bd.gif of it suicidal.gif , and compile a bit so i can see how many bytes were saved, and then Oh suprise 5000 taken down Shifty41s_beerhatsmilie2.gifmusic.gifsleepy.gif

And then i went testing and latter reading the topic i thought I'm cool.gif !!!!!!!!!! smile.gifsmile.gif That's it for the lesson, go on. Go to the playground and have fun kids !!!!!!!!!!!!!!!!

 

 

As I'm passing around here I'd like to say to @jonc, yes I got crazy doing that and it really would be more usefull and effective with a program. But back then it was more simple and actually quicker (I've spend some nearly as much time with point (without any success) as i did manually). I might didn't have search right, but i haven't seen any topic about point on mission coding forum, so it might be usefull to create one (that will be a topic at it's right place for this time at least...) like there is for sanny builder to get more feedback from more people, and maybe a little tutorial for newbies and people who get skin allergy when they got to do something with a command line, and maybe explain the .beach .sand files....

And hoping it will get compatible with other decompiler/compiler so anyone can use it and anyone open the file compiled.

 

Have fun

Share this post


Link to post
Share on other sites
Demarest
I do still believe it to be a bad idea to arm somebody with the ability to make a release that can only be used and not deconstructed for the sake of tweaking, expanding upon, learning from, etc.

you have it backwards. point isn't doing anything; its output can be disassembled just fine, if the bugs in those tools are fixed. in fact, it was already possible to crash/confuse the disassemblers & the builder with valid code, before point ever existed.

I don't have it backwards. I totally understand that in this case, MB is at fault. But to come into a thread and say that a program will auto-slim it for you is going to have a bunch of people using it to slim their code, then wondering why they can't decompile it in the prog they're used to. My original intent was to let others know that this would not be possible. You know better than anybody that I'm constantly wrestling with MB's shortcomings. However, this is for the benefit of others and I wanted them to be aware that they would be encountering a one-way street to use the program just to slim it down... especially after the work to do so manually has already been fronted in part.

 

@tomworld10: Just be careful becuase sometimes separate if checks are separate for a reason wink.gif

Share this post


Link to post
Share on other sites
Un3462

ah, but when you're optimizing, you already sacrifice savegame compatibility, so returning to the builder after using point offers zero advantage. so nobody would be crazy enough to do that. ;)

Share this post


Link to post
Share on other sites
Demarest
It's not about savegame compatibility. Person A uses Point to make his mod trimmer and releases it. Person B downloads the mod, wishes to know how something is done and wants to avoid being told to check the code, so he reaches for a decompiler and SPLAT! So Person B asks Person A for the code. Well assuming nobody mentioned that using Point's optimization renders the SCM undecompilable by current decompilers, he can't get back to his own code. Again, I was mentioning it so people who weren't thinking that far ahead are aware. That doesn't mean they can't use it at all. It's just that if I was offering a program to do something and such a side effect existed, I would feel obligated to include it. Maybe jonc didn't, so I did. What I said was the truth. What anybody does with it is up to them. smile.gif

Share this post


Link to post
Share on other sites
Demarest
Not sure why I didn't realize this before. Unique Jumps in SA are defined by a single opcode.
0814: define_unique_jump -72.3  17.7  8.4132  2.81  2.4  4.48 -65.1154  35.7188  16.6432  7.71  2.0  7.27 -70.4781  55.8289  15.0868  500

They're 80 bytes in weight and there are 70 of them. There's 5,600 bytes since they do nothing for the game cool.gif

Share this post


Link to post
Share on other sites
Opius

So what you're saying is that progress should be stopped outright so we can continue to support an archane program that is NEVER going to be added to again, ever?

 

If the scene was like that from the start we'd still be using hex editors.

Share this post


Link to post
Share on other sites
Demarest
So what you're saying is that progress should be stopped outright so we can continue to support an archane program that is NEVER going to be added to again, ever?

If you can find where I said that at all, I'll dignify your question. If you could also explain what part of the fact that I was stating something that IS TRUE for the sake of information is confusing, I can maybe clarify better. For somebody that once defended the way I made sure others were well-informed, this is uncharacteristically manipulative of you. So let's recap:

 

What I said is true. A SCM compiled by Point cannot be decompiled by any current tool.

 

It doesn't mean anybody has to do ANYTHING differently. If they want to use Point, more power to them. Now they can do so with the knowledge that there SCM won't be decompilable. Unless there's an issue with an educated public? If anything, this could spare us a few "I can't decompile my script" gripes later on. Where's the problem? You know, I originally made the statement in hopes that I'd be proven wrong. It would be GOOD to hear that a decompiler exists that could handle the optimizations made.

 

Now you know why I don't get into fave sports teams discussions; people take things as bad-mouthing their team and get defensive wink.gif Maybe save trying to correct me until I misstate something. Like when I used proprietary in the abstract.

Share this post


Link to post
Share on other sites
jonc

Wow.. that caused quite a few posts. I don't have anything to add to CyQ's responses, so I'll leave those points alone.

 

@demonj0e: Yes, Point performs optimizations both when it decompiles and when it recompiles (so that the effects of those optimizations are hidden from the user). For example, when it decompiles, a jump to a jump is removed. And then when it recompiles, it will do even crazier optimizations like making loops out of a series of assignments. (These are hidden from the user since they make the code much harder to read and modify, but they save quite a bit of space.)

 

@Demarest: This is true, MB can't decompile it as is. (Maybe Sanny Builder could? I've never tried.) However, it can output natively in GTAMA format. Only problem is CyQ never wrote an assembler for SA in it.

 

@tomworld10: There was a very old post about it in the Mission Coding forum before it could decompile R*'s scripts. Since then I've been working with a few people privately to get it up to spec so everyone in the community would be able to use it easily. That time is almost here, but I just couldn't resist posting in this thread about it (it's hard not to be proud of what the optimizer can do).

 

That said, if you give it no parameters, it will launch a GUI environment which should be fairly self explanatory. Go to File, New, Imported Project and follow the dialog prompts for file paths.

Share this post


Link to post
Share on other sites
Demarest
@Demarest: This is true, MB can't decompile it as is. (Maybe Sanny Builder could? I've never tried.)

Me neither. In fact, I was kind of hoping that my comment would fish out that it is. Certainly didn't mean to cause a ruckus and I'm glad you didn't take it to heart. I'm also glad you explained a bit about how these optimizations are made. It's impressive to say the least. Don't mind me as my latest thing is savegame compatiblity, so I have to go where the DMA is is all.

Share this post


Link to post
Share on other sites
jonc

@Demarest: Certainly not, but I would suggest you haven't been checking changelogs. smile.gif v0.13 supports forced address allocation (with safeties so that an address isn't allocated twice) as well as unallocated memory usage (and unbounded by the variable block size). In my biased opinion, this is more powerful than DMA is. Although it still won't give you full compatibility since label offsets are likely to change.

 

In future releases, my goal is to allow full savegame compatibility without any user intervention whatsoever (I have some untested ideas for how to do this).

 

In fact, most of the optimizations are listed in the changelog, and I'll copy and paste them here:

* Collapse jump-to-return, jump-to-EndThread. (11k saved.)

* Roll sequential assignments (and other similar calls) on an array into a loop. (21k saved.)

* Remove __if opcode if only a single condition is present. (241k saved.)

* Collapse tail user function calls (__gosub followed by __return). (700b saved.)

* Collapse jumps-to-jumps and remove unreferenced code. (46k saved.)

* Minimize size of label parameters. (133k saved.)

(Those numbers are on the entire main.scm rather than the first-four-segments; otherwise it would have gone into negative size rather quickly as you can see.)

 

As you can see, none of those inherently make an SCM undecompilable; so I think the fact other tools can't open it is simply due to details of the SCM that Point generates different than other tools. I just tested disassembling using Sanny Builder and it got confused by a NOP Point placed in the source. By manually editing the SCM (I replaced that with a different 0-op instruction) it continues, but later (unspecified offset) it died saying %d format mismatch.

Share this post


Link to post
Share on other sites
random_download

Oh, the nop is supposed to be there? I just thought my decompilation had gone wrong as it had:

 

nopjump 183end_threadreturn

 

Is that how it is supposed to be?

Share this post


Link to post
Share on other sites
jonc

Yes, that is correct. The NOP is there because my own decompiler doesn't appreciate a jump being the first instruction of the main segment. And adding that was easier than fixing it. The return and endthread are used for the jump-to-X optimization I mentioned.

Share this post


Link to post
Share on other sites
random_download

Ah, good I will test with larger scripts to see if it still decompiles without errors.

Share this post


Link to post
Share on other sites
Demarest

In my biased opinion, this is more powerful than DMA is. Although it still won't give you full compatibility since label offsets are likely to change.
I agree with you. But that last part is where it escapes my interest. I'm glad you don't take that personally.

 

 

In future releases, my goal is to allow full savegame compatibility without any user intervention whatsoever (I have some untested ideas for how to do this).
The mere prospect is awesome. HOWEVER (and this is more of a difference in schools of approach with the same goal in mind), in my times focusing on such things, I've learned TONS about the format and other inner workings involved. Mostly I've discovered that it's not that hard, it just requires dedication. I've been tying to inspire others to take it up as well because I believe in education. Had I used your tool (as you sumise it will at one point be), I would be none the wiser. I rather appreciate the education as it has whet my appetite for a large number of things I needed the motivation for. Now if I could only find the time to pursue it wink.gif

 

Now that that's blown over (thankfully), I hope nobody missed the free 5,600 bytes I proposed. Quite large and dead easy colgate.gif

Share this post


Link to post
Share on other sites
jonc

Yep. And there's nothing wrong with knowing details (someone has to), but the general community really doesn't understand the issues involved and just wants a tool that does the right thing when they add their small changes.

Share this post


Link to post
Share on other sites
Un3462
Yes, that is correct. The NOP is there because my own decompiler doesn't appreciate a jump being the first instruction of the main segment. And adding that was easier than fixing it. The return and endthread are used for the jump-to-X optimization I mentioned.

i can see how jfs to returns/endthreads can be optimized with that (in fact, i did that in vc tetris), but for a jump to a return (or endthread), wouldn't it be shorter to simply replace the jump itself with a return?

 

also, you could probably jump over the return/endthread straight from the jump at the start of the segment right before the main code (and maybe you could even experiment with leaving out those final segments; in my exploration of the exe, i found it never seemed to load anything after the external thread segment).

 

 

dem; basically i already had this discussion with barton, when he said my arrays shouldn't be used because the disassemblers couldn't correctly deal with them. the solution is that everyone releases their source code, and if they don't, i'll disassemble and reconstruct their code on request. your position now seems to be though that you're just providing information, and not arguing anything, so i should probably let it rest. :p

Share this post


Link to post
Share on other sites
jonc

 

i can see how jfs to returns/endthreads can be optimized with that (in fact, i did that in vc tetris), but for a jump to a return (or endthread), wouldn't it be shorter to simply replace the jump itself with a return?

 

Yes, I should have been clearer, that is in fact how it works.

 

 

also, you could probably jump over the return/endthread straight from the jump at the start of the segment right before the main code (and maybe you could even experiment with leaving out those final segments; in my exploration of the exe, i found it never seemed to load anything after the external thread segment).

 

In theory yes, but I think that would break existing disassemblers even more and I already have enough people mad at me for that. smile.gif

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • 2 Users Currently Viewing
    0 members, 0 Anonymous, 2 Guests

×
×
  • Create New...

Important Information

By using GTAForums.com, you agree to our Terms of Use and Privacy Policy.