Quantcast

Jump to content

» «
Photo

[IV] ColAccel - reduce loading time by 50%*

41 replies to this topic
NTAuthority
  • NTAuthority

    hell, no, tunnel, no

  • Feroci
  • Joined: 09 Sep 2008
  • European-Union
  • Best Conversion 2014 [ViIV for GTANY]
    Most Knowledgeable [Tech] 2013
    Best Map 2013 [ViceCityStories PC Edition]
    Contribution Award [Mods]

#1

Posted 21 January 2014 - 01:38 AM

The problem

 

For CitizenIV, I'm intending to add other cities, which would be able to be flown to from the current city's airport. (the actual feature depends on me being able to write automated tools to convert at least VC and generate navigation meshes)

 

As I wanted that experience to be as seamless as possible, loading the game needs to be optimized to take less time. So, I took Process Monitor and looked at what happened from the point I selected to load the recently downloaded DLC (CitizenIV, again - a bug in my episode loader :) ) until I saw the scripts running (loading text) - in fact, the period during which the artwork loading screens appear. Out of the 18 seconds this process took on my system, around 9-10 were spent reading large chunks of data from various .img files.

 

I looked into the files at the offset shown by Process Monitor, and found RAGE resources of type 0x20 - collision models. I already know GTA III to SA were loading all collision meshes for various precalculation reasons during the game load, but seeing as this is around 400-500 MB of data in IV (which also has to be decompressed, as it is deflate'd, adding more time to the loading process) it takes fairly long - I wouldn't be surprised if V still does this, as it still loads slow on consoles...

 

The solution

 

As recently revealed in the mobile version of GTA:SA, Rockstar North had thought about this problem during the development of SA, and created a cache (internally named ColAccel, like my implementation is named as well) which stored the actual data read from the collision files (basically, the size of the collision models) in a cache, to load them without having to read through the full collision files. This feature never got used in SA, and clearly IV still did not have anything like that.

 

Well, except that I implemented something now. :)

 

ColAccel is a .asi plugin for GTA IV (without any external dependencies) which creates a 'colCache.dat' file on the first load, containing all the data actually used by the game from the collision meshes (basically, 580 kB with a lot of redundant data, nothing like 500 MB!) and accelerating your game load time by 50% or more.

 

Note that the plugin currently only works on GTA IV 1.0.7.0 - if demand is sufficient I'll make a version for the latest EFLC version as well.

 

In addition, you *should* delete colCache.dat any time you a) add additional .img files or b) replace any .wbn/.wbd files inside any .img, either directly or indirectly - if not, various bad side effects can and will happen.

  • Alt, lpgunit, cp1dell and 19 others like this

El Dorado
  • El Dorado

    Set the controls to the heart of the sun

  • Feroci
  • Joined: 07 Nov 2013
  • Brazil

#2

Posted 21 January 2014 - 02:04 AM Edited by format c:, 21 January 2014 - 02:07 AM.

This can in any way increase in-game performance (fps)?


Ash_735
  • Ash_735

    1 627 826 3789

  • Feroci
  • Joined: 15 Nov 2005
  • None
  • Contribution Award [Mods]
    Most Knowledgeable [GTA] 2013
    Best Map 2013 "ViceCityStories PC Edition"

#3

Posted 21 January 2014 - 02:41 AM

Nice, always looking to make IV more bareable, reducing initial load should help with that.
  • lpgunit, Ss4gogeta0 and snoops26 like this

Marty McFly
  • Marty McFly

    Madness is like gravity. All it takes it a little push!

  • Feroci
  • Joined: 07 Nov 2010
  • Germany
  • Contribution Award [Mods]

#4

Posted 21 January 2014 - 05:34 AM

Could you be so kind to do it for 1.0.4.0 as well? Patch 4 and 7 are mostly used and we enb modders prefer patch 4. Besides that: awesome work!
  • lpgunit and bst81 like this

ThirteenAG
  • ThirteenAG

    Soldier

  • Feroci
  • Joined: 29 Dec 2008
  • None
  • Contribution Award [Mods]

#5

Posted 21 January 2014 - 05:51 AM

Note that the plugin currently only works on GTA IV 1.0.7.0 - if demand is sufficient I'll make a version for the latest EFLC version as well.

Sign me up :) By the way, do you think it's possible to create a plugin, which will load last saved game automatically? Like in IV/V on consoles. Removing intro screens completely would be nice too.

lpgunit
  • lpgunit

    It's L, as in Lpgunit, not I.

  • Feroci
  • Joined: 24 May 2008
  • Philippines

#6

Posted 21 January 2014 - 06:28 AM Edited by lpgunit, 21 January 2014 - 06:34 AM.

Could you be so kind to do it for 1.0.4.0 as well? Patch 4 and 7 are mostly used and we enb modders prefer patch 4. Besides that: awesome work!


Yeah, it would be neato if you do one for 1.0.4.0 as well. Besides, most other people who play with patch 4 are those running the game on lower-end systems such as myself.
  • PacketOVerload_x64Bit likes this

BisP
  • BisP

    Ghetto Star

  • Members
  • Joined: 17 Apr 2011

#7

Posted 21 January 2014 - 12:20 PM

Please please do it for patch 4.  :colgate:

  • PacketOVerload_x64Bit likes this

Zer0w5
  • Zer0w5

    There are no stupid questions, just stupid people.

  • Members
  • Joined: 07 Dec 2008
  • None

#8

Posted 21 January 2014 - 06:08 PM

Tried it on my version of GTA IV 1.0.7.0 and it seems to do its job, thanks :beerhat:


buzzbass
  • buzzbass

    mayorista no limpio pescado

  • Members
  • Joined: 11 Jun 2011
  • Spain

#9

Posted 28 March 2014 - 03:23 PM

Thanks for this, a EFLC would be welcome too :D


T-ru
  • T-ru

    kamikadze

  • Members
  • Joined: 02 Sep 2009

#10

Posted 29 March 2014 - 05:06 AM

This's great, and yes make it for EFLC.

  • buzzbass likes this

bst81
  • bst81

    GTA V & Watch Dogs videos

  • Members
  • Joined: 09 Mar 2014
  • None

#11

Posted 29 March 2014 - 06:35 AM

Good. Please make for patch 4 too

Hopefully you will do it 


PacketOVerload_x64Bit
  • PacketOVerload_x64Bit

    ENB Whore

  • Members
  • Joined: 13 Mar 2010
  • Canada
  • Contribution Award [Mods]

#12

Posted 02 April 2014 - 05:37 AM Edited by PacketOVerload_x64Bit, 02 April 2014 - 05:38 AM.

Amazing. Many, many folks still on patch 4 for ENB. Most will never move from Patch 1040/1004 until Boris V. fixes skylighting at the internal D3D9.dll level, so if your ever inclined to, ColAccel for one of 1030/1040/1004 patches would be awesome. 

  • lpgunit likes this

BlackScout
  • BlackScout

    Snowy and Cold

  • Members
  • Joined: 08 Sep 2013
  • Greenland

#13

Posted 10 April 2014 - 01:38 AM

Is it able to work in your ViIV?


NTAuthority
  • NTAuthority

    hell, no, tunnel, no

  • Feroci
  • Joined: 09 Sep 2008
  • European-Union
  • Best Conversion 2014 [ViIV for GTANY]
    Most Knowledgeable [Tech] 2013
    Best Map 2013 [ViceCityStories PC Edition]
    Contribution Award [Mods]

#14

Posted 14 April 2014 - 12:43 PM

Is it able to work in your ViIV?

not currently as viiv uses more collision files than colaccel allocates; eventually the framework will contain it though (and somehow cache the img-loaded files/replace img streaming itself)

  • Mr. Jago and BlackScout like this

gajrajgchouhan
  • gajrajgchouhan

    tommycjnikojohnnyluis fan

  • Members
  • Joined: 08 Dec 2013
  • India

#15

Posted 24 August 2014 - 02:47 PM

please make it for eflc

cp1dell
  • cp1dell

    Big Homie

  • Members
  • Joined: 04 Jul 2008
  • None

#16

Posted 25 August 2014 - 08:37 PM

Wow this is great. Yes, please make this for EFLC. I don't see why you shouldn't.


InfamousSabre
  • InfamousSabre

    Harpocrates

  • Members
  • Joined: 03 Jan 2013

#17

Posted 18 November 2014 - 07:36 PM

Another vote for EFLC, here! Very cool script!

  • buzzbass likes this

Blackbird88
  • Blackbird88

    Not using Cinematic FPS

  • Members
  • Joined: 21 Jul 2008
  • Czech-Republic

#18

Posted 18 November 2014 - 08:00 PM

Yes EFLC please.

  • buzzbass likes this

Silent
  • Silent

    Moderating Cookie Monster

  • Feroci
  • Joined: 01 Feb 2010
  • Poland
  • Contribution Award [Mods]
    Best Script/Plugin 2014 [SilentPatch]
    Most Respected 2014
    Most Helpful [Mods] 2014
    Most Helpful [GTA] 2013
    Most Helpful [Mods] 2013
    Most Talented [Modding] 2013
    Best Map 2013 [ViceCityStories PC Edition]
    Best Vehicle 2013 [III Aircraft]
    Most Helpful [Mods] 2012
    Modder of the Year 2012

#19

Posted 18 November 2014 - 09:59 PM

Do EFLC, yes.
  • buzzbass likes this

LINK/2012
  • LINK/2012

    LIVIN' IN CODE

  • Feroci
  • Joined: 30 Jan 2011
  • Brazil
  • Best Tool 2014 [Mod Loader]
    Contribution Award [Mods]

#20

Posted 18 November 2014 - 11:38 PM

SA please

  • Androll, Fantomax and snoops26 like this

fefenc
  • fefenc

    Blue Dante

  • Members
  • Joined: 12 Apr 2014
  • Brazil

#21

Posted 19 November 2014 - 12:04 AM Edited by fefenc, 19 November 2014 - 12:15 AM.

I vote for an EFLC version, playing as Luis would be more funny if I could load the game faster o/

  • buzzbass likes this

gta.bullet
  • gta.bullet

    Player Hater

  • Members
  • Joined: 19 Jun 2010

#22

Posted 21 November 2014 - 06:24 PM

50s vs 20s for me. better than 50%.


nyolc8
  • nyolc8

    -

  • Members
  • Joined: 12 Oct 2009
  • Hungary

#23

Posted 10 January 2015 - 02:21 PM

EFLC version please :3 But how would that work? Because the two episodes have different files...

Blackbird88
  • Blackbird88

    Not using Cinematic FPS

  • Members
  • Joined: 21 Jul 2008
  • Czech-Republic

#24

Posted 10 January 2015 - 04:55 PM

EFLC version please :3 But how would that work? Because the two episodes have different files...

I already PM'd him about it. No response though so it's safe to say it's not happening now :(


ThirteenAG
  • ThirteenAG

    Soldier

  • Feroci
  • Joined: 29 Dec 2008
  • None
  • Contribution Award [Mods]

#25

Posted 10 January 2015 - 05:44 PM

Making the source code public would help, I believe.
  • Blackbird88 and buzzbass like this

nkjellman
  • nkjellman

    Boss

  • Members
  • Joined: 28 Aug 2008
  • United-States

#26

Posted 12 January 2015 - 08:05 AM

This will definitely be needed on GTA V. That game takes ages to load.

  • lpgunit likes this

Silent
  • Silent

    Moderating Cookie Monster

  • Feroci
  • Joined: 01 Feb 2010
  • Poland
  • Contribution Award [Mods]
    Best Script/Plugin 2014 [SilentPatch]
    Most Respected 2014
    Most Helpful [Mods] 2014
    Most Helpful [GTA] 2013
    Most Helpful [Mods] 2013
    Most Talented [Modding] 2013
    Best Map 2013 [ViceCityStories PC Edition]
    Best Vehicle 2013 [III Aircraft]
    Most Helpful [Mods] 2012
    Modder of the Year 2012

#27

Posted 12 January 2015 - 01:33 PM

But console GTAs have colaccel at least since SA.
  • lpgunit likes this

BlackScout
  • BlackScout

    Snowy and Cold

  • Members
  • Joined: 08 Sep 2013
  • Greenland

#28

Posted 21 January 2015 - 12:09 AM

But console GTAs have colaccel at least since SA.

PS2 Versions loading time used to be horrible, really horrible...

  • Marty McFly likes this

Blackbird88
  • Blackbird88

    Not using Cinematic FPS

  • Members
  • Joined: 21 Jul 2008
  • Czech-Republic

#29

Posted 22 January 2015 - 07:59 AM Edited by Blackbird88, 22 January 2015 - 08:00 AM.

 

But console GTAs have colaccel at least since SA.

PS2 Versions loading time used to be horrible, really horrible...

 

Because they have to load the game from DVD drive. I wonder if SA could be installed to PS2's HDD. Most games can't be.


AgentWD40
  • AgentWD40

    GLITCHED

  • Members
  • Joined: 02 May 2009
  • Philippines

#30

Posted 28 January 2015 - 09:59 AM

Wow, wished you made this sooner. Hopefully you could do the same with GTAV because those loading times are ridiculous and can't even play while waiting for matchmaking.





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users