God damn, you guys know your s*it that was it. Even though I'm using the PS2 loadout-Skygfx1, I could see the droplets for the brief moment when car hit the water, so without neo folder and colorcycle.dat file in my root dir, they couldn't be loaded and that caused the crash. Thanks!
Why you have to delete anything? You could just disable the raindrops from the ini file.
I can't really remember, since it was some time ago, but deletion of the mentioned folder solved it for me even though that droplets were turned off in the first place. Later on, and it took me a while to get to the bottom of this, the crash problem was caused by Fastman's Limiter. No matter what I tried, it was always this mod causing crashes, so I definitely deleted it for good.