Quantcast
Jump to content
Search In
  • More options...
Find results that contain...
Find results in...
    1. Welcome to GTAForums!   (85,382 visits to this link)

    2. News

    1. GTA Online

      1. Find Lobbies & Players
      2. Guides & Strategies
      3. Vehicles
      4. Content Creator
      5. Help & Support
    2. Crews

      1. Events
      2. Recruitment
    1. Grand Theft Auto Series

    2. GTA Next

    3. GTA V

      1. PC
      2. Guides & Strategies
      3. Help & Support
    4. GTA IV

      1. Episodes from Liberty City
      2. Multiplayer
      3. Guides & Strategies
      4. Help & Support
      5. GTA Mods
    5. GTA Chinatown Wars

    6. GTA Vice City Stories

    7. GTA Liberty City Stories

    8. GTA San Andreas

      1. Guides & Strategies
      2. Help & Support
      3. GTA Mods
    9. GTA Vice City

      1. Guides & Strategies
      2. Help & Support
      3. GTA Mods
    10. GTA III

      1. Guides & Strategies
      2. Help & Support
      3. GTA Mods
    11. Top Down Games

      1. GTA Advance
      2. GTA 2
      3. GTA
    12. Wiki

      1. Merchandising
    1. GTA Modding

      1. GTA V
      2. GTA IV
      3. GTA III, VC & SA
      4. Tutorials
    2. Mod Showroom

      1. Scripts & Plugins
      2. Maps
      3. Total Conversions
      4. Vehicles
      5. Textures
      6. Characters
      7. Tools
      8. Other
      9. Workshop
    3. Featured Mods

      1. DYOM
      2. OpenIV
      3. GTA: Underground
      4. GTA: Liberty City
      5. GTA: State of Liberty
    1. Red Dead Redemption 2

    2. Red Dead Redemption

    3. Rockstar Games

    1. Off-Topic

      1. General Chat
      2. Gaming
      3. Technology
      4. Programming
      5. Movies & TV
      6. Music
      7. Sports
      8. Vehicles
    2. Expression

      1. Graphics / Visual Arts
      2. GFX Requests & Tutorials
      3. Writers' Discussion
      4. Debates & Discussion
    1. Forum Support

    2. Site Suggestions

MickB

World To Screen Lag

Recommended Posts

MickB

Using the _WORLD3D_TO_SCREEN2D native I can successfully draw text over entities in the game world but there is an obvious lag as when moving the camera around the text will move a bit before snapping back to the entity. Obviously I'm running it in a loop every tick but I'm guessing that ScriptHookV threads are not actually called as often as every game frame is rendered which is causing this lag?

Share this post


Link to post
Share on other sites
InfamousSabre

That's just how it is. Nothing that you can do about it, unfortunately.

 

Read below :D

Edited by InfamousSabre

Share this post


Link to post
Share on other sites
CamxxCore

Using the _WORLD3D_TO_SCREEN2D native I can successfully draw text over entities in the game world but there is an obvious lag as when moving the camera around the text will move a bit before snapping back to the entity. Obviously I'm running it in a loop every tick but I'm guessing that ScriptHookV threads are not actually called as often as every game frame is rendered which is causing this lag?

I'm not sure if it will work with custom textures but you can use natives SET_DRAW_ORIGIN and CLEAR_DRAW_ORIGIN to draw without any stuttering/ lag. I ran into similar issues when making some scripts.

Share this post


Link to post
Share on other sites
InfamousSabre

 

Using the _WORLD3D_TO_SCREEN2D native I can successfully draw text over entities in the game world but there is an obvious lag as when moving the camera around the text will move a bit before snapping back to the entity. Obviously I'm running it in a loop every tick but I'm guessing that ScriptHookV threads are not actually called as often as every game frame is rendered which is causing this lag?

I'm not sure if it will work with custom textures but you can use natives SET_DRAW_ORIGIN and CLEAR_DRAW_ORIGIN to draw without any stuttering/ lag. I ran into similar issues when making some scripts.

 

Really? Wow, I'll have to try this.

 

Edit: Wow no sh*t. It's totally lag-less. Thanks, CamxxCore!

Edited by InfamousSabre

Share this post


Link to post
Share on other sites
MickB

 

Using the _WORLD3D_TO_SCREEN2D native I can successfully draw text over entities in the game world but there is an obvious lag as when moving the camera around the text will move a bit before snapping back to the entity. Obviously I'm running it in a loop every tick but I'm guessing that ScriptHookV threads are not actually called as often as every game frame is rendered which is causing this lag?

I'm not sure if it will work with custom textures but you can use natives SET_DRAW_ORIGIN and CLEAR_DRAW_ORIGIN to draw without any stuttering/ lag. I ran into similar issues when making some scripts.

 

 

Thanks looks promising. Pretty sure it won't support custom textures only the native draw functions.

Share this post


Link to post
Share on other sites
TriRozhka

Im trying to draw helicopterhud hud_corner but it look like square instead off L. Maybe it doesnt like orange color ?

Share this post


Link to post
Share on other sites
mmuaz

 

Using the _WORLD3D_TO_SCREEN2D native I can successfully draw text over entities in the game world but there is an obvious lag as when moving the camera around the text will move a bit before snapping back to the entity. Obviously I'm running it in a loop every tick but I'm guessing that ScriptHookV threads are not actually called as often as every game frame is rendered which is causing this lag?

I'm not sure if it will work with custom textures but you can use natives SET_DRAW_ORIGIN and CLEAR_DRAW_ORIGIN to draw without any stuttering/ lag. I ran into similar issues when making some scripts.

 

I am trying to get just 2d screen coordinates. I don't want to draw them on the screen. Is it possible that if I could only have non-lagged 2d screen points? Because when I try to use _WORLD3D_TO_SCREEN2D, box lags a lot especially when I move the camera really fast.

Edited by mmuaz

Share this post


Link to post
Share on other sites
K^2

Yeah, that's a standard issue in game-dev. You run the script, get the screen coordinates, then camera updates, and only then do draw operations happen. As the result, screen positions lag a frame behind. Last game engine I was working on had a special post-update pass that ran after the camera update and allowed screen-space FX to get the up-to-frame coordinates.

 

There's some smoothing you can do by filtering the positions and estimating next frame position. But that won't completely remove the lag. Just make it less jumpy. To fix it completely, you have to either rely on native functionality designed specifically to make this work, or write a work-around. If your script runs sufficiently late, and it's just the matrix that's out of date, alex8b's suggestion of reading matrix directly from memory might work.

Share this post


Link to post
Share on other sites
CamxxCore

Why not look at what I suggested above already? Using those functions seemed to eliminate the lag entirely. Edit: Oops. Sorry I didn't read.

Edited by CamxxCore

Share this post


Link to post
Share on other sites
LeeC2202

Yeah, that's a standard issue in game-dev. You run the script, get the screen coordinates, then camera updates, and only then do draw operations happen.

 

I have never actually seen that problem before... that's not me being my usual argumentative self, I have honestly never seen that problem.

 

When I saw it happen in this game, I presumed it was because the script processes are being hooked and might be being queued for the next redraw, rather than being done at the end of the current one... which is where I would expect all overlay processing/coordinate tracking to be done.

 

Has anyone checked to see if SET_DRAW_ORIGIN kicks in an associated priority change that forces elements to be processed during the current update by the game, rather than being queued? I notice the final parameter in that native is always 0, does changing that affect anything? Does that 0 mean zero delay? I can't see a reason for that final parameter to be there, unless there is a default setting that is different. **see Edit**

 

I mean, the fact that changing the origin cures the problem, does seem to indicate that the resolving of worldspace to screenspace is being accurately handled in the current frame. If it was being delayed, that origin would be being set to a location that no longer exists in the next frame, because it would have already moved... so the lag should remain.

 

Just to put this into perspective with something I am working on, I have a UI being drawn made from UIRectangles and UIDrawTextures. If I move a pair of elements (one of each) left or right, the Rectangles lag behind the DrawTextures, so it appears as though the sprites are being pushed ahead of the cursor. When I stop moving, both elements align perfectly, so it's not as if there are some calculation errors in the process. So I wonder if by default, certain elements are processed immediately and some are pushed back a frame and this SET_DRAW_ORIGIN forces them both to be done immediately. Maybe at the default setting, it is intended for elements that have no bearing on what is on-screen, like cash displays, road names etc...

 

I am mostly just thinking out loud here... but I know curiosity did kill the cat. :D

 

Edit: Being unable to resist, I made a quick mod to test a theory. Setting that final parameter to anything other than 0, seems to stop it working altogether. I tried 1, 2 and -1 but only 0 worked. So that's my theory busted...

Edited by LeeC2202

Share this post


Link to post
Share on other sites
K^2

I have never actually seen that problem before...

It's not as likely to manifest if you build the engine right the first time. The experience I was describing was with an engine that was written as a single-threaded task more than a decade ago, and has since become a massively multi-threaded beast of legacy code running on multiple platforms. When Tiamat fight landed on my desk, it was chugging at 3FPS on XBOne. I am not proud of things I had to do to fix it.

 

What usually happens is that the engine expects things to happen in certain order. Then you realize, you can get more performance by scheduling this on multiple threads if you change the order a little bit. And now, the character movement controller, which is what's going to adjust the camera matrix, might lag behind the FX update, but still lead the rendering thread. So you end up with FX screen positions a frame out of date. Naturally, if you've thought of it from the start, you could have simply deferred necessary operations. But if you didn't, you end up with an a FX late update pass as an ugly hack to try and fix the issue. It's not pretty, but this console port is already a month behind, and you still have a pile of race conditions to fix.

 

Now, I'm by no means advocating writing code like that, but you put 20 people who are way over their head on tight schedule, and that's the sort of code quality you're going to get. And that happens in game dev left and right, I'm sure you know. So just count yourself lucky you haven't had to deal with that particular mess.

Share this post


Link to post
Share on other sites
alex8b

Pure C# implementation of World to Screen projection via getting camera matrix

 

https://github.com/alex8b/gta5eyetracking/blob/master/Gta5EyeTracking/CameraHelper.cs

 

https://github.com/alex8b/gta5eyetracking/blob/master/Gta5EyeTracking/SigScan.cs

using System;using System.Diagnostics;using System.Runtime.InteropServices;using SharpDX;namespace Gta5EyeTracking{	[structLayout(LayoutKind.Sequential, Pack = 1)]	public struct CViewPortGame	{		public unsafe fixed byte _0x0000[588];		public unsafe fixed float mViewMatrix[16]; //0x024C 	};//Size=0x028C	public static class CameraHelper	{		private static IntPtr _gPViewPortGame = IntPtr.Zero;		private static IntPtr GetViewPortGame(IntPtr baseAddress, int length)		{			if (_gPViewPortGame == IntPtr.Zero)			{				SigScan.Classes.SigScan sigScan = new SigScan.Classes.SigScan(Process.GetCurrentProcess(), baseAddress, length);				IntPtr matricesManagerInc = sigScan.FindPattern(new byte[] { 0x48, 0x8B, 0x15, 0xFF, 0xFF, 0xFF, 0xFF, 0x48, 0x8D, 0x2D, 0xFF, 0xFF, 0xFF, 0xFF, 0x48, 0x8B, 0xCD }, "xxx????xxx????xxx", 0);				if (matricesManagerInc != IntPtr.Zero)				{					var offset = Marshal.PtrToStructure<int>(new IntPtr(matricesManagerInc.ToInt64() + 3));					var ptr = new IntPtr(offset + matricesManagerInc.ToInt64() + 7);					_gPViewPortGame = new IntPtr(Marshal.PtrToStructure<long>(ptr));				}			}			return _gPViewPortGame;		}		public static Matrix GetCameraMatrix()		{			IntPtr baseAddress = System.Diagnostics.Process.GetCurrentProcess().MainModule.BaseAddress;			int length = System.Diagnostics.Process.GetCurrentProcess().MainModule.ModuleMemorySize;			var viewPortGamePtr = GetViewPortGame(baseAddress, length);			if (viewPortGamePtr != IntPtr.Zero)			{				var viewPortGame = Marshal.PtrToStructure<CViewPortGame>(viewPortGamePtr);				unsafe				{					var matrix = viewPortGame.mViewMatrix;					return new Matrix(matrix[0], matrix[1], matrix[2], matrix[3], matrix[4], matrix[5], matrix[6], matrix[7], matrix[8],						matrix[9], matrix[10], matrix[11], matrix[12], matrix[13], matrix[14], matrix[15]);				}			}			return Matrix.Identity;		}	}}
        public static bool WorldToScreenRel(Vector3 entityPosition, out Vector2 screenCoords)        {            var mView = CameraHelper.GetCameraMatrix();            mView.Transpose();            var vForward = mView.Row4;            var vRight = mView.Row2;            var vUpward = mView.Row3;            var result = new SharpDX.Vector3(0,0,0);            result.Z = (vForward.X * entityPosition.X) + (vForward.Y * entityPosition.Y) + (vForward.Z * entityPosition.Z) + vForward.W;            result.X = (vRight.X * entityPosition.X) + (vRight.Y * entityPosition.Y) + (vRight.Z * entityPosition.Z) + vRight.W;            result.Y = (vUpward.X * entityPosition.X) + (vUpward.Y * entityPosition.Y) + (vUpward.Z * entityPosition.Z) + vUpward.W;            if (result.Z < 0.001f)            {                screenCoords = new Vector2(0, 0);                return false;            }            float invw = 1.0f / result.Z;            result.X *= invw;            result.Y *= invw;            screenCoords = new Vector2(result.X, -result.Y);            return true;        }
Edited by alex8b

Share this post


Link to post
Share on other sites
mmuaz

Thank you everyone for your response and healthy discussion.

@alex8b, I am also trying to get 2d points using matrix as you are doing in code. Is it working fine for you?

As what I have tried, you can see in this image that 2d projection of

e.Position // where e = Game.Player.Character.CurrentVehicle;

(in green) using your code is not accurate as well.

So what exactly I am doing is I take a screenshot of the game at each tick and save it on 852 x 640 resolution. And then draw following point on the screen.

Math.Abs(my2dPoint.X * 852)Math.Abs(my2dPoint.Y * 640)

Whereas, the 3d bounding box seems to be perfect calculated using this repo.

Also, components of 2d projected points are mostly negative. I don't know what wrong I am doing.

Edited by mmuaz

Share this post


Link to post
Share on other sites
alex8b

WorldToScreenRel returns screen coordinates normalized from -1 to 1.

 

In order to convert them to UI coordinates, use the following code:

 

var uiCoords = new Vector2(UI.WIDTH * 0.5f + screenCoords.X * UI.WIDTH * 0.5f, UI.HEIGHT * 0.5f + screenCoords.Y * UI.HEIGHT * 0.5f);

Edited by alex8b

Share this post


Link to post
Share on other sites
Cuky

 

Pure C# implementation of World to Screen projection via getting camera matrix

 

https://github.com/alex8b/gta5eyetracking/blob/master/Gta5EyeTracking/CameraHelper.cs

 

https://github.com/alex8b/gta5eyetracking/blob/master/Gta5EyeTracking/SigScan.cs

using System;using System.Diagnostics;using System.Runtime.InteropServices;using SharpDX;namespace Gta5EyeTracking{	[structLayout(LayoutKind.Sequential, Pack = 1)]	public struct CViewPortGame	{		public unsafe fixed byte _0x0000[588];		public unsafe fixed float mViewMatrix[16]; //0x024C 	};//Size=0x028C	public static class CameraHelper	{		private static IntPtr _gPViewPortGame = IntPtr.Zero;		private static IntPtr GetViewPortGame(IntPtr baseAddress, int length)		{			if (_gPViewPortGame == IntPtr.Zero)			{				SigScan.Classes.SigScan sigScan = new SigScan.Classes.SigScan(Process.GetCurrentProcess(), baseAddress, length);				IntPtr matricesManagerInc = sigScan.FindPattern(new byte[] { 0x48, 0x8B, 0x15, 0xFF, 0xFF, 0xFF, 0xFF, 0x48, 0x8D, 0x2D, 0xFF, 0xFF, 0xFF, 0xFF, 0x48, 0x8B, 0xCD }, "xxx????xxx????xxx", 0);				if (matricesManagerInc != IntPtr.Zero)				{					var offset = Marshal.PtrToStructure<int>(new IntPtr(matricesManagerInc.ToInt64() + 3));					var ptr = new IntPtr(offset + matricesManagerInc.ToInt64() + 7);					_gPViewPortGame = new IntPtr(Marshal.PtrToStructure<long>(ptr));				}			}			return _gPViewPortGame;		}		public static Matrix GetCameraMatrix()		{			IntPtr baseAddress = System.Diagnostics.Process.GetCurrentProcess().MainModule.BaseAddress;			int length = System.Diagnostics.Process.GetCurrentProcess().MainModule.ModuleMemorySize;			var viewPortGamePtr = GetViewPortGame(baseAddress, length);			if (viewPortGamePtr != IntPtr.Zero)			{				var viewPortGame = Marshal.PtrToStructure<CViewPortGame>(viewPortGamePtr);				unsafe				{					var matrix = viewPortGame.mViewMatrix;					return new Matrix(matrix[0], matrix[1], matrix[2], matrix[3], matrix[4], matrix[5], matrix[6], matrix[7], matrix[8],						matrix[9], matrix[10], matrix[11], matrix[12], matrix[13], matrix[14], matrix[15]);				}			}			return Matrix.Identity;		}	}}
        public static bool WorldToScreenRel(Vector3 entityPosition, out Vector2 screenCoords)        {            var mView = CameraHelper.GetCameraMatrix();            mView.Transpose();            var vForward = mView.Row4;            var vRight = mView.Row2;            var vUpward = mView.Row3;            var result = new SharpDX.Vector3(0,0,0);            result.Z = (vForward.X * entityPosition.X) + (vForward.Y * entityPosition.Y) + (vForward.Z * entityPosition.Z) + vForward.W;            result.X = (vRight.X * entityPosition.X) + (vRight.Y * entityPosition.Y) + (vRight.Z * entityPosition.Z) + vRight.W;            result.Y = (vUpward.X * entityPosition.X) + (vUpward.Y * entityPosition.Y) + (vUpward.Z * entityPosition.Z) + vUpward.W;            if (result.Z < 0.001f)            {                screenCoords = new Vector2(0, 0);                return false;            }            float invw = 1.0f / result.Z;            result.X *= invw;            result.Y *= invw;            screenCoords = new Vector2(result.X, -result.Y);            return true;        }

 

I can't run your code. In WorldToScreenRel, Visiual Studio 2017 is saying, that var result = new SharpDX.Vector3(0,0,0) is wrong. But new SharpDX.Mathematics.Interop.RawVector3(0, 0, 0) works. Is this because of different SharpDX versions?

 

And in CameraHelper.cs, there is no variable type Matrix. When I change it to SharpDX.Mathematics.Interop.RawMatrix,

 return new SharpDX.Mathematics.Interop.RawMatrix(matrix[0], matrix[1], matrix[2], matrix[3], matrix[4], matrix[5], matrix[6], matrix[7], matrix[8],                        matrix[9], matrix[10], matrix[11], matrix[12], matrix[13], matrix[14], matrix[15]);

I get the compiler error, that there is no such constructor! Can you please tell me how to fix this?

 

Thanks.

Edited by Cuky

Share this post


Link to post
Share on other sites
Cuky

Since the last update I have a weired problem with transforming 3D points into 2D like shown in the images.

1. It seems to work normal, like it should: https://ibb.co/nvuZhR

2. but as soon as the camera passes a point, the transformation gets wrong. The back corners are projected to the wrong direction: https://ibb.co/iF1ap6

 

I tried to transform with alex8b code and also with another transformation, but the result is the same.

 

I'm struggling for a couple of days with this problem. Does someone has an answer for me?

Share this post


Link to post
Share on other sites
Billion

Using the _WORLD3D_TO_SCREEN2D native I can successfully draw text over entities in the game world but there is an obvious lag as when moving the camera around the text will move a bit before snapping back to the entity. Obviously I'm running it in a loop every tick but I'm guessing that ScriptHookV threads are not actually called as often as every game frame is rendered which is causing this lag?

off topic but can you make a video of this ? Very curious

Share this post


Link to post
Share on other sites
Cuky

Since the last update I have a weired problem with transforming 3D points into 2D like shown in the images.

1. It seems to work normal, like it should: https://ibb.co/nvuZhR

2. but as soon as the camera passes a point, the transformation gets wrong. The back corners are projected to the wrong direction: https://ibb.co/iF1ap6

 

I tried to transform with alex8b code and also with another transformation, but the result is the same.

 

I'm struggling for a couple of days with this problem. Does someone has an answer for me?

 

Am I really the only one having this issue?

Share this post


Link to post
Share on other sites
alex8b

Hi!

 

Try checking x y z coords of the 3d object.

Try doing the same logic in a different part of the map.

Share this post


Link to post
Share on other sites
Cuky

Hi alex8b.

 

Unfortunately, this happens everywhere on the map. I'm not good at 3D calculations and transformations etc. Can you tell how to do that exactly?

I calculate the 8 corners of the car and apply your transformation on every corner. But as soon as a corner is behind the cam, it gets projected to the wrong side. This happens on scripted cams and the gameplay camera.

Edited by Cuky

Share this post


Link to post
Share on other sites
alex8b

Of course it is. It is behind the cam :)

Share this post


Link to post
Share on other sites
Cuky

Of course it is. It is behind the cam :)

:cool: I already understand this.

 

What do I have to do to prevent this? Something like 3D line clipping?

1. Calculating planes of frustum - how?

2. Checking every line etc? - how?

:/

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×

Important Information

By using GTAForums.com, you agree to our Terms of Use and Privacy Policy.