Quantcast

Jump to content

» «
Photo

Getting depth stencil and segmentation data

4 replies to this topic
mmuaz
  • mmuaz

    Player Hater

  • Members
  • Joined: 25 Sep 2017
  • Germany

#1

Posted 27 September 2017 - 03:40 PM Edited by mmuaz, 27 September 2017 - 04:26 PM.

I am trying to run this repository umautobots/GTAVisionExport/native. I have compiled and built it successfully following instructions on the repo. 

I posted some issues on this link under name muaz-git. Does anyone have ever tried to run this repository and could help me in running this repository. I am pretty sure that I am either missing some driver or library on runtime otherwise it wouldn't have compiled properly. Also for most of the people it works properly or at least to some extent. 
 
I am kind of new to C++ and not really sure how to debug and figure out what I am doing wrong. here are my complete PC Specs. I would really appreciate if anyone could help me in running this repository. I am struggling on it for months now and finally decided to post here. I will really appreciate your help.
 
Actually I am not able to get depth maps of the game after pressing the `L` key. After bit of manual debugging I found out that in this line is getting executed for me all the time. Besides that whenever I try to save depth map, the conditions here are not satisfied which is I think the reason that I am not getting depth map. Specifically, `Format` of `D3D11_TEXTURE2D_DESC` doesn't match sometimes also `Width` and `Height` of `D3D11_TEXTURE2D_DESC` are not always greater than `1000` (which is the reason that condition will be trued).

K^2
  • K^2

    Vidi Vici Veni

  • Moderator
  • Joined: 14 Apr 2004
  • United-States
  • Best Poster [Technology / Programming] 2016
    Best Poster [Programming] 2015
    Most Knowledgeable [Web Development/Programming] 2013
    Most Knowledgeable [GTA Series] 2011
    Best Debater 2010

#2

Posted 28 September 2017 - 05:28 AM

Do you need to monitor these continuously for the mod, or are you just looking for a snapshot for some external use? Because in the later case, I would recommend to just use RenderDoc.

mmuaz
  • mmuaz

    Player Hater

  • Members
  • Joined: 25 Sep 2017
  • Germany

#3

Posted 28 September 2017 - 11:58 AM

Well, I wanted to use it for external purpose. As I plan to collect segmentation data in order to use them for my thesis. I have looked in to renderdoc as per the guys from Dramstadt have worked on. But as suggested by them, it's kind of semi-automatic. I am looking for something, which should be completely automatic. 


azathoth42
  • azathoth42

    Player Hater

  • Members
  • Joined: 4 weeks ago
  • Czech-Republic

#4

Posted 4 weeks ago

Well, I wanted to use it for external purpose. As I plan to collect segmentation data in order to use them for my thesis. I have looked in to renderdoc as per the guys from Dramstadt have worked on. But as suggested by them, it's kind of semi-automatic. I am looking for something, which should be completely automatic. 

Hi, I am now too using the umautobots/GTAVisionExport as part of my thesis. Few days ago, I managed to get working the automatic gathering of screenshots with depth and stencil data. I proposed pull request with fully working automatic driving and collecting both screenshots to tiff and saving metadata to PostgreSQL database.
Btw, I am racinmat on the github.

How are you proceeding with it? I am now struggling with displaying bounding boxes over the screenshot. Did you figure that out? And what about the depth data? Does it work for you properly?


Cuky
  • Cuky

    Player Hater

  • Members
  • Joined: 23 Sep 2017
  • Germany

#5

Posted A week ago

Hi guys,

the umautobots/GTAVisionExport is working fine for depth and stencil. But what about full image segmentation? I can't use renderdoc....according to this article https://hackernoon.c...ge-6d0b7fb4c781and the screenshot of the code, I assume that they can extract some kind of a vertex buffer, thus the shapes?!

 

Another approach could be to "map" a pixel of an image to a 3D point in game with raycast.

So first calculating for every pixel in image file, e.g. 1920 x 1080, the 3D coordinates in game, make a raycast to this point and get hash or object class or something like that and save that along pixel and 3D point (maybe usefull for depth info) for every frame.

But this would be over 2 million raycast for a single frame, I don't know if this will work!

 

Do you have any idea how to solve this?





1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users