Quantcast

Jump to content

» «
Photo

Parse XML from web without lag

18 replies to this topic
LetsPlayOrDy
  • LetsPlayOrDy

    Too fast for ya

  • Members
  • Joined: 24 Aug 2013
  • United-States

#1

Posted 24 June 2014 - 02:55 AM Edited by LetsPlayOrDy, 24 June 2014 - 03:13 AM.

(Title is supposed to be "Get XML from web without lag/pause".)

Hi everyone.

I'm trying to create a script which gets the current weather condition in your specified city, but I've got a problem. The game lags/pauses when I'm getting the XML. When I say pause, I don't mean like the pause menu, I mean the game stops doing anything until the script receives the XML.

Here is my function that I have now:

public static string GetCurrentConditions(string location)
        {
            try
            {
                string condition = "No data";
                XmlReader xmlReader = XmlReader.Create(string.Format("http://api.openweathermap.org/data/2.5/weather?q={0}&mode=xml", location));
                while (xmlReader.Read())
                {
                    if ((xmlReader.NodeType == XmlNodeType.Element) && (xmlReader.Name == "weather") && xmlReader.HasAttributes)
                    {
                        condition = xmlReader.GetAttribute("number");
                        break;
                    }
                }
                return condition;
            }
            catch (Exception ex)
            {
                return ex.ToString();
            }
        }

It works perfectly, but has the lag, which is annoying and I wouldn't want to release a mod that does that. I looked up the fastest way and I found that XmlReader was the fastest (I was using XmlDocument before). I don't think the parsing of the file is the cause of the lag because the XML is a small file. Just look at this.

Is there any other way to get the XML without this lag/pause?

 

P.S. I have never done anything related to XML, so if you think I sound like a noob or something, this is why.


LordOfTheBongs
  • LordOfTheBongs

    Deceased

  • Members
  • Joined: 21 Sep 2013
  • None

#2

Posted 24 June 2014 - 04:25 AM Edited by LordOfTheBongs, 02 July 2014 - 05:27 PM.

...

  • Wiebrendh likes this

LetsPlayOrDy
  • LetsPlayOrDy

    Too fast for ya

  • Members
  • Joined: 24 Aug 2013
  • United-States

#3

Posted 24 June 2014 - 01:15 PM

 

you need to get the weather in a new System.Threading.Thread and then when that is finished... it can set some stringvariable and maybe trigger a bool to let you know the weather is ready for your script to use...

using System.Threading;
...
string weatherInfo = "Retrieving weather...please wait...";//you can be drawing this as text in a per frame draw event while the thread is running
Thread t = new Thread(new ThreadStart(() =>
{
    //get weather here!!
    //dont write any gta related code in here like calling natves
    weatherInfo = yourParsedAndFormattedString;
}));
t.IsBackground = true;//this thread running will not prevent the app gtaiv.exe from closing
t.Start();

Thanks! I tried it and it seems to be working. I'll post if otherwise ;)

  • Wiebrendh likes this

InfamousSabre
  • InfamousSabre

    Harpocrates

  • Members
  • Joined: 03 Jan 2013

#4

Posted 24 June 2014 - 08:58 PM

Ha! had the same idea as me :p I just finished mine a few days ago. used wunderground API
Wont be releasing though. GTA IV's weather just isn't dynamic enough for me to be pleased with the result.
Hope yours goes well :)


LetsPlayOrDy
  • LetsPlayOrDy

    Too fast for ya

  • Members
  • Joined: 24 Aug 2013
  • United-States

#5

Posted 25 June 2014 - 01:32 AM

Ha! had the same idea as me :p I just finished mine a few days ago. used wunderground API
Wont be releasing though. GTA IV's weather just isn't dynamic enough for me to be pleased with the result.
Hope yours goes well :)

Yeah, I know. I wish it was more dynamic too :/

I'm still going to release mine though :p


InfamousSabre
  • InfamousSabre

    Harpocrates

  • Members
  • Joined: 03 Jan 2013

#6

Posted 25 June 2014 - 03:34 AM

in .net4+ you can also do this instead of creating a new thread
 

System.Threading.Tasks.Task.Factory.StartNew(() =>
{
   //get weather here!!
   //dont write any gta related code in here like calling natves
   weatherInfo = yourParsedAndFormattedString;
}));

Haven't personally compared it against a new thread, but have read that its supposed to be less taxing on cpu/ram
*shrug* give it a shot if you'd like.


LordOfTheBongs
  • LordOfTheBongs

    Deceased

  • Members
  • Joined: 21 Sep 2013
  • None

#7

Posted 25 June 2014 - 02:41 PM Edited by LordOfTheBongs, 02 July 2014 - 05:27 PM.

...


NTAuthority
  • NTAuthority

    member_title

  • Feroci Racing
  • Joined: 09 Sep 2008
  • European-Union
  • Most Knowledgeable [Technology] 2013
    Best Map 2013 "ViceCityStories PC Edition"

#8

Posted 26 June 2014 - 08:04 PM

actually the thread pool is primarily meant for short-lived work objects; longer-term tasks are still better off on a manually-managed thread or passing a specific flag to the native thread pool functions that doesn't happen to be exposed in the BCL/FCL.

  • LordOfTheBongs likes this

LordOfTheBongs
  • LordOfTheBongs

    Deceased

  • Members
  • Joined: 21 Sep 2013
  • None

#9

Posted 26 June 2014 - 09:21 PM Edited by LordOfTheBongs, 02 July 2014 - 05:27 PM.

...


LetsPlayOrDy
  • LetsPlayOrDy

    Too fast for ya

  • Members
  • Joined: 24 Aug 2013
  • United-States

#10

Posted 26 June 2014 - 11:18 PM

oh and i never mentioned your parsing is not the lag, waiting for the server to respond and converting the stream to text (i assume xml reader does this internally but i dont know unless i go look and the wrapped code) is what is lagging. So it depends on the server and it's traffic.

 

Ive parsed huge web pages just using string methods and it is always super quick...

 

getting the response stream and loading into a reader, then doing reader.ReadToEnd()... (better to read lines in a loop anyway)... on a large page will lag... im curious and may took a look at what xmlreader does

 

I already basically knew it isn't the lag. I mentioned that in my post :p


LordOfTheBongs
  • LordOfTheBongs

    Deceased

  • Members
  • Joined: 21 Sep 2013
  • None

#11

Posted 27 June 2014 - 12:16 AM

 

oh and i never mentioned your parsing is not the lag, waiting for the server to respond and converting the stream to text (i assume xml reader does this internally but i dont know unless i go look and the wrapped code) is what is lagging. So it depends on the server and it's traffic.

 

Ive parsed huge web pages just using string methods and it is always super quick...

 

getting the response stream and loading into a reader, then doing reader.ReadToEnd()... (better to read lines in a loop anyway)... on a large page will lag... im curious and may took a look at what xmlreader does

 

I already basically knew it isn't the lag. I mentioned that in my post :p

 

u sounded unsure so i thought id tell u what i experienced before


LetsPlayOrDy
  • LetsPlayOrDy

    Too fast for ya

  • Members
  • Joined: 24 Aug 2013
  • United-States

#12

Posted 27 June 2014 - 01:15 AM

 

 

oh and i never mentioned your parsing is not the lag, waiting for the server to respond and converting the stream to text (i assume xml reader does this internally but i dont know unless i go look and the wrapped code) is what is lagging. So it depends on the server and it's traffic.

 

Ive parsed huge web pages just using string methods and it is always super quick...

 

getting the response stream and loading into a reader, then doing reader.ReadToEnd()... (better to read lines in a loop anyway)... on a large page will lag... im curious and may took a look at what xmlreader does

 

I already basically knew it isn't the lag. I mentioned that in my post :p

 

u sounded unsure so i thought id tell u what i experienced before

 

 

Ah ok


LetsPlayOrDy
  • LetsPlayOrDy

    Too fast for ya

  • Members
  • Joined: 24 Aug 2013
  • United-States

#13

Posted 27 June 2014 - 01:39 PM

When I use a thread and put a loop in it to keep checking for the weather, it freezes the game when the game starts. Does anyone know how to fix this? If you need my code just tell me.


LordOfTheBongs
  • LordOfTheBongs

    Deceased

  • Members
  • Joined: 21 Sep 2013
  • None

#14

Posted 27 June 2014 - 03:01 PM Edited by LordOfTheBongs, 02 July 2014 - 05:26 PM.

...


LetsPlayOrDy
  • LetsPlayOrDy

    Too fast for ya

  • Members
  • Joined: 24 Aug 2013
  • United-States

#15

Posted 27 June 2014 - 03:44 PM

^ Don't know if you meant you wanted to look through the code or not, but I'll post it anyways.

 

This is the code I have right now. It runs when the script starts.

Thread thread = new Thread(new ThreadStart(() =>
            {
                while (!done)
                {
                        if (rwEnabled)
                        {
                            string getCondition = GetCurrentConditions(sLocation);

                            if (currentCondition != getCondition)
                            {
                                currentCondition = getCondition;
                                Game.Console.Print("Real Weather: Updated weather: " + CurrentConditionToName(getCondition));
                            }
							Thread.Sleep(60000);
                        }
                }
            }));

            thread.IsBackground = true;
            thread.Start();

GetCurrentConditions gets the weather XML then parses it to get the condition (which is a string).

CurrentConditionToName converts the current condition to a name (like "thunderstorm" or "drizzle").

I don't think those are what are causing the freezing, as before I added the thread these functions worked fine.


LordOfTheBongs
  • LordOfTheBongs

    Deceased

  • Members
  • Joined: 21 Sep 2013
  • None

#16

Posted 27 June 2014 - 04:21 PM Edited by LordOfTheBongs, 02 July 2014 - 05:26 PM.

...


LetsPlayOrDy
  • LetsPlayOrDy

    Too fast for ya

  • Members
  • Joined: 24 Aug 2013
  • United-States

#17

Posted 27 June 2014 - 04:33 PM

see what happens if u comment out the printing to the console, it needs to access the console object and this object i guess needs to be serializable to pass between threads... im not 100% certain though, just from my basic understanding of threading

Well that worked. Thanks :D


LordOfTheBongs
  • LordOfTheBongs

    Deceased

  • Members
  • Joined: 21 Sep 2013
  • None

#18

Posted 27 June 2014 - 04:44 PM Edited by LordOfTheBongs, 02 July 2014 - 05:26 PM.

...


LetsPlayOrDy
  • LetsPlayOrDy

    Too fast for ya

  • Members
  • Joined: 24 Aug 2013
  • United-States

#19

Posted 27 June 2014 - 04:47 PM

ok cool, yeah i think he needed to add a Serializable attribute to the console class to make it be able to be serialized and rebuilt in the new thread so it could be used in your thread... anyways, just use a bool or whatever u like to trigger the code in your actual script running on the script thread

Already did ;)

  • LordOfTheBongs likes this




1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users