I know we're all frustrated but wanted to offer some info from an

[MrB UK]

Hi all,

 

I know that we're all frustrated with Online - and the general consensus is mudslinging at R*... I'm an infrastructure engineer myself - and build out server environments myself (albeit for apps for corporate enterprises rather than games). I've been lurking on here for a little while but finally joined as I think I can offer a perspective that is often overlooked.

 

Sure - what I have to say won't fix the problem. It won't make everything suddenly work - and it won't answer everyone's questions or make people feel better... But I'm guessing understanding what's going on a bit does help.

 

So - one of the big criticisms of R* is that they "should have been prepared" and "they've had an extra two weeks since launch", etc, etc. I hear ya - but what you have to remember is that R* is a business and they can't have enough capacity for every game sold to all play at once for the next 10 years. If that were to happen, then each game would probably cost in the region of $500 - $1000 a copy.

 

The simple fact is that only some people will play the game, and for various reasons - time zones, work / school schedules, eating habits, family and other commitments - not everyone will be playing at once. So... R* needs to provide enough capacity with servers - and just as importantly bandwidth to those servers - to enable people to play.

 

In the first few hours and days, we're all clambering to play as early as possible - heck, we've already had to wait an extra few weeks, right? The thing is that there's more people trying to play now then there will normally be. What should R* pay for? Should they pay for capacity for everyone to play now and then have all that extra capacity afterwards when numbers level out? If so, that has to be paid for from somewhere.

 

Of course, hopefully R* will be utilising some form of "elastic" cloud servers and networking. Apologies for those of you that knows that this means that they can add more servers and bandwidth, etc "on demand" (I'm not trying to teach people to suck eggs, just set a level of understanding for everyone reading).

 

Whilst this allows on-demand upscaling / downscaling of servers available - it also means that disk space and the building out of templates to allow the quick provision of servers needs to be accounted for. If there's too much demand then new servers may need bringing online at a new datacenter, or where hardware hasn't got access to templates to build new servers. This means that extra time is needed to bring up new servers in new locations.

 

Also, if a larger number of servers are needed then it can sometimes have to wait until the datacenter has physically got the resources available to provide (I'd hope that this wouldn't happen though!) - and in some cases if an agreed threshold is surpassed it can mean that until a new Purchase Order is signed off by the customer (in this case R*) with the host/datacentre, new servers can't be deployed.

 

Also you have to remember that not everyone wants to play online - and so a huge guessing game would have had to have taken place with R*. Admittedly it would have been an educated guessing game, based on number of sales... But very little else!

 

Another of the criticisms being levelled at R* today is that people don't like being "beta testers" and that's what we are...

 

When any new service goes live - no matter how much testing has been done - there will always be issues. A testament to a company and the services they and their suppliers provide is how they react to those issues.

 

R* *could* have done a staggered roll-out - they could have made people sign up to R* Social Club, take their place in a queue and opened up to say a thousand people a day to keep capacity issues at bay. People would've complained about that though, and there would've still been issues that wouldn't be experienced until you had a critical mass of users.

 

Alternatively they've gone the other way - they've opened up to everyone. The game's sold beyond their wildest dreams. They knew it was gonna be popular - but they didn't expect it to be by this much. It's easy to say "oh, they've had two weeks in the know" - but the fact is that they haven't - the games continued to sell even between now and opening day. For all of the reasons above, I hope it's easy to see that you can't just bring enough capacity online within minutes.

 

Let me try and put an analogy on it. You invite a load of people to dinner. A few weeks out you have a pretty good idea of who's coming and who isn't, but you haven't quite heard from everyone. A few more people ask to come and you think you'll be okay so say yes. You then make sure you have enough food ordered, and know how you're going to cook it all. The evening before you get a phone call from one of your kids asking if another 3 people can come because they've got a power outage. You say yes, dash to the store only to find out that they're out of some of the food you need - but that there should be a delivery in the morning. The big day comes, but you have to wait for the store to open - and your guests have started arriving before the store takes delivery... What do you do?

 

For me - I'm frustrated like everyone else. But it's where we are in a couple of days that matters. If R* have it sorted, great... If not then I think it's fair to start asking bigger questions.

 

I can imagine those involved with infrastructure capacity at R*, along with engineers with their suppliers, are run off their feet, will be working very long hours today and for the next few days, and are doing their level best to get everything sorted for us all....

[Black Rabbit]

Great post, but prepared to be bashed on by the hoards of douche.

[koolvid12]

Have you seen how terrible this community is? 95% won't be able to handle such a long post to read.

[kingcool52]

Great post and just wanted to say I'm one of the few that actually bothered to read the whole thing . I agree, we are not giving Rockstar enough time plus it's the first day, obviously everyone is going to be on therefore overloading the servers. By around 2 weeks from now, it should be much better

[NothingPersonal]

I order 20 pizzas...maybe rent a film to entertain people as they wait for the food...or perhaps take them to a restaurant...

 

But seriously, in the world of data centers, isn't it possible to rent servers temporarily? Like, with sites, when there's an unexpected surge of traffic, or there's a DDoS attack, there are measures to temporarily increase bandwidth levels to endure the effects.

[DeadstarHorizon]

An intelligent and informative post. But as the others have said, It will be more like

 

 

HURRRR ROCKSTAR DOESNT CARE YOU A DOOSH WORST GAME EVER

[aR2k]

Great post, and while I have been defending R* practically all day (even tho im upset).

i must say this: 6 Hours after launch and the game is not even playable.... not good, and it doesnt seem to be just overpopulation on servers, there seem to actually be some coding errors, especially for matchmaking on the PS3.

There are allso a few gamebreaking bugs, like vehicles not going into the impound garage, but stacking up in a parkinglot taking damage, and so forth.

 

Ive been to so many game launches in my years, WoW and its expansions, Diablo3, SC2, Fifa09-14 (and EA aint good at launches), Battlefield1942- Bf3, and the list goes on.

I dont think ive seen a launch quite as bad as this, i could perfectly live with bugs in game, i could live with getting dced, and server restarts. But its sad that 6 hours after the game has launched, I have yet to even enter it.

 

I have had my characters wiped (not that it matters when i havent played them), but there are allso people reporting having their Singleplayer savegames corrupted, now THATS bad

Edited October 1, 2013 by aR2k

[stee_vo]

Thanks Obama..

[MrB UK]

Hey all, cheers for joining the debate. :-)

 

Nothingpersonal,

 

Ordering 20 pizzas sure is a great idea - and that'll probably take your local pizza store an hour to deliver - thus creating a wait time.... Now multiply the problem to hundreds instead of say 20 or 30... You could phone multiple pizza stores, but they're still gonna run out. Eventually you can get the whole situation under control, but it's all gonna take time.

 

Yes - you can rent servers temporarily. And you can bring them online pretty quickly in some cases - this is the "elastic" bit in my last post. :-) You're still likely to need to pay for these servers for a period of time though.

The problem is though that first of all you have to build out the servers as you need them - you can have a template and do this automatically... But that template has to be available physically to the hardware that you want to bring up.

 

The big problem you have is knowing what demand will be, and how much kit to have on standby - and having data centres able to have that kit ready and waiting for you... Or bring up more if it goes over that (and to do so you then need to get the template in place, etc).

 

Another quick analogy - if you wanted to rent 100,000 cars from Herz next Wednesday, but then only wanted 2000 the following day, do you think Herz would have enough cars available for you to do that? Probably not - and if they hadn't then they might be willing to buy in another hundred or so cars, but they're not going to buy 10's thousands just for you for a day.

[Nasir_Jones]

Thanks Obama..

you must be from iFunny.

[MkIIJay]

Regardless, if Rockstar sell 15 million copies of GTA, they should have the infrastructure to allow them to play.

 

I suppose a football team can sell 50,000 tickets even if their maximum capacity is 30,000? The other 20,000 will just have to wait until the others aren't there...

[MrB UK]

aR2k - I hear ya... I'm the same, I've "got in", but can't do anything, can't do the racing mission... Any launch of any "service" this big will have problems. Some should've/could've been sorted before hand, others get (annoyingly) get missed.

 

I think they picked a rather strange time to release personally - the US was waking whilst Africa & Europe were heading towards afternoon and kicking out of school followed by people finishing work, it was evening east of Europe in Asia. There's never a "right time" in a 24 hour world, but evening US-wise would've at least put it past midnight in Europe and still overnight in a large chunk of Asia.

 

6 hours in is frustrating - but as I say, I'm willing to give them a bit longer. EA & Sim City was the biggest car-crash launch I've ever seen. I was still having issues trying to play a single player game over a week after launch....

[hewentthatway666]

Hi all,

 

I know that we're all frustrated with Online - and the general consensus is mudslinging at R*... I'm an infrastructure engineer myself - and build out server environments myself (albeit for apps for corporate enterprises rather than games). I've been lurking on here for a little while but finally joined as I think I can offer a perspective that is often overlooked.

 

Sure - what I have to say won't fix the problem. It won't make everything suddenly work - and it won't answer everyone's questions or make people feel better... But I'm guessing understanding what's going on a bit does help.

 

So - one of the big criticisms of R* is that they "should have been prepared" and "they've had an extra two weeks since launch", etc, etc. I hear ya - but what you have to remember is that R* is a business and they can't have enough capacity for every game sold to all play at once for the next 10 years. If that were to happen, then each game would probably cost in the region of $500 - $1000 a copy.

 

The simple fact is that only some people will play the game, and for various reasons - time zones, work / school schedules, eating habits, family and other commitments - not everyone will be playing at once. So... R* needs to provide enough capacity with servers - and just as importantly bandwidth to those servers - to enable people to play.

 

In the first few hours and days, we're all clambering to play as early as possible - heck, we've already had to wait an extra few weeks, right? The thing is that there's more people trying to play now then there will normally be. What should R* pay for? Should they pay for capacity for everyone to play now and then have all that extra capacity afterwards when numbers level out? If so, that has to be paid for from somewhere.

 

Of course, hopefully R* will be utilising some form of "elastic" cloud servers and networking. Apologies for those of you that knows that this means that they can add more servers and bandwidth, etc "on demand" (I'm not trying to teach people to suck eggs, just set a level of understanding for everyone reading).

 

Whilst this allows on-demand upscaling / downscaling of servers available - it also means that disk space and the building out of templates to allow the quick provision of servers needs to be accounted for. If there's too much demand then new servers may need bringing online at a new datacenter, or where hardware hasn't got access to templates to build new servers. This means that extra time is needed to bring up new servers in new locations.

 

Also, if a larger number of servers are needed then it can sometimes have to wait until the datacenter has physically got the resources available to provide (I'd hope that this wouldn't happen though!) - and in some cases if an agreed threshold is surpassed it can mean that until a new Purchase Order is signed off by the customer (in this case R*) with the host/datacentre, new servers can't be deployed.

 

Also you have to remember that not everyone wants to play online - and so a huge guessing game would have had to have taken place with R*. Admittedly it would have been an educated guessing game, based on number of sales... But very little else!

 

Another of the criticisms being levelled at R* today is that people don't like being "beta testers" and that's what we are...

 

When any new service goes live - no matter how much testing has been done - there will always be issues. A testament to a company and the services they and their suppliers provide is how they react to those issues.

 

R* *could* have done a staggered roll-out - they could have made people sign up to R* Social Club, take their place in a queue and opened up to say a thousand people a day to keep capacity issues at bay. People would've complained about that though, and there would've still been issues that wouldn't be experienced until you had a critical mass of users.

 

Alternatively they've gone the other way - they've opened up to everyone. The game's sold beyond their wildest dreams. They knew it was gonna be popular - but they didn't expect it to be by this much. It's easy to say "oh, they've had two weeks in the know" - but the fact is that they haven't - the games continued to sell even between now and opening day. For all of the reasons above, I hope it's easy to see that you can't just bring enough capacity online within minutes.

 

Let me try and put an analogy on it. You invite a load of people to dinner. A few weeks out you have a pretty good idea of who's coming and who isn't, but you haven't quite heard from everyone. A few more people ask to come and you think you'll be okay so say yes. You then make sure you have enough food ordered, and know how you're going to cook it all. The evening before you get a phone call from one of your kids asking if another 3 people can come because they've got a power outage. You say yes, dash to the store only to find out that they're out of some of the food you need - but that there should be a delivery in the morning. The big day comes, but you have to wait for the store to open - and your guests have started arriving before the store takes delivery... What do you do?

 

For me - I'm frustrated like everyone else. But it's where we are in a couple of days that matters. If R* have it sorted, great... If not then I think it's fair to start asking bigger questions.

 

I can imagine those involved with infrastructure capacity at R*, along with engineers with their suppliers, are run off their feet, will be working very long hours today and for the next few days, and are doing their level best to get everything sorted for us all....

your wrong

[snacker111]

you f*cking idiot do you not realize they cant drop servesr after a while. Its not like they are committed to keeping all the servers for 10 years. heck they can remove servers by the end of the week if they needed.

[chainsoar]

Fantastic post. It blows my mind to see all the people complaining that Rockstar should have had a newly designed service - the likes of which has never been tried before - functioning perfectly from the day they roll it out. A brand new service that no matter how hard they work on, they don't have in excess of 1 million staff members to stress test.

[MrB UK]

Regardless, if Rockstar sell 15 million copies of GTA, they should have the infrastructure to allow them to play.

 

I suppose a football team can sell 50,000 tickets even if their maximum capacity is 30,000? The other 20,000 will just have to wait until the others aren't there...

 

But if that's the case, do you want to pay $500 for the game?

 

The simple fact is that if 15 million copies are bought, not 15 million people will play online. Let's just pluck a figure out the air and say 12 million would want to play. Not all 12 million would want to - or could due to other things in life - play at the same time.

 

To have capacity for 15 million people to play due to 15 million games being sold - would've delayed the game by even longer than now (to ensure that capacity) and would've also wasted a whole heap of electricity and resulted in paying for services that were never used.

 

Lemme use yet another analogy. If the London Underground sells 1 million travel cards next week, should there be capacity on every line individually to carry 1 million people at once, just in case?

[MrB UK]

you f*cking idiot do you not realize they cant drop servesr after a while. Its not like they are committed to keeping all the servers for 10 years. heck they can remove servers by the end of the week if they needed.

 

Haha, welcome to the debate. :-)

 

I do fully realise this, yes - I've even talked about elastic hosting directly and in analogies in several posts.

 

My point is though - that if people expect there to be enough capacity for every copy sold to always play online, then yes there does need to be servers available for the life of the game.

 

My further point is that R* really didn't know how many people were going to play online. They had to try and guess how many people would want to go online. If you think that they should have enough capacity for every game sold up front, then reduce the servers they have, then sure this is possible. But only to an extent. If you go to a data hosting company and say "We want enough servers to cater for 15 million seats for a week, then we're unsure how much we'll need after that - we might only need 10 million" - it's likely that the hosting company will go "sh*t - we have spare capacity off the bat in our centres for 1 million extra seats (but that's our operational capacity, we'd need to replace it) - so we need to build out capacity for another 15 million seats... But they only want a third of that afterwards. Sod it, we'll charge them for 15 million seats as we'll have capacity for 5 million that we're not going to be able to charge for afterwards....

 

As I say - thanks for joining the debate, even with the name calling. Doesn't bother me - but remember... This IS my industry (IT infrastructure). This IS what I do. Daily. For enterprise corporates. Globally. As a gamer I understand the frustrations.... I'm just trying to share a perspective, trying to help. :-)

[Collekt]

I find it hilarious that people are so up in arms about the issues going on right now. R* themselves warned you ahead of time that there would be issues at launch, and these things happen for all online games. Just calm down and give it a little bit of time for things to balance out.

[Erebosgaminghd]

So can the amount of players trying to connect influence mission bugs etc?

 

Not so savvy when it comes to servers and what not.

[JonRenemy]

 

Lemme use yet another analogy. If the London Underground sells 1 million travel cards next week, should there be capacity on every line individually to carry 1 million people at once, just in case?

This.

 

Perfect analogy.

 

End of discussion folks. Some people believe they're not quite as dumb as they really are. Its a real pain. Especially online.

[SkunkonToast]

Thanks MrB for restoring my faith in these forums I was beginning to think everyone here was unable to think with any logic. I am yet to try the online yet and probably won't for a week or so when things have calmed down a bit (besides with the amount of people just on this site who say this is the worst game ever it may quieten down sooner than that) c'mon guys we have waited 5 years for this game, what's another week going to matter. I'll stick to the single player for now.

 

Remember people patience is a virtue.

[A-Lonely-Dragon]

just think... GTA V not even out in Japan yet ;D hah

[smokey__]

Your all not seeing that for some reason, xbox servers are running almost bug free, and playstation nobody can even play at all....

 

 

Big difference

[cash68]

Great post! However, you're ignoring a few things. I'm a designer, and I deal with aesthetics, user-interfaces, user interaction, and building consistent brands. Rockstar is one of, if not the, biggest game developers in the world. They KNEW they had a home run on their hands; the sales numbers (if not pre-orders) should have clued them in. And one important aspect of brand respect, is a complete and utter failure to plan, like the situation we are in. Should they have spent billions on servers? No. But they could have shut down ALL their other game servers for a day, to deal with this "surge". Would some complain? Probably. But more importantly, it would have been a great "launch".

 

They failed to plan/accomodate a great launch, and now they have this: A game that has been scoring off the charts, sales off the charts, with a GIANT black eye in terms of how they handled the online release. This was a piss poor decision, and it will ultimately cost Rockstar a lot of money to regain confidence from their consumers.

[MrB UK]

smokey__ - I'm an Xbox player, and I can't get to the race. :-( That said, I see what you're getting at and agree with your point/sentiment... I much prefer the infrastructure over P2P route!

 

cash68 - I see what you're saying. But problems would still occur. It goes beyond servers - it goes to server placement, bandwidth need... Where players are coming from is as important as anything else in capacity planning.

 

Pre-orders is a great indicator, but it still doesn't tell you which ISPs your customers are using, and it still didn't give them anywhere near the final number of sales in the first weekend, let alone the first two weeks....

 

Shutting down other games isn't the answer to me - it wouldn't protect your brand. I personally think it would just do more damage... There would still be issues (as per last two paragraphs, plus other bugs/problems that would only show themselves when you get a critical mass of users).

 

On top of that, you'd piss off customers on all your other games - and you'd show ALL of your customers (past games and new customers with GTA alike) that you only give a toss about the newest, shiniest game, and as soon as the next release comes round you'll screw them over... Thus damaging your brand and reputation with your entire existing customer base and planting seeds of doubts in your new customer's minds. It's a very dishonourable way to do business where I'm standing from.

 

To me, if R* can sort this inside a week I'll be happy. For the scale of the game and number of customers that would be fair and reasonable.