Return Home
3rd January, 2016

How Does Rockstar Deal With Severs And Glitches?

As a daily contributor to the online world of Grand Theft Auto 5, i was surprised that i after playing for a year or so now i have never thought about how they handle the game glitches and traffic.

Servers are a keen interest of mine and working out just how Rockstar produces a strong enough platform for hundreds to connect to whilst still enabling the user to save on the fly and recover personal vehicles etc.

So i had a poke around and this is my findings.

First things first, Rockstar don’t host “Servers” in the classic sense people understand, that is “Minecraft Servers” or “Half Life Servers” and so on, meaning your console (and others) do not in fact all join one server hosted by Rockstar and play together on Rockstar’s servers.

Rockstar employ a “meshed session” multiplayer system. With this system, Rockstar don’t need to operate expensive and bandwidth intensive game servers, this saves them literally hundreds of thousands (possibly even millions) of dollars a month. They instead operate a number of “support” services, this includes things like SocialClub content (license plates, crew emblems, crew membership, etc), next are the “Global” servers, these host things like the loading tiles (text, and graphics) configuration files (eg, tunables.json) and other stuff needed by the game which isn’t session specific.

Now, the important servers are the “Matchmaking” servers. These servers have multiple parts to them, their role is to choose a session for you, and put you in it. They take information from the support servers for crew/friends stuff, they also take into account your level, skills, etc and does its best to place you in a session that “averages” the total session playerbase (from newb/low level up to skilled/high level).

The way it does this is that it constantly maintains a database of sessions and players within those sessions, when you join Online (Box 2 in the image) you contact your consoles network, this “authenticates” you for Rockstar, they check your status (bans, bad sports, etc) and then they begin the matchmaking process.

The matchmaking process, basically has an algorithm that selects a session from it’s session list, and then contacts the players in that session and feeds them your details, it also feeds you the details of the other players in that session and your console (the game itself) begins the “enmeshing” process, whereby your console, and the consoles of all the others in that session begin making contact with each other. Once successful contact has been made, you zoom down the clouds, and begin playing. If any of that contact fails, you get the dreaded errors (cannot join session, etc).

So the tl;dr is that there are no servers, the consoles talk to each other directly.

Now, the Pros for doing this:
• Saves Rockstar massive amounts of money
• Potentially improves “quality of experience” be eliminating single point of weakness (a single game server hosted by Rockstar)
• Player failures in a session don’t bring down the whole session.
• Saves costs permitting “free” multiplayer.
• Now, the Cons … this is where I explain all the “glitchiness” and how cheating is relatively untraceable.

Everyone already has a copy of the GTA “world” on their console, it’s needed for single player, so to make it Online is a relatively trivial task of removing unnecessary entities (fences, wildlife, reduce NPC numbers, etc) and enable the sharing of “entity tracking” across the network so things can be drawn and interactive with simultaneously by players.

The flaw with the meshed topology is that everyone has a copy of the server …

Cheating: To explain this, cheating is a bitch, the best way to combat cheating is with a series of servers which do server side checking, these are outside the reach of the clients, so if the client attempts something “against the rules” then the server catches this behaviour and bans the player.

When you are both the server and the client, it’s much harder to prevent cheating because those with the necessary abilities can “bend” the rules in their favour, and unless other consoles have anti-cheating “rules” to look for cheaters, then they go unnoticed. This is precisely what led to the tunables/DNS fiasco, Rockstar (very stupidly) never imagined people would find/abuse tunables … and when they did, they were entirely unprepared for the damage. This is because there were never any “rules” (also called sanity checks) on the activity of players, especially financial transactions, and levelling, as well as hit/damage/death ratios. In short, people could modify things and the game did exactly what it was designed to do.

To fix this, Rockstar implemented sanity checks on money transactions, bounty limits, and other factors for rudimentary cheat detection. They may have also employed third party cheat detection, but it’s not confirmed to what extent this detection is for.

Glitching: The game engine (RAGE) is geared to provide the player with a world to play in, it’s designed by humans, and is largely “broken” by design. The game engine provides models and physics for interaction. Wallbreaches are when the models have incomplete collision models. Collision models are “dumbed down” models of the object to prevent you from walking through it. You don’t use an exact replica of the model because it wastes memory reducing performance.

Next, we have physics glitches (gateswings, tank wallbreaching, etc) these are also relatively simple in design, some physics “suspend” the rules in order to provide a player with an experience, they also ignore the physics properties of an object (mass, etc) causing the glitch.

Lastly, we have all your most often abused glitches like duping, selling, etc. These are all to do with entity, and entity status tracking. The engine tracks the status of entities (players, objects, NPC’s, etc) and applies rules to those entities, during certain sequences (Cash Store, Garage Animations, etc) those rules are suspended and the engine forgets to track things, from here it gets very complicated to explain, however it can only be explained that some rules override other rules, which “break” things.

Take the classic “dupe” glitch, this works because (in the case of the blocked garage), the engine expects to go through a series of sequences for exiting the garage, one of which includes the garage opening. Each sequence does things in engine (create entity, destroy entity, etc) so in the case of the blocked garage, what’s happening is that because the exit garage sequence can’t be played, the “garage” entity (that is the room with all your cars) isn’t destroyed properly in engine, and the sequence for transferring the “personal vehicle” status to your newly spawned car isn’t enabled … this “duplicates” your car. This is a very simplistic explanation, but it’s sufficient to give you the idea, a lot of other “rules” are also in effect which enable the glitch.

So that’s basically it … the problem with a meshed session system, while it saves Rockstar a lot of money, and potentially improves the experience of the players, it also allows for cheating quite easily because the player controls both the “server”, and the client.