Robocalypse Now: Using Deep Learning to Combat Cheating in CSGO [video]

http://www.gdcvault.com/play/1024994/Robocalypse-Now-Using-Deep-Learning

by doppp • 8 years ago

84 32 8 years ago

gdcvault.com

RenegadeEagle• 8 years ago

I haven't watched this talk yet, but as an avid CS:GO player, this isn't doing much to combat cheaters.

NamTaf• 8 years ago

This is a really amusing comment after watching the 9-12 minute segment of the video. :)

For what it's worth, I very rarely run into cheaters. I couldn't specifically identify the last time I ran into one, in fact.

tehlike• 8 years ago

It sounds like WAI.

efini• 8 years ago

Those who cheat and don't get caught are very good at hiding it. When all it takes to toggle is a hotkey like NUMPAD1 to activate aimlock/walls and NUMPAD0 to turn all cheats off, it makes it almost impossible to identify a cheat.

dvt6• 8 years ago

I built (and tried to get funding for) http://gameref.io/ a few years ago. Many critics argued that a deep learning or AI solution would be better. For what it's worth, I also played Counter-Strike at a professional level from '04-'07 -- my CS:Source team was ranked #7 in North America in 2005 (according to GotFrag.com).

After some research and a few POCs I am 100% certain a ML solution will not work in detecting cheats other than completely trivial "spinbots" or "aim-lock" aimbots. For example, it by definition cannot catch an entire class of cheats (wall hacks), but it even falters when trying to catch aimbots; the presentation here argues for a "heuristic-based" method, but, again, these methods often fail when (if you look at the source), aimbots are often coded to (a) move smoothly/naturally and (b) introduce random noise.

In my case, I tried to at the very least combat one class of cheats with close to 100% confidence. These kinds of talks are great at conferences, but, as they say, the proof is in the pudding. Cheaters are very hard to catch, and cheat coders are much smarter than generalized AI methods.

rudedogg• 8 years ago

gameref looks like a really clever and simple solution as a hardware device.

After some research and a few POCs I am 100% certain a ML solution will not work in detecting cheats other than completely trivial "spinbots" or "aim-lock" aimbots.

I'm curious why you say this. I'm a machine learning noob, but it seems pretty straightforward that it would work. Relating to wallhacks, they can use your last 1,000 matches, and if you're never/rarely surprised by a camping enemy, you're probably cheating. Your camera position moments before interaction with an obscured enemy is probably another indicator.

dvt6• 8 years ago

I'm curious why you say this.

Well the answer is surprisingly simple. Suppose you only have one person that's wallhacking on your team. He's calling out positions for everyone else. He can play terribly, but since the calls are going out, your team still wins. (This is not a hypothetical and has actually happened in high level matches before.)

metjm• 8 years ago

I'm failing to see a use case for gameref. At the big tournaments the computer is provided by the organizer. For online tournaments, wallhacks and maphacks won't be detected by this device.

I'm curious why you say this. I'm a machine learning noob, but it seems pretty straightforward that it would work. Relating to wallhacks, they can use your last say 1,000 matches, and if you're never/rarely surprised by a camping enemy, you're probably cheating.

Exactly. And not just that, wouldn't ML in theory also be able to detect subtle reactions to changes that the player shouldn't see? For example if a cheater has a map hack that only shows dots on the minimap, and does some small action like twitch with the mouse when an enemy he's following and can't see or hear ingame, changes direction?

trinitry3• 8 years ago

Relating to wallhacks, they can use your last 1,000 matches, and if you're never/rarely surprised by a camping enemy, you're probably cheating. Your camera position moments before interaction with an obscured enemy is probably another indicator.

Do you actually play CS:GO proficiently? Because it doesn't sound like you do.

netheril96• 8 years ago

(a) move smoothly/naturally and (b) introduce random noise.

Moving naturally is not that easy to code. Otherwise Google would not be able to tell if you are a robot or not simply by analyzing how you click a checkbox.

LoSboccacc• 8 years ago

it's also hard to define "move naturally" - if you look for smoothness you'd get a ton of false positive. check this out https://streamable.com/swz3y - twitchiness is a byproduct of scouting for target in motions.

mynewtb• 8 years ago

Well, Google can't. Or maybe I am a robot.

makomk• 8 years ago

Google can't. They use your history of online activity across their services to decide if you're a robot. If you click the checkbox the exact same way as usual, but do so in an incognito window without any established Google cookies, they require you to go through a captcha.

ygra2• 8 years ago

It should be noted, though, that this ML model is based on Overwatch (that's where they get their training data), which is only good at detecting blatant cheats as well. So they probably are fairly sure already that this system won't be able to catch 100 % of cheaters.

However, as John noted in the beginning, those blatant cheats mostly have one reason to exist: to actively and obviously spoil the fun of others involved, so to improve things to players of that game, it's a very good starting point.

dvt6• 8 years ago

However, as John noted in the beginning, those blatant cheats mostly have one reason to exist: to actively and obviously spoil the fun of others involved, so to improve things to players of that game, it's a very good starting point.

This is very true (and very important), but I don't think you need AI to detect blatant cheats. IMO, simple heuristics can do the job better.

switz• 8 years ago

To clarify for anyone reading, "Overwatch" is not referring to the Blizzard game, but an in-game Counterstrike tool in which players review recorded gameplay of a suspected cheater.

When enough reviewers agree that a player is cheating, that person is banned. Each player has a hidden successful conviction rating and on occasion the system will send you professional players or non-cheating players in an effort to verify your ability.

This is only meant to convict the most blatant of cheaters. It's fairly successful for what it's intended to do, but of course, it does not even remotely catch every cheater.

leetcrew• 8 years ago

what team if you don't mind me asking?

sturadnidge• 8 years ago

I’d be curious to know if there was a difference in engagement from Overwatchers. I basically stopped doing cases because there was so much junk in queue (rage reporting, skilled players playing against unskilled players, aka ‘smurfing’, etc). But after the introduction of VACnet the case quality went through the roof, i no longer felt like i was wasting my time doing cases and thus do more of them now.

narrator• 8 years ago

I have long said that the future will be our AI bots vs their AI bots with the humans stuck in the middle of it all just trying to have a life.

Hydraulix989• 8 years ago

Now cheat by using a generative network to combat the deep learning that combats cheaters.

CNJ7654• 8 years ago

What a time to be alive

Robocalypse Now: Using Deep Learning to Combat Cheating in CSGO [video]

Related Stories