Artificial intelligence is paradoxically both humanity’s greatest opportunity for advancement and its greatest threat. On the one hand, AI could be used to augment our own abilities and to solve global problems like poverty and disease. On the other, top scientists, including the late Stephen Hawking have issued dire warnings about AI being the potential downfall of humanity.
Video games have a part to play in this as well. OpenAI, a non-profit research program spearheaded by Elon Musk and Sam Altman, was founded in 2015 with the mission of advancing AI in a way that’s safe for humans. The researchers have discovered that complex video games like Valve’s Dota are actually perfect testing grounds for the AI they’ve been developing.
Last month, OpenAI described in deep technical detail how its team of five neural networks (OpenAI Five) managed to defeat amateur players in Dota 2. Now the goal for OpenAI Five is to go pro and win the esports competition at The International in August. OpenAI co-founder and CTO Greg Brockman told GameDaily that this lofty goal may not be achievable by August, but that the AI systems his team has worked on have continually surprised everyone.
“The one thing that I think is illustrative of [our] progress is that our goal for probably two months was to beat just a pretty good Dota player here at OpenAI in a 1-vs-1 [match],” he explained. “And every week, we would push that goal back by an additional week. And it was just like, ‘This is the week we’re going to do it. We’re totally going to do it.’ And we never did, until I think the end of June of last year, and then we beat our [first] semi-pro player [last] July 7th.
“And then one month later is when we played it in pros for the first time and we beat them. Two days later, we had more training and we played the second best player in the world and we beat him. And then one more day of training and that was enough. We were able to beat the top player. So the way that these systems go is that they just don’t work at all and look really unimpressive and you’re just doing all this nitty gritty [stuff] and trying to get anything to happen at all. And then suddenly, you have this incredible rapid performance. And we see the exact same time thing as we shifted from, not just 1-vs-1 but to this year’s 5-vs-5.”
Human beings may play a solo match very differently from something that requires intensive teamwork and planning, but Brockman noted that for an AI, the approach that the robot takes in 5-vs-5 is actually not all that different from solo match-ups.
If you look at the milestones in this field, people have always used games, with games like chess or Go. People have marching up this ladder of harder and harder games with more and more complexity. If you think about the real world, it’s really just this very, very complicated game with many degrees of freedom
“You’ve got all these different heroes, all these different things. And the funny thing is that our method doesn’t care,” he said. “Our method pretty much acknowledges the fact that, sure, you’ve got more complicated environments. Just show me the data and I’ll figure out what to do. So the way that our 5-vs-5 system works is that you have five separate neural networks. Each one is observing the game state and the only thing that we do in order to develop a sense of teamwork is that we just reward them with this parameter called team spirit.
“With team spirit at zero, which is what we do at the beginning of training, they’re [taught] to be selfish and optimize for their own reward. But then with team spirit at one, we’re telling them [to] be cooperative and just care about the team’s reward. And then over the course of training, you transfer from that zero to one and then we start to see them just like being these utterly selfless players and they have this crazy team coordination.”
But why did Brockman and the OpenAI team select Dota as a testbed to begin with? As it turns out, games in general are a great construct for testing AI’s abilities, but complex esports matches could be the ultimate trial.
“If you look at the milestones in this field, people have always used games, with games like chess or Go. People have marching up this ladder of harder and harder games with more and more complexity. If you think about the real world, it’s really just this very, very complicated game with many degrees of freedom,” Brockman explained.
“And as you start to move to something like Dota, you start to capture more aspects of the real world you just don’t get in these other milestones; things like long-term planning with continuous actions and continuous time and you can’t just go and do a bunch of searches. You can’t just look into the future. You have to somehow use intuition and be able to do things that I think match a lot of real-world extremes.”
Most people in the field doubted OpenAI’s methods, according to Brockman. The thinking from other engineers was that new algorithms would be needed, but that hasn’t been the case. It was more of a matter of scale. OpenAI Five plays 180 years worth of games against itself every single day.
“We had these very general algorithms but they have all these flaws,” Brockman said. “As a human, the way you think about your day, you don’t think about every movement you’re going to make as you go from appointment to appointment. If you think about your career, you don’t think about every single day. You’ve got some sort of hierarchy in how you think about planning and some abstraction you can do. You need that. You need algorithms to capture that and [the] algorithms we have today don’t do anything like it.
“The surprise for us is we actually haven’t had to invent new methods. Today’s methods, if you scale them up, you run them on massive, massive queues then they are able to solve these incredibly complex problems in ways that people just thought were impossible. So I think that’s another really interesting, eye-opening thing for us.”
OpenAI is not concerned with game development per se, but Brockman and his team do concede that the advancement of AI could have significant implications for gaming experiences. The more sophisticated AI becomes, the more effectively it can be utilized to offload certain game development processes — tasks that would have typically taken months or years of human labor. Moreover, the possibilities for immersion only become greater when virtual characters can respond to players in more human-like ways.
“The possibilities for storytelling, the possibility for personalized content is really opening up,” Brockman said. “The content side is something that I see happening probably actually quite fast. [AI] can [generate] faces right now. If you just show it a data set of a bunch of things that you want to be able to generate.
“I think that you should be able to be in a world where everyone can have their own customized appearances and things like that in a way that today you just have to have human artists go in and do it.”
Conversation and personalities is another big area for AIs to play around with, which could make the video games of tomorrow quite engaging.
“We’re starting to be able to do natural language in a way that we just couldn’t before,” Brockman noted. “And so you start to look at, ‘What are the possibilities for making games that are more immersive and that are more reactive to you?’ If you think about systems like OpenAI Five you can really start to have these very smart NPCs. You can imagine your friend disconnects and then you just have a bot takeover for them playing at the same level as them and potentially even the same play style as them. All of that starts to be within the realm of possibility.”
Brooke Chan, an Open AI engineer, added that one of the interesting side effects of working on AI to challenge human Dota opponents is that human players are actually learning new tricks from the bots.
“One of the things that I think is really cool is that we have the opportunity to sort of learn from the AI,” she commented. “If you look at our 1-vs-1 [matches] from last year, our bot ended up playing by a different play style that wasn’t typically used a lot. And, recently, when we had our professional counselor come in they were talking about how the bot actually altered the way that pros nowadays play 1-vs-1, because it sort of stumbled upon this technique that was actually not only quite strong but pretty learnable by professional players. So I feel like going forward, specifically with Dota, this has the opportunity to stumble upon things that professional players haven’t found just because the AI’s able to play so many games.”
What OpenAI has accomplished thus far is thoroughly impressive. Microsoft co-founder Bill Gates praised the team for reaching a true AI milestone. But the underlying goal, to create what’s known as Artificial General Intelligence (AGI) — an AI that can perform human-level intelligence tasks and pass the Turing Test — is equal parts terrifying and exhilarating. That’s why OpenAI is trying to ensure that when AGI is achievable that it’ll align with human values and be to the benefit of everyone. Brockman testified before congress last month about this very issue.
“The most important thing I wanted to talk about in the testimony is just the fact that people have been talking about AGI for certainly as long as I’ve been alive and it hasn’t happened yet,” he said. “So, how should we think about the timeline and can it happen on a reasonable timescale? And the reason that we can’t rule it out anymore is very simple; it really comes down to the hardware.”
Brockman explained that with the three main pillars of AI progress (data, compute and algorithms) AGI could be achievable a lot faster than you might believe. “The dream is, of course, you just go and consume the whole internet and learn from that. [Our AI is] not quite there, but we’re starting to make strides towards data just not being a problem or enough of a bottleneck,” he noted.
The compute part of the equation is accelerating at an amazing pace, as well.
“We actually just released a study showing that for the past six years the rate of improvement in the amount of compute thrown at the largest AI training labs has been increasing with a doubling period of 3.5 months; Moore’s Law by comparison is 18 months. And so over that time, we’ve seen a 300,000 times increase and our projections are that it’s going to continue for the next five years,” Brockman said.
“To put that in perspective, it’s just like if your phone battery, which today lasts for a day, suddenly started to last for 800 years and then you waited five more years and then it started to last for a hundred million years. And so the amount of compute that’s coming down the pipes, it’s just astronomical… And the question is, will that new computational power combined with near term improvements in algorithmic understanding — will that be enough to build AGI?”
No one knows the answer to that question, and it’s a matter of fierce debate among some scientists. But humanity must be prepared. It’s certainly in the collective consciousness right now and has permeated pop culture, with shows like HBO’s Westworld, AMC’s Humans, or of course the classic Terminator franchise. Brockman certainly doesn’t want to be responsible for creating a real-life Skynet. But there are teams of engineers across the globe working towards AGI, and they may not share OpenAI’s mission.
“There are some causes for optimism, and some causes for caution. I think that it certainly keeps me up at night to think about the ways that things could go wrong,” Brockman admitted.
“The core danger is that AGI has the potential to cause rapid change. Whether that’s a malicious actor supporting deploy systems, whether that’s a system that just doesn’t do what it was designed to do that goes a little off the rails, or whether it’s really just an economy that’s just being owned by a very small number of people that has just kind of grown for its own sake and isn’t resulting in improvements in human lives.”
Imagine that there’s an arms race here between different countries trying to get to AGI first and if that happens there’s going to be a lot of pressure to deploy at the expense of safety.
Rapid change can be tough to comprehend and adjust to. What we may witness in the future could almost be compared to someone from the 1950s trying to understand the internet, Facebook, Uber, smartphones and other new tech.
“I think that would be almost incomprehensible,” Brockman said. “And now imagine that you have to take that same [amount of progress] for the past almost seventy years and compress that time frame into a much smaller period. And suddenly that’s the kind of thing that we’re talking about.”
The problem is that some scientists may throw caution to the wind. The AGI race is on.
“Imagine that there’s an arms race here between different countries trying to get to AGI first and if that happens there’s going to be a lot of pressure to deploy at the expense of safety,” Brockman cautioned. “I think that that’s one place where we might end up being forced into cutting corners on this thing that is the most important thing, which is making sure that systems align with our values and do the thing it’s supposed to do.
“The more that there is a race, the worse things are likely to go. And so you can start having some sort of coordination, some sort of conversation before everyone’s truly in the race and really going for the end. And so that’s how I think about it; we’re still kind of early. I think that things are going to be transitioning from abstract to concrete and as they do it would be good if we had as much [dialogue] as possible.”
OpenAI isn’t sure if it’ll be testing its neural networks on other games in the future, but the team intends to continue working with Dota. Brockman has pondered testing AIs within big open-world gaming environments, where “you build environments that are fun for humans and also fun for AIs, that end up being places where AI can learn lots of behavior and lots of complexity and can do something useful with that.”
“So we’re still at pretty nascent phases of thinking about this problem, but I think that’s kind of the next step for a lot of these technologies,” he said.
I quipped that Brockman had just said “fun for AIs,” hinting that he’s gotten a bit too close to his work and is already anthropomorphizing his bots. But with the goal of making human-like AI, that’s a slippery slope many engineers may take.
“It’s interesting seeing how much our thought processes, ‘Oh, this would be a more fun game for me and oh it’s also easier for the AI to learn,’ end up correlating,” Brockman added.
Assuming AGI is achieved, humanity will have to confront its very existence. What is consciousness? What does it mean to be human? These are philosophical questions that our species has grappled with for centuries, but AGI will force us to rethink our nature.
“I think that as time goes on, as we start to see systems like OpenAI Five and other ones that are performing these very advanced [tasks] at or above the level of the best humans in a particular domain, I think we’re going to start learning a lot more about ourselves in some ways and what the world’s going to look like once we get to a place where you have that sort of performance but across a huge variety of tasks,” Brockman said.
“I think it’s also really good that today people’s first encounter with AI is in the context of a video game. It’s a [safe space] to start understanding this other, this thing. And I think that you’d rather have that than have your first encounter be with a self-driving car or with a robot walking around or something. So it’s good not just from a technical perspective, but from a societal perspective for people to be able to interact with these systems in low stakes environments.”
GameDaily.biz © 2024 | All Rights Reserved.
