Would EA have been better off labeling its demo weekends as 'beta tests?'
Over the weekend, BioWare rolled out the red carpet for EA’s first big game of 2019, Anthem. In the first of two demo weekends, anyone who had pre-ordered the game, as well as VIP invitees, attempted to take the game for a test drive. Turns out that the royal reception wasn't as regal as people expected it to be, and many players (and likely EA) spent the weekend sparring with the same question: why wasn't this labeled a beta?
The demo was plagued with connection issues, a dreaded "infinite loading" bug, and initial bouts of rubber-banding that kept players from reliably playing the game solo, let alone with others. To their credit, EA and BioWare were highly communicative throughout the weekend, updating Anthem fans along the way to let them know that their concerns were being heard and that the teams were working as hard (and as fast) as they could to resolve them one by one.
Day 1 of the VIP Demo didn't go as planned. We'd like to let you know what happened and what we're doing to fix it.— Anthem (@anthemgame) January 26, 2019
Note from our BioWare Head of Live Service, Chad Roberston: https://t.co/0bLV0nEp4h
Day two of the VIP demo wasn't nearly as tumultuous as the first, but there were a number of issues that continued to prevent players from experiencing the game in a squad, which is how Anthem is supposed to be enjoyed. The rubber-banding was gone. The servers were standing up and cooperating. But it wasn't enough and the damage, it seemed, had been done.
But what happened with Anthem's VIP demo problems were the result of more than just the technical concerns: it had everything to do with messaging and positioning.
Demo versus Beta versus VIP Demo - it's all about the messaging
For a long time, there had been a collective understanding about what a demo was: it was a vertical slice of a game that allowed players to experience a small piece of what the full game was going to be. If you log into the Nintendo eShop, you'll see a number of games that have playable demos. These demos are self-contained and polished to shine.
A beta, on the other hand, is meant to have bugs. In traditional software release schedules, the beta is what helps solidify the final product -- you put the product out into the world in its beta form to gather feedback about what works and what doesn't, all the while expecting to run around putting out fires when things break outright. The closed beta precedes the open beta, of course, giving exclusive access to the software to a small circle of trusted users. The open beta is open season for bugs.
With the advent of early access games becoming the norm for roguelike single player games and most multiplayer experiences, these lines between alpha, beta, demo, and launch have become blurrier and blurrier. It's due in part to the evolving nature of the development cycle, with an emphasis on the minimum viable product, iteration, and moving as quickly as possible through prototyping and into initial product release.
Anthem was positioned and marketed as a demo for paying customers (either via pre-order or an EA Access or Origin Access subscription). EA did cast a broader net, offering each person in the VIP demo three access codes for friends. If players didn't secure access prior to the VIP demo, they could get in on this coming weekend's open demo, but that would be it. Part of what left some players angry over the weekend is that they had paid to play in this demo but… the demo wasn't a demo.
It was a closed beta.
The phrase "beta" is lost— Rami Ismail (@tha_rami) January 27, 2019
Everything is a beta now
Don't call anything you work on a beta because the expectations range from "it's broken so it'll be much better later" to "it should be perfect because beta is the last thing before release" and both are bad for your game
Rami Ismail's comment isn't wrong -- but it does point to a larger issue in the industry.
Companies need to effectively manage player expectations before setting them loose in the game. By labelling the first very public interaction with Anthem as a demo, BioWare communicated that the game would be relatively flawless; it would be a vertical slice that is as polished as users have come to expect from demos.
And, on the flip side, marketing it as a closed beta (while positioning it as a paid experience) is just as damaging. This isn't about mere early access on Steam or the Epic Games Store -- EA marketed the public's first look at Anthem as a "VIP" demo, implying it would be special.
Infrastructures are built for what market research determines as "reasonable" loads
Building for worst case scenarios when rolling out a piece of software, especially if it's "always on," like in the case of Anthem, is often more folly than it is helpful. There's only so much prep work that can be done before the network buckles under its own massive overload and the game is crippled by its own popularity.
Because I occupy a unique space in that I have the IT infrastructure and the CM/games experience, I can tell you definitively - you can QA and stress test all you want, but NOTHING prepares you for CCU hits from actual production go-live. It's systemic because it's inevitable. https://t.co/NxLDp5Bfdm— Frank (@PhoenixRed) January 26, 2019
Different publishers and developers approach infrastructure in different ways. Prior to Destiny's original open beta, which took place during the summer of 2014, was a full closed alpha that allowed a group of dedicated Bungie fans to get their hands dirty in the new game. It was invite-only. Destiny's open beta was a full month of stress testing servers and ironing out issues.
Anthem had two days of an inaccurately positioned demo.
A game I was working on expected maaaaaybe 50k people at once. We planned on 100k MAX. For the first live Beta over 120k showed up. Turns out the Canadian Marketing team went a little overboard. No amount of virtual stress tests will show off what really happens at scale— Tara J. Brannigan (@kindofstrange) January 27, 2019
Beyond fixing the technical problems, EA has its work cut out this week. There are likely VIP demo players who have decided not to give Anthem a chance at launch. There is understandable hesitation among those who want to play for the first time in the open demo that the same issues will exist for a second weekend in a row. And if EA can’t deliver a relatively stable demo experience, it’s a blow to confidence for the game’s earliest release two weeks later on February 15 (for EA Access players on Xbox One and Origin Access members on PC).
EA doesn’t just need to sell the public on Anthem (releasing in the most crowded window of 2019 so far). It needs to convince players that it has the IT chops to solve technical issues and deliver a solid experience on day one.
For more stories like this one delivered straight to your inbox, please subscribe to the GameDailyBiz Digest!/* =$comments; */?>