Under the Hood – Testing, Testing, 1, 2, 3


Testing, Testing, 1, 2, 3
By The Warden

Developing games is not a simple task, though it may seem so at first. As someone who’s been working on just one game alone for over a year, I can testify to this like a choir at church. What starts as a core mechanic shifts into a complex system put to the test when you present it to your table and the prized players with dice and cards in hand. It’s much like a hand of poker. When you first look at your cards, you see two pair and a rush of adrenaline surges through your chest. It takes all your willpower not to stand up and scream as you look at the pot to find over $500 lying there, winking at you, but you have to keep your emotions under control to avoid alerting others too soon. Because even though you might have the best hand of the night, another player might have an even better hand or be willing to psych you into believing they do. When it comes down to it, you never know if you have the winning hand until you win the hand.

I’m a firm believer in playtesting and thoroughly enjoy it as an activity all its own. At least twice a month, I have a group who does nothing but playtest upcoming material we’ve designed ourselves and there’s an inherent freedom in it. One late November evening found us playing the game for five minutes and debating alternate dice mechanics the rest of the night. (Does this make us geeks among geeks?) This may be a wicked thing to say, but a good playtester intends to bend the game until it breaks, because that is the ultimate goal of playtesting. Only when the game holds its own can it be ready to hit the market with guns blazing (or swords swinging, if that’s your genre).

During a playtest I ran late last year, the players apologized for “criticizing” and dissecting the game to what they considered the Nth degree. I laughed and said “But that’s what you’re here for. I won’t know if this game is ready unless we break it, because only then can it be fixed.” Because of that criticism, I’m more confident about the material than ever for one simple reason: I only have one brain. Playtesters run the game through a hive mind of simulation and conjecture from an outside point of view without the personal attachment to the project.

At this moment, eager players are wrapping up an entire weekend of playtesting and feedback for the upcoming iteration of D&D and you can bet they’ll be tearing that sucker apart. And that’s exactly what the designers will be looking for, though that statement may be a bit simplified. It leads to the old saying “It’s easy to be negative; being positive takes work.” It’s all about constructive criticism, not just criticism alone, and the dissection of how the game plays out in your mind, to your expectations, and engages your imagination.

Playtesting as Marketing
Playtesting is a valuable asset to all publishers, from the big budgets of Paizo and Fantasy Flight Games to the independent scale of Cubicle 7 and Evil Hat Productions. In this era of viral marketing, playtesting has become more than a review of a game’s current foundation, but a means to create buzz as playtesters brag about giving an upcoming game a whirl (even if they signed a Non-Disclosure Agreement). If anything, playtesting is crucial to those independent publishers attempting to break into the market or compete with some of the big guns in the field. The upcoming Marvel Universe RPG from Margaret Weis Productions, for example, has utilized their playtesting as a means to harness responses and feedback as well as plug the product with tweets and posts from comic book and RPG fans alike.

Like the publishing field as a whole, game production is looking to streamline costs and staff while outputting the same quality of product for the consumer. Many industry experts, such as Ryan Dancey, believe the roleplaying industry’s standard of large companies employing dozens of people will give way to one- or two-person home-based operations depending on freelancing and viral marketing to get the job done. Playtesting may be perhaps the best – or only – tool to gain attention for an upcoming product for one reason: it’s free. In troubled economies, getting people to check out your work without paying anything for it is a given.

Playtesters themselves are generally an eager bunch and take on the role (to whatever capacity they can handle) with fervor, never expecting any form of payment. All they ask is for credit and the bragging rights to say they’ve been playing the next edition of a favorite game (with that bragging enhanced by the fact they “can’t talk about it because I signed a NDA”). Playtesting is the focus group of the industry, the equivalent of loaning the newest model of hybrid cars to the general public in exchange for detailed feedback rather than just running it through a computer simulation or feed off questionnaires on a form.

More Than Just a Thumbs Up or Down
There is no better method for playtesting than actually sitting down at a table and playing it as if the game were already in full production and picked up from a store. While glancing through and reading over the rules, character concepts, and expansions can offer a hint of what’s to come, it’s no better than judging a film based on the latest trailer. Particularly when the game is a truly original endeavor; you might be able to gauge a lot about an adventure with working knowledge of the core rules, but even that robs the designers of the crucial information they need.

Better feedback requires detailed analysis. It is a lot to ask for considering the lack of pay commonly involved with playtesting, just think of it as the cost to gaining access to the material months in advance. How many players were at the game? Which characters were played and at what level? Did you run a sample encounter/adventure included with the material or did you write your own? On that front, was there anything you changed during the game? Why? Was it out of necessity (being that the rules made no sense or the concept felt unbalanced) or simply out of spontaneity? What was the overall reaction at the table? And more importantly, would you play it again and if not, why?

There’s an easier method to gaining all this information and that’s alpha testing, but even that can only go so far. The game has to leave a designer’s hands at some point and time – Monte Cook can’t sit down and run every single game for us, as much as we’d love it – and a successful game is one where the material speaks for itself. It needs outsiders.

In my own years of game design and development, I’ve had dozens of groups sign on to playtest material I was working on, from adventures to supplements to original systems. Without any actual numbers at hand, I think it’s fair to say only 10% of those respondents on average actually wrote back with anything, let alone comments and suggestions. To rub salt in the wound, even securing answers from a prospective playtester for this article turned out to be a challenge. This is perhaps the biggest risk – and annoyance – of public testing, yet it is inevitable. From a design standpoint, we can only dread that the response must have been horrible and the copy was burned in a satanic ritual to wipe out demons brought into the tester’s house.

Patron Projects
Some have found an interesting tool for inciting both feedback and funding in the form of patronage. Open Design has been doing so for years, putting out an open call for interested parties to contribute funds in exchange for access to the material as it’s being developed (or better yet, a chance to actually contribute, should you cough up enough). This has been taken to a more global step with Kickstarter, the latest online phenomenon, allowing backers to offer funds in exchange for rewards.

The irony of patronage is rather than paying people to test out your game, they pay the publisher instead. It’s much like the chef asking for the check and then taking you to the kitchen to help peel potatoes… but when something works, why question it?

Looking to the Future
Playtesting an upcoming game, supplement, or adventure may seem like a thankless task on the surface and perhaps many do not realize the importance of their work. It’s also an incredibly difficult concept for executives to understand (though I must admit my opinion is rather biased on this), letting the material out in the world for free during the age of online piracy. The success of a final product can be measured with sales and dollar signs. The only means for a developing game to know where it stands is with feedback, be it positive or negative.

With so much talk of doom in this industry – from outside and within – is it not reasonable to assume the future of role-playing games depends on the design of the games themselves? If the publishing world as a whole shrinks to become more economical, playtesters will take on more importance. Or let’s put it this way. Do you want to bitch about the game before or after it’s been released?

Share this post:

Related Posts

Leave a Comment