My wife designs the visuals and UI/UX for websites. Last week, she was telling me about something she’s learned over her years doing this. She said that the real core of doing it well is finding the balance point that serves multiple different user types, or “personas”, without letting any of those different users feel like the site was made for someone else.

She also mentioned that she’s learned to approach iterative design with the mindset of “failing faster”. You don’t spend time in the early stages of design refining details when you’re still building the underlying structure. Instead, you use broad strokes with the knowledge that they’re just faster to execute, and you expect a big portion of those broad strokes to not be especially close to the final target. You learn more from failure than from hitting home runs on your first swing, so you might as well embrace the early misses as educational experiments.

My head immediately went to a game playtesting journal I was once gifted by a coworker. It was literally called “Fail Faster”, and was created by veteran tabletop game designer Jay Cormier. It gave really sound advice on how to collect data from playtests and iterate on that data, and then supplied a structured journaling section for tracking everything you’d learned, changed, and retested in your design journey.

It’s a great learning tool, and I’ve since sent copies to friends who were just starting out as game designers. When I told my wife about it, she looked it up and remarked how applicable a lot of it was to what she does as a web designer every day. Turns out, designing stuff for other people to use is pretty much the same regardless of specific product type.

Jay covers a lot of ground in his book, so if you’ve read it, anything I’ll attempt to add to it today may sound redundant. I’ve had some experience running playtests and managing playtesting teams too, and want to share a few observations I’ve made along the way.

Over my nearly 20 years of designing board games and TCGs, I’ve run a LOT of playtests. Something I’ve seen over and over is that there are two vastly different fundamental kinds of tests: First Impressions and Break/Balance This Game. If you’ve read want to get the most out of your testing mileage, you need to do both, in that order, over and over.

If I had to draw a parallel between my wife’s experiences and my own playtesting cycles, I’d say the Persona testing is most like First Impressions tests, and Failing Faster is closer to the Break-and-Balance approach. The second format will happen more often and at a faster frequency than the first (as its name kind of implies), but both are critical to run multiple times.

Here’s a rundown of what you’re looking for and some tips on how to run each.

First Impression Tests

You may only get one shot to make a first impression, but you can make lots of shots if you show your game to a bunch of different people. Seems obvious, but the sheer breadth of what’s needed for effective First Impression testing is easy to overlook.

The trick is to make sure your testing pool has a bunch of different player archetypes you’re trying to tune your game for, and that you reach out to all of those archetypes several times over your game’s development cycles. You need fresh eyes every time though, so be ready to call in a lot of gamers (or non-gamers, if that’s part of who you’re trying to appeal to).

When I ran the playtests and development cycles on a recent contract gig, I ran four rounds of First Impression tests, all at least two weeks apart, and as many as six weeks apart for some.

Round One: Six players with mixed levels of experience with similar games, and mid-to-low experience with the game’s IP.

We learned that regardless of investment in the IP, players found the overall theme appealing, and felt that it tied well to the mechanics of the game. We saw places in the rules that needed clarification or reworking, and found things that seemed counterintuitive to new players. We determined certain things probably needed to be simplified or cut entirely to allow more focus on the elements players really liked. Estimated development progress: 20%

Round Two (two weeks later): Roughly 24 testers in seven different groups, all heavy gamers, varying experience and interest in similar games models. All heavily invested in the game’s IP.

This time, we learned that the players most invested in the IP were very surprised to see the themes of the world they loved applied to a whole new kind of game. Some really liked it, some were initially disappointed in the moment that it wasn’t closer to their early expectations. (We intentionally hadn’t told them how much of a departure this new game was from what they knew.) Final notes all came back that the game worked, needed tuning (we knew that), and that they could see themselves sharing the game with a broader audience than their usual IP-specific play group. We were really happy with these results, and turned our focus to the Break/Balance crowd — more on that later. Estimated development progress: 30%

Round Three (Six weeks later): Heavy gamers with mixed experience with the IP, lots of experience with similar games models models, and TONS of experience playtesting and improving games. Sidebar, if you’re designing games independently, find yourself a local Unpub organization and get their feedback. Every Unpub playtester I’ve met has been friendly and insightful.

This group was analytical to the core, and was maybe most surprised by the simplicity of the questions I asked at the end. They liked the way the theme worked with the mechanics, saw lots of places to dissect and optimize strategies which they enjoyed. They had thoughts on places to polish and tighten up the highlights of the game, which were great to get from a team that hadn’t been privy to all the Break/Balance conversations. Their comments about the flow and overall experience told us we were not only on the right track, we were well on target and in the home stretch. Estimated development progress: 80%.

Round Four (four weeks later): Eight players in two groups of four. All heavily invested in the IP, most very familiar with similar games. Expressed similar surprise to round two group in regards to seeing the IP on a new kind of game.

High fives all around. Could not have asked for better news at this stage. The players found rules they could understand quickly, a game that was light enough to play fast the very first time, and enough depth and strategy that both groups immediately reset and played a second time, before I could even ask for their initial thoughts. Estimated development progress: 95%

Now, I promised earlier some tips for running First Impressions tests. The two biggest things that come to mind that I haven’t already covered are these:

• Keep it casual. You want players to feel like they’re trying out a new game in the usual conditions that they’d get together for other games. I wouldn’t necessarily playtest something brand new immediately after playing some finished games, but make it feel like that sort of get-together.
• Keep the questions really simple. I absolutely recommend starting with, and maybe even ending at, High/Low/Buffalo. There’s an explanation of the system about halfway down the page in my Playtest at the Disco post, and this is the perfect place in your iterative development process to put it to use. Don’t inundate your testers with questions about details (card functions, deeper strategies, etc.), just broad notes like “are the rules generally clear enough” and “if you imagine this as a finished game, what would you compare it to?”. There will be time later to dig into specific individual game component interactions.

As important as running multiple rounds of First Impressions tests are, don’t stress about them. Listen for the tone of the song, rather than whether the lyrics are right. You’ll have plenty of time to lose sleep angsting in the Break/Balance phases.

Which gets us to…

Break/Balance This Game

This is where the deeper nuts-and-bolts of development happen. You’ll need to find a group of really experienced and committed gamers for this, and should even consider compensating them in some way (many will gladly accept credits in the rules doc and a copy of the finished game, but it can never hurt to be more generous too).

I’ve assembled Break/Balance teams of up to 40 players at once, but can say from experience that a group of 20+ players offering a few hours a week can be very effective. And if you’re at the point where you can put money into paying people to focus on a game part- or full-time, your team can run a tighter ship with fewer players. When I began working on Lorcana, there were fewer than 10 full-time playtesters on-site. Regardless, you’ll need your team to collectively log many, many games, possibly in the hundreds.

This is a long-term process. Be very transparent with them as testing — including First Impressions games — goes on. Give them the tools to play frequently, and to record their findings easily. Make sure you’re capturing as many details as you can without overwhelming them. Make your prototype updates as frequently as you can, based on subjective data first, but also objective data.

To these ends, I recommend making your game prototype available online via Tabletop Simulator or Tabletopia (there are other systems like ScreenTop too, but these are the two platforms I know best). Build Google Forms that can feed into automated spreadsheets for data collection and analysis. Create a Discord server where playtesters can meet up, find opponents, launch games, and discuss their experiences with the game.

Your goal is to find the specific points where rules and components are making the game harder or less fun to play, and to solve those issues as quickly as possible. It will be stressful, especially if you have a deadline attached to the process. You need to be able to manage a lot of playtester personalities, and know what sort of notes need critical attention, and which can be a lower priority.

It’s a combination of following your gut here, and following the math. Get skilled at understanding which one is more important in any given moment.

There’s a common question that comes up in game design circles: which is more important, making your game balanced, or making your game fun? Ideally, if your Break/Balance testing is providing good, actionable data that you follow up on, you’ll be getting closer to both optimal balance and fun.

How do you gauge those leaps in balance and fun? That’s where you shift back to First Impressions testing with fresh players who haven’t seen the game before. Gather their big-picture feedback, and share it with your Break/Balance team. They may disagree with what your latest First Impressions group saw, but by this point they’re looking at the game with very different eyes.

Once you reach the point with your First Impressions groups that they feel like they can play it a second time with no help or prompting (like the Fourth Round groups earlier in this post), you’ll know your game is close to done. Start wrapping up the things you’ve been tuning with your Breaker/Balancers, and begin really thinking about the next steps. That may be art, or pitching to publishers, or stepping up your infrastructure for self-publishing.

———

It’s a lot to chew on, I know. I’ve done it many times, and it took me years to dial in my process. Hopefully this has given folks new to the process something to build their own system from.

I’ve used this methodology mostly for tabletop board game design, but it’s equally as applicable to Trading Card Game design, especially in the rigid, “building the framework” part of the design process. The proportions of First Impression versus Break/Balance can be a little different for TCGs; as I said in my previous post, there are really two major stages of design for those, and the needs of those two stages are considerably different. Ultimately though, both use the same two types of testing.

Good luck, and by all means, if you’ve got any follow-up questions, I’m always happy to talk shop. Ask away!

---