Level 12: Solo Testing

At this point you have some ideas, and you have some feedback. As with many things in life and in game design, you could sit here forever contemplating which is the best choice… but at some point you’ll need to start working towards your goal. Choose a direction and go with it, even if it might not be the best one. Trust that you will be able to use the iterative process to fix any mistakes you make along the way.

Today I’d like to cover the general concept of playtesting. As we will see, there are many different kinds of playtesting, and it is important to be able to differentiate between them in order to get maximum value from our time.

 

Readings

There are no readings for today, other than this blog post. Take the time that you would have spend reading, and use it to work on your Design Project.

 

Different Kinds of Playtesting

The word “playtesting,” like the word “game,” is overused and can mean different things to different people. In general, the term covers any activity where you are playing a game in progress for the purpose of improving it. But different playtests may have different goals, and it is important to know what your goals are before you do anything.

I’ll be playing a bit fast and loose with terminology here, so in this case the concepts are more important than the labels I’m giving them.

Bug Testing (or Quality Assurance)

The purpose of QA is to find errors in the game’s behavior relative to its design. “Fun” does not enter the equation. If the designer says that the game should do one thing and it actually does another (even if what the game is doing may be superior), that is a bug that needs to be identified.

Normally, we think of bug testing as specific to video games. Board games do have a corresponding kind of testing, where the purpose is to find holes in the rules and dead ends in gameplay – gaps in the game that the designer did not cover.

Focus Testing

In a focus test, you bring together players that are part of the target audience’s demographic in order to determine how well a game serves their needs. This is normally done for marketing purposes, but if game designers are involved it can also help to make the game more enjoyable for that particular demographic.

Usability Testing

In a usability test, players are given specific tasks to accomplish in an attempt to see whether they understand how to control the game. This is done frequently in the greater software industry to make sure that a piece of software is easy to learn and easy to use. Video games can take advantage of this as well, and results from a usability test can be used to either change the controls or modify the early levels to teach those controls more effectively.

In board games, usability is doubly important, because there is no computer to respond to player input for you. If you misunderstand how houses work in Monopoly and place them on Community Chest spaces, the game will not stop you. By observing players who are trying to play your game, you can learn a lot about how to design the various game bits so that they are easy and intuitive to use.

Balance Testing

A fun game can quickly become boring if some kind of play exploit exists that lets a player bypass most of the interesting choices in the game. If only one strategy can win and it is just a matter of which player follows that strategy the best, it is not as interesting as if there are multiple paths to victory. Likewise, if one player has a clear advantage over the others, it is important to identify that so that players do not feel the game is being unfair. The purpose of this kind of test is to identify imbalances in the game so that the designer can fix them.

Fun Testing

A game can be usable, balanced and functional and still be uninteresting. That elusive “fun factor” may be hard to design intentionally, but when people are playing the game it is pretty obvious whether they are having fun or not. Certain aspects of the game may be more fun than others, so it is also important to figure out what parts of the game need to stay the same… not just what to change.

 

All of these forms of testing have some elements in common. Best practices are similar if not identical. All are important to the success of a project. So why make a distinction?

The reason is that each is appropriate at different stages of completion in a project. Each kind of testing has different goals, and you need to know what your goal is before you can achieve it.

 

Order of Effects

When should you do which kind of playtesting? What order do you do them in? A lot depends on your particular project, so some of this will be up to your judgment as the designer. However, there are some rules of thumb.

• Very early on in the project, you need to make sure your project will meet its design goals (usually the “design goal” is to make a game that’s fun to play). Testing for fun is necessary to make sure you do not spend a lot of time building on the wrong foundation. If you are making a game for a specific market, focus testing may be involved at an early stage as well, simply to ask the target audience if a game with a particular concept sounds interesting to them at all.
• Once you know that you have something, you need to solidify the mechanics. Design the whole game, making sure that all the details are taken care of. Test for “bugs.” (Note that bug testing in software projects is often done continually throughout the project, increasing in intensity toward the end. Non-digital games are easier to “debug” though, and a “bug” can stop a playtest in its tracks, so it is important for us to have a complete set of rules early in the process.)
• Once the game is fun and the design is complete, gradually shift from testing for fun to testing for game balance. Make sure that all the numeric values and player abilities are where you want them to be.
• When the game is working and balanced, towards the end, you’ll want to think more about the usability of the game. When you change usability you are not changing any mechanics, merely the way those mechanics are presented visually to the players. This is an important step that is often neglected. If you’ve ever encountered a game that you could only learn by being taught by another player (as opposed to reading the rules yourself), that is the kind of usability failure you want to avoid in your own projects. You may also do additional focus testing at this time, to make sure that the theme and visual elements of the game appeal to the target audience.

As I said, these are just guidelines. If it is incredibly important that your game be well received by a particular demographic, for example, you may be doing focus testing throughout the project at all stages. Do not let this order of things be your master.

 

Different Kinds of Playtesters

As there are different kinds of testing, there are also different kinds of testers. Each kind of tester has their own strengths and weaknesses, and some are more important for some kinds of testing than others.

• Yourself. You are your own most valuable playtester. Do not forget your ability to play your game on your own. You know your game better than anyone.
• Other game designers. If you are lucky enough to personally know some other skilled game designers, you can get some very useful testing done through them. They are able to critically analyze your game and propose design solutions. (If you do not know any professional designers, perhaps you can at least make contact with other participants of this course.)
• Close friends, family, and confidantes. People close to you who are willing to provide their time to test your game are very useful. They are approachable and can make themselves available as a favor to you. Take good care of them, and do not abuse their kindness. Note that these people may not fall into any of the other categories, so while they are good for early tests, they may not be appropriate in more focused testing for bugs or balance since they may not know what to look for.
• Experienced gamers. Skilled game players are great at finding exploits and dominant strategies in a game, and are appropriate for balance testing.
• Complete strangers. People in your target audience are appropriate for focus testing and usability testing, and they are absolutely critical when testing for fun. Finding them can be tricky, though, because it is not in most of our natures to just walk up to someone we’ve never met and ask them to play a game. We will talk more about this in the coming weeks.

 

Order of Familiarity

In general, you will want to go through testers in order from more to less familiar. Test with yourself first, then with close friends, then with acquaintances that are useful (because they are designers, gamers, or part of the target market), and then with strangers.

If you show your work to other people too early, it will likely be in such a rough state with multiple design flaws and holes in the rules that it will waste their time and frustrate them, and you want to treat your playtesters better than that. Also, if you start playtesting with strangers too early in the process, you may not get useful feedback – if your game prototype is in a rough state with only crude art and components, for example, the playtesters may be so busy commenting on the poor quality of the pieces that they will not be able to concentrate on the gameplay.

At this point you might be tempted to just do all of the playtesting by yourself, so that you don’t need to rely on other people or keep track of them. In practice, the designer eventually gets too close to their own project and is so familiar with the game’s systems that they can miss some really obvious flaws. If you keep the same set of playtesters for long enough, they will suffer from this problem as well. You need to bring in fresh sets of eyes to look at your game on a continuing basis throughout the project.

 

Playing By Yourself

In the early part of playtesting, when you are playing the game on your own, here are some things you should be looking for:

Does the game meet your design goals?

Is it fun, at least for you? While you are not the ideal playtester to judge effectiveness most of the time, if you are not having fun then most other people will probably not either.

Are there any holes in the rules?

A “hole” is a situation where the rules simply do not say how to proceed. For example, perhaps one of your rules is that a player’s army can attack another player’s army, but you don’t yet have rules for resolving the attack. What happens in this case? In practice, what happens is that the players sit around and wait while the designer figures out what to do!

As an example, consider these rules for Tic-Tac-Toe played on a 4×4 grid:

• Players: 2
• Objective: Get a straight line of symbols.
• Setup: Draw a 4×4 square grid.
• Progression of play: On your turn, place your symbol (“X” or “O”) on an empty square.
• Resolution: If either player on their turn has a set of four of their symbol in a straight line (across, down, or diagonally), they win.

If you try to play this game just following the rules, you’ll quickly realize that you can’t even start – nowhere does it say which player is X or O, or who takes the first turn! To fix this, you would add a situation to handle this. For example:

Setup: Draw a 4×4 square grid. Choose a player to go first, who is assigned the symbol “X”. The other player is given the symbol “O”.

Are there any dead ends?

A “dead end” is a game state where there is no way to proceed further, but the game is not resolved. Consider our revised 4×4 Tic-Tac-Toe rules above. Suppose that both players fill up all squares on the board without anyone winning. At this point the game cannot proceed, because the rules say a player must place their symbol on an empty square. There is no empty square, so the player cannot take a turn. But there is also no resolution, because neither player has won. In this case, a new rule would have to be added (such as: in the resolution, if neither player can make a legal move and no one has won, then the game ends in a tie).

Are any of the rules unclear?

It is natural for us to assume things that are in our head, to the point that we often forget to write them down in our rules. Try to look at your rules and see if there is anything you are assuming that your players might not.

Are there any really obvious rules exploits?

Is there a single strategy that wins the game easily? Try to find it. It’s much less embarrassing if you find and fix it yourself, as opposed to having it discovered by your playtesters (or worse, your players after you release the game). Clarity and exploits are often hard to find in your own game; you tried to design this game to not have any problems, after all. Still, make an honest effort, and sometimes you will be rewarded by finding and fixing errors early (which saves a lot of time in the long run, leaving you more time to iterate on other parts of your design).

You might think that looking for exploits is something to do later in the project when balancing the game. Sometimes it is. It is a matter of degree. If an exploit is so powerful and so obvious that it prevents your playtests from giving you real information about your game, fix it now.

 

Solo Test Difficulties

There are a few things that are hard to test alone:

• Realtime multiplayer games, such as games where you must slap a card or say an answer faster than your opponent.
• Hidden information games, where each player has information that only they know and that is important to keep secret from the opponent.
• Trading, negotiation, and auction games, where each player must place a value on an item, and different players may value things differently (and especially when players can artificially extort high prices or drive up the cost of an item at auction just to make their opponent pay more).

For the latter two, it is possible to play anyway, by simply limiting your actions to what you think you would do if you were in each player’s situation, knowing only what they would reasonably know. Some people find this more difficult than others.

The simplest answer here is, for the purposes of this project, to not use mechanics that you can’t test yourself. The alternative is to bring in another player or two early on in this case only, after you take things as far as you can on your own.

 

Let It Grow

Experienced designers often talk of a game “making itself” – as if the game has a life of its own, and the designer is merely guiding it rather than creating it. On the surface, this seems strange because really, the game is just sitting there and doing nothing unless the designer is playing it or changing it. What’s going on here?

I think that what is really happening is that the creation of a game is a learning process. You may have some idea of where you want your game to end up, but the final version may be very different from what you originally envisioned. The reason why it changes is that at the beginning, you don’t know very much about your game. You have some basic ideas, but you don’t actually know how the mechanics will interact, or what the actual dynamics and aesthetics will be. As you playtest, you learn more about how your game’s systems are working. As you learn, you become more able to predict the effects that changes will have on the system.

Right now, though, you don’t have that experience… at least not with this game. Playtesting on your own is your first act of discovery. As you discover, it may seem as though your game wants to grow in a new direction, as if it has a life of its own. If you feel that, go ahead and listen to your game. See where the process of discovery takes you.

 

Homeplay

As the nature of the work at this stage is solo, you do not have to post anything on the forums. Work alone until Monday, at which point we will start involving others.

Before Monday, August 10, noon GMT, create a playable prototype of your game. You may find it useful to review the section of Level 4 on prototyping. Remember that you can make a prototype in about 15 minutes – it doesn’t have to be pretty and it doesn’t even have to be complete (yet), but it does need to be at the point where you can sit down and play it by yourself.

Then, play the game by yourself, at least once. In your idea notebook, write down any problems you encountered or questions that you ran into. Trust me, no matter how obvious the things are that you need to fix, you will forget them if you don’t write them down.

Finally, write down the rules once your game is at least somewhat playable. This is also for your own reference, so that you do not forget.