Game designers need fresh eyes to see if their game works as designed. There is some overlap in understanding with the terms
focus testing and
market research, but according to
Steve Jackson, the rules of blind testing are pretty simple. If I find my notes, I can illuminate them further. This is what I took away from Steve's
GDC 2007 talk:
- Frequently get naive users to play your game and observe the playing session personally.
- Do not help the users ever except to get them unstuck.
- Do not alter the users' impressions of the game. That is your data.
- Note everything that is unintended, and act on it in a new design iteration.
-
Interview the users about their experience and cut through the
impressions and advice to get to the core design problems. Do not use
leading questions that may destroy your data.
Interesting to note that
Warren Spector's company does this once a week. According to Steve Jackson, this was mandatory at one point, and when the restriction was removed, everyone still attended.
The Monolith team had a presentation in
GDC 2007 where they mentioned
focus testing and
market research. Using some kinds of play testing, they exposed several weaknesses in
F.E.A.R..
More telling than what they changed based on feedback was that they
admitted that the main weaknesses of the game (slapped on story system,
for example) were never tested with naive users. I think they probably
would have done better if they had augmented these types of testing
with blind testing.
Blind testing is frequently informally performed at work, but to see it integrated into the schedule would be interesting.