GameDevBlog

Blah blah blah

Last night I dreamed I was protecting Earth from mechanized tripod invaders, but I was running low on ammo.  In short, videogames have become my dreams.  Which reminds me - I read somewhere that videogamers are more likely to have lucid dreams than normals.  I did an informal poll of the three people in my office, and none of them can lucid dream.  So I'm not sure I buy that research.

Bloglines

I'm hip to bloglines now:  my favorite thing about it is your information is stored server-side, so I can view my blogs from any computer.  No more trying to sink up RSS readers on my laptop, home machine, and work machine. 

And, in case anybody cares, here's what I'm currently subscribed to. I'm probably missing several important ones...

Variance, Part 2

Ok:  so where am I?

Yet another example of variance:  a strategy game that uses die rolls to determine outcomes of battles is higher variance than a strategy game that does not.

Somebody out there might be saying, "*I* can beat the first boss in Ninja Gaiden every freaking time.  It's not such a high variance game."  What I mean is - (wishing my blog could support graphics right now) - if you were to graph the probability of succeeding a challenge against player skill, a low variance game would show a steep jump from 0% to 100% when player skill reached a certain point, whereas a high variance game would show a gradually increasing slope.  My Ninja Gaiden skill puts me at around a 10% success rate against that first boss - yours may put you near 100%, but that doesn't mean it's a low variance game.

High variance games violate Scott Miller's "God concept" - even God (assuming he doesn't cheat) won't always win at poker.

My first instinct - because I usually dislike high-variance games - is to say, "Variance is bad."  In fact, I used to think of variance as "slop" - mushy, chaotic randomness that got in the way of a game's true expression.  (And I was pleased with myself, because that gave me a formal aesthetic with which I could argue that Tony Hawk 4 was a superior game to SSX3...albeit not as pretty.)

But this, as with all else, really is a matter of taste, and a matter of whether the variance fits the game. 

Because there are good things about variance:  in a low-variance game, once you've solved a challenge once, there's no reason to ever play it again.  In a low-variance game, you can memorize patterns - and the game stops being about solving problems on the fly and starts being about memorization.  High-variance can dump you in situations you aren't prepared for and get your adrenaline pumping.  One of the exciting things about SSX3 is that your runs don't usually go as planned, and you make do with what you end up with.  Spider-Man 2 is the same way - subtle differences in timing can mean you land on a completely different building than you intended to - now you have to make do with the hand you've been dealt.

In fact, one could argue that low-variance is boring, predictable, and that we should always strive for some variance because it adds replay value.

A good case in point is Mercenaries - (The good people at Pandemic are my new heroes, by the way, now that Ion Storm has bitten the dust.  First Full Spectrum Warrior and then Mercenaries.  Game design geniuses over there.) - Mercenaries is high variance.  With one of the ace missions it took me a dozen tries or so to get to the first objective;  then I died;  and then it took me another dozen tries to get there again!  Was I frustrated to be doing the same stuff over and over again?  Yes, I was, but I probably would have been even more frustrated if that stuff was predictable, because it would mean however many minutes of being bored for each replay.  Each time I played the situation shook down a different way, creating a varied gameplay experience.

What I need to do is pin down why the Ninja Gaiden and Devil May Cry boss fights bother me, but the SSX3 runs and Mercenaries missions do not.  It's possibly in the kind of variance.  With SSX3 and Mercenaries and poker the variance is more situational.  SSX3:  subtle differences in timing and action put you on a different part of the slope where different moves are going to be effective.  Mercenaries:  subtle differences in where the enemies go after you engage them creates different warzones.  Poker:  differences in the hands dealt create different landscapes to play in.  The variance in Gaiden and DMC is usually just:  which attack is the boss going to do next? (And is he going to be on camera when he does it or am I going to get blindsided?)  My thoughts here are still murky, even to me.  I'm certainly not suggesting that we take away the randomness of the boss attacks and replace them with patterns.  Shudder.

One last note:  high-variance games are easier to develop than low-variance ones.  In particular, 3d videogames bring a lot of chaos with them that their 2d brethren didn't necessarily have, and tricks to make 3d less variant usually involve constraining the gameplay into two or one dimensions (the wall running of Prince of Persia, the rails of Tony Hawk, Jet Grind Radio, or Sly Cooper).

Variance

Something I haven't seen a whole lot of discussion of in the videogame developer literature is variance in games.  Poker is a high variance game, so there's plenty of discussion of the phenomenon in gambling literature.

I think variance is one of those things that a game designer should give serious thought to when designing, along with feedback and emergence and learning curves. 

So what am I talking about?

Variance is the amount of randomness in the outcome of your game.  Poker is extremely high-variance:  even the best player in the world will lose regularly.  Chess and Go are very low-variance:  Kasparov will beat me in chess 99,999 times out of 100,000...maybe even more.  (Which means if we play one game an hour...let's say ten games a day...I'll need to play for something like 30 years to expect to beat him once...of course, if I play 10,000 games of chess with Kasparov I might get a little better at chess and improve my odds.  Okay, digression over.)

How does this apply to videogames?

The SSX series is high variance, whereas the Tony Hawk series is low variance.  When I play Tony Hawk, with most of their challenges, if I can beat a challenge once, I can go back and repeat my performance almost every time.  This is because of the Tony Hawk rail system is really tight:  you can use rails in Tony Hawk to line up shots and create repeat performances.  With SSX, on the other hand, slight differences in timing and pressure on the analog stick can cause you to miss the rail completely and end up doing a totally different run than you might have originally planned.

Devil May Cry and Ninja Gaiden have high variance in their boss fights:  when I finally beat the first boss of both of these games, I was pretty proud of myself.  "I figured out how to beat the boss!"  I thought to myself.  Later, I went back and tried to beat that same boss again, and failed.  Apparently I hadn't "figured it out" - I had simply gotten good enough to beat the boss some small percentage of the time.  (Or maybe there was some kind of dynamic difficulty adjustment going on, making the bosses easier and easier until I finally passed them.)

With Zelda, on the other hand, once you've figured out a boss, you can beat it again almost every time.

I've got to go;  I will continue this essay later.

Notes on a bunch of games I haven't finished yet

I do intend to go back to these games and finish them at some point - usually I don't let myself play a new game until a previous game is finished - and I only let myself buy a new game if I don't have any games that I haven't played yet.  But somehow the system broke down.

Anyhow.

Mercenaries:  The pandering to the player (take a rocket propelled grenade in the face and you live;  make sure to hide behind a rock and catch your breath, though, otherwise you might not be able to take a second one to the face) is perfect for what they're doing here.  The desturctible environment is unparalleled - if I was a younger man with more free time I might see if I could destroy every single building in the game.  Pave the earth...a little like back when I filled the ocean in Ultima II with ships...I completely lost count of the number of enemies I killed in this game so far but it must be over a thousand.

Resident Evil 4:  Finally, a Resident Evil that I can stand the controls of.  It's still unweildy;  you still drive your character a little like a truck;  but it fits in with the game.  The zombies are coming at you slowly, so the slow controls seem to harmonize.  And it has the continuous world and the RPG-lite shopping and this treasure-hunting side event...kind of neat.  One thing it's not, however, is scary.  You need Fatal Frame 2 or Eternal Darkness for that.  My kill count is over 400 so far.

Wik and The Fable of Lost Souls:  Discovered this thanks to the Gama postmortem that they published today.  Obviously, I'm partial to swinging as a gameplay mechanic.  So this is right up my street.  I think it may also be the best example of controlling a character in a 2d game with a mouse I've ever seen.  Also a good example of a small number of different elements creating a wide variety of interesting levels...interesting choices abound, all the time, hectically.  http://www.wikgame.com.  I forgot to count kills - I'm sure it's in the hundreds.  Enemies--in this case, bugs--are only one or two clicks to kill, and you can sometimes kill lots with one stroke, so I expect to push a thousand before I'm done.

Batman:  Vengeance:  Coming to this a little late.  (Four years late.)  Hey, it got mediocre reviews, so I figured...why bother?  It's actually not bad, so far.  I like the art direction and level design;  particularly the rooftops of Gotham.  Hope they have more of those.  Only beat a few dozen enemies so far.

Seems like I like everything, doesn't it?  Maybe I need to refine my tastes.

Continuing my Ill-Advised Project

Spider-Man 2:  300 enemies.

This is assuming you're playing the game the way our focus testers did, by going straight for whatever bright and shiny dot is on your HUD rather than doing any of the  side missions or trick challenges. 

Only a quarter of that number is in story missions, so it's technically possible to win the game after beating only 80 or so enemies.

I determined this number empirically, by playing the game at work the last couple of days when I should have been working.  The way Mark put it:  "Jamie's like an expensive, inefficient tester."

Somebody asked me who Mark Nau is, btw:  he's one of the brainiest guys at Treyarch.  We're sharing the Creative Director position on our yet-to-be-announced project;  he's the brains, and I'm the...I don't know what...the guy who likes Spider-Man and third person action-adventures a lot.  We share an office, so I frequently hear his opinions, and can share them with you.  His CV:  http://www.mobygames.com/developer/sheet/view/developerId,26235/.  He also writes a Secret Blog that's not about games that I don't actually know the link to but the intrepid reader may be able to find it for themselves.

Both the Prince of Persia numbers and this are kind of eye-opening for me:  if you'd asked me, "How many enemies do you think you fought in those games?" I would have told you around a hundred or two.  The reality is two or three times that, so in this one sense the project has already been worthwhile, giving me a more accurate idea of just how much fighting is really in these games.

What is "Quality", anyhow?

A lot of people get bugged when I use gamerankings as an indication of the "quality" of a game.  I don't blame them:  like them, I have a list of a dozen or so games that are rated 90+ that I for the life of me can't see why.  And then there's Metal Gear Solid 2: almost nobody I know likes that game;  most hate it;  and there it is, 90+.  Clearly, there are whole groups of people not being represented by gamerankings.

And yet, I persist.

Why?

There's a world of difference between saying, "X is a great game" and "I love X."  If you say "X is a great game", you're implying that it's objectively great.  How can you back up such a claim?  Have you developed the one true formal aesthetic for that kind of game?  And your aesthetic is correct, whereas the aesthetic of Joe Assjack Game Reviewer is obviously flawed?  I know I have my own aesthetic, and it makes for interesting conversations by the water cooler as I try to explain why Galleon is in fact the Best Platformer Evar and I am shocked and alarmed that nobody else can see it.  Still, I recognize that I'm just blowing smoke.

Well, what other ways are there to back up the claim?  "X is a great game because everybody likes it."  We could do surveys.  Full-on marketing research type surveys are going to be more accurate than gamerankings and suffer less from the nonrepresentative sample problem.  Good data.  Unfortunately, most of us don't have access to a marketing research department.

How about online user surveys?  Again, you get the nonrepresentative sample problem.  You get the group of users that reads a particular site.  I actually had high hopes for Amazon.com's user ratings, figuring that Amazon.com is the closest thing we'd get to an online representative sample, but on close inspection I discovered their system is broken:  if you look carefully, on some products, you'll find that a lot of people give great reviews in the text and then give the product zero stars.  It turns out their interface is flawed, and it's very easy to give a product zero stars by accident.  So much for that. 

User reviews also suffer from the problem that well-known games get a zillion reviews and games you've never heard of get 14, so you end up with *Hitman:  Blood Money* being the #20 game of all time as far as Gamerankings users are concerned.

Okay, how about using sales as an objective measure of game quality?  Don't make me laugh.

How about the Game Designer's Current Most Popular Method of Measuring Game Quality:  "How much do I and my friends like it?"  This is an easy trap to fall into, because my friends and I kind of think alike, and when I say, "Viewtiful Joe is the Best Game Evar" they don't disagree with me.  But my friends are an even less representative sample than Gamerankings.

See where I'm going here?  As much as they suck, Gamerankings and Metacritic are the best we've got as far as trying to find some semi-objective data as to how the world feels about a game. 

Maybe we should just give up.  Throw out the idea that there can be an objective measure of game quality, and stop saying things like, "Such-and-such is a great game."  And instead of trying to make games that appeal to the mass market, we make games for ourselves, because that's all the data we have.  So I'm going to get back to work on my Ancient Domains of Mystery/The Sims hybrid, and I'm sure it will sell great.  Did I mention it's going to be all text?

Or we can accept Gamerankings as a useful data point.

One other thing:  JP pointed out that game reviewers can be bought.  Yes, true.  But to get that really kick-ass score on Gamerankings you have to buy all of them.  Apparently Atari got caught buying some good reviews for Driver 3.  It still didn't get Driver 3 a good score on Gamerankings.

JP also pointed out that game reviewers are as susceptible to hype as consumers.  Which I'm fine with, as long as those consumers, in return, are as susceptible to hype as the reviewers.  If everybody shares the same illusion that such-and-such is a great game...is that really an illusion?

Back to the title of this entry.  What is "quality"?

Total Quality Management would say that Quality is an absence of defects.  That  doesn't apply to video games, because a video game that has nothing wrong with it is still not necessarily worth playing.  This attitude helps lead to the "highly polished turd" phenomenon.  Unless we start writing bug reports that say things like, "This game lacks Tim Schafer's genius.  To reproduce:  1)  play game;  2)  note absence of genius."

So I know one thing that Quality is not, but I can't actually tell you what it is.  Woah.  I just realized it took me an hour to write this entry.  Do I really spend that much time blogging?

Number of Enemies

You've heard of that study they did on videogame jump duration, where they measured how long it took to do a jump in a bunch of succesful games and decided that the best jumpfeel was .56 seconds or whatever?  I can't find a link to that article now...if anybody can find it, let me know.  (Side note:  Spider-Man 2's jump was much longer than that and still the game was somehow succesful.)

I'd kind of like to do a similar study for the number of enemies in third person action adventures.  I'll start with Prince of Persia.

With the help of this FAQ (the only one I saw that actually listed enemy numbers, thank you "The Lost Gamer") I counted the number of enemies in Prince of Persia, making guesses at composition...so these numbers aren't terribly accurate, but are probably accurate enough for my purposes:

Human guards:  10

Red sand fiends:  130

Sand Beetles:  20

Female sand fiends:  70

Blue fiends:  70

Sledgehammer fiends:  50

Birds:  25

Chain fiends:  40

Sand creatures after you get the one-hit-kill sword:  40

TOTAL:  415

That's a whole lot of killin'...take into account dying and reloading and you're probably killing around 600 critters.

Still, this one game doesn't really leave us with enough data to start drawing conclusions.  And it's a dodgy idea anyway - some games the monsters are much easier to kill than others, so how do you compare their enemy counts?  Apples and oranges.

Bad data is better than no data, I say.  Mark Nau would say, "No.  No it isn't."