GameDevBlog

TDD

The last few days I've blown the dust off of a prototype and gotten it up-and-running again to discover that it's embarrassingly buggy.  In fact, as I've been trying to resurrect the project I've been spending more time fixing bugs than making it cooler.  One bug in particular I was having trouble reproducing the situation in which it occurred.  And then I thought -
Maybe this is a good time to try out Noel Llopis's free unit test framework!
I made this decision around 7:30 tonight.  Since then, I downloaded the stuff, compiled and ran the sample, got it working with my project (which required seperating most of my game code into a static library, a surprisingly painless task - that would have really kicked my ass ten years ago), wrote the code to set up the situation I was trying to repro (laced with CHECK() statements), and wrote the test.
Voila.  My fix worked. 
But is it fixed in the wild?  I can't be 100% sure, but the code isn't *that* crazy, so I'm confident.
So this was all done before 10:30.  Three hours.  And now I have a testing framework.  One of the things I resurrected with the project was its old testing framework...which didn't compile.  (I found a note on my buglist:  "Unit testing framework should actually...work.")  So, going forward, I'm going to give test driven development a try. 
I haven't had the greatest luck with unit testing in the past.  I've even mentioned it before.  It's not one of those cut-and-dried things, like source control or bug tracking software where once you implement it you wonder how you ever lived without it.  For me, at least.  Even on Clinton Keith's webpage he mentions that velocity went down when they implemented it.  In theory, it will pay for itself in the back end.  In theory, I will get my prototype to the point where I'm spending more time making it cool and less time fixing legacy bugs.  I sure as hell hope so, because the bug-fixing part of making a prototype is no fun.

Can I Have A Law Named After Me?

[For anyone who's coming to here recently because of the IGDA forum, let me say right up front that after working with this a while I've changed the factor to a more optimistic 0.8 -this number, btw, will vary from studio to studio, depending on various factors - but I'm going to leave the text of the original post intact:]

Looking at a preview and then hitting "back" erases your post?  WTF?  Okay, starting over...

I had a friend, George, in high school who was jealous of guys like Planck and Avogadro for having their own constants named after them.  So he decided he was going to have a constant and it was going to be 5.  5 was George's Constant.  Whenever our math teacher wrote 5 on the chalkboard, George would pipe up, "That's my constant!"  It seemed pretty funny at the time.

Anyhow...

We've all heard of Brooks' Law, right?  From The Mythical Man-Month:

Oversimplifying outrageously, we state Brooks' Law:  adding manpower to a late software project makes it later.

Turns out Brooks' Law isn't really true.  Hey, he said it was an outrageous simplification.  I've been on multiple projects where we realized we weren't quite going to make it so we added a guy or three and hit our date.  Still, the spirit of Brooks' Law is true - adding more people to a project makes it less efficient, as communication overhead goes up.  Instead of coding, the guys are asking each other what to code, how to code; telling each other what to code, how to code;  going to meetings, training, trying to understand each other's code, and so on.

I read, in I forget which software engineering magazine, sometime over a decade ago, that we could estimate the effect of "Brooks' Law" by taking the square root of the number of coders on the team.  So, 4 people can only do the job of 2;  9 people can only do the job of 3;  and so on.

That's too conservative.  Maybe if management is absolute crap:  there's no lead, no source control, no planning, nothing.

On the other hand, most people estimate as if there's no communication inefficiency at all.  This is not something that's modeled into MS Project.  Add a guy to your Project and you don't see all of your bars automatically getting slightly bigger.  Some teams might estimate the inefficiency by not assigning the lead any tasks - so n programmers are now doing the jobs of (n-1) programmers.  That's too optimistic.

We need some middle ground.

So, simplifying not-as-outrageously, we state Fristrom's Law:  if there are n people on your team, expect (n ^ .75) people's worth of work out of them.

This law roughly matches my experience:

• Die By The Sword.  8 people.  8 ^ .75 = 4.75.  DBTS had 8 levels.
• Draconus.  16 people.  16 ^ .75 = 8.  Draconus had 15 levels.
• Spider-Man.  30 people.  30 ^ .75 = 12.8.  Spider-Man had around 20 levels.

The "levels" may be an apples-an-oranges thing, really I'm mostly going by feel.  Spider-Man felt about 50% bigger than Draconus.

This law is useful for planning.  Say you've made a rough estimate of your entire game - and you think it'll be 240 man-months of work.  That means you could finish it all by yourself in 20 years.  How many people will you need to get it done in a year?  If there was no communication overhead at all, it would take 20 people.

But, applying Fristrom's Law: 

n ^ .75 = 20
n = 20 ^ ( 1 / .75 )
n = 54

Fifty four people - to do only twenty man-years worth of work.

You might say, "That's a pretty rough estimate!  You want me to bet my project on that?"  To that I'd say, "Studies have shown that rigorously planned estimates are only marginally more likely to be accurate than rough estimates.  (Schwaber, *Agile Software Development With Scrum*) So why waste the extra effort?  Besides, most teams don't even take this factor into account, so we're being extra-conservative."

You can also use Fristrom's Law if you know your team's velocity, feel it's not good enough, and want to figure out how much staff to add to get to the velocity you're looking for.  I'll leave that as an exercise for the student.

There are some corollaries here:

As you add more staff your scope-for-buck goes down dramatically.  Yes, you're making a marginally bigger game, but your cost went up linearly.  People will be less impressed.  You'd think, because of this, that publishers would want to make 100 games with 10 man teams instead of 10 games with 100 man teams, but the best game is the one that everyone buys...the second best not-so-much.  So that marginal additional scope may have been worth it, if it means you beat out your competitor.

Time is more valuable than money.  Scope increases almost linearly with time invested.  Two guys working for one year make a smaller game than one guy working for two.  Too bad that one guy's game looked a little outdated by the time it hit the shelves.

As the industry trends towards larger projects, demand for talent goes up, because of the reduced efficiency.  So although these big budget projects may be bad for the industry, they're good for the individual.  As our companies flounder and our projects get canceled, we'll all still be able to land well-paying jobs.

So, there it is, Fristrom's law.  Unless somebody beat me to it.  Remember when I thought I invented the term "Heisenbug"?

Lucky

Coming across this article on the joys of leaving game development make me think how lucky I've been in my career.  The short list that keeps Danc away hasn't applied to me:  I've been on projects that had decent project management, where the bulk of the project was spent working forty hour weeks;  my projects have often involved interesting new game mechanics;  once you count bonuses and options and whatnot my pay has been competitive with other industries, I think;  and I've had time to spend with my family, because of the aforementioned forty-hour weeks.
Not only that, but most people in the industry get shuffled from one project in crisis mode to the next:  I've never had to do that.  I've generally gotten to see projects through from beginning to completion.
My projects have usually been fairly agile (Ken Schwaber even said in one of his Scrum books, something like, if you've been on a succesful project, chances are it was agile whether you meant it to be or not.)
And...as for making the world a better place...games do make the world a better place.  I'm grateful that I live in a world that has chess and Magic and Guitar Hero and Spider-Man 2 and Grand Theft Auto and Kohan and Age of Empires and X-Com and Civilization and The Sims and on and on and on.  Maybe they don't save lives - although I do sometimes entertain the crackpot idea that the virtual worlds we escape to relieves some of the pressure of the overpopulation in our urban environments and just may be the thing we need to survive the oncoming crisis where the human population gets too big to sustain itself - but they make our lives better in a variety of ways.
Danc says that 95% of the industry is the sucky cesspit he describes, which means I've almost always been rolling 20s.
Here's one thought:  maybe I haven't had good luck;  maybe Danc had bad luck.  So he was on a crappy team and there are a lot of crappy teams out there, but there are also a lot of good teams - sometimes there can even be good teams and bad teams at the same company.  The good teams are less vocal then the bad teams but they're out there:  Infinity Ward, Neversoft, High Moon...
Here's yet another thought:  maybe it's not all luck.  There have been times where I was asked to put in overtime and I refused;  there have been times when there was peer pressure to put in overtime and I didn't participate;  I would never let other people schedule for me;  and I would bring good project management practices with me to the team.  And somehow I never got fired. 
Jason Della Rocca said that improving quality of life has to be an executive-level decision, but it can be an employee level decision, as well:  if you're getting ready to leave the industry because you hate the situation at your current employer, there's a couple of steps you can take first.  One, you can stop crunching and ask for some changes where you work.  Hey, the worst that'll happen is you'll get fired.  Probably what will happen is they'll realize they'd rather have your 100% than your 0%:  they won't spite you for not giving the 110%.  Two, you can look for another job in the industry where things are a little saner.
The workers control the means of production, yo.

Bigger Than Jesus

So, maybe they were once, but not anymore:

http://www.google.com/trends?q=beatles%2C+jesus&ctab=0&geo=all&date=all

But you know what IS bigger than Jesus?

http://www.google.com/trends?q=warcraft%2C+jesus&ctab=0&geo=all&date=all

Dressing vs. Salad Redux

From Greggman: 

If your going to abstract them that far then you can keep going until almost all games are about pushing a button and seeing a reaction. Not a very useful abstraction. Even if it's only imagined, stealth games put the player into the role of being stealthy and this means their perception and enjoyment of a stealth segment of a game is vastly different to they way they feel avoiding a stomper on a scroller. That difference is extremely important and your comment suggests it isn't.

Which is pretty much what I said when I first read Raph's book.  And Raph said this:

I don't intend to come across like the dressing is unimportant, or only relevant to mass market games. Chapter 10 is all about how experience design matters, for example. I even use almost your chess example, only with go instead. I also say "The dressing is tremendously important. It's very likely that chess would not have its long-term appeal if the pieces all represented different kinds of snot." ;)

In the book I make the distinction between the person in charge of the formal abstract parts of games (what I called the "ludemographer") and the person in charge of the game experience (call them the director, if you like).

The avenues of enjoyment beyond ludemography that the director taps into are well understood. They are story, they are art direction, they are music, etc. The thrill from getting the headshot, the tactile feel of real go beads on a wooden board. And even though they are well-understood, they aren't EASY--so I think it's perfectly valid for you to spend a lot of time thinking about those things.

I argue that the fun brought by pure ludemography is the core of gameplay and the part that is not as well understood as it should be, and that's what the bulk of the book is about.

Which is why this "Game Mechanic Map" might point out that a "stealth game" with guards and patrolling is just over the hill from a platformer with stompers in the same way that meringue is just over the hill from an omelet.  This "Game Mechanic Map" would be just for the salad AKA style AKA mechanics and we'll leave the dressing AKA substance AKA color for other books.

Actually, I believe color is more important than the mechanics.  The reason I'm addicted to Guitar Hero is because the color taps into a core fantasy;  yes, the formal rules are slightly different and possibly improved from Amplitude & Frequency but that's not what gets my glands pumping.  (And imagine if these games played random tones instead of music when you got the notes right.)

Still, like Raph, I think it's useful to strip away color when discussing mechanics if we're going to figure out why one mechanic "works" better than another.

Applying Best Practices To Game Development

So a lot of people talk about applying good software industry practices to game development.  I just had another idea:  how about we apply good food industry practices to game development?

It just so happens I've read three books on cooking and restauranting, all of which I recommend:  Anthony Bourdain's Kitchen Confidential, Alton Brown's I'm Just Here For The Food, and America's Test Kitchen's The Best Recipe.

From Kitchen Confidential, we learn the importance of speaking other languages so we can work with our immigrant help;  the importance of firing people if they're late to work and don't call in first;  and the importance of crunch time.  Sounds like being a cook is much less pleasant than being a game developer.  Good thing they're so well paid.

Okay, so I'm being snarky.  Seriously, there is some stuff in the book that I think does apply:  he talks about the difference between craftsman and artist.  This is something that Pete used to talk about at Treyarch;  although Die By The Sword was something of an "art project" for us, after that we were pure craftsmen...(although you could argue that the swinging system in Spidey 2 was "artsy" we still had our eyes on reviews and sales - if the People didn't like it, it would get cut)...and by craftsmen I mean we were trying to do a high bang-for-buck job that our clients would appreciate and that would be commercially succesful.  Chris Crawford is an artist.  A fine artist.  We were craftsmen.

Bourdain also has a chapter on firing, one that's fairly merciless on those who coast, cause trouble, or can't cut it.  His argument is that it's a dick thing to do--he argues that it's almost as bad as murder--but you've got to do it anyway:  if someone can't help you bail water out of the lifeboat you've got to throw them over the edge.  I agree with that one too.  (Let the flame wars begin?)

From The Best Recipe we mostly have just recipes.  What's interesting there is the approach:  they experiment with dozens of different variations on a given recipe, doing blind taste tests, and then publish the most succesful result.  What you end up with are recipes where, if you mess with them, you're fairly likely to make them worse.  That's been my experience, anyhow.  Another thing about this book is they have recipes for all of these common-man foods:  hamburgers, steak, chicken, turkey (this is the recipe to use for Thanksgiving, yo), meatloaf, pasta.  I started using these recipes and I became a much better cook overnight.  All from a book.  (I've said it before and I'll say it again:  books are good.)  So what we learn here is that playtesting is important.  We knew that already, but does anybody playtest as extensively as America's Test Kitchen tests its recipes?  Imagine every time you tweak or tune a parameter of your main character you have a group of volunteers rate the experience!

And, finally, I'm Just Here For The Food. 

You've never been to my house so you ask for directions.  I fax you a very precise list of instructions designed to get you where you're going.  Distances are calculated to the tenth of a mile and landmarks are described in Proustian detail.  You arrive without a hitch.

But do you know where you are?  If a tree had fallen in the road or a road suddenly closed, would you know what to do?  Unless you have a GPS in your pocket, I'm eating lunch alone.

If only I'd send you a map instead.

That's what's wrong with recipes.  Sure, they can get us where we're going, but that doesn't mean we know where we are when we get there.  And it would be a real shame to make it all the way to a souffle without realizing that scrambled eggs are just over the hill and meringue's just around the corner.

The rest of the book is his "map" of cooking.  He describes all the basic kinds of cooking, the principles behind why they work, how to do them and how they relate.

Could such a book be written for game development? - and I'm thinking specifically right now of game mechanic development.  In A Theory Of Fun Raph Koster shows us how certain game mechanics relate to each other, and on one page even has a map of a handful of games and their relations, but it isn't the sort of all-encompassing map of the terrain that I'm Just Here For The Food is.  Zimmerman's Rules of Play has the sort of scope I'm imagining, but goes far beyond rules & mechanics.  The game design pattern wiki could become this, but it's in its infancy.  Such a work would be encyclopediac, much larger than Brown's book, I think.  And would it be useful?  The way I imagine it in use is you're working on, say, a stealth game, and your kleenex testers are telling you they're not enjoying it, so you turn to the "stealth game" section of this game mechanic map, and there would be a list of various rules that different stealth games have, and why they work together or don't work, and you might say, "Aha!  My patrol paths are too long;  the player spends too much time doing nothing;  it's right here in black and white."  [Side note:  stealth mechanics are usually just tweaked timing mechanics;  there's little difference between sneaking behind a guard in Beyond Good and Evil and avoiding a stomper on a timer in a side-scroller.]  Just like Alton Brown's book tells you what temperatures work for different kinds of cooking. You'd never come up with a Tetris from using such a reference - it would purely be a tool for commercial game makers who are doing evolutionary experiments in established genres.

You can apply any field of learning to any other field of learning with a little creativity, I guess.

Seattle

So I live in Seattle now, and although it may have a smaller population of developers than Los Angeles, it's still gotta be in at least the top 3 game developer cities in the States, and it's definitely a gamer's town, no doubt due to the influence of Wizards of the Coast.  I haven't seen CCG's in LA in ages, but here they've got CCG's in Target and I spotted a couple of guys playing Magic in a Starbucks the other day.  And the Starbucks turns out to be down the street from a Games Workshop retail outlet.

I also wonder if LA or Seattle has a higher number of world-class developers.  LA has Naughty Dog, Insomniac, Infinity Ward...Seattle has Valve, Bungie, Sucker Punch...

So there's been much more activity on my auxiliary blog since I've been talking about the move and stuff.

Just Something I Noticed

I know it's a little late for a GDC report with E3 bearing down upon us...but for the second year in a row, Will Wright was my favorite speaker.  This shouldn't surprise anyone - it seems like he's pretty much always considered the best speaker at the show.  Interestingly, he breaks a couple of conventional wisdom rules about public speaking:  he says "um" and "uh" a lot - very substance over style.  Also, we're told you shouldn't have a zillion slides for an hour long talk, but he makes it work.

Also, when I see him walking around the show, he never has an entourage of rabid fans.  Why is that?