GameDevBlog

Look At My Beautiful Code

Mat Noguchi turned me on to a talk by James Coplien.  The purple book ("Advanced Styles and Idioms in C++") used to be my bible, so I checked it out - one of the things that struck me besides his "How do you know computer science isn't a science?  Because it has 'science' in the name." joke was him talking about beauty and asking "How many of you have opened a source file and said 'that is some beautiful code'?"

Damn that's rare.  Usually I start working on a legacy project and I react with horror.  I've seen some API's I consider beautiful - the dreamcast rendering API and XNA Game Studio (if you ignore the Xact stuff) - but code?  One thing that comes to mind is when we went from Tony Hawk 1 to Tony Hawk 2, Neversoft overhauled their 2D rendering and that was beautiful, though I'm not sure I would have appreciated it if I hadn't seen how it was before.

Producers hate to hear from their coders about beautiful code.  "Is that why it took so long?" they'll ask.  And who can blame them, because the next coder that comes along and looks at that 'beautiful' code will probably think it's hideous.

But maybe, like Christopher Alexander says, we can find some area of agreement.  Maybe most coders find most of the same things beautiful.  

My Question To You:  what is some code you consider beautiful?  It has to be code you didn't write yourself.  And preferably open source, so we can all take a look at it.  

Here's the talk, btw:

http://archive.oredev.org/toppmeny/download/downloadseminarsday2/jamesocoplien.4.5bd7fa0510edb4a8ce4800024169.html

I Am Happy

I feel like the chef in T. C. Boyle's "Sorry Fugu" when he gets Willa Frank to say something nice about him.

Of course, I have to agree that everything he says in Spidey 2 is crap is crap.  Even the missions, which I made some of.  What can I say?  Movie license, strict release date, excuses, excuses.  At least we got something right:

http://www.escapistmagazine.com/videos/view/zero-punctuation/584-Spiderman-Web-of-Shadows

Australian Notation - Globals And Multicore

So Bryan says you can't have globals if you have a multicore architecture.  

Schizoid was multicore (to my chagrin, because it doesn't look like the sort of game that should have needed more than one core to run fast, but that's what the XNA game studio guys recommend), so let's talk about how we did that and had globals and survived.

One thing we did was close to what Bryan describes - before and after game states.  But how we did that was quite ghetto - since our multicore was a retrofit, we had oldPos and newPos, oldRot and newRot, in the entity states.  Sometimes, there'd be cosmetic variables that could get bashed by the wrong thread and would come out different on different executions, but I didn't particularly care.

But what about singletons?  And here I'm talking about logical singletons.  Variables or classes that there are only one instance of, that can be read to or written from, that don't make sense to duplicate.  The renderer...the audio engine...things where I felt performance didn't matter...the UI manager...player macro-game states (what levels were unlocked or not).  You can wrap these things up however you want - put layer of layer of indirection on them, and you still have the problem that if more than one thread tries to access them at the same time your results are undefined.  Whether they're globals syntactically is irrelevant.  

And not just singletons, there's a kind of variable that's sort of 'logically' global - which means that it's fairly easy and probably intended for any module to be able to access it, whether it's syntactically global or not.  Maybe the module has to access it through an Inst() method or call gApp->GetRenderer()->GetDisplay()->GetWhatever(), but it's still fairly easy and if you're doing it from multiple threads you'll get in trouble.  So any member variable that has, say, both Get... and Set... accessors could be 'global' in this sense...

And we don't have the tools to automatically protect ourselves at compile time, or even really at runtime, from the sorts of heisenbugs that can crop up when we use these things.

So what helped me was notation.  Spolsky once wrote an article on what Hungarian notation was *really* for - it's down at the bottom of http://www.joelonsoftware.com/articles/Wrong.html - and what it was for was for helping you tell if the program is doing what you think it's doing.  I thought, instead of using notation to remember if something is a bool or not, we can use notation to remember if something is logically global or logically a singleton or what.  And I broke it down like this:

There are constants.  These things are read-only.  They might not be syntactically const - sometimes I like to store them as mutable variables so I can change them in the debugger at runtime, for example.  But when we ship, they'll be read-only.  These get the letter 'c', and they're safe to read from on any thread.

There are full-on globals.  You can read or write to them from anywhere.  They may not be syntactically global, they might be a member of a singleton.  These get the letter 'gl' (global lock) and you have to remember to use locks if you're messing with them.

There are thread-safe globals.  These are classes that have thread-safety built into them - they lock and unlock their members appropriately when you use their functions, so you can't hurt yourself using them from multiple threads, though they're not necessarily fast.  These get 'gts'.  (global thread safe.) 

There are partially thread-safe globals.  These are classes where some of their member functions are thread safe and others aren't.  These get 'gpts' and you have to read the documentation when you're working with one of these to make sure you don't screw up anything.

Globals that were only supposed to be accessed by the main thread - gmt.

Globals that should only be written to by the main thread, but can be read from any thread - gwmt.  (global write main thread)

Member variables that were fairly protected or private, that didn't have Set accessors or could be otherwise violated (a Get that returns a reference, for example) would get the familiar 'm' designation.

You might say, 'Sounds like almost nothing in my program is m' but it's about how the variables are *supposed* to be used rather than how they *could be* used.  If the compiler could protect us from using them wrong, we wouldn't need the notation.

And I called it "Australian Notation" because I'm half-Australian.

Random additional note:  part way through the project I realized that using prefixes didn't really work too well with Visual's autocomplete feature.  Although I'd remember the name of a member, I usually wouldn't remember if it was global or a member or what kind of global it was - that was why I invented the notation in the first place, so I wouldn't have to remember.  After Schizoid, I started making the notations suffixes instead of prefixes, so 

  

  IRenderer* gRenderer;

would become

  RendererI* rendererG;

And then I discovered that the latest Visual Assist does something fuzzy where prefixes can be ignored, so the point is moot.  Do whatever you want.  As long as you're consistent.

Globals underrated continued

Sorry, in the last post, instead of saying "you don't want any global accessed from anywhere" what I meant was "you don't want every global accessed from everywhere."  You do want some globals accessed from somewhere.  It's going to be really hard to convince me that Mat Noguchi's proliferation of pointers is a good thing, unless they're smart or shared.

How would I enforce layering without a proliferation of pointers?  One could use code review, etc, to get all the coders on the same page about the ideal you're trying to live up to, but if you want to do something in code, and the layering has the same level of coarseness I talked about in the last post, I'd use libraries.  Game.lib (or exe) would depend on both Renderer.lib and GameMechanics.lib, but Renderer and GameMechanics would not depend on each other.

You might wonder why I'm keen on separating game mechanics from the renderer.  Reason A is because I'd like to make games like Civilization Revolution that run the same game mechanics on wildly disparate platforms (DS & Xbox).  Reason B is it appeals to the game designer in me - I like to think of the game mechanics as being a sort of pure form separate from interface.  (Though in reality no game can be separated from its interface, it is a fun and useful thought exercise.)

Globals underrated - fight!

Ok, I need to refine.  I don't like Gang of Four singletons either; I tried them way back when on Draconus and never wanted to use them again because of order-of-initialization surprises.  But I do like the syntactic sweetness of static Inst() accessing a static member, as long as we control initialization order ourselves, as Noel Llopis suggests.  That's why I said "variation of a popular singleton pattern" - key word being variation, don't use it as written.

Yeah, it's true, proper forward declaration doesn't make the circular dependencies *too* bad.  Though, windows.h isn't too bad either, IMO, as long as you use precompiled headers.

And yeah, using a language like C# or Java that gets rid of headers altogether is heavenly.

Globals can make it harder to test, but I've usually found you can mock out whatever globals are required pretty easily.  

It sounds like in Noel's perfect world, we have to copy our game state every frame - our update function would take the old game state as its input and spit out the new game state.  That would be my ideal too, come to think of it, though I've never lived up to it.

Speaking of ideals, even if you have globals, you don't want every global to be accessed from everywhere.  You do want proper layering.  My ideal world, here, which I've only lived up to on homebrew projects, and never managed to in shipping code, is something like Patrick Buckland describes for Magic.  The game state carries no cosmetic information;  you have a sort of document / view paradigm with your game state.  Your low-level renderer never talks to your game state or vice-versa.  You have a high-level graphics engine that looks at your game state and describes it to the renderer.  In my production engines, I've always let this ideal slide and stored information about meshes, textures, and purely cosmetic animation and effect states in the game entities themselves.  

But all I'm really trying to say with the last post is that code that does this:

  gGraphics->DrawMesh(myMesh);

or this:

  Graphics::Inst()->DrawMesh(myMesh);

is no worse than

  gApp->GetGraphics()->DrawMesh(myMesh);

I suspect Noel isn't too fond of that last one either (gApp still being a global) and wonder what his alternative is.

For Gregg's thing, the ServiceLocator, I don't immediately see the advantage to adding the complexity.  Not for testing, anyway.  Assuming I have the source to what gGraphics points to, even if that's an untestable class that does lots of file loading and takes over the hardware, it's not too much work to change it into a GraphicsI* for which I then write a mock (I've done this on two different legacy projects, took about half a day each time) and then my tests don't look much different from Gregg's:

gGraphics = new GraphicsMock();

classThatUsesgGraphics->DoTest();

And if I'm using some kind of static Inst() function in my renderer, I move that into GraphicsI.

Globals are underrated

One of the first things they taught me in my college Pascal class (I already knew some Pascal, but had to take it for the credit...on the bright side, that's where I learnt about pointers and recursion, so it was worth it...) was that global variables are bad.  And in most cases this is true, but I think it's easy to take this philosophy too far.  

Suppose you're designing a game architecture.  And so you have classes for the app, the sound engine, the graphics engine, the input device, the game world.  These are all singletons.  Nothing's stopping you from making them globals or using some sort of singleton pattern (where they're still effectively global;  any class can access them from anywhere).  But say you say to yourself, "I know globals are bad.  I'm going to have all of these singletons be members of App, and App will initialize and deinitialize them in the proper order."

Now you're stuck with a couple of possibilities:

Everything that wants to refer to one of these classes has to talk to App.  They're still effectively global if you do this - anyone can access them from anywhere.  But you've also incurred the overhead of your App .h file including all your other .h files, and you'll probably start to find that a lot of your .h and .cpp files are including App.h, giving you a nasty nest of circular dependencies, where whenever you change an .h file you're rebuilding half the program.

Or, you can do what Grady Booch suggests:  when you instantiate a class that needs to refer to one of your singletons, you pass the instance a pointer to your singleton.  Yay, no globals and you can use forward declaration to prevent .h files including .h files.  What you do have, though, is Duplication of State.  Supposing you later decide you want to be able to destroy and re-instantiate your singleton, perhaps to reset it or for some other purpose.  Imagine all the bugs that could introduce.  Besides, it's just plain ugly bulking all your classes up with all those pointers.  I've never seen an engine carry this through to the full extreme - I've only seen engines where it seemed like some coders preferred philosophy A while others tried philosophy B.

If you just made those singletons global, or used some variation of a popular singleton pattern, then you can make sure that only the .cpp files that use your singleton need to include its .h files, and your code doesn't get as gunked up.

Clarification: Quatloos vs. Man-Days

Couple questions answered about the last post:

Quatloo is from Star Trek: TOS, "The Masters of Triskelion" later parodied on the Simpsons when Burns bets quatloos on the newcomer.  Not intended to be pejorative:  I still measure projects in quatloos myself (well, "Torpex Work Units", I call them), multiplying by various fudge factors to try and get a date and budget.

When I say scrum doesn't measure velocity in quatloos, that's from Cohn's *Agile Estimating And Planning* - they measure the whole project, or the release burndown, in story points, but each sprint they re-estimate the tasks in man-days, and that's what they track velocity in.  Double checking:  yes, it's in Schwaber's book too:  the project backlog burndown chart uses coarse-grained estimates multiplied by a fudge-factor, then each sprint re-estimates. 

Looking it up, I just noticed Schwaber does suggest using the project burndown chart to estimate when the project will complete (p. 11 of Agile Project Management With Scrum), so it is the quatloo-velocity that really matters to him too, it seems.  But that's the velocity that I find flattens out over time.

And...I don't know what to say about Gregg's experience.  I'm jealous, and maybe should go work with him.

'Theory of Drag' detractors

I've been thinking about the kind of people who disagree with the 'theory of drag' - the idea that as a project progresses your velocity slows down, the 'pushing a snowball up a hill' metaphor - and wondering what their beef is when it's, well, so obviously true. 

I've rarely met these people in person, though I do have anonymous feedback from talks I've given - but I have a couple of theories about who these people might be.  

One is the 'go team!' producer.  I heard a producer say the other day that the primary number-one job of a producer is to make the team feel like they can ship.  Keep the morale up.  "We can do this!  Go team!"

The 'theory of drag'...which, summed up, says 'Your project is more screwed than you think it is' isn't what that kind of producer wants to hear, or spread to the rest of the team.  

I don't know if that sort of producer is as wrong as they sound.  An optimistic, happy team will be more productive than a "we're doomed!" team.  Like JFK saying we would put a man on the moon within the decade:  you can galvanize a team that way.  

But the ideal is to embrace "the Stockdale paradox" from Good to Great:  always have faith that you're going to make it while simultaneously being able to confront the brutal facts of your current situation.  And someone embracing the stockdale paradox would want to always keep drag in mind.

Another kind of detractor is the scrum guy.  I was talking to a scrum expert, and I told them that in my experience velocity always decreases as a project goes along, and they said, "Really?  In my experience, velocity always increases, as the team gets more used to their tools and technology."

Frankly, I think it isn't becuase they're mastering their tech--seriously, in my experience the gains from mastering tech are always buried by feature creep and the drag from project bloat--but because scrum has some dishonesty built into the scrum process.  When doing scrum, you have the project backlog for the entire release, which is estimated in arbitrary time units...let's call them quatloos.  And each sprint, when a team commits to the work they'll do for the sprint, they more rigorously estimate the tasks they're about to undertake, in man-hours or man-days.  The thing is, scrum guys measure their velocity in terms of these man-hours or days that they re-estimate every month, so what I think is happening is this.

Sprint 1:  team takes on a chunk of work.  It takes a little longer than expected.  Their velocity comes out at, say, .9.

Sprint 2:  team consciously takes on a smaller chunk of work.  Because of drag, it takes a little longer than expected, but because it's a smaller chunk, their velocity appears to go up to say .95.

Sprint 3:  team consicously takes on a still smaller chunk of work.  Etcetera.

I think you'd find if you looked at the velocity in quatloos, you'd find it was continually decreasing, as they both got fewer quatloos done due to drag and added more quatloos to the project due to feature creep.  They're committing to fewer quatloos every sprint.

Measuring a project in quatloos obviously sucks, because it's so damn hard to predict how long a feature is going to take a year down the road, and that's part of the reason drag increases near the end of a project.  But I think if you want to be honest about your velocity it's the only option.  Also, if you're a contractor who needs to promise a certain feature set by a certain date, it's all that really matters.  You can't go to your client or publisher and say, "Well, we've gotten to the last sprint and discovered that we can't commit to all the backlog we originally promised.  Sorry, but we're doing scrum, so that's how it goes."

Instead, if you measure every project in quatloos, although those first couple of projects may shock you with the massive amounts of drag you experience, after a while you'll start to expect it and be able to bid accurately on contracts - and cheesy rules of thumb like "try to be 90% done 50% of the way through the project"--in other words, get 90% of those quatloos done at the halfway point--may save your ass.

And if you haven't seen my posts on the theory, here they are plus.

Folder Hierarchies

One of the guys on our project wanted to change the way sound folders are organized, and I realized I have fairly strong feelings about the subject.  Most of this is just curmudgeonly ranting, it is not Truth, but still:

My experience is keeping folder hierarchies flattish cuts down on minor daily annoyances, both with source control and navigating: say you’re editing the raw content data in one folder, then you go to save, so you have to click up, up, then down, down to get to the proper destination directory;  whereas scrolling through a long list of files usually just requires a keypress or three.

To make this work, when using Explorer, everyone will want to do View->Details, otherwise sure, the screenfuls of icons would be overwhelming.

Keeping the raw content separate from the cooked (content you're actually going to ship in the distribution) is important, to simplify not just the authoring/final build process but the process of giving publishers, associates, marketers builds.

One idea that occurs to people frequently is to use the directory structure to describe the files.  So instead of zombieAttack.qrx and zombieIdle.qrx you'd have a \zombie folder that contains Attack.qrx and Idle.qrx.  This is appealing at first but it's nice to know if you've accidentally dragged the zombie attack sound into the yeti folder.  And if the code can assume that every sound name is unique that can gives you an advantage - you can use the asset name as a unique ID.  (I don't believe there's any reason any more to have an asset manager that builds .h files with asset codes baked into them - maybe I'll post on how we get around that someday.)

It's not *that* hard to rearrange your hierarchy later (though source control can make it harder) so don't fix a theoretical problem now ("This might get messy");  wait for it to become an actual problem.  

When would I start subdividing a folder?  When it hits around 500 files, or when all of the files of a subcategory no longer fit in a couple of pagedowns.  That is, once you've got enough zombie_xxx files that it isn't just a keypress to find the one you want.

One last thing:  I don't use spaces in file and folder names, I use underscores or intercaps.  Using the command line to do things with files has become vanishingly rare, but typing in those quotes to find a file with a space in the name is still annoying on those rare cases where I need to do it.

Curmudgeonly ranting over.