More GDC takeaways

I don't have my notebook with me, so this is from memory, so...on the bright side, you're only getting some of the last most memorable points.

Automated Robot Testing - Paul du Bois.

I've done 'random monkey' testing of various stripes ever since Die By The Sword.  Some places where Paul's system beats any of the ones I've done:

- uses scripting language to set up and execute tests - a test lead can write your automated tests

- can automatically run on any idle dev kit in the office - a single test server machine dispatches tests.  Now that's cool.

Don Daglow always gives inspirational talks that sometimes get me a little misty eyed.  One thing that got my attention:  he's very psyched about distributed virtual teams, which was good to hear, since Schizoid was a virtual team and it's always an awkward conversation with publishers when they find out Torpex is mostly distributed.

I almost missed the Bioware talk on the level process for ME2, the last talk of the show, but I couldn't get in touch with my ride so I decided to go.  Glad I went!  Bioware has done it, they've broken down the silos, gotten the artists and designers together in one room to build levels.  So instead of this constant back-and-forth (designer lays out blueprint, artist models, designer gives feedback, artist remodels, designer scripts, artist makes pretty) where they're throwing levels back-and-forth over the wall like bricks, you get real teamwork.  This is something I've been trying to make happen ever since Spider-Man 1 and always failed.  With Spider-Man 1 we put artists and designers in the same room, but they weren't talking.  Rather than work out why not, I lost the battle and we went back to scripters in one room and artists in another.  The reason it didn't work is obvious in hindsight - we put them in the same room but they weren't working on the same levels!  Duh.  Spider-Man 2 we tried virtual teams - put them on the same e-mail list, had regular meetings, and although this helped, it still wasn't great:  still, one person would typically be working on the level at a time, and resources such as sound and QA that wouldn't be tapped for a while were coming to the meetings but unable to participate.  Things got even worse during my short stint at the beginning of Spidey 3, when all the leads started wanting to come to all the level meetings, so you'd have these comical meetings where the two people who mattered (an artist and a designer) would be discussing their level while one or two art directors, one or two creative directors, a lead designer, and a producer watched.  (On ME2, they do regular level review meetings where the leads check out the level and provide feedback, but they're not always in your dish.) My ultimate solution was to quit and start making a small game to feel like I was working on a Team again.  I still didn't fully "get" Bioware's system - they said they'd put three artists and three designers in the same room - with our tech, it was really inconvenient for more than one artist and one designer to work on a level at the same time - I thought Unreal was just as bad, but maybe I'm mistaken.  Or maybe the mixed teams are working on 2-3 levels at once.

Something that I've noticed a few places lately is timeboxing level development.  High Moon and Bioware Austin do kanban;  the ME2 team doesn't do kanban but they do timebox.  They take their level through various stages of completion with a regular heartbeat, and I forget the actual stages which is a good thing because you'll want to tailor your own stages for your own shop, but it's something like

- concept (writing, concept art, 2d map)
- block world, with dialog and encounters put in as big text boxes (you can check pacing, feel of level already)
- untextured mesh world, combats at "alpha" - no cover yet, enemies don't even shoot back, animatics
- dialog is in, combat is at "beta" (cover & AI active), music in (he said you needed music at this point or game felt anemic - I do the same thing when I prototype, you have to have music in your prototype even if it's just a programmer-art white-box prototype)
- could ship it if you had to - voiceover is in (they bring in their main actors [to a Hollywood studio] many times during production - not something you can afford to do when you hire, say, Tobey Maguire, but it works for them) - texturing, lighting, animations
- "Bring The Awesome" - an extra timebox to give a level that special something.  But they also said they haven't gotten a level to this stage yet.

The length of the timeboxes they arrived at empirically, by putting some levels through the process and using that as a benchmark.

They said it was worthwhile to do deep-dives on some levels while also building all the levels to playable, because upon playing the whole game even in blockworld it revealed the interest curve of the whole game needed a bigger bang at the ending. 

Being a programmer, I always used to rankle at the idea of timeboxes.  "How do I know when it's going to be done to the level of quality we want?"  Over the years, I've come to accept that artists are totally okay with timeboxes.  With designers, it's okay if they're "in the box", placing encounters and drops and doing stuff by-the-book, but for a special-case gimmick level a timebox is probably trouble.  Of course, special case gimmick levels are bad ideas, period, (and Kaplan said the same thing in his talk) - create systems and let the designers work those systems (which is what we did with Schizoid) rather than have a bunch of special case scripts where each level is kind of its own game (Spider-Man 1 and 2, where the level quality was variable to say the least.)  So I'm ready to give timeboxing level development a try.

Comments

[Lee Winder]

I was really excited about viewing the 'Automated Robot Testing' presentation as it's something I've toyed with in the past, but it doesn't seem to be available on the my GDC Vault site yet, which is a real shame.

Having a scripted monkey can save a vast amount of time, and I really do see a time when QA create these simple scripts themselves, especially to catch hard to reproduce bugs or those with long repro steps.

It's also the first step to real visual unit testing, with QA and engineering fully testing their work again and again (probably as a large scale nightly build step).

I especially like the idle dev kit step.