« Notes on Galleon | Main | Two Degrees Of Separation »

September 06, 2004

Comments

Noel Llopis

This idea has been brought up repeatedly in other forums (The Chaos Engine for example). It is really appealing, but I don't think it would work:
- Truly anonymous postmortems are worthless (disgruntled employee, hearsay, someone with an axe to grind, etc).
- It would be difficult to write a meningful, truly anonymous postmortem. I mean, teams aren't that large, we'll know what game it's referring to, everybody knows everybody in the industry, put two and two together and you have it. People could get fired for that.

Don't get me wrong. I'd love to see more meaningful postmortems. I just don't think it's going to happen any time soon.

Michael P.

The best post-mortems I've read are researched and written independently by journalists such as Geoff Kneighly (www.gamespot.com/features/btg/). Hopefully once more journalists become interested in the process of making games, budgetary information and internal complaints will flow more freely. However, I don't think that post-mortems written by the game's creators are inevitably dishonest, either. With the appropriate amount of signal bullshit, a clever writer can communicate to industry insiders just how fucked up their project was and still appease the censors.

Now that I think about it, why not change your approach to "Fucked Project" and instead develop a code language that disguises specific information. For example, the phrase "modest budget" means "$4 million," "decent budget" means "$10 million," etc.. "Visionary art director" could mean "dictatorial smartass who made our environments too small," and so on. And an advantage is that these post-mortems would be unknowingly approved by the company.

This might be more useful than anonymous ramblings, for if the author's identity is ever discovered (even the most thickly disguised description of a fucked project could be recognizable to insiders, as Noel points out), he or she is screwed.

JP

I find it really sad that developers *aren't* allowed to say what they really mean to say with postmortems, which are ostensibly about letting them talk about what went right, what went wrong, etc so you can ACKNOWLEDGE AND LEARN FROM MISTAKES. I can't remember the last one I read that wasn't just a bunch of self-backslapping, worthless platitudes and very guarded statements.

There is not nearly enough self-analysis going on in the industry right now.

Bo Gus

Funny, the SM2 postmorteum has had lots of BS in it from the first revision. And it has been untouched by executive/legal hands!

Robert 'Groby' Blum

It's just fine for developers to learn from their own mistakes. That's not the point of a published post-mortem, though. It's - like any other published information - marketing material. If you take that into account, you can still learn.

Unfortunately, the main lesson from most post-mortems is that the industry as a whole refuses to learn.....

On average, we don't have tight control over schedule/scope, we're prone to add features last second, and we're doing quite a bad job communicating w/ people outside the team. Or even our own domain (Need I say artist/programmer dialogue?). Oh, wait - we also think working insane hours is heroic.

Did I miss any of the major themes?

So what's the point in additional post-mortems if we don't react to the existing ones?

m to the vizzah

I think you missed the most important one: it's really fucking hard to know how long certain things will take until they're done. From designing /bug-free/ systems in code to making levels 'fun', we have really hard time scheduling this stuff. About the only thing we know seim-accurately is how long it takes to create meshes and textures, but they're the bottom of the process pipeline, so they often have to do more work because of changes in the tools ("we're moving to a parametric building representation, we need to rebuild the city in a month") or design ("I didn't realize I'd need these two new characters for this mission").

Here at Treyarch we have good control over our scope and schedule. We don't add significant features (in terms of work) late--instead we're still working on the ones we planned months (years) ago. We sure as hell don't think working insane hours is heroic. Half our staff on SM3 is married with kids, and our one modest victory on SM2 is that we didn't work any Sundays until the last two weeks.

I'm interested in processes that worked, and processes that failed. The latter are usually easier to cite, but I'll take whatever I can get. GDMag's post-mortems are sanitized and lifeless, I want something with more candor and even vitriol if that's what it takes.

m.

Robert 'Groby' Blum

I'm glad to hear you have a process that is working for your company. I happen to be lucky enough to be in a company that has (mostly) managed to achieve this, too. I think we can agree it's not exactly industry-standard, though. How is this relevant to the discussion? That means that a lot of post-mortems are rather useless, because not even the basic obstacles have been tackled.

Yes, the GDMag ones are sanitized - but that doesn't change the fact that most of them could be about the same project. I think that (at least in the short run) shining a spotlight on the blatant shortcomings of the whole industry is helping us more than a postmortem or two.

But maybe what you're seeing as "sanitized" is the same thing that I see as "they're all the same", and people are just repeating "accepted" mistakes instead of really taking a hard look.

I think the only way to get a candid look at projects is to talk to people directly involved. Official material is always marketing (and cleared through legal&marketing), and anonymous data is not trustworthy. (Just look at fatbabies :)

Maybe we should just meet in person and compare notes... If you're in L.A. and think that's a good idea, let me know.

russell

I don't feel the 'what went wrong/what went right' approach is prescriptive enough. It allows the writer to pick an angle to discuss an area from. It becomes highly subjective.

You could put together an updated post-mortem 'format' that embodies the key aspects of project planning and execution. We know traditional PM methodologies work (don't we?), and surely there can't be many people around still relying on that old 'but games are different' line. Sheesh...

The 'accepted mistakes' that happen over and over again - e.g. the old classic 'sporadic uncontrolled change to an ill defined scope' - could be reported this way. Just ask the questions:

1. List all and describe sample cases of features having their scope increased / reduced - what happened, why, the resultant effect on the product etc

2. Describe the initial work carried out to define the range and scope of features, how you prioritised areas to work on, and how you controlled change

Any attempt to answer these sample questions will highlight the scale of attention paid to scope definition as part of a production plan, and the resultant effect it had.

Treyarch sound like its at the top of their agenda - smart guys...


btw, I'm 'not from around these parts', so forgive me for gatecraching your blogfest :]

JasonR

Do other industries have post-mortems? It seems like the way it is is a fact of life and it can't really change. I think GDMag should just get rid of them. You really will only know the dirty talk about companies by being friends with the employees.

scottj

I agree Noel's skepticism about the project. But perhaps some of the bigger publishers are mature enough to produce candid postmortems that are shared within the publisher's development studios? When I hear about a project within my company developed half way around the world getting delayed, cancelled, or 100% ratings on the major game review sites, I'd love to know what happened to get those results. And I would think company management would have a vested interest in helping me to know why the other team succeeded or failed.
And then those postmortems could be leaked to the Fucked Project site. ;) Hmmm. Perhaps I just figured out why I've never seen anything like that...

Nathan McKenzie

At least from my own experiences, most of the fucked up projects I've seen have an awful lot to do with specific people on the project, usually in key positions...

It seems like it would be really hard to make honest post-mortems in those kinds of situations, because it rapidly devolves into a rotating blame game.

tuvalu

Well there is already thechaosengine and fatbabies for that.

Obi Busta Nobi

I just read the SM2 postmortem. I know, I know... where have I been? I've been in a videogame coma you could say. Well, now I'm awake and I have plenty of questions. Like.. What was left out? Did the Actvision higher ups have anything to say about what could and couldn't get published? I can understand their need to protect their property, but how did that affect the article? I found myself wanting to read more. Like a good story I suppose I wanted to see how it ended (great, if you judge by sales). The swing system seemed to be the highlight of the article. Justifiably so. But, what about scheduling, planning, and keeping folks happy as they gruel through the dreaded "crunch time." "The process" I'm sure is what everyone is salavating for in order to avoid pitfalls and assure a great end product that ships on time. I suppose we can get some of that in Jamie's Gamasutra articles in respect to Spidey, but it would be nice to see more postmortems get into some real details without what Nathan describes above as the "rotating blame game." I've seen that before and it ain't pretty. Our industry has grown up so much from when I first started. Yet, it seems we still are looking for that Holy Grail of the perfect plan or system that will ensure success. I often wonder if companies like EA, who is way ahead of many other developers and publishers, have a secret doctrine that only key Producers know about and can measure up to that standard. Or, are they "winging it" and have been extremely lucky all this time? Anyway, I suppose the clues are somewhere in these postmortems and articles. At the very least, they're food for thought...

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been posted. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Your Information

(Name is required. Email address will not be displayed with the comment.)

Jamie's Bragging Rights

  • Spider-Man 2
    The best superhero games of all time Game Informer
    Top five games of all time Yahtzee Croshaw
    Top five superhero games of all time MSNBC
    Top 100 PS2 games of all time Official Playstation 2 Magazine
    1001 Games You Must Play Before You Die Nomination for Excellence in Gameplay Engineering Academy of Interactive Arts & Sciences
  • Schizoid
    Penny Arcade PAX 10 Award
    Nominated for XBLA Best Original Game
    Nominated for XBLA Best Co-Op Game