« Why Are Games Important? One Of Many Reasons | Main | TDD addendum; end-to-end builds; soak tests »

August 23, 2007

Comments

joshlee

This was pretty much my exact experience with TDD. Mediocre as a QC tool, but invaluable as a coder's tool. I found that writing tests first meant that I was writing a client for a class before writing the class itself, which forced me to design interfaces that made sense from the beginning. As a result, the interfaces were so clean that I didn't even really feel bad about exposing extra bits.

The real win was when I had to start working on code that someone else had written. Turns out that running and inspecting unit tests is a much quicker way to learn a class's interface than by digging through twisted source code or out-of-date docs.

YourMacGames

Great post and that's very nice to share your experience on this hot topic!
But "false positives", "worst code quality", "90% of bugs not caught"... I will really need to have more positive points than "better code understanding" to spend time on TDD! ;)

Parveen

The important question is:

If you didn't employ TDD, would Schizoid be better, worse, or the same?

greggman

Or would it have been done faster without TDD?

Sean Barrett

What about its role as a safety-net, where you can be more willing to refactor the code because you have a safety net "guaranteeing" that you'll catch refactoring bugs? Even if it doesn't catch any bugs (because you refactor too well), it can still have an effect this way.

Jeff

I have to agree with Sean. The TDD cycle (test, code, test, refactor, test) gives you the safety net late in development when you have to refactor or optimize libraries. A failed test immediately sends up a red flag that the refactored function does not work the same way as the new funtion. Even if your assumptions have changed, you'll know to look carefully for any places in the code that may have been depending on that assumption.

In my mind, TDD is all about making sure some of the more simple bugs never get introduced into the code base because of either bad assumptions between programmers or because of hurried bug fixes late in development. Simple bugs waste tester time and they waste coder time. It's better just to make sure they never hit the code base (if that's possible).

Lastly, TDD for me (on a large project anyway) is just one piece of the puzzle. It serves as the initial smoke screen for the stability of your build, but big projects should also think about add integration testing, monkey testing, and the possibility of an automated in engine test (having the game play through several levels automatically). That way, you can make a pretty reasonable guess as to the stability of your build before it even goes to testers, wasting less of their time, and less of the programmer's time.

About code coverage: have you thought about using code coverage tools to let you know what code branches may not be tested? I know I always forget to test failure states / assertion states. A code coverage tool can usually tell you how much of your code isn't being tested.

Jare

The benefits you describe seem to be 100% in line with what you should get using the good old assert() in a smart way. asserts are a very good way to document your assumptions, expectations and understanding of what the code should be doing.

asserts have degenerated into a poor (and wrong) technique to validate runtime conditions. To me that is a real shame.

Jamie Fristrom

Well...a unit test fails immediately when you violate it, but an assert only fails once that code is later executed, which could be days or weeks later, if you're running a debug build, and you have to rely on whoever happened to see the assert to then report it. Or some sort of automated assert logging system. Which is cool, too. Love those asserts! But not a substitute for unit testing or vice-versa.

Soren Johnson

Jamie, how strictly do you adhere to "official" TDD practices? Every time I read up on it, usually from a non-game developer, it just seems so heavy-weight and somewhat backwards that I have a hard time seeing how I could actually use it for game development. I'd love to see some actual examples of it in use.

martin

Having shipped games that passed Sony PSX QA with zero bugs... no A, B, C bugs... I can honestly say... TDD is a crock of horse manure. There are far more powerful methods to prevent bugs which are automated. Hand writtening tests, that need to be re-written each time you refactor is anything but automated. Learn to use assert, as the comment by Jare is dead on. Oh, and little secret here... sice you already decouple your render logic from game update logic... run the game at like 1000 fps, with render turned off. Let it play itself overnight, each time you go home. It will find bugs and leaks you never even dreamed of. I'm willing to try TDD... but first, and I've been searching I need to find a project that finished using it correctly, and showed some benefit. And no 20 guys 2 years to make a basic console game is not "successful" unless it is gears of war.

The comments to this entry are closed.

The Games

  • Energy Hook
    3D grappling-and-swinging-and-running-on-walls-and-doing-tricks ... with a jetpack ... for style!

Jamie's Bragging Rights

  • Spider-Man 2
    The best superhero games of all time Game Informer
    Top five games of all time Yahtzee Croshaw
    Top five superhero games of all time MSNBC
    Top 100 PS2 games of all time Official Playstation 2 Magazine
    1001 Games You Must Play Before You Die Nomination for Excellence in Gameplay Engineering Academy of Interactive Arts & Sciences
  • Schizoid
    Penny Arcade PAX 10 Award
    Nominated for XBLA Best Original Game
    Nominated for XBLA Best Co-Op Game