« So Why Do I Like The Name 'Torpex' So Much? | Main | Speaking Engagements »

October 02, 2006

Comments

most

"Premature abstraction is the root of all evil."

I like this, and I'm going to use it.

Even apart from template use, some programmers become so enamored of C++ classes that they place interface and common implementation classes in front of every last meaningful piece of code. Performance and readability are obvious casualties here, but they're not the biggie.

In both the template and the class stacking cases, the real cost is the difficulty of debugging. Both lead to a lot of crawling up and down the call stack (and the template representational equivalent thereof). Encountering three or four do-nothing hops down the stack pane and back up again is about as distracting as having to get up and do jumping jacks at every call/return pair. It's enough for anyone to lose mental context.

ketan

I don't see how you can generalize that template is evil or scares newbie! Most of the grad/under-grad learns STL and to some extent templates while learning C++.

How do you write basic math/physics library without vectors? Duplicate code?

Template (apart from meta-template stuff etc.) basically offers ability to avoid repeat coding. Which is used when size and type based specialization helps.

Jason Olson

"Premature abstraction is the root of all evil."

That is an awesome quote!

Anyways, this problem not only exists in C++, but I see it becoming more and more
"abused" in .NET now that we have Generics. Yes, a generic collection or dictionary is good. But, outside of that, how many times do you _really_ need to use generics or templates? Not much, I'd guess.

It reminds me of the abuse I see with inheritance in OOP. I think because of the way OOP is taught (a good number of teachers focusing on "is a" rather than "has a"), young developers tend to instantly jump to inheritance rather than composition. IMHO, composition is where the power lies. First, if done properly, it becomes easier to write re-usable components. Second, with inheritance, people often overlook the fact that your child class now is _dependent_on_ and _tightly_coupled_to_ your parent class (meaning, changes in either could break the other if not careful).

That's a bit off topic though and is perhaps another post for another day :).

To concur, I believe premature abstraction is just as horrific and pain-causing as premature optimization can be.

proj

I'm not a big fan of STL for game programming. Often times the goals of generalization is contracdictory with performance and usability.

STL requires customization to do the right thing in a constrained environment, it's a pain to debug issues that come up and the implementation varies from platform to platform.

I also dislike all the copy construction and such that is hidden in the STL on iteration etc.

There are a few useful templates in the STL and these are not hard to write, nor hard to understand for a newbie if the interface is minimal and consistent. Does STL yet define a standard hashtable type container?

Most of the issues I have with STL can be solved by using STLport/boost but that's a pretty big dependency to pull in.

In most of the C++ code I write I use my own templates Array<>, Hashtable<>, StringHash<>, SList<>, DList<>, String<>. Because I wrote them to be minimal and performant they may not do everything the STL does but they are fast and do predictable things with memory. I use them for writing cell phone games as well as win32 3d code. I also have ref counted versions of these templates. I can quickly calculate in my head the memory overhead and exact performance characteristics of each implementation. They are also closely follow the Java/C# container paradigms which make them easy for a new user to understand.

Writing templates is not hard but you're right don't overdo it. Everything in moderation right?

M to the Vizzah

C++ is a terrible language, and the STL container classes are a barely mediocre implementation of such. They fail in providing good control of type alignment, allocation, and you end up rewriting them to support efficient serialization, anyway.

m.

billy_zelsnack

It's all about moderation. Templates and STL are tools. Sometimes/often people become enamoured by their tools and try to use them to solve problems that don't exist.

Sean Barrett

I generally avoid STL containers (except vector<>) because they don't sanely define iteration over changing data structures. If they're going to go to this work to create such a heavyweight design (with iterators and everything) there's really no excuse to not have tackled the change-while-iterating problem.

Well, I also avoid them because 99% of the time I either want a vector, an embedded linked list, or a hash table, and STL is no good for the latter two.

jason jones

One reason to reconsider using STL for all containers is explained in detail here:
http://www.codefarms.com/publications/intrusiv/intr.htm
STL does nothing to address the problem of relationships between data and thats its achilles heel...

Bryan McNett

Once again, STL != containers. Containers are just one small part of STL, whose best goodies (like nth_element) are just as happy with naked arrays as linked lists.

I'm not sure if I agree about premature abstraction being the root of ALL evil. Take that line of thinking too far, and you are developing games with infinite monkeys at typewriters, which *will* produce Hamlet but also lots of other things you don't want!

Templates and macros are dangerous, and should not appear in interfaces that junior programmers use. typedef foo bar; isn't enough. Interfaces for general consumption should be ANSI C where possible, and "vanilla " C++ otherwise.


Nathan McKenzie

For some reason, it often seems like template programming (which of course can be uesful and intellectually interesting in its own right) often becomes aggressively championed by the sorts of programmers who are a lot more interested in rewriting code over and over and over instead of adding new ideas or new functionality. Same with all sorts of dodgy ever-shifting UML diagrams of class hierarchies. I'm not against refactoring, and sometimes it can be the best thing in the world, but there's definitely a cut-off point where work is being replaced by mucking.

Sometimes I think the greatest sin of C++ (particularly in comparison to ANSI C and to Python) is that it gives programmers so many different ways to write roughly the same code, with each way having its own set of pros and cons. In theory that provides for an awful lot of control, but in practice it can result in an infinite regress of you swapping out the devil you know with the devil you're about to.

Nathan McKenzie

(Apologies if this double posts)

For some reason, it often seems like template programming (which of course can be uesful and intellectually interesting in its own right) often becomes aggressively championed by the sorts of programmers who are a lot more interested in rewriting code over and over and over instead of adding new ideas or new functionality. Same with all sorts of dodgy ever-shifting UML diagrams of class hierarchies. I'm not against refactoring, and sometimes it can be the best thing in the world, but there's definitely a cut-off point where work is being replaced by mucking.

Sometimes I think the greatest sin of C++ (particularly in comparison to ANSI C and to Python) is that it gives programmers so many different ways to write roughly the same code, with each way having its own set of pros and cons. In theory that provides for an awful lot of control, but in practice it can result in an infinite regress of you swapping out the devil you know with the devil you're about to.

Dimiter "malkia" Stanev

To Jason Jones, Code Farms...

I've stopped reading the article one I got to the following point:

"Fig.3: A container-based list may include objects of different types. However, this is a poor design. Use a list of common base objects in order to take advantage of the safe typing in C++."

There is a fine point between what is a premature optimization, and what is not.

I much preffer OpenStep (Cocoa) NSArray, in Common Lisp's (array t), in Lua's tables, as a collection, rather than what's being offered in STL, or in any other type of collector in C++.

All of them are forcing you to derive from a base class, which is a premature optimization (OpenStep enforces you the same - NSObject, but it enforces you for all possible objects, in that sense it works, but if it's only for half-of the object it doesn't).

So C++ is the language for making premature optimization calls all the way. You can't change or add a memmber, or at least new method to any class, without the need to recompile the whole goddman project, or rebuild your DLL's to get them working.

Really, when you type in C++ you are already making lots of premature optimizations - location of your members in the structures, function arguments, names. Class inheritance (you can't overload class at runtime, can you?), and many many other things (no reflection, no introspection, no evaluation, no code generation, no good macro system). It's your God's assembler, because God, and only God can make intelligent designs, where there is no premature optimization.

I'm not sure what Jesus was writing in... I guess probably C or assembler... Yeah...

Andrei

#define for_each( head, type, iter ) for( type * iter = head; iter; iter = iter->next )

struct A
{
int data;
A * next;
};

for_each( head, A, iter )
f( iter );

* No need for extra allocations for "list nodes"
* Same data structure can be stored in the cooked image without having to manually link objects into an STL list on load and then remove them on unload
* Too simple for practical purposes. Also Must Have Mashable Containers. (That's sarcasm.)
* I don't recommend hiring programmers that can't write a bug-free linked list.

- Andrei

Bilbo Baggins

Templates aren't only for abstractions. They are useful for metaprogramming (which I realize is a form of abstraction). Metaprogramming is basically writing code that works on types. Most of the time a type is known at compile time, allowing you to make appropriate decisions at compile time, e.g. type traits and static asserts help you catch invalid types the same way an ASSERT helps you catch a NULL pointer at runtime.

Template metaprogramming has its uses for things like loop unrolling. It's easy enough to write a generic loop unroller using a template metaprogram. A loop unroller is an abstraction the way that a new function is an abstraction, so you can apply the same rules.

I agree that using templates with justification is asking for trouble, but writing code of any sort without justification is asking for trouble. In the words of Thoreau: "Simplify, simplify, simplify!"

Paul Sinnett

"STL good. Templates bad."

I understand that's a gross oversimplication of the kind you need for a good rule of thumb. But I don't think it's a particularly good rule of thumb.

I advocate STL, template, and even macro use, but with the caveat: use only when the alternative is cut and paste coding. It's theoretically possible to use STL, templates, or macros for almost all of your programming needs - but in most cases it's far simpler to avoid them.

I don't buy the "root of all evil" in programming rhetoric. There is one, and only one, root of all evil in programming: the programmer. Ignore that at your peril ;-)

eh

Attempts to reduce things to simple rules like "premature abstraction is the root of all evil" (a ridiculously hyperbolic statement) are not as productive as some might think. Programming smart does not mean carrying around a collection of pithy sayings and trotting them out for speaking engagements or to awe junior programmers. If you examine the above "abstraction" statement you will see that it sounds a lot deeper than it is. The word "premature" already implies that the abstraction is an error, so the rest of the statement is redundant (ignoring the drama). You might as well just say "don't do unnecessary work", but that should be obvious. A lot of the quality of a programmer depends on how well they recognize what needs to be done and what doesn't.

Paul Sinnett is right on when he criticizes the "root of all evil" statements that people seem to like to make.

"I think it was Kent Beck who said you should refactor once you've written the same code three times. I think with templates I'd go as far as four..."

This makes no sense to me. The purpose of refactoring is to simplify existing code without changing its function. If a change is a simplification, why not do it? (barring time constraints, etc.) Even if it involves a dreaded template that causes these apparently brain dead new hires to commit suicide. On the other hand, if a change is not a simplification, and adds no other benefit (as implied by the term refactor), why even discuss doing it? Obviously you should not. There's no need for a goofy rule like "If you rewrite it three times or four times or 2.6 times". Again, it's just "don't do unnecessary work".

We can see again this tendency to over-simplify things with these pithy rules with the statement you seem to want to make: "writing your own templates is bad". No - the unnecessary work done by these "clever" programmers is what is bad. Templates are just a tool. I can screw up and complicate code using one of many C++ language features, including STL, if I want. It makes no sense to single out templates as people so often want to do.

Jamie Fristrom

Eh, you're generalizing yourself, there, implying that all attempts at inductive reasoning are oversimplifications made to awe junior programmers and make ourselves look good at lectures.

Inductive reasoning is fallible. I know that. I'm still a big fan of "rules of thumb" as a first line of defense against bad ideas. The conversation at the workplace would go something like this: Coworker: "I'd like to use a template for this." Me: "Speculative generality...premature abstraction...been burned many times...?" Coworker: "No, this time it really is good, because x, y, z" Me: "Tell you what, how about you get x done without, and if y crops up, *then* template it."

Which brings us to the refactoring point, where I must not have been clear, because it's really very simple, and it doesn't necessarily have anything to do with templates. You're writing some code - at this point, you may anticipate a future use for this code. You could isolate it out as its own method and give it an extra parameter it won't need until later - this is where you've commited a mild case of speculative generality. The majority of the time you may even correctly anticipate and win.

I don't do this. I write the code so it does what's needed at the moment. Later, I discover myself writing the same - or similar - code again. At this point I might refactor, extracting the orignial code out into a method.

But Kent Beck would say to wait until I'm writing the code for the third time and refactor then.

And I'd say that if the refactoring involves making a template, because this is code that needs to work on ints, floats, and several other weird datatypes to be named later, I might wait until the fourth time I'm writing this similar code before I abstract out the commonality. And the reason I say four is because of some specific templated code I've worked with - animation code that we used to animate scalars, vectors, and rotations. At the time, it seemed like templates were a great idea, and we were excited about the possibilities of animating any future parameter we could come up with. But we never did come up with another data type to animate, and the original coder and I were a couple of the only coders in the company who liked to work with that code.

Again, in case it's not clear, the point at which refactoring happens is not after you've written the code three or four times, and, as far as you know, you're done - that would be potentially unnecessary work - it's when you're about to write the code again for the n+1th time. You refactor first so that n+1th time is trivial.

eh

"Eh, you're generalizing yourself, there.."

Eh, I think you are reading too much into my statements. They should be taken at face value. I can't be held responsible for what you think I'm implying.

"it doesn't necessarily have anything to do with templates"

Huh? Isn't that what I said? And how can your point about waiting till 4 rewrites till refactoring specifically when using templates not have anything to do with templates?

"But Kent Beck would say to wait until I'm writing the code for the third time and refactor then."

Do you have a reference for this statement by Kent Beck? It doesn't sound wise to me, though it seems like he has made sense in the past. Refactoring makes things simpler by re-using common code, in general; why would you not always do that, even with just two cases?

So, as far as I can understand, your reasoning for waiting until "4" to refactor code with a template is that the number of items in the set "scalars, vectors, rotations" is 4 - 1, in this particularly traumatic case in the past you refer to.

Just so that you don't think I'm being totally negative, I do approve of your stance on speculative generality. Although in most of the code I see the tendency is to go much farther in the other direction. Sometimes I think I might rather work on a codebase with a little speculative generality than one that has huge amounts of cut and pasted crap.

Nathan McKenzie

This discussion brings up a tool question I've had for a while... but it's going to take some prefacing:

To me, the thing that can make template code hard to read is, generally, the process of templatizing involves adding many new parameters and, crucially, adding many new functions (generally in files left to be discovered by the reader). Properly templated code often jumps around like crazy from inline function to inline function to do its deeds, sometimes further intertwined with other templated classes elsewhere. Often, I'm not convinced such code can even be read sanely without debugging it (where the debugger is kind enough to take you from context-sensitively-labeled function call to context-sensitively-labeled function call to context-sensitively-labeled function call to...)

For writing code, it's still really nice to have all of certain kinds of code isolated to a single spot. And even for re-reading code, if I already know the system and how it's laid out and which classes are built to do specifically what, it can be a fine way to work as well. But for reading new code... well, eh. There's just _so_ much jumping around, and so much of whatever I'm interested in seeing or reading or understanding being constantly hidden away elsewhere, over and over and over, behind function names that are too generic (like Init or operator->) to be meaningfully searchable, in files to be determined at link time.

I like writing template code. I like reading my own template code, if I've written it recently. The template code of others, however, is a constant source of frustration. Which makes it sound an awful lot like Perl (or, if I want to be honest, it just makes it sound like code)

Notably, my griping here also applies to inline functions, getters and setters, new classes and abstractions and encapsulations, defines and constants, functions, gigantic horrifying inheritance heirarchies, interfaces, and the concept of indirection itself...

...which is a problem, given that indirection is the solution to the question, "How do we not repeat ourselves, and how to do localize change?"

Okay. So here's the actual tool question; I'm convinved the problem here isn't indirection, but rather that programming takes place mostly in single flat text files. The problem with all this indirection isn't that it takes an action that used to be localized in one place and smears it into 30 places throughout 10 files; the problem is that I have to read and make changes to that action in one file in one place at a time.

Are there any tools that break this paradigm? Are there, specifically, any tools that, more or less, let you inline code that is found elsewhere in a single viewing area? Basically, think of it as the opposite of code folding found in Eclipse or MS-DevStudio, or think of it as exactly what compilers do when they inline code at compile time. Are there any IDEs that, if you come across this:

Image< float, 4 > newImage( "angry_parade" );
imageList.push_back( newImage );

will let you replace it with

Image< class pixelType = float, int pixelStride = 4 > newImage( char* newName = "angry_parade" ){
this->name = newName;
this->Load< pixelType, pixelStride >( newName );
};
imageList.push_back( newImage );

?

I know that looks all mucked up in the text box here, so my apologies in advance...

Jamie Fristrom

I was wrong, it wasn't Kent Beck, it was Martin Fowler, in *Refactoring* - he calls it the "rule of three", and it didn't explicitly prohibit refactoring at two or one. (Of course, at one, it's not refactoring, is it.)

Turns out I'm something of a hypocrite, also, since I just wrote a template function yesterday that I've only used once. I feel pretty confident it will be a win, though, since it's a function I used a lot in another language. And I did have an argument with myself before I wrote it.

"Although in most of the code I see the tendency is to go much farther in the other direction." For the most part, not my experience - I guess that's why I like the hyperbolic pithy saying.

I suppose some programmers tend to be speculative generality guys and others tend to be cut-and-paste guys and it's very rare for someone to guess right how much generality the code is going to need on the first iteration.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been posted. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Your Information

(Name is required. Email address will not be displayed with the comment.)

Jamie's Bragging Rights

  • Spider-Man 2
    The best superhero games of all time Game Informer
    Top five games of all time Yahtzee Croshaw
    Top five superhero games of all time MSNBC
    Top 100 PS2 games of all time Official Playstation 2 Magazine
    1001 Games You Must Play Before You Die Nomination for Excellence in Gameplay Engineering Academy of Interactive Arts & Sciences
  • Schizoid
    Penny Arcade PAX 10 Award
    Nominated for XBLA Best Original Game
    Nominated for XBLA Best Co-Op Game