On Managing Complexity

There’s a recurring theme on this blog when I talk about programming. It’s a skill you don’t learn in school, but which separates recent graduates from employees with 5 years of experience: how to manage complexity.

The reason you never learned how to manage complexity in nearly 20 years of school is that you never actually encountered anything of sufficient complexity that it needed managing. That’s because at the beginning of each term/semester/year you start with a blank slate. If you only ever spend up to 500 hours on a school project, you’ll never understand what it feels like to look at a system with 10,000 engineering hours logged to it.

That’s when the rules of the game change. You’ve only ever had to deal with projects where you could hold all the parts and variables in your head at one time and make sense of it. Once a project grows beyond the capability of one person to simultaneously understand every part of it, everything starts to unravel. When you change A you’ll forget that you had to change B too. Suddenly your productivity and quality start to take a nosedive. You’ve reached a tipping point, and there’s no going back.

It isn’t until you reach that point that you understand why interfaces are useful and why global variables are bad (not to mention their object-oriented cousins, the singleton).

The funny thing is that you learn about all the tools for managing complexity in school. When I was in high school I was taught about structured programming and introduced to modular programming. In university my first programming class used C++ and went through all the features like classes, inheritance and polymorphism. However, since we’d never seen a system big enough to need these tools, most of it fell on deaf ears. Now that I think about it, I wonder if our professors had ever worked on a system big enough to need those tools.

The rules of managing complexity are:

  • Remove unnecessary dependencies between parts of your system. Do this by designing each module against an abstract interface that only includes the functionality it needs, nothing else.
  • Make necessary dependencies explicit. If one module depends on another, state it explicitly, preferably in a way that can be verified for correctness by an automated checker (compiler, etc.). In object-oriented programming this is typically done with constructor injection.
  • Be consistent. Humans are pattern-recognition machines. Guide your reader’s expectations by doing similar things the same way everywhere. (Five-rung logic, anyone?)
  • Automate! Don’t make a person remember any more than they have to. Are there 3 steps to make some change? Can we make it 2, or better yet, 1?

It would be interesting if, in university, your second year project built on your first year project, and your third year project built on your second year project, etc. Maybe by your fourth year you’d have a better appreciation for managing complexity.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.