So it begs the question, why are comments in code not helping?

With current tools and technologies there are so many layers that the code is scattered all around the place, in xml files, in os scripts, in java, stored procedures.

Fundamentally the Intention of Design being conveyed comments, or even as class diagrams are insufficient, therefore wasteful.

There are a number of actors and forces at play

• The wrong things are asked by management to be documented. Things that are most likely to change are given prime focus in the documentation. Unstable or fast changing parts of the code base, things that are easily inspected directly in code and tests (you have tests right?).
• Developers don't know any better, they are documenting, what they have produced, in the easiest way that they can, because - they know no one will read it - or if it is read the readers won't pay much attention to it anyway ie. they don't trust it

One approach to improve this situation could be learn from other fields of study, ones that rely a lot on communication of intent rather than ticking off items on the project plan

Last year, I spent a little time thinking about this problem, as we had the problem of improving inter team communication with as little overhead as possible. They already had a typical template for the usual type of things, general design, building, testing, technical details, including implemenation details.

But as a reviewer, what really struck me was the fragility of this documentation, a lot of things that were most likely to change, composed a reasonable chunk of the document.

Lucky for me, I had recently read a book "Sources of Power" [Klein1998]. It so happens that the armed forces represent a very large organisation with large communication issues. Orders need to be carried out and passed down the chain of command.

Considerations in Communicating Intent

(From pg. 225, ever so slightly modified)

1. Purpose.

   • What are the High Level goals and within what context.

2. Objective of task.

   • What is the desired outcome.

3. Plan.

   • Sequence of Steps in the Plan.

4. Rationale.

   • Why is this plan good, what makes this approach a better one over other approaches.

5. Key Decisions.

   • Explicit expectation that there is uncertainty in the plan. Some anticipated key decisions will need to be made at more appropriate and beneficial times as progress.

6. Unwanted Outcomes.

   • Anti Goals. There are many unwanted outcomes, often these may be things that immediately spring to mind. ie. that doing the simplest thing possible would be bad as it will attribute to an unwanted outcome. But in fact developers may unknowingly compromise the design of the system.

7. Constraints and other considerations.

Things I like about these considerations:

• Things that are most likely to change less frequently are documented.

  • Purpose, goals within a business context, can last 12-18mths.

• Ample opportunity to highlight "tradeoffs" that occurred.

  • Things that "come back to haunt you" should be mentioned in something with a longer memory than your brain. Perhaps when the next team get it they don't become "Angry Monkeys". Eg. Due to "abc" constraint we took "sdf" workaround OR when you touch this next make sure you fix "xyz".
  • That sort of information can take hours or days to re-learn next time, not to mention the fact that someone may learn it after planning has occurred. Yet all it takes is a one or two lines to actually make documenting deliver some value.

• Plan.

  • must not be a duplication of some other list, it needs to convey meaning and include the important bits for discussion, it is not a replacement for story or task cards or gantt charts.

• Developers will often get focused on a small item.

  • In a broader context that small item may have very little relevance.
  • Context and purpose get considered and documented to help keep it real.

• When circumstances change.

  • as they often do, developers have some contextual information to make informed decisions.

• As the project progresses and we learn more about the project, tools and users, we will discover better ways of achieving goals or ways of eliminating steps in the plan.

• There is a larger "surface area" for less experienced developers to gain some insight into factors that are discussed and decided upon during design and to raise questions.

As I work amongst several project teams, I strive to get the leads to write about these considerations. Even if they don't make it for prosperity we are sure to have the conversations.

These considerations fit in with the primary aim of that book, which is Decision Making, but I'll leave that for another time.

The thing to remember is with your documentation is, what are you trying to achieve. Follow the DRY principle and don't reiterate stuff that people should already know.

[Ford2008]

Neal Ford, The Productive Programmer, O'Reily Media, Sebastopol, 2008

[Schwartz2004]

. Barry Schwartz, Paradox of Choice. HarperCollins, New York, 2004.

[CockburnAmbler]

Scott Ambler sourced from Alistair Cockburn

[Klein1998]

Gary Klein, Sources of Power, Massachusetts Institute of Technology, 1998