brettdargan.com » productivity

Considerations in Communicating the Intent of Software Design

admin — Wed, 07 Jan 2009 14:17:28 +0000

I finished reading Neal Fords book "The Productive Programmer" [Ford2008] It wasn't quite what I expected, but none the less it was well worth the purchase.

While there are a few topics I agree with but not to the same extent as him, the topic I strongly disagree with is on the generation of large amounts of documentation.

Documentation in this form merely serves as another one of those checklist items on a project plan.

"Conveying understanding" should be the underlying purpose for which usually wiki pages or documentation is produced in order to communicate with others through time and space. Documentation is not just about information.

Consider your own experience when it comes to picking up documentation produced by others no longer working on the systems.

Most documentation is brain dumped with little thought given to the actual writing or conveying of information. Anyone can describe the complicated in a complex manner, it takes more skill to take a complicated details and make them simple to understand.

Large volumes of javadoc, don't help me, it is another layer of indirection and as Robert C. Martin says regarding code, "all comments are lies". Deep down everyone knows this, even though many people are still in denial.

Rarely did I get documentation that was:

reasonably accurate in terms of what it was meant to do
mostly it differed significantly in how it was actually implemented
usually had typos in crucial lines that didn't aid understanding at all
large gotchas aka time-bombs are not mentioned
building and configuration where slightly covered
the best you usually wish for is some clues for which you can pull out your best Dr. Holmes hat and get to work with

The choice of conveying understanding via documentation is one of the least effective mediums for communication. Refresh your memory of Alistair Cockburn's [CockburnAmbler]; analysis of various communication mediums and it's effectiveness.

So it begs the question, why are comments in code not helping?

With current tools and technologies there are so many layers that the code is scattered all around the place, in xml files, in os scripts, in java, stored procedures.

Fundamentally the Intention of Design being conveyed comments, or even as class diagrams are insufficient, therefore wasteful.

There are a number of actors and forces at play

The wrong things are asked by management to be documented. Things that are most likely to change are given prime focus in the documentation. Unstable or fast changing parts of the code base, things that are easily inspected directly in code and tests (you have tests right?).
Developers don't know any better, they are documenting, what they have produced, in the easiest way that they can, because - they know no one will read it - or if it is read the readers won't pay much attention to it anyway ie. they don't trust it

One approach to improve this situation could be learn from other fields of study, ones that rely a lot on communication of intent rather than ticking off items on the project plan

Last year, I spent a little time thinking about this problem, as we had the problem of improving inter team communication with as little overhead as possible. They already had a typical template for the usual type of things, general design, building, testing, technical details, including implemenation details.

But as a reviewer, what really struck me was the fragility of this documentation, a lot of things that were most likely to change, composed a reasonable chunk of the document.

Lucky for me, I had recently read a book "Sources of Power" [Klein1998]. It so happens that the armed forces represent a very large organisation with large communication issues. Orders need to be carried out and passed down the chain of command.

Considerations in Communicating Intent

(From pg. 225, ever so slightly modified)

Purpose.
- What are the High Level goals and within what context.
Objective of task.
- What is the desired outcome.
Plan.
- Sequence of Steps in the Plan.
Rationale.
- Why is this plan good, what makes this approach a better one over other approaches.
Key Decisions.
- Explicit expectation that there is uncertainty in the plan. Some anticipated key decisions will need to be made at more appropriate and beneficial times as progress.
Unwanted Outcomes.
- Anti Goals. There are many unwanted outcomes, often these may be things that immediately spring to mind. ie. that doing the simplest thing possible would be bad as it will attribute to an unwanted outcome. But in fact developers may unknowingly compromise the design of the system.
Constraints and other considerations.

Things I like about these considerations:

Things that are most likely to change less frequently are documented.

Purpose, goals within a business context, can last 12-18mths.

Ample opportunity to highlight "tradeoffs" that occurred.

Things that "come back to haunt you" should be mentioned in something with a longer memory than your brain. Perhaps when the next team get it they don't become "Angry Monkeys". Eg. Due to "abc" constraint we took "sdf" workaround OR when you touch this next make sure you fix "xyz".

That sort of information can take hours or days to re-learn next time, not to mention the fact that someone may learn it after planning has occurred. Yet all it takes is a one or two lines to actually make documenting deliver some value.

Plan.

must not be a duplication of some other list, it needs to convey meaning and include the important bits for discussion, it is not a replacement for story or task cards or gantt charts.

Developers will often get focused on a small item.

In a broader context that small item may have very little relevance.

Context and purpose get considered and documented to help keep it real.

When circumstances change.

as they often do, developers have some contextual information to make informed decisions.

As the project progresses and we learn more about the project, tools and users, we will discover better ways of achieving goals or ways of eliminating steps in the plan.

There is a larger "surface area" for less experienced developers to gain some insight into factors that are discussed and decided upon during design and to raise questions.

As I work amongst several project teams, I strive to get the leads to write about these considerations. Even if they don't make it for prosperity we are sure to have the conversations.

These considerations fit in with the primary aim of that book, which is Decision Making, but I'll leave that for another time.

The thing to remember is with your documentation is, what are you trying to achieve. Follow the DRY principle and don't reiterate stuff that people should already know.

[Ford2008]

Neal Ford, The Productive Programmer, O'Reily Media, Sebastopol, 2008

[Schwartz2004]

. Barry Schwartz, Paradox of Choice. HarperCollins, New York, 2004.

[CockburnAmbler]

Scott Ambler sourced from Alistair Cockburn

[Klein1998]

Gary Klein, Sources of Power, Massachusetts Institute of Technology, 1998

Tweet This Post

reSTructured Text articles for wordpress

admin — Thu, 01 Jan 2009 22:55:56 +0000

I'm pretty keen to write more this year, but I'd like to do it in a format that leaves me plenty of options, so the reSTructured format appears to suit my needs.

To avoid wasted effort, I'll need to have wordpress blogs/articles using this format.
To get this installed on site5, was a little bit of a pain.

mkdir ~/bin

download docutils.sf.net
mkdir -p ~/bin
untar in ~/bin

python setup.py install --prefix $HOME #no sudo required

.bashrc, PYTHONPATH=$PYTHONPATH:$HOME # so you can test on cmdline

try a test value, rst2html.py should work.

solve any problems their first.

Now onto wordpress, install the rest.php plugin at launchpad.net.

Edit settings at the top of the file, for $prefix which needs to be the same value as setup --prefix and $rst2html to rst2html.py

Now I was stuck on the problem with wordpress executing it with environment variables that don't seem to include the PYTHONPATH that I require.

so a cheap hack around that is to just dynamically update the python search path in rst2html.py

sys.path.append('/your/python2.4/site-packages')

Now I have proper results being displayed, except the mode line, which marks the entry for processing by the rest.php filter, is also being processed.
-*- mode: rst -*-

This needs to be filtered out, either during the rst2html process or after,
so I went with just embedding it within a reST comment and all is well.

.. -*- mode: rst -*-

Tweet This Post

Simpler Serverside Application Deployment and Management

admin — Tue, 25 Jul 2006 01:15:58 +0000

Why aren't my release notes to sysadmins and testers:
sudo yum install mypkgname

Which downloads mypkgname from yum repository with all of it's dependencies.

That obviously wouldn't always be possible, but it is achievable with only a little pain .

Use RPM for managing components of your System

RPM is not ideally suited to deploying java applications and all of it's dependencies, but it can work.

I've used it to package up an installation of activemq, with my application config and start/stop scripts.
This was then extended to have ant build it so we could eventually have cruise create rpm artifacts for a project which could eventually get automatically sent to a yum repository.

RPM is not that great but when you have to wrestle with RedHat AS atleast it gives us

Dependency Management. Requires that mysql >= 5.0 must be installed.
Hooks for pre/post install/uninstall Scripts. %postin let me create a os user if required, set appropriate permissions, create a database, db user, add tables and insert data.
Ability to relocate the package on installation

Creating yum repositories sounds easy, but I haven't gotten to this yet.

What to do

Create your ~/.rpmmacros file
Create a specification file, don't forget it will has access to some of it's own types of variables that are defined in .rpmmacros file.
package up your application of choice in a mypkgname-version.tar.gz. This is your main input into rpm, Source0: in your spec file must exactly match this file
rpm -bb mypkgname.spec
check your pkg info with: rpm -qpi mypkgname-version-noarch.rpm and your list of files with rpm -qpl mypkgname-version-noarch.rpm.
sudo rpm -i mypkgname-version-noarch.rpm

Tips

Understand %files .I found it easier to build my own file.list, by using %files -f file.list. Defaulting attributes with %defattr(-,name,group) and %config to specify your config files are good to know about.
You don't have to use the root owned default database, you can specify your own with the --dbpath option
How can you tell if a package is being installed or updated?
$1 of the scriplets (%pre, %postin...) contain the number of pkg's affected, so during an update you affect 2 pkgs, as it is an uninstall, followed by an install operation.
Scriptlets share the same environment, but that environment is special. We had issues running pre-packaged scripts that depended on env variables setup in .bashrc becaused it exited if it wasn't an interactive shell. ie. [ -z $PS1 ]
Keep multiple copies of ~/.rpmmacros files and replace the one you are interested in
If you are packaging ruby, python or java applications, then specify it as non architecture specific
Make it Relocatable if possible
Someone had a problem with BuildArch: noarch on FC4 x86-64, that noarch was not recognised as a valid architecture. We needed to add a line to some file in /var/lib??. Moral of the story is to try not to use debian based distro instead.

If you know almost nothing about RPM

RPM can be used to manage src (c) or binary rpms.
It works by unextracting a tar of the source/binary then building and executing scriptlets at the right moments.
configuration files, identified by %config, like /etc/mypkgname are renamed when you erase a package to mypkgname.conf.rpmsave.
Relocatable pkgs require all your packaged files to have the same prefix.

Thanks to co-conspirators Ajit George, Sam Jordan, Peter Klein, Robert Watkins.

Tweet This Post

What is the cost of a build breakage?

admin — Mon, 20 Feb 2006 16:47:18 +0000

Has anyone done a study to cost of a build breakage to an organistation?

Let's do some back of the envelope costings, assume a long running build of 30+mins, with 10 devs, probably 50% want to commit during the period or during the period after the build was broken and before a rollback or fix has been committed.

A broken build has distracted atleast 5 people including the build master and they will have to wait or be careful of how they progress so on average that may be about 5hrs of lost productivity.

THAT's EXPENSIVE!!

Sure there are lot's of arguments like make your build faster, well the reality is whatever time your build is, you have disrupted the flow of your team members that has resulted in potentially significant wasted effort.

And in your rush to fix the build you may have not really addressed fundamental issues of the breakage, but just put a quick fix in to get things going.

Tweet This Post

Quick Calculations

admin — Sun, 19 Sep 2004 23:28:24 +0000

Today we had to do some estimates for potentially caching a large number of objects in memory, using some quick calculations (or Back of the Envelope) calculations, we quickly calculated a very big number, indicating we would immediately need to start considering alternative implementations.

Further analysis was needed and later it was confirmed that initial estimates were pretty close to the mark.

It reminded me of a good book that is probably not on alot of programmer's reading list. Programming Pearls is an old style book of computer science articles from the 80's. Column 7 about "Back of the Envelope" estimation, using very rough order of magnitude guesses to arrive at an approximate value for an amount, or response time. I originally read this from following some of the references from Code Complete. It is an Addison-Wesley book, but not available on safari , which is exactly the sort of book that should be provided

Tweet This Post

Catch html designer bugs early

admin — Tue, 20 Jul 2004 00:59:28 +0000

We had a few recurring problems that could not be found in testing, only in production. Mostly to do with the url routing issues that our client doesn't have a development environment for.

Enter a Grep ant task to help us fail early in the dev process. There are other similar jsp based checkers, but this is generalised utility that we use across other known issues in the project.

It is very useful for new team members.

It hasn't been added to an opensource project yet, could probably post it to ant-contrib or something like that.

Tweet This Post