Skip to content

Query vs. Matrix Params or Path Params

I've come across a few posts about Matrix Parameters recently and there is a lot of misunderstanding.

There are several old posts that discuss and describe these things:

I've summarised what I think is important:

Tip: If you're not sure if you should use them then don't.

Matrix parameters are used with

Caching Issues

Intermediaries (proxies) won't cache any url with a query parameter in the url
  • this is because in the early days of the web, they didn't trust the Cache control information from dynamically generated pages.
  • Freshness and Validation information are very important pieces of information and the proxy admins took a cautious stance and didn't cache any of it.

As the web has matured the views of devs/admins are changing, the new versions of Squid will default to cache any urls with query parameters

Obviously this could have a very large impact, I've heard google maps image tiles had this issue with some proxies at one time.

Relative URI calculations

Relative URIs have different affects, as relatives URLs are determined by appending to the Base URI. For Query Parameter requests they are stripped from the Base.

For those still interested I would recommend checking out the following in addition to the RFC-2616 of course.

Post to Twitter Tweet This Post

Posted in RESTful, Web. Tagged with , , .

By admin
January 16, 2009

6 comments

HATEOAS for legacy object models

One of the best aspects of RESTful orientated services is getting your resources connected, possibly even achieving HATEOAS. Making that jump with with legacy object models or data can be tricky, depending on how intrusive you want to be.

There is such a large amount of legacy data in various forms, usually with minimal links/relations that could be leveraged.

So it is a worthy challenge to try and do it in the least intrusive manner possible.

This is one approach that is reasonably decoupled from an existing legacy model.

There are three main parts to this technique:

  1. Enhance your model objects so they have a placeholder/collection so you can accumulate links
  2. Annotate your model at interesting points with link templates, around attributes or methods or classes
  3. Use a custom ContainerResponseFilter class of jersey to scan for annotations of interest, then evaluate your link templates (at run time), using the data of your models and any request based information you may require.

I chose to enhance my model classes using AspectJ at compile time.

Download the source

mvn glassfish:run
./curl_test #will hit two urls

and you should see some results, with links like this:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<country xmlns:ns2="http://www.w3.org/1999/xlink">
  <links>
    <link rel="" ns2:type="locator"
        ns2:href="http://localhost:8080/hateoas4legacy/rs/cities/Brisbane"/>
    <link rel=""
        ns2:type="locator" ns2:href="http://localhost:8080/hateoas4legacy/rs/cities/Sydney"/>
    <link rel=""
        ns2:type="locator" ns2:href="http://localhost:8080/hateoas4legacy/rs/cities/Melbourne"/>
    <link rel=""
        ns2:type="locator" ns2:href="http://localhost:8080/hateoas4legacy/rs/countries/AU"/>
  </links>
  <cities>
    <city>
      <links/>
      <name>Brisbane</name>
    </city>
    <city>
        <links/>
        <name>Sydney</name>
    </city>
    <city>
      <links/>
      <name>Melbourne</name>
    </city>
  </cities>
  <isoCode>AU</isoCode>
  <name>Australia</name>
</country>

All countries Results

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<countries>
<country xmlns:ns2="http://www.w3.org/1999/xlink">
  <links>
    <link rel=""
        ns2:type="locator" ns2:href="http://localhost:8080/hateoas4legacy/rs/countries/AU"/>
  </links>
  <cities/>
  <isoCode>AU</isoCode>
  <name>Australia</name>
</country>
<country xmlns:ns2="http://www.w3.org/1999/xlink">
  <links>
    <link rel=""
        ns2:type="locator" ns2:href="http://localhost:8080/hateoas4legacy/rs/countries/USA"/>
  </links>
  <cities/>
  <isoCode>USA</isoCode>
  <name>United States of America</name>
</country>
</countries>

It is very crufty, but does work in the simple cases. I haven't had a chance to demonstrate all of the features yet.

I'm likely to further flesh out this example as it is interesting in other RESTful related ways.

Post to Twitter Tweet This Post

Posted in Patterns, RESTful, Web. Tagged with , , , .

By admin
January 14, 2009

6 comments

Considerations in Communicating the Intent of Software Design

I finished reading Neal Fords book "The Productive Programmer" [Ford2008] It wasn't quite what I expected, but none the less it was well worth the purchase.

While there are a few topics I agree with but not to the same extent as him, the topic I strongly disagree with is on the generation of large amounts of documentation.

Documentation in this form merely serves as another one of those checklist items on a project plan.

"Conveying understanding" should be the underlying purpose for which usually wiki pages or documentation is produced in order to communicate with others through time and space. Documentation is not just about information.

Consider your own experience when it comes to picking up documentation produced by others no longer working on the systems.

Most documentation is brain dumped with little thought given to the actual writing or conveying of information. Anyone can describe the complicated in a complex manner, it takes more skill to take a complicated details and make them simple to understand.

Large volumes of javadoc, don't help me, it is another layer of indirection and as Robert C. Martin says regarding code, "all comments are lies". Deep down everyone knows this, even though many people are still in denial.

Rarely did I get documentation that was:
  • reasonably accurate in terms of what it was meant to do
  • mostly it differed significantly in how it was actually implemented
  • usually had typos in crucial lines that didn't aid understanding at all
  • large gotchas aka time-bombs are not mentioned
  • building and configuration where slightly covered
  • the best you usually wish for is some clues for which you can pull out your best Dr. Holmes hat and get to work with

The choice of conveying understanding via documentation is one of the least effective mediums for communication. Refresh your memory of Alistair Cockburn's [CockburnAmbler]; analysis of various communication mediums and it's effectiveness.

So it begs the question, why are comments in code not helping?

With current tools and technologies there are so many layers that the code is scattered all around the place, in xml files, in os scripts, in java, stored procedures.

Fundamentally the Intention of Design being conveyed comments, or even as class diagrams are insufficient, therefore wasteful.

There are a number of actors and forces at play
  • The wrong things are asked by management to be documented. Things that are most likely to change are given prime focus in the documentation. Unstable or fast changing parts of the code base, things that are easily inspected directly in code and tests (you have tests right?).
  • Developers don't know any better, they are documenting, what they have produced, in the easiest way that they can, because - they know no one will read it - or if it is read the readers won't pay much attention to it anyway ie. they don't trust it

One approach to improve this situation could be learn from other fields of study, ones that rely a lot on communication of intent rather than ticking off items on the project plan

Last year, I spent a little time thinking about this problem, as we had the problem of improving inter team communication with as little overhead as possible. They already had a typical template for the usual type of things, general design, building, testing, technical details, including implemenation details.

But as a reviewer, what really struck me was the fragility of this documentation, a lot of things that were most likely to change, composed a reasonable chunk of the document.

Lucky for me, I had recently read a book "Sources of Power" [Klein1998]. It so happens that the armed forces represent a very large organisation with large communication issues. Orders need to be carried out and passed down the chain of command.

Considerations in Communicating Intent

(From pg. 225, ever so slightly modified)

  1. Purpose.
    • What are the High Level goals and within what context.
  2. Objective of task.
    • What is the desired outcome.
  3. Plan.
    • Sequence of Steps in the Plan.
  4. Rationale.
    • Why is this plan good, what makes this approach a better one over other approaches.
  5. Key Decisions.
    • Explicit expectation that there is uncertainty in the plan. Some anticipated key decisions will need to be made at more appropriate and beneficial times as progress.
  6. Unwanted Outcomes.
    • Anti Goals. There are many unwanted outcomes, often these may be things that immediately spring to mind. ie. that doing the simplest thing possible would be bad as it will attribute to an unwanted outcome. But in fact developers may unknowingly compromise the design of the system.

  7. Constraints and other considerations.

Things I like about these considerations:

  • Things that are most likely to change less frequently are documented.
    • Purpose, goals within a business context, can last 12-18mths.
  • Ample opportunity to highlight "tradeoffs" that occurred.
    • Things that "come back to haunt you" should be mentioned in something with a longer memory than your brain. Perhaps when the next team get it they don't become "Angry Monkeys". Eg. Due to "abc" constraint we took "sdf" workaround OR when you touch this next make sure you fix "xyz".
    • That sort of information can take hours or days to re-learn next time, not to mention the fact that someone may learn it after planning has occurred. Yet all it takes is a one or two lines to actually make documenting deliver some value.
  • Plan.
    • must not be a duplication of some other list, it needs to convey meaning and include the important bits for discussion, it is not a replacement for story or task cards or gantt charts.
  • Developers will often get focused on a small item.
    • In a broader context that small item may have very little relevance.
    • Context and purpose get considered and documented to help keep it real.
  • When circumstances change.
    • as they often do, developers have some contextual information to make informed decisions.

  • As the project progresses and we learn more about the project, tools and users, we will discover better ways of achieving goals or ways of eliminating steps in the plan.

  • There is a larger "surface area" for less experienced developers to gain some insight into factors that are discussed and decided upon during design and to raise questions.

As I work amongst several project teams, I strive to get the leads to write about these considerations. Even if they don't make it for prosperity we are sure to have the conversations.

These considerations fit in with the primary aim of that book, which is Decision Making, but I'll leave that for another time.

The thing to remember is with your documentation is, what are you trying to achieve. Follow the DRY principle and don't reiterate stuff that people should already know.

[Ford2008]Neal Ford, The Productive Programmer, O'Reily Media, Sebastopol, 2008
[Schwartz2004]. Barry Schwartz, Paradox of Choice. HarperCollins, New York, 2004.
[CockburnAmbler]Scott Ambler sourced from Alistair Cockburn
[Klein1998]Gary Klein, Sources of Power, Massachusetts Institute of Technology, 1998

Post to Twitter Tweet This Post

Posted in General. Tagged with , , , , .

By admin
January 8, 2009

No comments

Question Marks and Speculation Marks

Is there a single character that we can use to indicate "Speculation", in the same what the question mark indicates a question?

If there was would you prefix your statements/queries instead of postfix?

Post to Twitter Tweet This Post

Posted in General.

By admin
January 2, 2009

No comments

reSTructured Text articles for wordpress

I'm pretty keen to write more this year, but I'd like to do it in a format that leaves me plenty of options, so the reSTructured format appears to suit my needs.

To avoid wasted effort, I'll need to have wordpress blogs/articles using this format.
To get this installed on site5, was a little bit of a pain.

mkdir ~/bin

download docutils.sf.net
mkdir -p ~/bin
untar in ~/bin

python setup.py install --prefix $HOME #no sudo required

.bashrc, PYTHONPATH=$PYTHONPATH:$HOME # so you can test on cmdline

try a test value, rst2html.py <filename> should work.

solve any problems their first.

Now onto wordpress, install the rest.php plugin at launchpad.net.

Edit settings at the top of the file, for $prefix which needs to be the same value as setup --prefix and $rst2html to rst2html.py

Now I was stuck on the problem with wordpress executing it with environment variables that don't seem to include the PYTHONPATH that I require.

so a cheap hack around that is to just dynamically update the python search path in rst2html.py

sys.path.append('/your/python2.4/site-packages')

Now I have proper results being displayed, except the mode line, which marks the entry for processing by the rest.php filter, is also being processed.
-*- mode: rst -*-

This needs to be filtered out, either during the rst2html process or after,
so I went with just embedding it within a reST comment and all is well.

.. -*- mode: rst -*-

Post to Twitter Tweet This Post

Posted in General. Tagged with .

By admin
January 2, 2009

No comments

Sydney, Best City in the World?

Well, this post belongs in the rant/vent category, on the troubles trying to get out of the "Best City in the World".

After spending Friday assisting with an internal Architecture conference at The Swiss-Grand on Bondi Beach.

Things where going well, until the end of the day that is. Here is a long history of events:

  • 1730 - sms notification our 7pm Flight Cancelled
  • 1815 - next flight is 9pm
  • 1945 - Electrical Storm Hits, buckets down
  • Hail taxi, didn't take too long, except for treading water to get bags in the back
  • 2030 - board our doomed flight
  • 2115 - our plane is broken and we get kicked out
  • 2215 - at the other end of the airport we get shuffled on to a shiny new plane, when does that takeoff curfew end???
  • 2300 - crap the curfew is till 11pm, we are still on the runway, and my socks are really wet
  • phew, there is some other runway we can use that goes over the bay
  • boo, now it's windy
  • boo, now planes need to land
  • boo, now it's still windy
  • 23:?? - everyone gives up, we're screwed
  • 00:25 - we are back at the terminal and were stuck in the entry walkway, the doors are locked shut somehow and they can't open it up...
  • 00:40 - ok, we're inside. hmm, what is going on now, will virgin be finding us a hotel, now, since we are out of towners and they are partly responsible - No apparently not, they recommend we all line up and stay in the airport unless you have some friends you don't mind waking up and as long as you don't mind paying for a cab. 300 people, 1 staff member to reschedule everyone, that's going to be quick
  • hmm - we weren't the only plane, there are a few others around and about 3 qantas planes as well
  • 00:50 - we grab a cab, the first 3 hotels near by are chock full, apparently ppl have been grabbing accommodation for hours, BOO
  • We start ringing ahead for known hotels, hmm, still full
  • yay, finally we have a bed
  • 01:48 - yay sleep time
  • 07:30 - I ring virgin and arrange for another flight, bit of a problem, as far as they know I boarded
  • First flight I can get out is 1300, boo
  • 10:45 - Surely Virgin will give us complementary Lounge visit for our troubles
  • BOOO!!! - Virgin says No
  • Thumbs down to virgin
For me, besides missing my fam, I can just laugh it off, I feel really sorry for the unaccompanied minor who was also staying in Sydney for the night, poor little guy must have only been about 10.
What's going on with Sydney airport, how much slack is in the system for schedule slippage?
What were the real Cascading Failures (see Release It Post) , that led to the late night chaos, will I ever find out?

Post to Twitter Tweet This Post

Posted in General. Tagged with .

By admin
November 15, 2008

No comments

REST and ROA

REST and ROA in a nutshell

Post to Twitter Tweet This Post

Posted in Web. Tagged with .

By admin
February 8, 2008

No comments

Experimental Java Circuit Breaker Pattern Implementation

I read "Release It" recently, not a bad book, some good stories, I wish more people would pay attention to things like Stability and Capacity.

Anyway I liked the Circuit Breaker pattern, as I've seen a number of apps with highly coupled systems that barf once latency increases in the back ends then you get "Cascading Failures" and the webapp's request handler thread pool is exhausted and they all wait their mandatory 30 seconds for a response. Eventually the app either dies, or gets taken out of a load balance farm causing a "Chain Reaction".

Here is an Experimental Circuit Breaker Implementation (circuit_breaker.tgz) , NOT built or tested FOR PRODUCTION use in anyway. There is explicity no thread safety no the actual CircuitBreakerSimple class.

download, extract and run ant.

Play around with the parameters to see the affect. Generally it seems to thrash more with increased concurrent threads and interestingly in jdk1.6 sometimes reverts to the default timeout instead of the instance value.

Based roughly on the following logic:
When Circuit is Closed:
on call = pass through
call succeeds = reset count
call fails = count failure
threshold reached = trip breaker. Open State

when Circuit is Half-Open
on call = pass through
call succeeds = reset go. Close State
call fails = trip breaker. Open State

when Circuit is Open
on call = return/fail
on timeout = attempt reset. Half-Open State

Post to Twitter Tweet This Post

Posted in Patterns, Stability, Performance and Monitoring. Tagged with , .

By admin
December 15, 2007

7 comments

IE6 Operation Aborted not just for the DOM not being ready

Most discussions about Operation Aborted only mention when javascript is attempting to manipulate the DOM prior to it being ready.

I've seen it fail prior to any of your own js executing.

Try this for an experiment have an onload event log something but also return:

You have an large list of table results that are fed back to the browser with a slight delay per row (ie. an unfortunate logging statement per row)

What comes first?

Post to Twitter Tweet This Post

Posted in Web. Tagged with , , .

By admin
December 5, 2007

No comments

Web Directions South 2007

Highlights

  • John Allsop - openning comments
  • Sebastian Chan - Government are into mashups
  • Scott Berkun on the Myths of Innovation
  • Chris Willson - MS architect co-chair on HTML5 working group
  • Scott Buchanan & Ben Macguire - Harnessing the law to protect IP
  • Stephen Cox - Ethnography in the design process
  • Rashimi Sinha - Slideshare founder

What wasn't mentioned:

  • REST
  • java
  • testing

Google maps by Raul Vera

Bert Bos - CSS level 3, vertical text layout, like for japanese; new grid positions; advanced grid layout; rounded cornersScott Berkun

Great talk on the myths of innovation

Other worthy talks

tidbits:

http://www.sharedspace.org

http://www.slideshare.net/search/shiftlearning

http://www.earthfromabove.whereis.com

australian stock photos

Recommended books:

the double helix

vitruvius - tn books on architecture

Diffusions of innovation

Wisdom of crowds

emergence

ethnography for marketers

Post to Twitter Tweet This Post

Posted in Web.

By admin
December 4, 2007

No comments

Twitter links powered by Tweet This v1.8.1, a WordPress plugin for Twitter.