This is based on a cross post from http://www.innoq.com/blog/st/2009/01/link_and_self.html.

Subbu and <Stefan Tilkov have been discussing URI Equivalence, linking to self and Resource Identities, so here is my view.

Subbu's last remark asks the question

>>>
On January 18, 2009 4:32 AM, Subbu Allamaraju said:
Here is a longer response that is longer than a comment <img src='http://brettdargan.com/blog/wp-includes/images/smilies/icon_smile.gif' alt=':)' class='wp-smiley' />

http://www.subbu.org/blog/2009/01/uris-vs-identifiers-take-two

IMHO URI non-equivalence does not imply resource non-equivalence. And if that is really important to your application there SHOULD be ways to handle it.

I agree with Stefan on providing a canonical resource.

You can argue both ways that person and person with address book are either two representations of a person resource, or two different resources, that is the great thing about the Web.

For this case in the atom self link what about using the <b>rev tag</b> to identify the canonical resource that makes sense for your application.

Since rev also accepts space separated list of link-types you could mark it with both the type and the uri of the canonical resource.

<link href="http://www.example.org/person/abc?include=addressBook" rel="self" rev="canonical http://www.example.org/person/abc"/>

As to whether or not two different entities that returned from different URI are based on the same version of the canonical resource or not:

I would use an EntityTag that encoded some value of resource state and some value of the representation. Eg. template for xhtml representation may change without resource state and the ETag must change in order to reflect that.

To KISS.

If you had an ETag consisted of something like "resourceVersion=20,reprVersion={date}" Then your application could extract out the self links with identical rev tags and extract from the ETag the resourceVersion.

Tweet This Post

Looking at URI design, there are a lot of cross-cutting concerns across many a resource. It would be great to come up with some conventions for the most common.

These are some of the ones I've been percolating on.

You can see there are a broad range, including:

• different views or representations of a resource, around breadth/depth of it's data and relations
• user request based affects on a representation
• resource version
• application supporting mechanisms, returning fragments and forms or form fragments
• temporality

There are a few more, but the trick is having less, right.

See also ISO8601

Tweet This Post

I've come across a few posts about Matrix Parameters recently and there is a lot of misunderstanding.

There are several old posts that discuss and describe these things:

• Query Strings Considered Harmful
• w3c MatrixURIs

I've summarised what I think is important:

• urls with query params won't have their response cached by intermediaries/proxies (at present)
• matrix parameters may appear anywhere in path
• calculating the relative uri is different
• much more than uri convention
• query params are generally abused to add new verbs instead of using existing methods on resources
• matrix parameters are not resources, they are aspects that help reference a resource in an information space that is difficult to represent within a hierarchy

Tip: If you're not sure if you should use them then don't.

Matrix parameters are used with

• YUI paging widgets
• RoR /some/resource/id;edit
• J2ee fallsback to ;jsessionid=xxx when no cookies can be set.
• Jersey (JAX-RS RefImpl) has good support

Tweet This Post

I finished reading Neal Fords book "The Productive Programmer" [Ford2008] It wasn't quite what I expected, but none the less it was well worth the purchase.

While there are a few topics I agree with but not to the same extent as him, the topic I strongly disagree with is on the generation of large amounts of documentation.

Documentation in this form merely serves as another one of those checklist items on a project plan.

"Conveying understanding" should be the underlying purpose for which usually wiki pages or documentation is produced in order to communicate with others through time and space. Documentation is not just about information.

Consider your own experience when it comes to picking up documentation produced by others no longer working on the systems.

Most documentation is brain dumped with little thought given to the actual writing or conveying of information. Anyone can describe the complicated in a complex manner, it takes more skill to take a complicated details and make them simple to understand.

Large volumes of javadoc, don't help me, it is another layer of indirection and as Robert C. Martin says regarding code, "all comments are lies". Deep down everyone knows this, even though many people are still in denial.

Rarely did I get documentation that was:

• reasonably accurate in terms of what it was meant to do
• mostly it differed significantly in how it was actually implemented
• usually had typos in crucial lines that didn't aid understanding at all
• large gotchas aka time-bombs are not mentioned
• building and configuration where slightly covered
• the best you usually wish for is some clues for which you can pull out your best Dr. Holmes hat and get to work with

The choice of conveying understanding via documentation is one of the least effective mediums for communication. Refresh your memory of Alistair Cockburn's [CockburnAmbler]; analysis of various communication mediums and it's effectiveness.

Tweet This Post

Anyway I liked the Circuit Breaker pattern, as I've seen a number of apps with highly coupled systems that barf once latency increases in the back ends then you get "Cascading Failures" and the webapp's request handler thread pool is exhausted and they all wait their mandatory 30 seconds for a response. Eventually the app either dies, or gets taken out of a load balance farm causing a "Chain Reaction".

Here is an Experimental Circuit Breaker Implementation (circuit_breaker.tgz) , NOT built or tested FOR PRODUCTION use in anyway. There is explicity no thread safety no the actual CircuitBreakerSimple class.

download, extract and run ant.

Play around with the parameters to see the affect. Generally it seems to thrash more with increased concurrent threads and interestingly in jdk1.6 sometimes reverts to the default timeout instead of the instance value.

Based roughly on the following logic:
When Circuit is Closed:
on call = pass through
call succeeds = reset count
call fails = count failure
threshold reached = trip breaker. Open State

when Circuit is Half-Open
on call = pass through
call succeeds = reset go. Close State
call fails = trip breaker. Open State

when Circuit is Open
on call = return/fail
on timeout = attempt reset. Half-Open State

Tweet This Post

So Tom mentioned the ant-contrib tasks, Compile With Walls and VerifyDesign.

Compile with Walls is great, packages won't compile if layers are broken. It is marked as deprecated with Verify Design to replace it.

Verify Design works after compilation and can't detect the source code violations of constants imported from other packages.

Compile With Walls requires dependant packages to declare all dependencies and the dependencies of dependencies, which could get unwieldly, but still potentially useful. An option to not have to do that would be nice.

Tweet This Post

System Management Patterns Overview

Collect Correlated System Metrics

Gather and store correlated system metrics.

The essence of this pattern is enable collection of multiple correlated data points over time to provide a picture of the system components that can be analysed at all interesting layers. Enabling better informed business, architectural and design decisions about the system or it's components.

The necessity of this pattern grows as we handle the web's bursty traffic nature and rely on more distributed services. I cannot over state the significant value this provides that will flow through the organisation and it's partners.

How it Works

Identify what components make up your system, what dependencies there are and start from the bottom of the pyramid.

There are many mature tools and protocols that already provide monitoring at the Operating System level.

Depending on what container or virtual machine you have they may support the defacto industry standard for monitoring.

Track major data points within your application. For a Web based retailer there are particular areas that would be important, such as Inventory Searches, Purchasing, third party dependencies, like Payment System Providers.

Using It

• Collect correlated system metrics in real-time. Log file analysis has a longer feedback loop, can introduce correlation issues, easily lost or overwritten and doesn't scale well as more system components are added.
• Use it often, resolve problems with collection as a high priority, the value it provides decrease as reliability of results or collection.
• Prefer the collection of raw data points over aggregated data where possible as you need to make arbitrary choice of aggregation which may prevent the use of the metric with other metrics. If you were measuring response time for a service serving many concurrent clients, you could track, hitcount and total response time or you may track hitcount and avg. response time over what: period of time, or last x hits?>
• Think about coverage, do you have too much? Do you have too little? Don't get too much information, avoid paralysis by analysis.
• Think about your system, it is alive, it constantly changes, whether that is code, hardware, os or business usage, everyday is unique!
• Don't blindly trust the statistics you have, occasionally, independently verify important statistics, does not have to be exact, could use back of the envelope calculations.
• Serious usage may warrant a separate network.

Example

The technology to support the implementation of this has been around for years
and is mature in the infrastructure layer of a system. There are several network management applications that build upon SNMP and Round Robin Database Tool to collect real time data and specify a suitable granularity of aggregation to limit archive size, yet still allow queries to be performed.

SNMP monitoring has recently been added to the Java JVM itself. At the application level there are several snmp implementations that could be used to enable a poller, or statistics collector to query your app for certain metrics.

So it is possible to create a system view that spans the Operating System (cpu, disk, network adaptors), Virtual Machine, Container and your application.

Reap the Benefits

Knowledge of your approximate workload is the first step.

The benefits flow from the day to day operational aspects to the CIO assuming sufficient coverage has been implemented. Operations/Production Support have an increased ability to diagnose issues, they can identify cause and affect immediately after a change instead of guessing, to the strategic end of town. Better diagnosis of issues, may only be to the extent that significant time is not wasted looking in an area that is not causing the problem. In many cases it will highlight what component is or is not causing a problem and further analysis of the Correlated Component Snapshot needs to be investigated.

Developers can use those numbers to approximate workload and test large scale changes to the system, like changing operating systems, changing databases, doubling inventory etc.

• CIO, can get more confident answers to:
  What is the load sensitivity to the system, if we double inventory? What happens if we introduce a promotion that is likely to increase traffic 10 times for the length of the promotion? How much hardware do we need to purchase to support these loads?
• Technical Operations Manager, are my servers up? What happened at 13:00, to spike cpu usage by 20%? Is load within reasonable? limits? Are there errors on any devices? We changed disk arrays and performance has decreased?
• Production Support, should have a dashboard of important points usually spanning all layers. They can ask questions like: Hey we requested a that package xyz be upgraded and response time of service abc has doubled?
• Development Team, the real workload is alot different to what everyone anticipated. Do we need to focus on a different area? Do we need to add or tune caches?

Correlated Component Snapshot

Collect correlated detailed information of a System Component.

How it Works

Periodically collect detailed correlated information about your System Component . Let you travel back in time to see what happened to a component, maybe even to see the cause of the resulting affect. It is useful for diagnosing system problems in both production as well as determining and removing bottlenecks when Scaling Approximate Workload.

Using It

• Gather and store detailed correlated information of a System Component.
  Collected periodically based on a balance of component usage, gathering cost. 20 minute i is a good rule of thumb
• A listing of top running processes at that point in time.
• A database server, may have information such as locking transactions, long running transactions, database specific internal statistic reports.
• Typically recent data would be stored. Older data to be archived or removed.

Scaling Approximate Workload

Build tests that can approximate the real workload or your system.

How it Works

Having the ability to scale an approximate workload of your system that is monitored via Collect Correlated SystemMetrics provides the ability to really see what is happening to the system when changes occur. It provides the final metrics by which changes to components or workload can analysed and simulated.

This enables developers to simulate and answer "What If?" type of questions.

Using It

• Build read only tests and read-write tests of vital functions. Read-only tests will provide a further level of approximation without the extent of data management imposed by the read-write test.
• The tests must be written in a loosely coupled way, to enable simple scaling up of workload. They are intended to be used by a tool that allows simple manipulation of scaling parameters, such as number of clients, number of hits per hour etc.
• Approximate data usage as well as functionality usage. While randomness of data is important, significant data distribution should reflected in you most important tests. Don't randomly choose products if a product category represents significant proportions of browsing or searching. If 50% of products browsed are from 1 product category then weight it accordingly.

Ping

Is your service platform alive and what is the upper bound performance limit [Bulka] I could potentially achieve from a service on your platform at this time. This gives meaning to the efficiency of system components under real or approximate workloads when monitored via Collect Correlated System Metrics

How it Works

A simple service of your service platform that does nothing, but returns immediately with a suitable response. Requires that a Ping service on the service platform be implemented.

In the case of a Servlet this may be an empty html form with a 200 response code. In the case of a webservice it responds in a similar way, perhaps with application specific response codes as well. In the case of a virtual machine you may be pinging a synchronized resource, to check it's liveness [Lea].

Using It

• This can be used for internal components and more beneficially for external third party components.
• Very effective in helping to determine where limiting factors lay; virutal machines, operating systems, design or architecture of a system
• Dynamically adjust your system to reduce workload on third parties. As webservices adoption grows businesses will become even more reliant on responsiveness of third party partners. In an ideal world any third party systems will be able to handle any load your system can throw at it. But in site that is growing exponentially, partners may not have the inclination, contractual obligation or capabilities to scale as fast as your organisation. It is most likely the case that some messages are more important than others and that changing the workload you place on their system, or altering practices to accomodate cyclic degradation in their system could be very beneficial.

References

[Bulka] Java Performance and Scalability, Volume 1

[Fowler] Patterns of Enterprise Architecture

[Lea]
Concurrent Programming in Java(TM): Design Principles and Pattern (2nd Edition)

Tweet This Post