I imbue visions and ideas
One team or one deliverable at a time
The journey is long and difficult
The rewards are many

Success made possible
By conveying understanding
Persistence and boldness
My allies include Clarity; reason and spikes

Reality intervenes
Tradeoffs will be forged
Intentional not accidental
The team understand the goal
Where we can take things next time

Helping others learn
Challenge and stimulate me
Together we light paths forward
Casting aside
Perceived constraints of the mind

Continuous improvements we make
Eliminating waste as we go
Alien to the unenlightened
Liberating to the energetic few

Mistakes and conflicts will occur
Choose battles and timing
Many opportunities to learn
Sometimes the master
Always a student
My appetite for learning
Grows with time

Custodians of the future
Expand your mind
Use many lenses
Choose your journey
Evolve your systems
Deliver value with vigour
Take action now

- Brett Dargan

Not sure where this came from, I had a slighly poetic moment, not in the rhyming sense

Tweet This Post

Another good event, great speakers, lots of language talks, agile, architecture, scalability, databases ala megastore. highlights for me, in no particular order.

• Avi Bryant's new project, great eye candy had the crowd go "ooooooooohhh". When will an api be open to plug other data in???

  • Also nice talk on VM history, how we are working on VMs from algo's designed in the 80's. And the features or standard of VMs aren't available for all our favourite languages.

• Tiersen , good intro to sharding and google approach to scaling out

  • liked the probability graphs
  • read chubby, logserve paper
  • higher level of jonas discussion, but jonas did refer to tiersen counter example
  • When the web came along, our apps were read mostly.
  • These days, social sites, are moving away from this scheme, to one of shared data. Some of that shared data is written to a lot. but it is updated, the majority of it is always inserts only.
  • write fan outs.
  • combine the two operations in one, so write + read all counters at same time
  • counter example too simple, can bypass optimistic locking as counter should be monotonically increasing
  • combine websites with fragments of data/entities with different shards and shard policies.

• Jonas - great talk re Megastore

  • Treat everything as an insert, duplicates will occur make sure they are idempotent
  • Updates are a little harder, but what is an update really, when can you say it is actually done?
  • Confirmed for me that a current strategy i'm implementing is right and will work (well for two, maybe three nodes anyway).
  • To get to more will need a decent Paxos implementation, which takes smart ppl and time.
  • consistency, availability, entity groups and big table
  • web apps more read/write than before, lots of shared data, but mostly additions
  • consistency, reliability, storage, scalability and megastore scale, entity groups, transactionality, avoiding joins
  • consistency: user trust, none, eventual, entity group, global
  • consistency: "a harmonious uniformity of agreement among things or parts"
  • Sharding by entity groups.
  • storage vs. cost of loss
  • paxos vs. 2pc
  • papers jonas recommended: pat helland paper; jinquan dai, intel, james hamilton, MS
  • I have a lot more to say about this topic, as I cut my teeth on Oracle Performance Tuning and it's architecture, always providing a READ CONSISTENT transaction at a minimum, I use to think was a great idea, until I wanted to scale it further.
  • Disk Reads are about 10,000 slower than memory access, but not if you have to manage a lot of versions of different blocks in memroy. The overheads reduce the read in memory to ONLY 10 to 100 times faster than a disk read. That just isn't enough. See Milsap papers on Oracle scaling there are a number of them and James Morle's book on Scaling Oracle 8i which is a great book, the older print version can still be picked up as well
  • The approach of one single db instance creates new problems, now we need transaction logs and a hot standby and a DR data centre...

• Nygard (Release It! )- i missed the first one due to a conflict <img src='http://brettdargan.com/blog/wp-includes/images/smilies/icon_sad.gif' alt=':(' class='wp-smiley' /> .

  • Great stuff from what i hear, I've been pushing for some time implementation of his patterns.
  • motivated a number of devs at my company. Hopefully I'll see some badly behaving webservices and stability pattern implementations.
  • Here is a simple Java Circuit Breaker Pattern Implementation from a ways back.

• mike cannon-brookes - had to fill a tough set of constraints, but interesting story about atlassian.

• Clemens

  • ms unified component thingy. simliar to osgi. Nice talk about issues with design, component composition and how all of IT boils down to composition of things at varying levels.
  • Discussion about component composition and how state is always a problem, yes.
  • Advocate of service use, not reuse as it should be used "as is"
  • I prefer service use over code/component, especially with a services developed with RESTful intentions

• Dan North, telling project war stories based on experience and * observations of good architects * soa gone bad, wsdl

  • Listen, Listen, Listen
  • technical problems aren't the biggest issue, silo communication
  • replacement of tools sqlserver to oracle, not solving the real problem
  • the nameless quality
  • vision, inspiration, enabler
  • project shaman
  • empathise
  • self belief, a sense of conviction and humility

• Joshua Bloch - great stuff

  • checkout MapMaker

• Eastman - apache mahut. Dirichlet clustering, hmm, that algorithm wasn't in Collective Intelligence

• Douglas Crockford

  • javascript inspired by self, scheme, perl and java
  • prefer === over == doesn't do type coercion
  • he no longer uses ++ and -- anymore, implicated in buffer overflow exploits
  • lambda, dyn objs, loose typing and object literals.
  • refreshing to discuss languages and language features again
  • the memoizer example was good, check the slides for recursion usage
  • use functions to make objects
  • functional inheritance
  • be rigorous and use jslint

Tweet This Post

I don't think I have confused HATEOAS with "connectedness", I have a reasonable understanding of resource state, application state and HATEOAS.

See http://www.stucharlton.com/blog/archives/000141.html and this discussion, http://tech.dir.groups.yahoo.com/group/rest-discuss/message/12497, which I see you have contributed to.

I think this final point made by Roy is pretty applicable here:

A REST API should be entered with no prior knowledge beyond the initial URI (bookmark) and set of standardized media types that are appropriate for the intended audience (i.e., expected to be understood by any client that might use the API). From that point on, all application state transitions must be driven by client selection of server-provided choices that are present in the received representations or implied by the user’s manipulation of those representations. The transitions may be determined (or limited by) the client’s knowledge of media types and resource communication mechanisms, both of which may be improved on-the-fly (e.g., code-on-demand). [Failure here implies that out-of-band information is driving interaction instead of hypertext.]

My point is you can't adhere to the HATEOAS constraint, without providing links between resources so the server can guide the client to other application states.

I agree that forms are a better example for explaining state transitions, especially since resource state is likely to be changed by providing forms (as will the User Agent, as Roy mentions); or equivalent form templates/prototypes for other media types.
But to me providing forms or equivalents for other media types is a different and easier problem to solve.

I'm very interested in how to evolve existing systems; I want to leverage the vast islands of information that already exist.

I just want to do it in an elegant way from a code and a web perspective, including the fact that uri's shouldn't change.

URITemplateAspects; LinkAwareModels and LinkTemplateProcessor

In a standard, layered design, you would have these components (see Image):

1. Code-On-Demand
2. View
3. Controller
4-6. Contains the Proposed Abstractions
7. Model
8. Persistent Store - Db; file; jcr; webdav
9. Proxies

Here are some options I have for adding links around some existing system (there are likely more)

Due to the time constraints, the Pros, will be sparse.

Option 1: Add Links via Code-On-Demand.

Add links to my html page via javascript.
I could take my existing html representation and use dhtml to add links.

Cons:
* code-on-demand not available for all user agents or may be disabled
* Proxies can't cache the entire representation
* leak of possible application states to clients that don't need to know
* Detailed knowledge of resource and subresources required to provide "link rich" representations
* Alternative media types may require alternative languages or structure to perform add the links. ie. use of xsl for xml; javascript with svg.

Option 2: Add Links via Views

Cons:
* We all know this is bad
* Detailed knowledge of resource and subresources required to provide "link rich" representations
* different views/media types require the same logic and detailed knowledge of the model
* different views/media types may require some application state knowledge. an admin user may have other links than anon.

Option 3: Add Links via Controllers

Pros:
* Can handle the inclusion/exclusion of links depending on the current application state, regardless of the media type to respond with

Cons:
* Detailed knowledge of resource and subresources required to provide "link rich" representations

Option 4: Add Links via Models

Pros:
* Detailed knowledge of the model and related models belong in the Model

Cons:
* Detailed knowledge of resource links aren't the domain of the models
* Not all possible related models are likely to be represented; to balance performance with types of usage, often some relationships are not modelled, so as to not eager load, very large graphs etc.
* Links are very static, The "next available" application state, still needs to be determined in another layer, like Controller

Option 5: Add Links via Persistent Store

Store partial links in the db

Pros:
* Db driven, easy to update
* Tables have relationships
* easy adoption path
* easy to link to external resources, or resources outside your db schema
* easy to share links at db level, if several systems integrate at that level
* Good for sparse relationships.

Cons:
* Update to change a link, will not play nice with intermediaries nor "cool uris, that don't change"
* Not great for consistent dense relationships, you probably want to model that differently. So with a Parent Child relationship, you wouldn't want something in addition to existing foreign key relations, to redundantly specify that to get to the children of a parent you have a list of /child/xyz links. but by having that specified in the URITemplateAspect then you are managing that once, close to the data.
* Relationships are not always defined in Foreign Keys
* Links are very static, The "next available" application state, still needs to be determined in another layer, like Controller
* Only a partial solution for more dynamic aspect of determining what application states make sense, still needs to be done in another layer.
* May be invasive, links stored in columns or other tables, no good if you've got a locked down schema.

Thoughts

Models shouldn't have relationships to hardcoded resources, like "/resource/50", but they could have a semantic relationship to "/resource/{id}" or "/resource/{id};role={role}" under certain conditions.

There is a place for such LinkAwareModels (Component no. 5), as long as the links are isolated and only contain semantic relationships, they are not urls.

The Links from Models to Resources are isolated and encapsulated within its own abstraction; which gives us component no. 6 URITemplateAspects (maybe just simplify this to LinkTemplateAspect).

Possible "Application States" are dependent on the request, even though a request must be STATELESS, possible "Application States" may still be determined by the security access; restrictions of media-types or client, which should all be part of the request. If you do stuff with user roles, then make sure you encode within your url.

Component no. 4 is our LinkTemplateProcessor(s) and its responsibility is to evaluate URITemplates within the LinkAwareModels/entities and to determine appropriate state transitions based on the request.

This leads us to the final Option ( well, for this post )

Option 6: Semantic relationships of a "Model to resources" definied via URITemplateAspect and a LinkTemplateProcessor to evaluate the relationships (URITemplates) and sub-entities to determine other "application states"

Pros:
* Detailed knowledge of the model and related models belong in the Model
* Detailed knowledge of the model to resource of interest is a concern isolated in the URITemplateAspects
* Any possible relationships to other resources can be defined
* Links are very dynamic. The possible state transitions can be determined by the LinkTemplateProcessor, but it can do so decoupled from the detailed relationships of the model to other resources.

The semantic relationships that are defined in the LinkAwareModels, can include uri and other attributes, like rel. The LinkTemplateProcessor can evaluate a lot of the current application state (but not all as the User Agent may have representations from other servers or representations that have been modified by Code-On-Demand) in combination with the semantic relationship to determine the ultimate value of the link to use (relative preferred, of course) and it can determine if that link is a valid state transition. Only if you are an admin do you see this link; or the link retains the current media-type used for this request; so a request for /country/AU.html includes links to cities.html, not just cities. The default negotiation might be something if that is not specified.

* The Processor can operate on objects, prior to rendering to a particular media-type, so if you have a single object model that can render to multiple media-types, you could have less code

Conclusion

What do you call, this, maybe Rich Interconnected Model View Resource RIMVR.
Rich in relation to having deep links within a resource representation and from the applicability of those links. That actual uri templates are evaluated late and with some context of the request and application state.

The use of these abstractions, in no way make an application RESTful, that is up to the developers, but hopefully by following some abstractions, like these we will get some clean code and some better understanding of RESTful APIs.

I have some old code, to show this working.
One day I'll get a chance to clarify terms and simplify the code

Tweet This Post

Some comments about a standard "architecture" book of one pager pearls of wisdom.

There is not much too this book, then again, "less is more" .

#21 - An Architect knows something about everthing. An engineer knows everything about one thing.

"An architect is a generalist, not a specialist. "

This is consistent with becoming a Generalising specialist.

#29 Being process-oriented, not product-driven, is the most important and difficult skill for a designer to develop.

I take "process-oriented" in this context to mean improving your own processes and not the general interpretation of burdensome processes.

Pragmatically this "process-orientated" statment is about focusing on the process of design and improving those skills, in place of speed of product development.

Another aspect on this is balancing learning efficiency with task efficiency, which Jason Yip mentioned in a recent presentation.

Tip #29 has nine sub points, here are a few of my favourites:

• seeking to understand a design problem before chasing after solutions;
• making design investigations and decisions holistically
• working fluidly between concept-scale and detail-scale
• making design decisions conditionally - that is, with the awareness that they may or may not work out as you continue toward a final solution; See also "Considerations in communicating the Intent of Design".
• Knowing when to change and when to stick with previous decisions
• Always asking "What If...?" regardless of how satisfied you are with your solutions.

#45 Three Levels of knowing

Simplicity

world view of the child or uninformed adult.

Complexity

the ordinary adult world view. An awareness of complex systems in nature and society but an inability to discern clarifying patterns and connections.

Informed Simplicity

an enlightened view of reality. It is founded upon an ability to discern or create clarifying patterns within complex mixtures. Pattern recognition is a crucial skill for an architect, who must create a highly ordered building amid many competing and frequently nebulous design considerations.

#16/26 - Parti and Changing Parti

"Parti derives from understandings that are nonarchitectural and must be cultivated before architectural form can be born."

Parti has been in "architecture" for a long time, it also gets mentioned in ux articles.

At its most ambitious, parti derives from matters more transcendent than mere archtiecture.

I like the abstract notions behind it.

#48 If you can't explain your ideas to your grandmother in terms that she understands, you don't know your subject well enough.

"Some architects, instructors, and students use overly complex (and often meaningless!) language in an attempt to gain recognition and respect.
Try not to let them get away with it and whatever you do don't imitate them."

When this happens to you, be critical, instead o blind obedience, practice selective observerance

When you are accused of this, take the time to try and draw a picture or explain it in whatever form suits you best.

#51 Beauty is due more to harmonious relationships amount the elements of a composition than to the elements themselves

"Build a car out of the most beautiful features of teh most stunning cars ever made. See if your friends will be seen in it with you."

#78 "The success of the masterpieces seems to lie not so much in their freedom from faults - indeed we tolerate the grossest errors in them all - but in the immense persuasiveness of a mind which has completely mastered its perspective."

Virginia Woolf.

#98 The Chinese symbol for crisis is comprised of two characters: one indicating 'danger', the other 'opportunity.'"

Whille it sounds good, the interpretation is false, see: pinyin.info and wikipedia

Tweet This Post

This is based on a cross post from http://www.innoq.com/blog/st/2009/01/link_and_self.html.

Subbu and <Stefan Tilkov have been discussing URI Equivalence, linking to self and Resource Identities, so here is my view.

Subbu's last remark asks the question

>>>
On January 18, 2009 4:32 AM, Subbu Allamaraju said:
Here is a longer response that is longer than a comment <img src='http://brettdargan.com/blog/wp-includes/images/smilies/icon_smile.gif' alt=':)' class='wp-smiley' />

http://www.subbu.org/blog/2009/01/uris-vs-identifiers-take-two

IMHO URI non-equivalence does not imply resource non-equivalence. And if that is really important to your application there SHOULD be ways to handle it.

I agree with Stefan on providing a canonical resource.

You can argue both ways that person and person with address book are either two representations of a person resource, or two different resources, that is the great thing about the Web.

For this case in the atom self link what about using the <b>rev tag</b> to identify the canonical resource that makes sense for your application.

Since rev also accepts space separated list of link-types you could mark it with both the type and the uri of the canonical resource.

<link href="http://www.example.org/person/abc?include=addressBook" rel="self" rev="canonical http://www.example.org/person/abc"/>

As to whether or not two different entities that returned from different URI are based on the same version of the canonical resource or not:

I would use an EntityTag that encoded some value of resource state and some value of the representation. Eg. template for xhtml representation may change without resource state and the ETag must change in order to reflect that.

To KISS.

If you had an ETag consisted of something like "resourceVersion=20,reprVersion={date}" Then your application could extract out the self links with identical rev tags and extract from the ETag the resourceVersion.

Tweet This Post

So Tom mentioned the ant-contrib tasks, Compile With Walls and VerifyDesign.

Compile with Walls is great, packages won't compile if layers are broken. It is marked as deprecated with Verify Design to replace it.

Verify Design works after compilation and can't detect the source code violations of constants imported from other packages.

Compile With Walls requires dependant packages to declare all dependencies and the dependencies of dependencies, which could get unwieldly, but still potentially useful. An option to not have to do that would be nice.

Tweet This Post