Another good event, great speakers, lots of language talks, agile, architecture, scalability, databases ala megastore. highlights for me, in no particular order.

• Avi Bryant's new project, great eye candy had the crowd go "ooooooooohhh". When will an api be open to plug other data in???

  • Also nice talk on VM history, how we are working on VMs from algo's designed in the 80's. And the features or standard of VMs aren't available for all our favourite languages.

• Tiersen , good intro to sharding and google approach to scaling out

  • liked the probability graphs
  • read chubby, logserve paper
  • higher level of jonas discussion, but jonas did refer to tiersen counter example
  • When the web came along, our apps were read mostly.
  • These days, social sites, are moving away from this scheme, to one of shared data. Some of that shared data is written to a lot. but it is updated, the majority of it is always inserts only.
  • write fan outs.
  • combine the two operations in one, so write + read all counters at same time
  • counter example too simple, can bypass optimistic locking as counter should be monotonically increasing
  • combine websites with fragments of data/entities with different shards and shard policies.

• Jonas - great talk re Megastore

  • Treat everything as an insert, duplicates will occur make sure they are idempotent
  • Updates are a little harder, but what is an update really, when can you say it is actually done?
  • Confirmed for me that a current strategy i'm implementing is right and will work (well for two, maybe three nodes anyway).
  • To get to more will need a decent Paxos implementation, which takes smart ppl and time.
  • consistency, availability, entity groups and big table
  • web apps more read/write than before, lots of shared data, but mostly additions
  • consistency, reliability, storage, scalability and megastore scale, entity groups, transactionality, avoiding joins
  • consistency: user trust, none, eventual, entity group, global
  • consistency: "a harmonious uniformity of agreement among things or parts"
  • Sharding by entity groups.
  • storage vs. cost of loss
  • paxos vs. 2pc
  • papers jonas recommended: pat helland paper; jinquan dai, intel, james hamilton, MS
  • I have a lot more to say about this topic, as I cut my teeth on Oracle Performance Tuning and it's architecture, always providing a READ CONSISTENT transaction at a minimum, I use to think was a great idea, until I wanted to scale it further.
  • Disk Reads are about 10,000 slower than memory access, but not if you have to manage a lot of versions of different blocks in memroy. The overheads reduce the read in memory to ONLY 10 to 100 times faster than a disk read. That just isn't enough. See Milsap papers on Oracle scaling there are a number of them and James Morle's book on Scaling Oracle 8i which is a great book, the older print version can still be picked up as well
  • The approach of one single db instance creates new problems, now we need transaction logs and a hot standby and a DR data centre...

• Nygard (Release It! )- i missed the first one due to a conflict <img src='http://brettdargan.com/blog/wp-includes/images/smilies/icon_sad.gif' alt=':(' class='wp-smiley' /> .

  • Great stuff from what i hear, I've been pushing for some time implementation of his patterns.
  • motivated a number of devs at my company. Hopefully I'll see some badly behaving webservices and stability pattern implementations.
  • Here is a simple Java Circuit Breaker Pattern Implementation from a ways back.

• mike cannon-brookes - had to fill a tough set of constraints, but interesting story about atlassian.

• Clemens

  • ms unified component thingy. simliar to osgi. Nice talk about issues with design, component composition and how all of IT boils down to composition of things at varying levels.
  • Discussion about component composition and how state is always a problem, yes.
  • Advocate of service use, not reuse as it should be used "as is"
  • I prefer service use over code/component, especially with a services developed with RESTful intentions

• Dan North, telling project war stories based on experience and * observations of good architects * soa gone bad, wsdl

  • Listen, Listen, Listen
  • technical problems aren't the biggest issue, silo communication
  • replacement of tools sqlserver to oracle, not solving the real problem
  • the nameless quality
  • vision, inspiration, enabler
  • project shaman
  • empathise
  • self belief, a sense of conviction and humility

• Joshua Bloch - great stuff

  • checkout MapMaker

• Eastman - apache mahut. Dirichlet clustering, hmm, that algorithm wasn't in Collective Intelligence

• Douglas Crockford

  • javascript inspired by self, scheme, perl and java
  • prefer === over == doesn't do type coercion
  • he no longer uses ++ and -- anymore, implicated in buffer overflow exploits
  • lambda, dyn objs, loose typing and object literals.
  • refreshing to discuss languages and language features again
  • the memoizer example was good, check the slides for recursion usage
  • use functions to make objects
  • functional inheritance
  • be rigorous and use jslint

I don't think I have confused HATEOAS with "connectedness", I have a reasonable understanding of resource state, application state and HATEOAS.

See http://www.stucharlton.com/blog/archives/000141.html and this discussion, http://tech.dir.groups.yahoo.com/group/rest-discuss/message/12497, which I see you have contributed to.

I think this final point made by Roy is pretty applicable here:

A REST API should be entered with no prior knowledge beyond the initial URI (bookmark) and set of standardized media types that are appropriate for the intended audience (i.e., expected to be understood by any client that might use the API). From that point on, all application state transitions must be driven by client selection of server-provided choices that are present in the received representations or implied by the user’s manipulation of those representations. The transitions may be determined (or limited by) the client’s knowledge of media types and resource communication mechanisms, both of which may be improved on-the-fly (e.g., code-on-demand). [Failure here implies that out-of-band information is driving interaction instead of hypertext.]

My point is you can't adhere to the HATEOAS constraint, without providing links between resources so the server can guide the client to other application states.

I agree that forms are a better example for explaining state transitions, especially since resource state is likely to be changed by providing forms (as will the User Agent, as Roy mentions); or equivalent form templates/prototypes for other media types.
But to me providing forms or equivalents for other media types is a different and easier problem to solve.

I'm very interested in how to evolve existing systems; I want to leverage the vast islands of information that already exist.

I just want to do it in an elegant way from a code and a web perspective, including the fact that uri's shouldn't change.

URITemplateAspects; LinkAwareModels and LinkTemplateProcessor

In a standard, layered design, you would have these components (see Image):

1. Code-On-Demand
2. View
3. Controller
4-6. Contains the Proposed Abstractions
7. Model
8. Persistent Store - Db; file; jcr; webdav
9. Proxies

Here are some options I have for adding links around some existing system (there are likely more)

Due to the time constraints, the Pros, will be sparse.

Option 1: Add Links via Code-On-Demand.

Add links to my html page via javascript.
I could take my existing html representation and use dhtml to add links.

Cons:
* code-on-demand not available for all user agents or may be disabled
* Proxies can't cache the entire representation
* leak of possible application states to clients that don't need to know
* Detailed knowledge of resource and subresources required to provide "link rich" representations
* Alternative media types may require alternative languages or structure to perform add the links. ie. use of xsl for xml; javascript with svg.

Option 2: Add Links via Views

Cons:
* We all know this is bad
* Detailed knowledge of resource and subresources required to provide "link rich" representations
* different views/media types require the same logic and detailed knowledge of the model
* different views/media types may require some application state knowledge. an admin user may have other links than anon.

Option 3: Add Links via Controllers

Pros:
* Can handle the inclusion/exclusion of links depending on the current application state, regardless of the media type to respond with

Cons:
* Detailed knowledge of resource and subresources required to provide "link rich" representations

Option 4: Add Links via Models

Pros:
* Detailed knowledge of the model and related models belong in the Model

Cons:
* Detailed knowledge of resource links aren't the domain of the models
* Not all possible related models are likely to be represented; to balance performance with types of usage, often some relationships are not modelled, so as to not eager load, very large graphs etc.
* Links are very static, The "next available" application state, still needs to be determined in another layer, like Controller

Option 5: Add Links via Persistent Store

Store partial links in the db

Pros:
* Db driven, easy to update
* Tables have relationships
* easy adoption path
* easy to link to external resources, or resources outside your db schema
* easy to share links at db level, if several systems integrate at that level
* Good for sparse relationships.

Cons:
* Update to change a link, will not play nice with intermediaries nor "cool uris, that don't change"
* Not great for consistent dense relationships, you probably want to model that differently. So with a Parent Child relationship, you wouldn't want something in addition to existing foreign key relations, to redundantly specify that to get to the children of a parent you have a list of /child/xyz links. but by having that specified in the URITemplateAspect then you are managing that once, close to the data.
* Relationships are not always defined in Foreign Keys
* Links are very static, The "next available" application state, still needs to be determined in another layer, like Controller
* Only a partial solution for more dynamic aspect of determining what application states make sense, still needs to be done in another layer.
* May be invasive, links stored in columns or other tables, no good if you've got a locked down schema.

Thoughts

Models shouldn't have relationships to hardcoded resources, like "/resource/50", but they could have a semantic relationship to "/resource/{id}" or "/resource/{id};role={role}" under certain conditions.

There is a place for such LinkAwareModels (Component no. 5), as long as the links are isolated and only contain semantic relationships, they are not urls.

The Links from Models to Resources are isolated and encapsulated within its own abstraction; which gives us component no. 6 URITemplateAspects (maybe just simplify this to LinkTemplateAspect).

Possible "Application States" are dependent on the request, even though a request must be STATELESS, possible "Application States" may still be determined by the security access; restrictions of media-types or client, which should all be part of the request. If you do stuff with user roles, then make sure you encode within your url.

Component no. 4 is our LinkTemplateProcessor(s) and its responsibility is to evaluate URITemplates within the LinkAwareModels/entities and to determine appropriate state transitions based on the request.

This leads us to the final Option ( well, for this post )

Option 6: Semantic relationships of a "Model to resources" definied via URITemplateAspect and a LinkTemplateProcessor to evaluate the relationships (URITemplates) and sub-entities to determine other "application states"

Pros:
* Detailed knowledge of the model and related models belong in the Model
* Detailed knowledge of the model to resource of interest is a concern isolated in the URITemplateAspects
* Any possible relationships to other resources can be defined
* Links are very dynamic. The possible state transitions can be determined by the LinkTemplateProcessor, but it can do so decoupled from the detailed relationships of the model to other resources.

The semantic relationships that are defined in the LinkAwareModels, can include uri and other attributes, like rel. The LinkTemplateProcessor can evaluate a lot of the current application state (but not all as the User Agent may have representations from other servers or representations that have been modified by Code-On-Demand) in combination with the semantic relationship to determine the ultimate value of the link to use (relative preferred, of course) and it can determine if that link is a valid state transition. Only if you are an admin do you see this link; or the link retains the current media-type used for this request; so a request for /country/AU.html includes links to cities.html, not just cities. The default negotiation might be something if that is not specified.

* The Processor can operate on objects, prior to rendering to a particular media-type, so if you have a single object model that can render to multiple media-types, you could have less code

Conclusion

What do you call, this, maybe Rich Interconnected Model View Resource RIMVR.
Rich in relation to having deep links within a resource representation and from the applicability of those links. That actual uri templates are evaluated late and with some context of the request and application state.

The use of these abstractions, in no way make an application RESTful, that is up to the developers, but hopefully by following some abstractions, like these we will get some clean code and some better understanding of RESTful APIs.

I have some old code, to show this working.
One day I'll get a chance to clarify terms and simplify the code

This is based on a cross post from http://www.innoq.com/blog/st/2009/01/link_and_self.html.

Subbu and <Stefan Tilkov have been discussing URI Equivalence, linking to self and Resource Identities, so here is my view.

Subbu's last remark asks the question

>>>
On January 18, 2009 4:32 AM, Subbu Allamaraju said:
Here is a longer response that is longer than a comment <img src='http://brettdargan.com/blog/wp-includes/images/smilies/icon_smile.gif' alt=':)' class='wp-smiley' />

http://www.subbu.org/blog/2009/01/uris-vs-identifiers-take-two

IMHO URI non-equivalence does not imply resource non-equivalence. And if that is really important to your application there SHOULD be ways to handle it.

I agree with Stefan on providing a canonical resource.

You can argue both ways that person and person with address book are either two representations of a person resource, or two different resources, that is the great thing about the Web.

For this case in the atom self link what about using the <b>rev tag</b> to identify the canonical resource that makes sense for your application.

Since rev also accepts space separated list of link-types you could mark it with both the type and the uri of the canonical resource.

<link href="http://www.example.org/person/abc?include=addressBook" rel="self" rev="canonical http://www.example.org/person/abc"/>

As to whether or not two different entities that returned from different URI are based on the same version of the canonical resource or not:

I would use an EntityTag that encoded some value of resource state and some value of the representation. Eg. template for xhtml representation may change without resource state and the ETag must change in order to reflect that.

To KISS.

If you had an ETag consisted of something like "resourceVersion=20,reprVersion={date}" Then your application could extract out the self links with identical rev tags and extract from the ETag the resourceVersion.

Looking at URI design, there are a lot of cross-cutting concerns across many a resource. It would be great to come up with some conventions for the most common.

These are some of the ones I've been percolating on.

You can see there are a broad range, including:

• different views or representations of a resource, around breadth/depth of it's data and relations
• user request based affects on a representation
• resource version
• application supporting mechanisms, returning fragments and forms or form fragments
• temporality

There are a few more, but the trick is having less, right.

See also ISO8601

I've come across a few posts about Matrix Parameters recently and there is a lot of misunderstanding.

There are several old posts that discuss and describe these things:

• Query Strings Considered Harmful
• w3c MatrixURIs

I've summarised what I think is important:

• urls with query params won't have their response cached by intermediaries/proxies (at present)
• matrix parameters may appear anywhere in path
• calculating the relative uri is different
• much more than uri convention
• query params are generally abused to add new verbs instead of using existing methods on resources
• matrix parameters are not resources, they are aspects that help reference a resource in an information space that is difficult to represent within a hierarchy

Tip: If you're not sure if you should use them then don't.

Matrix parameters are used with

• YUI paging widgets
• RoR /some/resource/id;edit
• J2ee fallsback to ;jsessionid=xxx when no cookies can be set.
• Jersey (JAX-RS RefImpl) has good support

One of the best aspects of RESTful orientated services is getting your resources connected, possibly even achieving HATEOAS. Making that jump with with legacy object models or data can be tricky, depending on how intrusive you want to be.

There is such a large amount of legacy data in various forms, usually with minimal links/relations that could be leveraged.

So it is a worthy challenge to try and do it in the least intrusive manner possible.

This is one approach that is reasonably decoupled from an existing legacy model.

There are three main parts to this technique:

1. Enhance your model objects so they have a placeholder/collection so you can accumulate links
2. Annotate your model at interesting points with link templates, around attributes or methods or classes
3. Use a custom ContainerResponseFilter class of jersey to scan for annotations of interest, then evaluate your link templates (at run time), using the data of your models and any request based information you may require.

I chose to enhance my model classes using AspectJ at compile time.

Download the source

mvn glassfish:run
./curl_test #will hit two urls

and you should see some results, with links like this:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<country xmlns:ns2="http://www.w3.org/1999/xlink">
  <links>
    <link rel="" ns2:type="locator"
        ns2:href="http://localhost:8080/hateoas4legacy/rs/cities/Brisbane"/>
    <link rel=""
        ns2:type="locator" ns2:href="http://localhost:8080/hateoas4legacy/rs/cities/Sydney"/>
    <link rel=""
        ns2:type="locator" ns2:href="http://localhost:8080/hateoas4legacy/rs/cities/Melbourne"/>
    <link rel=""
        ns2:type="locator" ns2:href="http://localhost:8080/hateoas4legacy/rs/countries/AU"/>
  </links>
  <cities>
    <city>
      <links/>
      <name>Brisbane</name>
    </city>
    <city>
        <links/>
        <name>Sydney</name>
    </city>
    <city>
      <links/>
      <name>Melbourne</name>
    </city>
  </cities>
  <isoCode>AU</isoCode>
  <name>Australia</name>
</country>

All countries Results

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<countries>
<country xmlns:ns2="http://www.w3.org/1999/xlink">
  <links>
    <link rel=""
        ns2:type="locator" ns2:href="http://localhost:8080/hateoas4legacy/rs/countries/AU"/>
  </links>
  <cities/>
  <isoCode>AU</isoCode>
  <name>Australia</name>
</country>
<country xmlns:ns2="http://www.w3.org/1999/xlink">
  <links>
    <link rel=""
        ns2:type="locator" ns2:href="http://localhost:8080/hateoas4legacy/rs/countries/USA"/>
  </links>
  <cities/>
  <isoCode>USA</isoCode>
  <name>United States of America</name>
</country>
</countries>

It is very crufty, but does work in the simple cases. I haven't had a chance to demonstrate all of the features yet.

I'm likely to further flesh out this example as it is interesting in other RESTful related ways.

I've seen it fail prior to any of your own js executing.

Try this for an experiment have an onload event log something but also return:

You have an large list of table results that are fed back to the browser with a slight delay per row (ie. an unfortunate logging statement per row)

What comes first?

• John Allsop - openning comments
• Sebastian Chan - Government are into mashups
• Scott Berkun on the Myths of Innovation
• Chris Willson - MS architect co-chair on HTML5 working group
• Scott Buchanan & Ben Macguire - Harnessing the law to protect IP
• Stephen Cox - Ethnography in the design process
• Rashimi Sinha - Slideshare founder

What wasn't mentioned:

• REST
• java
• testing

Google maps by Raul Vera

Bert Bos - CSS level 3, vertical text layout, like for japanese; new grid positions; advanced grid layout; rounded cornersScott Berkun

Great talk on the myths of innovation

Other worthy talks

tidbits:

http://www.sharedspace.org

http://www.slideshare.net/search/shiftlearning

http://www.earthfromabove.whereis.com

australian stock photos

Recommended books:

the double helix

vitruvius - tn books on architecture

Diffusions of innovation

Wisdom of crowds

emergence

ethnography for marketers

Both parties were interested in avoiding iframes and it's baggage.

Ideally we were looking for a solution that involved:

• minimal (html,css,js) code on affiliates pages. After all who would want deployment dependencies across organisations?
• dynamic json data served from our site
• long lived highly compressed components
• efficient use of connections to our server

We evaluated dojo and google web toolkit

Dojo won out, among it's many features it has:

• Nice Custom Widgets
• Cross Domain IO via ScriptSrcIO
• compression and packaging into a single js. create your own profile listing your dependencies and dojo buildscripts
• XDomain package loading. You need to build it from source, Although this wasn't required for us in the end.

Widget Issues

Unfortunately widgets are not completely self-contained. CSS is a bit of a problem. Typical custom widget will result in your css being inlined into your html widget, but this isn't required or always the bestway to go.

CSS relative image references are resolved relative to the host the base page was served from not our site.

We decided to keep our relative image references and have our stylesheet added to our affiliate pages dynamically.

Give namespaces are changing in new version we didn't create our own and just used dojo.widget...

Should Dojo be called Sumo? (says teagles)

Most ajax libraries are fairly heavy, you are looking at atleast 100K with the average closer to 200K to get simple io, events, effects and dom manipulation functionality. Disclaimer, it was a pretty quick scan a couple of weeks ago, of mochikit, dojo and scriptaculous.

After various packaging experimentation the Animation.js heavyweight seemed pretty core if you want to use custom widgets.

Why GWT wasn't for us

• binary format
• can't use 5.0 features
• not all features supported, ie new ArrayList(size)
• active monitoring
• components served off google
• ...

GWT does have some upsides though:

• testability
• small footprint
• cross browser compatibility verified by google