JAOO 2009 – Brisbane

Another good event, great speakers, lots of language talks, agile, architecture, scalability, databases ala megastore. highlights for me, in no particular order.

• Avi Bryant's new project, great eye candy had the crowd go "ooooooooohhh". When will an api be open to plug other data in???

  • Also nice talk on VM history, how we are working on VMs from algo's designed in the 80's. And the features or standard of VMs aren't available for all our favourite languages.

• Tiersen , good intro to sharding and google approach to scaling out

  • liked the probability graphs
  • read chubby, logserve paper
  • higher level of jonas discussion, but jonas did refer to tiersen counter example
  • When the web came along, our apps were read mostly.
  • These days, social sites, are moving away from this scheme, to one of shared data. Some of that shared data is written to a lot. but it is updated, the majority of it is always inserts only.
  • write fan outs.
  • combine the two operations in one, so write + read all counters at same time
  • counter example too simple, can bypass optimistic locking as counter should be monotonically increasing
  • combine websites with fragments of data/entities with different shards and shard policies.

• Jonas - great talk re Megastore

  • Treat everything as an insert, duplicates will occur make sure they are idempotent
  • Updates are a little harder, but what is an update really, when can you say it is actually done?
  • Confirmed for me that a current strategy i'm implementing is right and will work (well for two, maybe three nodes anyway).
  • To get to more will need a decent Paxos implementation, which takes smart ppl and time.
  • consistency, availability, entity groups and big table
  • web apps more read/write than before, lots of shared data, but mostly additions
  • consistency, reliability, storage, scalability and megastore scale, entity groups, transactionality, avoiding joins
  • consistency: user trust, none, eventual, entity group, global
  • consistency: "a harmonious uniformity of agreement among things or parts"
  • Sharding by entity groups.
  • storage vs. cost of loss
  • paxos vs. 2pc
  • papers jonas recommended: pat helland paper; jinquan dai, intel, james hamilton, MS
  • I have a lot more to say about this topic, as I cut my teeth on Oracle Performance Tuning and it's architecture, always providing a READ CONSISTENT transaction at a minimum, I use to think was a great idea, until I wanted to scale it further.
  • Disk Reads are about 10,000 slower than memory access, but not if you have to manage a lot of versions of different blocks in memroy. The overheads reduce the read in memory to ONLY 10 to 100 times faster than a disk read. That just isn't enough. See Milsap papers on Oracle scaling there are a number of them and James Morle's book on Scaling Oracle 8i which is a great book, the older print version can still be picked up as well
  • The approach of one single db instance creates new problems, now we need transaction logs and a hot standby and a DR data centre...

• Nygard (Release It! )- i missed the first one due to a conflict <img src='http://brettdargan.com/blog/wp-includes/images/smilies/icon_sad.gif' alt=':(' class='wp-smiley' /> .

  • Great stuff from what i hear, I've been pushing for some time implementation of his patterns.
  • motivated a number of devs at my company. Hopefully I'll see some badly behaving webservices and stability pattern implementations.
  • Here is a simple Java Circuit Breaker Pattern Implementation from a ways back.

• mike cannon-brookes - had to fill a tough set of constraints, but interesting story about atlassian.

• Clemens

  • ms unified component thingy. simliar to osgi. Nice talk about issues with design, component composition and how all of IT boils down to composition of things at varying levels.
  • Discussion about component composition and how state is always a problem, yes.
  • Advocate of service use, not reuse as it should be used "as is"
  • I prefer service use over code/component, especially with a services developed with RESTful intentions

• Dan North, telling project war stories based on experience and * observations of good architects * soa gone bad, wsdl

  • Listen, Listen, Listen
  • technical problems aren't the biggest issue, silo communication
  • replacement of tools sqlserver to oracle, not solving the real problem
  • the nameless quality
  • vision, inspiration, enabler
  • project shaman
  • empathise
  • self belief, a sense of conviction and humility

• Joshua Bloch - great stuff

  • checkout MapMaker

• Eastman - apache mahut. Dirichlet clustering, hmm, that algorithm wasn't in Collective Intelligence

• Douglas Crockford

  • javascript inspired by self, scheme, perl and java
  • prefer === over == doesn't do type coercion
  • he no longer uses ++ and -- anymore, implicated in buffer overflow exploits
  • lambda, dyn objs, loose typing and object literals.
  • refreshing to discuss languages and language features again
  • the memoizer example was good, check the slides for recursion usage
  • use functions to make objects
  • functional inheritance
  • be rigorous and use jslint