matt.geek.nz

good onyas

New Zealand just got its own version of the online Oscars.  Last night’s Onyas ceremony, organised by the rock star Webstock crew and held at the Wellington Town Hall, was a classy event, with a red carpet at the entrance and a whole crowd of very clever people dressed up in ‘geek formal’ which, as it turns out, is anything from a dinner gown or tuxedo through to the Silicon Valley jeans and blazer combo.  The Town Hall looked amazing, with glorious lighting, great music and was amazingly well set up, considering that they’d only managed to clear out the Webstock conference goers an hour and a half earlier. Russell Brown was an excellent choice as compère, and Che Tamahori of Shift gave a great opening address which set the tone for the event: these awards matter because they’re being judged by people who work in this industry, at the coalface, every day.

Xero won three awards: Best Web Application, Best User Experience, and the supreme award of the ceremony: Most Outstanding Website. Craig Walker, CTO, accepted the Web Application award and has hopefully started a tradition of double entendre in acceptance speeches with his double entry wisecrack. Acceptance speeches were limited to 140 characters or less: adhering to the limitations imposed by Twitter, which made it easy to tweet your acceptance speech after you’d delivered it.

There was some strong competition.  Pocketsmith are doing very clever things with personal finance planning, Digital NZ and NZ On Screen are remarkable initiatives to preserve our digital treasures and present them in a compelling way, Eventfinder is a fantastic events database with an innovative heart, and Powershop are doing their bit to shake up the notoriously intractable electricity industry: which netted them the Most Innovative award. Every finalist was well chosen and as they played the showreels it was clear that they all deserved to be there.

After the ceremony, we were treated to a light show that was projected on to the interior ceiling and pipe organ of the Town Hall.  It transformed the room into, alternately: a diabolic cathedral, a steampunk gasworks, a blazing inferno, an underwater wonderland, and a galaxy of stars.  It was spectacular.  To see something similar you could check out this video of a recent light show in Hamburg, Germany.  It’s not quite the same, but it’s a similar concept: using projected light to transform architectural features into a shifting canvas.

A fantastic end to a fantastic conference.

wellington underground

Today I realised that you can actually travel a fair distance through Wellington completely indoors. It is possible to start from Hunter Street, enter a doorway, and to re-emerge in any of the following places without passing through an outside area:

• Victoria St
• Willis St
• Boulcott St
• Extra for experts: The opposite side of Lambton Quay

It's probably not news to many Wellingtonians ("Congratulations, Matt, you've discovered a mall.") but it was news to me, and kiwis are so starved of subways and underpasses that this sort of thing is a novelty.

This got me wondering: how far could you get from the corner of Hunter & Lambton Quay without crossing a road?  The answer, I think, is quite far. You can get from that point to Clifton Terrace, the corner of Lambton and Bowen Street, or to the corner of Boulcott and Willis.  What do you think, reader? Would it be possible to get any farther?  And what's the optimum route, from say, the corner of Lambton and Bowen all the way to Boulcott and the Terrace, where the following conditions are favoured in descending order:

1. Route does not cross a road 
2. Route or partial route avoids driveway entrances and carparks
3. Route or partial route is indoors
4. Route or partial route is under an awning

This map illustrates what I think are the extremes of where you can get to without crossing a road. (This is of course trumped by the waterfront: you can travel kilometres, though most of it is in the open air.)  Is the map correct?  Are there any underpasses that I've missed which open up a new block?

Useless information, I know, but maybe it'll be handy in the next zombie apocalypse.

pecha kucha wellington #6

I've worked out that there's no point in going alone to events with door sales. You have to time it right. Too late and you miss out, too early and you have to either foist yourself on people you only vaguely know (unfair) or stand in a corner pretending to read e-mails (uncool). Much better are ticket sales where you can turn up as people enter the theatre and avoid all the drama of forced networking. The idea of networking at Pecha Kucha is kind of hopeless anyway: most people arrive in groups and are just waiting for their seats. It would be a bit like trying to network at a stage play.

The event itself was good. Pecha Kucha is a killer format. A range of speakers, with each speaker presenting in front of twenty slides of their own choice with twenty seconds per slide. Last night there were writers, designers, artists, public speakers and architects, and a good mix of male and female perspectives too. There was some great work on display. Some speakers perhaps tried to cram too much into the few minutes that they had: there were ideas that deserved longer lectures as when abridged the subtleties were lost, and the grand ideas just seemed pretentious. And there were one or two speakers who lacked any sort of thesis at all: or worse, had only the most marginal of theses but attempted to disguise their tenuous conjectures under a veil of academic whispers. Bad move with such a savvy audience: they saw right through it.

But that's the nature of the format. You take the good with the not-so-good, and the not-so-good aren't around for so long that you get bored. It's fantastic. And the Downstage was the perfect venue for the Pecha Kucha. As Robyn pointed out, the set for Biography of My Skin was perfectly suited to the Pecha Kucha format.

There was a sense of display and a desire for validation that gave the whole thing an air of 'Creative Idol'. It wouldn't have been too out of place to have had a panel of judges (I'm nominating Hamish Keith, TJ McNamara and Dick Frizzell, for what it's worth) giving criticism in soundbites and choosing qualifiers for the next round. "This farrago of idle vanity and whim has no place in a gallery. Begone!"

Yeah, that's how I imagine art critics talk. At least, I hope they do.

I heard a couple of people complaining about the amount of self-promotion. At first, this bothered me too, but after reflection I realised that a) I would probably do the same and b) you'd have to be extremely fortunate to be successful as a creative in New Zealand without a thick skin and a drive to promote yourself. There's something in self-promotion that rankles the kiwi psyche.

What really stuck with me overall with most speakers (and perhaps this was on my mind from seeing the Yasoi Kusama exhibit earlier in the day) was the obsessive nature of the artist, the way in which a person can have an idée fixe and take it to such interesting and counterintuitive places, and above all, see it through to a work or works. My favourite speaker was the very last one, Meighan Ellis, an artist whose obsession was obsession itself. She described her art as a form of lepidoptery, and the word was well chosen: there was something of The Collector in what she was doing: it was creepy and captivating. I wouldn't mind sitting for her if I wasn't sure that I'd be too old and bereft of all those boyish qualities that she is singularly fascinated with.  I hope she exhibits soon.

three late film festival additions!

Morris

Gonzo director Harvey Boyce returns with this heartwarming look at the misunderstood world of Morris dancing. Boyce traverses the globe uncovering pockets of Morris enthusiasts in England, Canada, Sweden and Hong Kong. He steps behind the glamour and uncovers some of Morris dancing's great characters: Woolly, a German expatriate taught to Morris by the British soldier who shot his arm off in World War II; Alice (formerly Hassan), a post-op transsexual who had to leave her glittering Morris-dancing career behind to become a woman; and the Welsh Border Morris enthusiasts, a group that traditionally danced in blackface but who now find their favoured style under threat from an increasingly politically correct Britain.

Loveguts

Another controversial sensation at this year's Cannes Festival, Foley Gould's fetish epic invokes other sex-shockers such as Cronenberg's Crash and Chéreau's Intimacy. Ingenue Salome is drawn into an underground world of "navel-gazers", a disturbed community obsessed with sexual umbilicoplasty.  Hunted by the law and misunderstood by the world at large, the belly-bandits are forced into dangerous amateur surgical practices, culminating in the spectacular evisceration of Hugo: the hairy and obese tummy-gimp on whom their fragile world centres.

Preservar Mi Corazón (Preserve My Heart)

An evocative Spanish masterpiece that channels the visual effulgence of Almodóvar and the absurdist lyricism of Buñuel. When her grandmother dies, hard-as-nails city-girl Isabella inherits a Navarrese preservery in unremarkable Celigüeta. Isabella discovers spiritual reward amongst the pickles and jams of her ancestors, and when banderillero Javier returns home to Celigüeta in disgrace, love blossoms amongst the jars and demijohns of Isabella's new home. As Javier's attentions macerate Isabella's toughened heart, the lovers find hope for a future together: if only they can outrun their encroaching pasts.

presentation, then ...

Yesterday I attended Garr Reynolds' Advanced Presentation Zen workshop. I had attended his first workshop in 2008, and loved it, so I was very excited about the advanced session.

This was a refresher course. Although there was nothing particularly new in this class, it was great to have an opportunity to think everything over again, especially after having a year to apply some of Garr's ideas in practice.

His message is that the vast majority of presentations can be more engaging, and that when you incorporate design, story and emotion into your presentations you can do a much better job of persuasion than you can with badly designed slides covered in words and data. And although having slides that augment your message (rather than flat-out documenting it) is an improvement, your delivery is hugely important too.

In essence, it's the difference between a dry PowerPoint yawn-fest full of bullet points and verbatim delivery and compelling presentations like a TED talk or Al Gore's An Inconvenient Truth.

Garr went into some of the specifics around how great presentations are constructed, with some nods to the work of Duarte Design (who prepare talks for some of the world's best speakers), describing a process that involves going analog: brainstorming, chunking, storyboarding (preferably with post-its for easy re-ordering), and only then moving to slideware (PowerPoint or Keynote).

As part of this you would also create both an audience map (what are they like, why are they here, what keeps them up at night, how can I solve their problem, what do I want them to do, how might they resist?) and even a speaker map (what is their perception of me, how do I establish credibility without arrogance?).

In winning over your audience and creating sticky ideas, he referenced Daniel Pink's A Whole New Mind, which posits six traits that which future workers must utilise in order to thrive in an age of concepts: Design, Story, Sympathy, Empathy, Play and Meaning.

In relation to story: he recommended Robert McKee's renowned Story as a great primer (although aimed at screenwriters, it contains truths universally acknowledged amongst storytellers in many other disciplines) and pointed out Isabel Allende's 2008 TED talk Tales of Passion as a great example of story in action. "What is truer than truth? Story." He also singled out Steve Job's 2007 MacWorld keynote as an example of withholding information in order to build anticipation.

Regarding design, Garr's view is that design thinking is one of the most important differentiators you can have. We spent some time reflecting on the qualities that designers have (or stereotypically have) and the benefits to communication that effective design brings, through the use of things like contrast, simplicity and emotion.

I do wonder whether those students who had been in the first workshop only a day earlier would have got as much out of it as those of us who had had the benefit of a year's worth of reflection. And, perhaps surprisingly for a Presentation Zen Master, Garr had to jump through large chunks of his material. This was okay though, Garr is very much about 'naked' delivery, relying less on the strongly scripted approach and channeling the adaptive, reflexive approach of jazz musicians and stand-up comedians. And he talked about how to deal with situations where you have prepared an hour's worth of material and have been told you only have ten minutes, so his own presentation provided a good example in practice!

In short, the workshop was a recap and review of ideas presented in his book: Presentation Zen, but with the benefit of having Garr provide some more recent examples and some practical tips and pointers. Is it worth it? In truth, most things he covered are conveyed in his book, and in greater depth. But personallly, I found that the opportunity to spend six hours in the company of the author and talking through his ideas with others extremely valuable, and galvanic in a way that only good presentations can be.

better than book club

One of the things I look forward to is a semi-regular get together with friends that's centred around music. We call it a Music Appreciation Society, and the concept's pretty simple. About a week before we meet, we select a theme (everyone gets a turn) and then we all bring along three songs that have some relationship to that theme. On the night, we play a song each, until all the songs have been played, maybe with a spiel about how the song relates to the theme.  We also have dinner and a few glasses of wine. It's great. And it's much better than book club because it's a much smaller time commitment than it is to read a book that one person might like and the rest will hate.

You could consider it a collaborative version of Bob Dylan's Theme Time Radio Hour.

Our group has fairly diverse tastes: there's a guy into hip-hop, reggae and alt-country, a girl into 80s hits and female singer-songwriters, another who's into show tunes, ballads and classical, and then there's me, who's pretty much into anything, but I probably focus more on alt-rock and experimental more than anything else. Past themes have included Time, Face Cards, War, Colours, Animals, and Heroes.

Here's the tracklisting for our Covers night:

• This Land Is Your Land - Bruce Springsteen and the E Street Band (Woody Guthrie cover)
• I Just Don't Know What To Do With Myself - The White Stripes (Burt Bacharach cover)
• Ceremony - Radiohead (New Order cover)
• When Doves Cry - The Be Good Tanyas (Prince cover)
• Everybody Wants To Rule The World - The Bad Plus (Tears for Fears cover)
• Straight Outta Compton - Nina Gordon (N.W.A. cover)
• Hand On Your Heart - Jose Gonzalez (Kylie Minogue cover)
• You're Gonna Make Me Lonesome When You Go - Ben Watt (Bob Dylan cover)
• Superstar - Sonic Youth (Carpenters cover)
• Fast Car - Xiu Xiu (Tracy Chapman cover)
• Helpless - k.d. lang (Neil Young cover)
• Cars - Katzenjammers (Gary Numan cover)
• Love Will Tear Us Apart - Jose Gonzalez (Joy Division cover)

Our next theme is Sex. Any suggestions?

abel tasman: day 4

Breakfast in the morning is chaos: tables are unwiped, the brewed coffee is spilt and forming rings around all the unused mugs. We check out, and are confronted with the bill for our three nights. We feel a bit short-changed overall: the level of service was not commensurate with the prices we were charged. It's tempting to say that an easygoing attitude to kiwi hospitality is fine, and that this should not excuse inattentiveness and complacency, but most of the staff were actually foreigners, and I suspect that it's really mismanagement that is to blame. That said, the food was good, and the setting was spectacular. While waiting for our transport back to Kaiteriteri we sit in front of the open fire and read.

The taxi is full, and we head straight back to Kaiteriteri, stopping at Tonga Island for another look at the seals and at Torrent Bay to drop off a few day trippers. Off the coast of Sandy Bay the skipper turns down the engine because of a 'strange vibration'. He is unable to fix it, and is worried about the drive belt. He proceeds at a slower speed, but five minutes later there is a load bang, and the drive belt has snapped. There is a lot of racing about, and then he tries a secondary motor, but it gets us moving no faster than we are drifting, so he turns off the motor and lets us drift back towards Kaiteriteri.

Another taxi shows up in half an hour: offering to take the most urgent passengers back to Kaiteriteri. We have a bus to catch, so we, along with an American family, take up the offer and transfer boats.

We arrive back at Kaiteriteri, and head to a cafe there. The food is bad, but it is a nice setting, and I amuse myself listening to a salt-of-the-earth granddad dote on his grand-daughters. He talks about shares with his son and the way things used to be in Abel Tasman before things got all 'upmarket'. He makes plans to take his family out on one of the larger taxis one day. As he talks, our formerly stranded taxi limps into shore and the passengers we left behind disembark.

A short bus ride later, and we're back in Nelson. We go in search of another hot chocolate for L, and then as a concession to me we end up at the Mac brewery over the road for a couple of beers. We watched the locals do their thing: they are friendly but fearsome with their tattoos and beanies. A couple of bikies stop in for a drink before heading out on their hogs. A bloke brings in his younger brother for a beer, and they attempt to chat up a couple of German tourists. Two people seated outside appear to be drinking faster than anyone else: they take turns ducking inside for the next round, a black-singleted thug and his shifty-faced companion. While all this is going on, L finishes her book.

At five we catch a shuttle. The driver despairs of the new ownership of the lodge, and preferred the days before it got all 'upmarket', when it was just a stall in the bush that sold scones. We caught our flight and then got back home to our cat, who is standoffish to begin with, but it is not long before all is forgiven, and we are greeted with great affection.

abel tasman: day 3

In the morning the phone rings: apparently a kayaking guide has been found who will take us. L and I get dressed and grab a quick breakfast (where I have to tell the wait staff to see to the coffee) before meeting with the other three kayakers at reception, the older woman from the night prior and a young honeymooning Australian couple. We are told to walk to Onetahuti Beach to meet our kayaker guide, so we all set out together.

After forty minutes of walking through beautiful native bush we emerge at the top end of Onetahuti Bay, only to find that the high tide blocks our passage with the inward flow of a deep lagoon. It's short enough to swim across, but we're unsure of whether to wait or send someone ahead. As we consider this, we spot a lone pied shag fishing in the lagoon. It emerges every thirty seconds or so in a different spot. Eventually it emerges quite near us with a fish in its beak. It disappears, only to be replaced by a lone seal pup who is also fishing, though with less success and a greater amount of time spent submerged.

After thirty minutes, and a mere five minutes away from my resolution to swim across to find help, an water taxi appears and pulls into the lagoon to pick us up. He takes us on to the beach proper where we meet J, a charming kayak guide who is delighted to see us and apologetic that neither he nor us were given clear instructions on how to deal with the tidal situation. In any case, he gives us a quick safety briefing and we get in our kayaks and head out towards Tonga Island. L is in the front of my kayak, and I try to synchronise my strokes with hers, but they seem to me to be following their own erratic, half-distracted rhythm. At the island we watch as seals caper in the high water, while their mothers sun themselves on rock shelves. J talks about the history of the area: the Maori settlements, the first European contacts and their gradual displacement of Maori, the clearing of bush for farmland, the capitulation and return to immaculacy. On the shaded side of the island, less active seals sleepily watch us drift by.

We cross back to the mainland and follow the coastline south, past Mosquito Bay and into Bark Bay, wending our way in and out of rocks. We land in a small bay south of the main beach, where we pull our kayaks up into the sand and grab our packed lunches. We have a muffin and a sandwich each, though I swap my muffin for L's sandwich, as the carrot it contains puts L off the sandwich completely.

J tells us about some of the fauna, and the strange gossamer threads that can be found on some of the black trunked trees: these are the anal filaments of a particular type of insect, and the small bead of liquid at the base is their waste: a single drop of sugary water.

Our water taxi hasn't shown up to pick us up, so J gets on his radio to find out what is going on: the transport has been misbooked, and we aren't scheduled to be picked up until 2:30. It's 1:30, so we have an hour to kill. J's ride arrives however, so he leaves us on the beach. We walk around to the Bark Bay beach proper, passing through the forest along the way, where we find the anal filaments that J mentioned. I gather a few and try them: sugary, sticky and sweet. A few hundred and you'd have enough for a snack.

The taxi finally arrives, and we head back to the lodge. We pass through reception and the girl on the counter looks ready for a scolding, but I resist the temptation. We make a dinner reservation instead.

At dinner we find a table, and settle in, only to be told after five minutes that we need to move: apparently the table had been reserved for someone else (though there was no reserved sign to speak of). Once reseated, L had a steak, and I try the special: butter chicken, which is actually pretty amazing but lacks a naan bread: it is served only with rice. We skip entrees because of the Good Friday surcharge, which seems like a rort given the price of the meals and the rooms.

After dinner we borrow a torch and go in search of glow-worms. We find the grotto: it is dark and it has been a long time since I have been out in the wilderness in the darkness. With the torch off the glow-worms gradually begin to manifest: three or four brights ones, or are they merely a drop of moonlight on a serendipitously placed leaf? We aren't sure.

We head back, and as I cross into the lodge to hand my torch back, I hear noises. I swing the torch around the ground, and then up behind me. A possum is staring out at me from the tree, munching on conifer needles: black eyes, pink nose, and big ears. As I move closer he disappears back into the tree.

abel tasman: day 2

Our first morning in Abel Tasman is cold and misty. We wake and head to the lodge to get a cooked breakfast. It's pretty generous, but still overpriced. We cheekily smuggle out a pair of muffins to cover us for lunch: we feel as though we've paid for it.

At reception we inform the staff that our electric blanket is problematic too, so they promise to fix it. We tell them of our plans to walk to Mutton Cove and back, they suggest that the tides won't let us do that: we need to catch a water taxi up to Mutton Cove and then walk back, by which time the tides on the Awaroa inlet will allow us to cross it.

The staff on the desk are able to organise for us to catch a water taxi in the next quarter of an hour, so we accept.

On the water taxi, an older woman argues with the skipper. She has been told by someone at the lodge that she can catch a 4:00 boat back from Bark Bay, the skipper tells her the 4:00 boat is no longer running. It's an ugly situation, with the skipper telling the woman that she has plenty of time to catch the 2:30 boat, that his grandparents did the walk in sufficient time, but she is insulted and angry, and decides to go back to the lodge to confront the management. So we turn back after ten minutes and drop her off.

After setting out again we get to Mutton Cove, which is beautiful little bay about four hours north of Awaroa. We are dropped off with an American girl who is travelling alone: she sets out ahead of us while we are busy drying our feet from the landing. We walk fifteen minutes northwards through dappled forest, passing a Department of Conservation worker who is clearing a path with a pickaxe. We emerge at Separation Point, which is reached by a narrow path carved high up into the side of a bluff, with a steep climb down to a lighthouse. As we walk the path we hear the sounds of cavorting seals below us. In the distance we can see Farewell Spit.

We head back south, following the bush through the forest and out along the beaches. The track drops out of the bush at the top end of each beach, and we walk along the beach and rejoin the track at the bottom end. We stop in Anapai Bay to eat our muffins, along with a couple of easter eggs and a few potato chips. A lone gull eyes us gamely while sandflies gorge themselves on us.

We head back through the bush, peeking out through the trees into each departed bay, and descend into Totaranui, where the tide is almost completely out, but not wholly, and I have to carry L across the mud. At the Totaranui motor camp we fill our water bottle. Following our rule of looking for the track at the bottom of the beach, we lose it, and at L's behest we climb around the rocks from Totaranui into Goat Bay: I'm a little nervous about being caught out by the tide but at this point it's still going out so we have time on our side. We head back into bush to get to Waiharakeke Bay, where the American girl, who we thought was in front of us, appears behind us. A short walk through the bush and we get to the Awaroa Inlet, which is empty but requires us to take shoes off three times before we cross completely. The water forces us to take shoes off, the vast swathes of empty shells force us to put our shoes back on. The empty estuary is full of variable and pied oystercatchers, spearing their beaks into the wet mud in search of sustenance. L tires of taking off her shoes so I carry her over the last few muddy rivulets.

At the lodge we try to organise kayaking, but the management tell us that it's 'uneconomical', so we go to our room dejected and trying to come up with alternative plan for the next day. I head into the lodge early for a couple of beers by the fire. L joins me and we have dinner: her the salmon, me the steak. After dinner L has a chocolate torte by the fire while I nurse a whiskey that has been served on the rocks, in spite of my requesting it neat. An older woman asks us if we'd like to go kayaking: she has been trying to organise it too but has also hit a wall with the management. She conspires with us to leave a note at reception.

abel tasman: day 1

Our holiday began inauspiciously with an alarm that didn't go off. It put us thirty minutes behind schedule and we struggled to catch up.  We remembered most things, but in our rush we forgot the pasta that we planned to have for lunch, and the wine we planned to have after dinner.

Nevertheless, we arrived at the airport with ten minutes to spare, and I got one of Mojo's Manuka Smoked Bacon sandwiches to tide us over until lunch time: they're really good.  Even L, who 'hates sandwiches', really liked them.

On the flight I begin my book: Ogilvy on Advertising, which I immediately take a liking to.  L seems to as well, she sneaks glances at it from the pages of her book: the full colour ads draw her in.

We arrive in Nelson and get a shuttle into town.  We have an hour to spare so we go to a small cafe on Trafalgar Street, where L orders a Chili Hot Chocolate.  It's served with flakes of chili: a really weird flavour but L loves it.  She also has a carrot cake which she likes, but we're still hungry, so we cross the road and head to another cafe where we order bacon bagels. We're the first customers of the day and unfortunately the bagels are terrible: they're stale and hard and the cooked bacon is freezing cold. We don't have time to make a scene so we leave them half eaten and head up to the Nelson Christ Church Cathedral to enjoy the view before going to catch our bus.

I read all the way to Kaiteriteri so miss the view from the bus: I'm pretty absorbed with the book. L looks out the window.

There's a bit of confusion when we arrive: we are confronted with three different water taxi stalls lined up beside each other. I consult my notes and manage to find the right one. We pay and go to sit on the beach to await our ride.

We get on the water taxi and head north, skirting around Split Apple rock along the way. We visit Pinnacle Island, which is covered in spotted shags and has a small rock pool in which juvenile seals frolic. We also pass Tonga Island, where larger are seals are sprawled over rocks. Finally we head into the Awaroa inlet. By this point L is a bit nauseous, and when we're told to that we have to wade into shore, she grimaces. "Or," says the driver, "you can wear these." The driver produces ridiculous orange waders which slip over shoes, legs, everything. L and I put them on and we head into shore.

After checking into our room, we head into the main part of the lodge to get lunch. L orders a salad and I get an eye fillet. It's very nice but we're both pretty shocked at the prices. We let it slide because we had been forewarned that the prices were a bit higher than normal because of the difficulty in bringing supplies in: but there are no cheap options.

After lunch I fall asleep, and wake to find L upset that the heater isn't working. She wasn't able to sleep, even under an extra duvet and with an electric blanket on. After checking the blanket, I wonder whether it isn't working either. We go to reception and let them know about the heater before heading out for a walk.

The sun is setting: we stroll through the wetlands and hear bellbirds and walk through the reeds that are dotted with tufted white spider nests. The reeds give way to native bush, and we walk across an airstrip where six sheep graze, and then into a large inlet, emptied of the tide. We skirt around the outside, until we notice the moon and take forced perspective pictures of each other pinching it between our thumb and forefinger. We head back along the beach and back to the lodge.

The heater has been fixed in our absence. We head to dinner, where L has the steak and I have a lamb rack. We share a bottle of wine and finish with dessert and whiskey.  It's still early when we turn in.

the cost of genealogy

One of my interests is genealogy. It's a hobby that involves a lot of time and effort, but the web makes it far easier. A researcher's chances of making connections with distant relatives have exponentially improved. Over the past few years I've traced back four or five generations of family, and for some ancestors I've gone back as far as seven or eight. I've done this both through my own research and through the work of others that I have met online who have done their own research and are willing to share it.

My ancestors were all fairly ordinary: tailors, tree-fellers, builders, immigrants, middle children, scumbags, soldiers. They'd hold little interest for others, but they hold great interest for me, because they're mine. A genealogy is a kind of spiritual DNA: by looking back you can see echoes of who you are and where you've come from, and it gives you a weird kind of confidence. It binds you up with history, and even the smallest details hold some reward.

However, there is one thing that really bothers me about the world of genealogy. It's a really expensive hobby. In order to clarify and confirm your roots, you've got to get hold of 'vital records'. These are the birth, death and marriage certificates that became prevalent amongst most developed nations in the middle of the 19th century. Before that there were parish registers in most Euro-Christian nations. These are sketchy at best but can provide ways to trace your family back even further.

Parents and grandparents work in powers of two. So you have two parents, four grandparents, eight great-grandparents, and sixteen great-great-grandparents. If you go ten generations back, you've got 1024 eighth-great-grandparents. (Computer scientists may want to think of the set of eight generations of an individual and their parents as a genobyte!)

Vital records generally cost around $NZD20-26. If you wanted to get birth, death and marriage certificates for six generations (about as many as a 25-year-old would have after 1850):

2⁶-1 = 63 individuals (counting self)
Birth certificates = 63
Marriage certificates = 31 (assuming self unmarried)
Death certificates = 56 (assuming parents and grandparents still alive)
Total certificates = 150
Total cost = 150 x NZD$20 = at least NZD$3,000.

That's just the vital records. You can double that figure for each additional generation, and in order to find more about residences, occupations, and the make-up of families, you need to consult passenger lists, census data, and more. And if you want to trace descendants (e.g. find all the children of all ancestors), you can multiply that $3,000 figure several times (it would be unusual for a genealogist to purchase vital records for anything other than direct ancestors, but it does happen).

These records are held by governments. In New Zealand, the data is held by Births, Deaths and Marriages which is administered by the Department of Internal Affairs. They recently launched a new website which is a boon for genealogists across the country. It allows you to search for historical births, deaths and marriages, and to order certificates online. It also allows you to do interesting things like searching for a combination of a surname and a maiden name to find all the children of a marriage. This sort of research was, until recently, stuck in the dark ages, and involved heading to a library and poring through hundreds of microfiches, finding a reference number, ordering it from Internal Affairs, and hoping you got the right person.

It's disappointing that the government is still charging so much for these certificates and printouts. The website can only have made things more efficient for them, and will have at least got rid of genealogists queuing up in their offices in Boulcott St with stacks of paper forms (you used to have to fill in a paper form for every single certificate you wanted to order), and reduced the amount of searching that their staff would need to do.

I think this information, once it has passed into history, should be free, scanned, and online. I would be very surprised if they hadn't already done the scanning. Currently, ten different genealogists researching the same individual will be unaware of each other's existence, and will each have to pay $20 to get their own copy of the same record: a copy which (in the case of a printout) is nothing but an A3 photocopy. This information is arguably of value to all taxpayers, as everyone has a mother and father. Is Internal Affairs the right owner of this information? Beyond a point, it becomes part of our shared culture: I'd love to see the Ministry of Culture and Heritage incorporate this information into their collection of sites devoted to New Zealand history and heritage: genealogical information gives New Zealand history a sense of personal relevance.

auckland vs. wellington

Simon Wilson's feature in the March issue of Metro claims that Auckland is the new cultural capital. Apparently all the edgy, big budget productions and concerts and shows are happening in Auckland, and Wellington is in disarray: its theatre crippled, its music regional, and international acts giving it a wide berth.

Auckland's a huge, whirring economic engine, no one would deny that. But a cultural capital?  As Wilson himself points out, they lack four things: the self-confidence, strong champions, business support, and the topography. He sees most of these things as surmountable.  Personally, I think they're too schizophrenic to have the self-confidence, too bitchy to support their champions, too mercenary to get strong arts support from business, and too Auckland to be anything but Auckland.  Maybe they'll get over those things.  But in the next five or ten years? Unlikely.

I'm sure they have cultural life.  But where you might spend a weekend coming to Wellington to see the World of Wearable Arts and then head up to Martinborough for a day in the vineyards, in Auckland you spend half your time driving from Cultural Destination to Cultural Destination that it feels less like a Cultural Hub than it does pockets of culture in a sea of depressing spec houses.  When you visit Wellington for culture the destination is part of the package: like Nelson or Queenstown or Dunedin. Auckland is something you tolerate in order to be able to see Bob Dylan at Western Springs.  No one visits Auckland for Auckland.

In the article, Wilson stacks up the major cultural events of Wellington and Auckland, and what he's called a 'ledger': it's telling that in judging the relative culturedness of the two cities, Metro got their accountant to work it out for them. Unfortunately for him, the title of Cultural Capital is a qualitative one.  It's the whole experience, and Auckland is just too dire.

Compare "Dominion Road" from The Mutton Birds' eponymous debut with "Wellington" from Salty: the first evokes the rejected bloke in the halfway house in a city with "antennas in the hills" and  where the only natural beauty is the sun hitting the road which shines "like a strip cut from a sheet metal plate 'cause it's just been raining."  Clouds and mountains are symbols of struggle and aspiration.  And Jane, his wife or girlfriend, has moved south (probably to Wellington), in search of something better with one of his friends.

"Wellington" could be a catalogue of the fondest wishes of the protagonist of "Dominion Road".  It is a place where the weather is terrible (natch), but there's "so much more to do" than "the other end of the island".  There are cafes, bars, "the music and the theatre, and the old Cable Car."  It is a place "you can walk everywhere, 'cause nowhere's very far."  "Wellington" is a song of desire and wishing-I-was-there, "Dominion Road" is a song of bleak hope and of wishing-I-was-somewhere-else.

section 92a: it's not over

The blackout ends tomorrow, and the new Copyright law comes into effect next Saturday. Section 92A of the amended Copyright Act 1994 will probably remain, and it is what it is: it’s not a violation of rights, nor is it, in itself, a piece of legislation that “presumes guilt until proof of innocence."

It’s a terrible clause that is both too prescriptive and too vague. It starts to say something but then it doesn’t say enough. I joined the blackout, not because I think the law is wrongheaded: it’s not. I’ve joined because the amended Act is poor legislation and the faults in section 92A should either be hammered out and the law made more specific, or the section should be completely removed. In its current form it’s a mess and it relies on ISPs to clean it up. But if those ISPs don’t get it right, copyright holders could test them in court and ruin them. It’s a court case that ISPs can’t avoid without being very, very careful: they need to keep copyright holders happy.

Rhetoric about innocence until proof of guilt may not be all that relevant. Copyright infringement is usually a civil offence, not a criminal one. It can become criminal if you engage in piracy: making copies and selling them, or setting up an enterprise that sells products based on the creative ideas you don’t hold under copyright. But a private citizen who makes copies for personal use, or downloads a TV show from a torrent will not be tried as a criminal under current law or under any changes in this Act. They are, however, violating terms of use, both those tacitly agreed when (and if) the product was purchased and those applied implicitly by the 1994 Copyright Act. That’s a civil case between a defendant and a plaintiff. At the risk of being didactic, a civil case between a private enterprise and an individual isn’t covered under New Zealand’s 1990 Bill of Rights Act. Our Bill of Rights, which guarantees things like a fair hearing and presumption of innocence, is designed to protect you in the case of a criminal accusation.

If a Copyright holder accused you of a crime such as piracy, now or under this new law, your day in court would be guaranteed and you would get your trial and all the rights that go along with it. The burden of proof is on the Crown and it has to prove beyond a reasonable doubt that you’re guilty of the crime you are accused of.

In a civil case, as in the case of an alleged copyright infringement, things are different. The burden of proof is on the plaintiff but they only have to prove that you are more likely to have infringed than not. You both argue your case before a judge and the judge decides for either the defendant or the plaintiff. If they win, you would normally pay damages, commensurate with the magnitude of the infringement. If you win, you might get court costs and some compensation. For argument’s sake, and as others more eloquent than I have pointed out, something like the court-ordered termination of an internet connection would be very, very unusual.

Despite the standard of proof being much lower, civil cases against copyright infringers have proven hard for copyright holders to win. Not only that, they’re costly, nasty exercises that result in redresses that are way out of step with the actual losses due to infringements because they have to be tackled one at a time and in depth; they cause huge public relation debacles; and they’re not actually achieving what a successful civil case should be achieving: making the result for a citizen so punitive that others will be deterred from following their example. Despite all of these trials and the closure of many P2P networks and Torrent trackers, file-sharing is on the rise.

So where does that leave us with section 92A?

92A Internet service provider must have policy for terminating accounts of repeat infringers

“(1) An Internet service provider must adopt and reasonably implement a policy that provides for termination, in appropriate circumstances, of the account with that Internet service provider of a repeat infringer.

“(2) In subsection (1), repeat infringer means a person who repeatedly infringes the copyright in a work by using 1 or more of the Internet services of the Internet service provider to do a restricted act without the consent of the copyright owner.

On the surface, this intent of this law isn’t actually that bad. It says that an ISP should have a policy that allows for the termination of accounts of repeat infringers, a repeat infringer being a person that repeatedly infringes copyright in “a work”.

This clause seems to say that if you upload a Madonna song to YouTube once, you’re an infringer. It might be saying that if you upload it the same song three or four times, you are a repeat infringer. It’s not clear if you’re a repeat infringer if you infringe the copyright of many different works, and do this repeatedly, with each work being infringed once.

But okay, let’s say you’ve been running a file-sharing application, and your copy of Madonna’s Like a Virgin has been shared with many people. Or you’re using a torrent and you’re uploading to three or four different peers while downloading from three or four others. That probably constitutes repeated infringement in “a work”.

So do you get disconnected if a copyright holder catches you and accuses you? The answer is: it depends. It depends on a document that is being drafted right now: the Telecommunications Carriers’ Forum’s Draft Code of Practice, which is a template document which all of the New Zealand’s ISPs will crib from in order to write their own Code of Practice. It depends on the Code of Practice that your own ISP adopts. It depends on what that document says about how a repeat infringer is identified, and defined, and the level of evidence required for the ISP to accept that a user has repeatedly infringed using the ISP’s service.

The copyright holders and ISPs have tussled over this document like dogs over a bone. The problem is, if the ISP adopts a Code of Practice that is too lenient on users, or too strict on the standard of evidence it requires over infringements, copyright holders will take them to court and they will fight over the words ‘in appropriate circumstances’. ISPs, particularly small ones, will want to avoid that situation at all costs. They have to overcompensate or risk their business.

So what’s in the Code? Here’s what the Draft Code of Practice says about repeated infringement:

To avoid doubt, an Infringer need not Infringe repeatedly with respect to the same category of work under the Act or with respect to the same Copyright Holder, to qualify as a Repeat Infringer.

The Draft Code of Practice is clarifying section 92A (2) and saying that actually, a repeat infringer is someone who repeatedly infringes copyrights in any works, in any categories, from any copyright holders. So you’re a repeat infringer if you share a Madonna song, and a Rihanna song, and an Animal Collective song, and you only share each once. The Draft Code, at the behest of entertainment industry interests, has interpreted the statute in the widest possible way.

The Draft Code of Practice also defines the levels of evidence required for someone to have made an infringement. They are:

11.1 A judgement of a Court (interim or final) finding Infringement under the Act;

11.2 A Copyright Holder Notice which complies with this Code;

11.3 such other evidence as that Party is prepared, in its sole discretion, to accept would be sufficient to satisfy a Court that an Infringement under the Act has taken place.

11.1 says that if a copyright holder took you to court, and won, that’s satisfactory evidence of an infringement. 11.3 says that if the ISP is satisfied that the evidence presented to them would be sufficient to satisfy a court, that’s satisfactory evidence of an infringement. So either you go to court, or the ISP makes the judgement that the evidence is satisfactory on its own merits.

But under 11.2, a copyright holder can issue a Copyright Holder Notice, which contains information about the how a copyright was infringed, the method by which the infringement was detected, the time and date, and so on. The Draft Code of Practice says that this is acceptable evidence of an infringement. The Draft Code also says (in its current form) that a user can issue a Counter-Notice, disputing the allegation of infringement, but this is being debated, and copyright holders have proposed an alternative Counter-Notice procedure that puts the copyright holder themselves in an position where they themselves decide the validity of a Counter-Notice.

With the inclusion of 11.2 and 11.3, we will have to deal with ISPs being in the position of an adjudicator. If the alternative Copyright Holder Notice procedure is taken up, the copyright holders themselves are in the adjudication position. Copyright holders, who would be the plaintiffs in a civil case, get to play judge.

The alternative is that the ISP plays judge, but they and the entertainment industry have intermingled interests. ISPs are also their distributors. They fight over this but on other things they’re business partners. Would you be comfortable being the defendant in a court case if you knew the judge and the plaintiff were business partners? Would you really get the same consideration as you would get in a court or arbitration? I don’t believe you would. Users need the opportunity to state their case before an impartial judge in a proper court of law, (or a Copyright Court, as suggested by the Creative Freedom Foundation): anything else is corruption waiting to happen. They may trust themselves to be impartial, but that's arrogance and hubris, and just not good enough.

The Draft Code of Practice, assuming all ISPs adopt it unchanged, will be the law in essence. We may lose this fight, and through our government’s inaction, section 92A may come into effect. That’s why it’s hugely important to write a submission to the Telecommunications Carriers’ Forum to let them know that they cannot accept any evidence short of a Court judgement finding infringement, or an agreed impartial body, such as a Copyright Court, finding infringement.

You can find more information at the Telecommunications Carriers’ Forum site. Please make a submission to submissions@tcf.org.nz: they close on Friday the 6th of March, in two weeks. This matters. Beyond that, we will have to vote with our wallets, by choosing the ISPs whose actual Code of Practice offers us the most protection from false allegations of copyright infringement.

webstock '09: day 2: when sterling attacks

If Day 1’s topic was community, Day 2’s topic was data and hardware. Big, messy stacks of data, measured, unmeasured, captured by all the things around us and coming to the web. On Day 1, I took an average of one page of notes per speaker. On Day 2 it was closer to three pages. I remembered a couple of things I forgot to mention on Day 1. Jane McGonigal’s latest game is the Signtific Lab, a massively multiplayer thought experiment that was played throughout the conference, and Ze Frank’s youngme/nowme project is just plain cool.

Russell Brown kicked things off with a quick chat about s92a and why we should really be paying attention to the Draft Code of Practice (PDF): because what’s in there will actually be the implementation of the law. He then talked about the state of television, and how the biggest problem they have is the old-style distribution networks: it’s the reason why you’re not allowed to watch TV on Hulu in New Zealand (the Alec Baldwin superbowl ad is worth a watch), or why an American can’t watch TVNZ on Demand. That said, sites like NZ On Screen and Radio New Zealand are getting it right with no DRM and no geo-blocking. He believes traditional TV still has a role (after all, it’s infinitely scalable), but that the content has a long way to go if it wants to still be relevant: people (and particularly kids) just aren’t watching the style of programs they used to. Oh, and Media 7’s doing great, thanks very much: particularly via the web.

Derek Featherstone discussed accessibility, and was chiefly interested in the differences between standards and accessibility: meeting standards is no guarantee that your site is actually usable. Without some knowledge about how your users actually interact with your site, or without at least thinking about it, you run the risk of creating something that validates but nevertheless sucks. I’m a fan of the Cooper process, and I found Derek’s insights underlined a lot of Alan Cooper’s ideas. He talked about making maps and flash movies usable by embedding buttons instead of overlays and putting controls outside a YouTube clip that controls the clip via the YouTube API. He showed us a cool accessible crossword, and professed a lot of love for Ubiquity, a command line interface for the web, which, among other things, is an accessibility tool on steroids.

Annalee Newitz of Wired was awesome. She talked about science fiction metaphors and the vocabulary established in films and books like Tron and Neuromancer and The Matrix and Battlestar Galactica, and the fear that people have about computer interfaces that interface back. She discussed how we can surmount those fears through education and by being mindful of the resistances that people have, because when it comes to new and scary interfaces people have internal narratives that end in having their brains sucked out: even when it comes to things like Facebook. She talked about superheroes and super villains and how the internal development team of Google Android switched from a Cylon-styled booting screen to a more friendly R2D2-style brand when it was about to be introduced to the public. And she presented some cool work on gestural interfaces, wearable computers, exoskeletons and Brain Computer interfaces, referencing Iron Man and Batman Begins along the way. It was nerd heaven.

I was really inspired by Toby Segaran’s talk. As web developers we rely a lot on the relational database, and he took as through some of the technologies that are trying to improve on the database paradigms we’ve depended on for the last 35 years. He discussed the graph data model, which is the model of the semantic web: object-predicate-subject relationships, and how it is organic and allows for schema changes because it doesn’t really have a fixed schema. Storing data in this way with tools like Sesame and Exhibit allows us to use SPARQL to do efficient queries that in RDBMSes take dozens of SQL joins. If data is presented in this way on the web, through standards like RDF and others, it means we can get answers to queries that are not only accurate, but also smarter.

Matt Biddulph of Dopplr talked about hardware hacking, which is a quiet ambition of mine, and he showed us some cheap electronics that we can use to create cool stuff. He directed us to Make, and showed us the current standard microprocessors and wireless connectors in hardware hacking: the Arduino, the XBee. He believes that if you can’t open something, you don’t own it. He pointed us to some cool hardware projects: a pot plant that tweets when it needs water, the tea/no-tea teapot, both built out of cheap hardware components that were just wired together. He recommended Everyware and Making Things Talk as good texts about the craft and the culture.

Matt Jones, also of Dopplr, talked about the cities that we live in, and how the spaces that we inhabit are becoming augmented with information: he pointed to the early ideas of Archigram, Project Cybersyn and Sir Richard Rogers. The end result of this is that our environments are becoming spaces that are robot-readable: machines can navigate the world by the nascent data-ness of things. Riffing on Matt Biddulph’s earlier example of the twitter-powered pot plant, he pointed to James Chambers’ Has Needs, a pot plant that will put itself on a site like Craigslist if it’s neglected. But people are walking architecture, mobile data sources, and our interaction with cities gives us a whole swathe of psychogeographic information (cf. Guy Debord) that we can play with. He presented a lot of interesting ideas, the street as platform, the Sky Ear, the Helsinki laser cloud, situated software. How do we enable all of this? In the words of Eliel Saarinen: always design a thing in its next larger context.

Tom Coates of Fire Eagle was hilarious. He was a good follow-up to Matt, as he talked about personal informatics, and how we are beginning to see a world where people can (if they want to) record every miniscule detail about their lives and view the information as the wish, using personal sensors. FireEagle, as a geolocator, is a part of that, and he thinks we will see an explosion of services that manage other pieces of data (think last.fm, Twitter, Flickr, 23andme, Skydeck, Mint, Google Health, Nike+iPod, Google Power Meter ... besides being tools to explore and store personal information, they’re data gatherers, each gathering a single slice of data). Mobile is key to this stuff because it’s the most ubiquitous sensor we have. And real-time really excites him. Technology like XMPP is enabling this, but Tom’s view is that it’s data that’s driving new products, not technology. As more data comes available, more products become possible. Aggregators crop up, like Socialthing and Friendfeed, and for devices, Pachube. (We had our own Webstock example of real-time data aggregation: the SuiteSpot.) Tom pointed out the difference between personal information and private information: although we might talk about what should be personal and what should be private, the smartest thing to do is to put that decision in the hands of users. And a big privacy policy or terms of use doesn’t help. Be up front.

Bruce Sterling was next, but I’ll get to him in a second.

Damian Conway, the bossy little schoolgirl of web design (his words), talked about designers and users, and how we tend to think about the differences. He compared the relationship to that of Elois and Morlocks, and then to Elves and Orcs, but actually thought it was more helpful for us to think of ourselves as more akin to doctors. He proposed a Hippocratic Oath for web designers:

1. To learn and share knowledge
2. To always do your best work
3. Don’t kill the client’s business
4. Know your limitations, and get help if you need it
5. Always have the best interests of your clients in mind (often the best interests of your clients are the interests of their clients)
6. Be professional, keep your clients confidentiality.

He showed us some sites that have really got it wrong, told us most shopping carts suck (how much information do they really need to collect, six steps worth?), and was hilarious throughout. He also showed us a couple of pictures of some guys holding cats.

Bruce Sterling spoke before Damian, but I wanted to talk about him last because his presentation was something that stood alone. He was electrifying. He presented one slide, Tim O’Reilly’s Web 2.0 meme map, and tore it to shreds. It was a deconstruction; a wake-up call, he was trying to shake us out of the fog we were in. This isn’t permanent. We got here because creative people are attracted to places that lack rightness, and the web lacked a lot of rightness, and still does. Web 2.0 had a few useful ideas, but some of those ideas (the ideas that lacked technology) were just attitudes. It’s not a platform: you can’t build a platform on a web, just like you can’t build a castle on a cloud: it’s a fantasy. The whole thing rests on an economy that is in a state of collapse. Web 2.0 is a structure built from the bones and ashes of Web 1.0 technologies: it’s bricolage. JavaScript might be the glue that holds Web 2.0 together, but if a freighter runs over our cable across the bottom of the Pacific, that glue isn’t going to hold. Ubiquity is a long way off, we may have the concepts but power sources and bandwidth hasn’t kept up: it’s an art scene, it’s a hack scene. Permanence is a fiction: the sites he linked to years ago are all gone, in 404-land. We can’t trust the so-called collective intelligence like we can’t trust the invisible hand of the market: it’s not our benefactor; it’s a force of nature, and it could turn and flatten us at any moment. With the economy in freefall, we’re about to see a Web with great big holes in it, the Transitional Web, a cyberstructure that is poverty-stricken but that we will use to keep the rain off. He encouraged us to grow up to the scale of things, see the culture for what it is, and to simply be, but be aware.

To some, he would have appeared as a ranting preacher, raving that the end is nigh. To me, it was a welcome shot of realism. We work in the best industry in the world right now, no doubt, but permanence is by no means guaranteed. We should enjoy it while it lasts, for however long that might be. Sterling’s talk was a great end to the conference (thankfully tempered by Damian Conway’s positivity), and it played on my mind as we walked out into the rain and the darkening day. Maybe it was just me but the party, awesome as it was, felt Gatsby-ish: beautiful geeks, industry celebrities, a red carpet, a band played by robots. I had a great time, but it felt of-a-time, and I wonder: what happens now?

webstock '09: day 1

I had a great time at Webstock today. I am getting the sense that this year's event is primarily about online communities: attracting them, keeping them, inspiring them and moderating them.  Last year was more about design and frameworks, so this was something quite new.

Jane McGonigal spoke about her work with AvantGame and the Institute for the Future, how games make for a happier humanity and the role of games in making futures, not just predicting them.  She talked about how we're all creating games, and if we're not creating them, our competitors will.  Good games are structures that make people happy, and help people feel and become more awesome.  How many websites do that? In what ways can they do that?  I'm heartened by Jane's ambition to have a game developer win a Nobel Prize by 2032.   And Top Secret Dance Off (which featured on Close Up tonight) looks fun!

Nat Torkington of O'Reilly talked about the scientific method and the idea of a feedback loop (in science, in evolution, in the political process), how it applies to web and software design and development and how we don't actually get anywhere without allowing failure to happen. If it happens in small, testable increments, failure is a tool for innovation.  But too small, and we risk any chance for serendipitous innovation.  He talked about how big companies suffer from the Inventor's Dilemma: they've got too much to lose to innovate their most successful products.  This is exactly why they can be beaten by small companies.  But, on the side of corporates, it brought to mind an article I read in Idealog earlier in the week about the Black Room concept.

Derek Powazek spoke about the wisdom of crowds: what it's good for, and how it can be woefully misused.  His insights about relying on selfishness and the double-edged sword of using game tropes - winning the game has to help the community, or you're screwed - were worth the price of admission. He talked about the ways in which communities can be encouraged and the methods you can use to keep them flourishing.  He spoke about trolls - who they were, what motivated them, and what they wanted - and how to diffuse them.  I really liked the idea of the cone of silence: this is where a logged-in user who is an identified troll sees their contributions within the stream of comments, but they are the only user who sees them.   Their attempts at baiting are ignored by those that can't see them, and they eventually leave out of boredom.  Sites he mentioned that are worth a look: Favrd, Hot or Not!, and his own site, Fray.

It was heartening to see Meg Pickard from the Guardian UK speak about community participation as something that fundamentally changes a content producer's offer: it's not about a content producer putting out authoratitive comment and then everyone picking it to pieces in a completely separate (and largely ignored) silo, it's about bringing that content into the conversation to create something completely new: something better than simply authoritative content with comments tacked on.  To me it seemed like an obvious leap, but the reaction of others in the room suggests that a lot of people really aren't there yet.  It's nice to know that the Guardian is willing to meet the challenges of the new media, I hope there were people from APN and Fairfax in the audience.

David Recordon of Six Apart talked through a number of technologies that support the social web: he discussed the growing recognition amongst people that walled gardens on social networking sites suck, it's our data, and we need to find ways to make it easy to share the data that we want in the way that we want, and it'd be great if we didn't have to complete a sign-up process for every site on the web either.  There's a whole bunch of open source tools that could fix this: Microformats, OpenID, OAuth, XMPP, Open Social, FOAF, vCard, DiSo.  There's a great Django-based product that builds on top of a lot these tools: Pinax, and it sounds well-worth investigating further.  He also mentioned the SocialWeb TV, a video podcast.  I've subscribed already.

Adrian Holovaty impressed the hell out of me.  I'm a Django fanboy to begin, but Django didn't actually figure much in his presentation.  He talk about data, and using data in exciting and interesting ways.  His latest project EveryBlock gives you an RSS feed for what's happening in your neighbourhood: crimes, property sales, new businesses, restaurant inspections.  His presentation was quite detailed: talking about URL schemes and mapping technology (he recommended an article in A List Apart about rolling your own maps).  He suggested we encourage our government (national and local) to not waste time building badly thought-out sites for displaying data: give out the data in an API and let the public use it as they will.  It's our data after all.

Heather Champ is the community manager at Flickr. She spoke about the ways  she's found effective for shepherding her community.  She talked about some of Flickr's most public challenges: the YahooID switchover, the masturbating subway pervert, laptop thieves accidentally uploading their photos to Flickr, and solving downtime issues with a colouring competition.  Heather talked frankly about the mistakes they'd made and the lessons they learned, and how they've become much better at managing the community as a result.  A great talk.

Michael Lopp of Apple took the entertaining route.  He spoke about nerds, geeks and dorks, showed us a Venn diagram of the three and how they intersected.  He talked about his observations on how geeks think and work: they're systems thinkers who go deep in interesting ways: they're obsessed with understanding the rules of things.  He believes that everything's just a beautiful mess, but that you can fascinate a nerd by pulling bits out of that mess and getting them to make sense of it.  His talk was perhaps lacking in information, but nerds love hearing about themselves, and hearing someone who seems to be explaining the rules about understanding themselves, so he was very well received.

I really didn't know what to expect with Ze Frank, but I was impressed.  He talked us through some of the projects he'd been involved in over the years, and some of the strange and wonderful moments that have happened as part of his many projects: Atheist Game, Flowers, 52to48, Facebook me=u, Earth Sandwich, and Remixes for Ray.  He was funny, and he was smart, really smart.  He understands the internet in such a fundamental way, and more than that, he knows how to participate in it in ways that really inspire others and go out of their way to do some really silly things.  I'm a total fan.

Tomorrow's speakers have a tough job ahead of them to live up to the standard set today.  There wasn't a single session where I wasn't completely fascinated.  I just wish I could have seen the other break-out sessions. Full credit to the Webstock team!

generating short urls for django site urls

Today I developed the following mixin for generating short urls (à la tinyurl) for any model that generates a unique absolute URL for an instance.

It builds on top of django.contrib.redirects, so relies on that app being installed. Aside from that, integration is easy: just import the ShortURL mixin and add it to your model's class declaration:

class MyModel(models.Model, ShortURL):

The mixin code itself is:

from django.conf import settings
from django.contrib.sites.models import Site
from django.contrib.redirects.models import Redirect
from random import choice, seed
from os import urandom

SHORTURL_CHARS = getattr(settings, "SHORTURL_CHARS", "bcdfghjklmnpqrstvwxyz2346789")
SHORTURL_CHAR_NO = getattr(settings, "SHORTURL_CHAR_NO", 5)
SHORTURL_APPEND_SLASH = getattr(settings, "SHORTURL_APPEND_SLASH", True)

class ShortURLException: pass

class ShortURL(object):
    """
    A mixin that sets up short url redirects for models that have a get_absolute_url 
    method.  Requires django.contrib.redirects to be installed to create redirects, and
    django.contrib.redirects.middleware.RedirectFallbackMiddleware to use them. 
    """
    def __init__(self, *args, **kwargs):
        """
        Seeds randomiser 
        """
        seed(urandom(256))
        super(ShortURL, self, *args, **kwargs)
        
    def get_short_url(self, *args, **kwargs):
        """
        Finds the short url for the object's absolute url in the Redirects model objects.  
        If it doesn't exist, generate a short url and create a new Redirect object.
        """
        if not hasattr(self, 'get_absolute_url'):
            return None
        else:
            currenturl = self.get_absolute_url()
            site = Site.objects.get(id=settings.SITE_ID)
            redirects = Redirect.objects.filter(site=site, new_path=currenturl)
            
            for url in redirects:
                if len(url.old_path) <= SHORTURL_CHAR_NO + 1: #allow for leading slash
                    shorturl = url.old_path
                    break
            else: 
                shorturl = None
                
            if not shorturl:    
                # Check we've got at least a 9 in ten chance of not colliding or throw an exception
                if Redirect.objects.count() > (len(SHORTURL_CHARS) ** SHORTURL_CHAR_NO) / 10:
                    raise ShortURLException
                while True:
                    shorturl = '/'+''.join([choice(SHORTURL_CHARS) for char in range(SHORTURL_CHAR_NO)]) 
                    if not Redirect.objects.filter(site=site, old_path=shorturl):
                        # save shorturl without trailing slash so redirect middleware will find both forms
                        r = Redirect(site=site, old_path=shorturl, new_path=currenturl)
                        r.save()
                        break
            
            shorturl += '/' if SHORTURL_APPEND_SLASH else ''
            return shorturl

The default settings give you around 17 million short URL combinations, one character less will give you 600,000.  If you've only got a few thousand unique pages you shouldn't be worried about collisions (the script just keeps pulling out random strings until it finds a unique one), but if you've got a big site you'll probably want to rely on a hashing function rather than just getting lucky.

This code can also be found at Django Snippets.

UPDATED: 16 February

• Now throws exception if the chance of collision is getting too great.
• Only returns short urls and not just the first redirect it finds for a page.
• Stores redirect in DB without trailing slash so that the redirect works with or without the slash.

variable swapping with python

Python's great. It's the little things. Like swapping variables. In most other languages, a variable swap requires a temporary variable, for example:

tmpvar = var1
var1 = var2
var2 = tmpvar

Or if you're dealing with integers, you can use the XOR/addition and subtraction method:

var1 = var1 + var2
var2 = var1 - var2
var1 = var1 - var2

But in Python? Because all variables are references to objects, you can use tuple-packing/unpacking to achieve a swap in one command.

var1, var2  = var2, var1

Nice.

debugging django apps with eclipse and pydev

The standard PyDev plugin comes with a debugger that you can use to debug Python applications, allowing you to utilise breakpoints, step through code and watch variables.

It's possible to get the debugger working with Django, so that you can use your Django app in your browser and have execution of a HTTP request stop when you hit a breakpoint. 

Select your project's manage.py file within Eclipse and create a new Debug Configuration.  Under the arguments tab, you need to add two program arguments.

runserver does as you'd expect: it starts the django development web server on localhost:8000.  --noreload turns off auto-reloading of code as the server is running, but prevents the server process from forking, leaving your debugger attached to the wrong process.

google-powered site search

There are certain things that websites of reasonable size are expected to have. One of those things is a site search, a search box that sits on most pages and allows you to search the entire site for information. The search's scope might be just that site, though if you have a number of websites you may want to provide a search that covers all of them.

There are a number of custom solutions and crawlers out there of variable quality.  Of the many products available, I've heard good things about Solr and Sphinx in particular. These options and others like them require you to provision and set up a search service, either local to the website or on a separate environment.

With smaller websites, a hosted search solution is probably easier to get up and running quickly, and of those, Google is probably your first option.

With Google, there are a number of products you can plug into, and a couple of ways you can achieve a working site search.

Google AJAX Search API

The Google AJAX Search API submits queries to Google via an AJAX API and returns the results for you to process in-page. The AJAX Search API allows you to limit your search to a site via the setSearchRestriction method on a Search object, which is effectively like adding 'site:matt.geek.nz' into every query string.  There’s no reliable way to search over multiple sites: you could resort to tricks like appending multiple inurl: statements to the query string, but this gives some pretty poor results.

Skinning is possible through DOM manipulation. Your results come back in well-formed HTML so you can do what you like with them, provided you adhere to the Google's Terms of Service, which stipulates that you need to attribute Google via a statement that your search is 'powered by Google' near the search box and the results.

There are a couple of limits with the AJAX Search API.  You can only get a maximum of 32 results back per query.  Also, because you're accessing the search via AJAX, for the best results you need to display the results on the same page that the search is submitted.  You could manipulate things so that a POST request goes to a results page and the AJAX call is made as the page loads, but you may end up with an empty search results block until your AJAX call returns with the results. 

In summary:

• It’s free.
• It allows unlimited searches.
• It returns only 32 results per search, with basic pagination.
• It can be skinned, but you do need to attribute Google as described in the Terms of Service.
• It can be limited to search a single site.
• You should display results within the same browser page request.  You could arrange things to resubmit a HTTP POST variable to the AJAX search, or somehow call the AJAX service from the server, parse it there, and return it to the client, but these are probably violations of the Terms of Service and are certainly against the spirit of Google’s offer.
• In terms of results, you get what you’re given.  You can’t tweak them as they’re the same as what you’d get if you went to the Google homepage and submitted the search directly.

Google Custom Search Engine

The Google Custom Search Engine is a search service provided by Google that allows you to create your own search engine.  Through a control panel, you can add a list of sites to be searched, you can prioritise results, you can add refinements, categories and synonyms, and you can submit new pages to be indexed within 24 hours.

The difference here is that you’re using your own variation on Google’s main index.  You’re specifying what you want searched and more importantly, how you want it searched.

Within the category of Custom Search Engine, you have a few options:

• Google Custom Search Engine, which is free.
• Google Custom Search Engine for Not-for-Profits, which is also free.
• Google Custom Search Engine for Business (now known as Google Site Search), which begins pricing at $100US a year.

A key difference between these three choices is the degree to which you can skin them.  With the first two options, you will have access to search results from your custom search engine, but they are provided as pre-processed HTML and you include them on your site via an iframe or modal overlay.  With the free Custom Search engine, you get Google advertisements placed with your search results (you can arrange to get paid if they result in any clickthroughs).  With the not-for-profit version, you can optionally remove the ads, but you are not allowed to obscure the Google branding.

There is some potential for skinning: you can alter the some of the colors used within the iframe/model overlay via a control panel.  Make no mistake though, your search results will look unmistakably like Google search results, and if they don’t you are probably violating the Terms of Service.

Finally, there is the Custom Search Engine for Business or Google Site Search.  This gives you access to an XML API, which you can submit your queries to.  This means that the potential for skinning is unlimited: you can parse the XML on the server and present the results in almost any way you wish.  This is borne out by individual blog posts and answers in Google Support, however it doesn't look like their Terms of Service have been updated to reflect this freedom.

With all versions of the Custom Search engine and Google Site Search, there is a limit to 5,000 unique URLs that can be indexed, and there is a limit of 250,000 queries annually, but with the business edition (Site Search) you can increase these limits by paying more per year.

In summary:

• Google Custom Search Engine 
  • It’s free.
  • It allows up to 250,000 searches per year.
  • It will index up to 5,000 documents.
  • It can be limited to search a single site, or several sites.
  • You can tweak search results via a control panel.
  • You can view search statistics via a control panel.
  • It is supplied with ads, which you must display.
  • It is included in your site via iframe or modal overlay, and is skinned by Google, which you can customise in a very limited way.
• Google Custom Search Engine for Non-Profits
  • All of the above, except you don’t have to display ads.
• Google Site Search (or Google Custom Search Engine for Business)
  • All of the above, except you don’t have to display ads or branding, and prices start at $100US a year.
  • You can also retrieve search results via an XML API, which you can programmatically manipulate and display on your site in any way you wish.
  • By paying more per year, you can increase the number of pages indexed, and the number of search queries that are permitted annually. 

Combining the two

The Google AJAX API and the Google Custom Search Engine can work together.  You can create your own custom search engine and use the setSearchRestriction within the JS Search object to limit your search to that search engine by supplying that engine’s hashcode.  In all other respects the AJAX API is identical, and is subject to the same limits (32 search results returned, results displayed in page).

Other Google Options

If these options aren’t sufficient, and you’re committed to using Google as your search solution, you’ll need to look at the Search Appliance products.  These begin at $3,000US (which covers two years of support and software upgrades).