Incidental Contemplations

I’ve just been skimming the new report(pdf) from the UK Department for Business Enterprise & Regulatory Reform. It goes by the completely uninspiring name of “Models of public sector information provision via trading funds”, though the contents are anything but uninspiring. Broadly speaking, it outlines the economic benefits to society of making government data available at marginal cost (ie, $0; Ordinance Survey data by bittorrent? Note 19, p34).

So far I haven’t understood much beyond the executive summary (it’s full of the sort of maths and dry theory economists salivate over), but I believe this report will have a profound impact on the way governments handle their responsibility as custodians of information about society. It won’t be due to any altruistic motives, but rather down to the almighty dollar (or pound in this case). If you’ve ever had the misfortune to be ear-bashed by me over a few beers, you’ve probably heard me say “if you want to know what’s happening, follow the money”. Now the money has a reason to set the data free in a way we’ve previously intuitively understood but haven’t been able to prove. This report demonstrates some impressive economic returns that are possible from setting significant data sets free, such as the Ordinance Survey. It will take some time, but I believe we are now on a one way path towards liberalising the distribution of government generated data.

WALIS Forum has started, and we have our OSGeo booth. Many thanks to our sponsors Gaia Resources, Lisasoft, Maunsell and Lat-Lon. WALIS was started by the Western Australian Government in 1981 for coordinating across-government access and delivery of the geographic information held by WA Government agencies. I was going to write a few words about how great WALIS is, but Marnie Layborne, Director of WALIS says it so much better;

WALIS is all about participation and cooperation. Our role is to advance the spatial information industry in Western Australia and WALIS has traditionally coordinated the sharing and access of geographic information across government. We are now expanding beyond government to include the private sector, academia etc as the only way we can really realise a global vision is to include all the players. This is why we have as our theme for this forum, public private partnerships.

WALIS forum is held every 18 months, and has been consistently growing bigger. It’s just been announced there are over 800 registrations. Everybody who’s somebody in the spatial industry in Australia (and many from overseas, particularly NZ) is here. We’ve only just started, and already we have had healthy interest in not just what OSGeo is doing, but more importantly, what everyone can do with open source themselves.

Matt Assay has a couple of blog posts dealing with PostgreSQL v MySQL and Billion-dollar open-source opportunities . Both are typically good informative posts in themselves, but their juxtaposition provides added meaning for me given I hold PostgreSQL/PostGIS dear.

As a keen user of PostgreSQL (driven by PostGIS, the spatial extentions to postgreSQL) I’ve always been keenly interested in the ongoing viability of it, both as a project and as a solution in business. I’ve often wished it was GPL’d rather than BSD’d, as GPL projects on the whole (in my experience) tend to have much more active and vibrant communities. That’s not firm and fast though. There’s plenty of exceptions. Matt compared the popularity of MySQL against PostgreSQL, and noted what looks like a more active community with PostgreSQL (ok, there’s an exception right away), and implicitly wonders if EnterpriseDB may get gobbled up the same way MySQL did. Given MySQL has the LAMP mind and market share going for it, the valuation wouldn’t be anything like the $1B MySQL got, but surely the possibility of acquisition has to be on the cards. After all, Matt’s Billion dollar post confirmed the healthy market for open source companies.

When it comes to takeovers and valuations, I think the role of GPL as a strategic weapon is often under appreciated. If you’re top vendor dog in a GPL project, other players have a very hard time unseating you. That may sound counter-intuitive given world + dog has the code, but I don’t believe it’s such an advantage for competitors as most assume. Your lesser competitors in the same space have to share their plum developments with you. Sure, the top dog has to share his plums too, but when you’ve got the top plum growers in your own yard (to push a metaphor too far), you get to go to market with the best solutions first. If you can keep your plum growers happy, and can do your business execution right, you’re in a very strong position.

With BSD projects on the other hand, solution providers tend to go to market with proprietary solutions. You can’t force your competitors to share their plums. You don’t share your own (at least, not till they’re getting a bit old and withered). The competitive maneuvering follows a more traditional proprietary model. Being top dog doesn’t stop the competition accruing some distinct proprietary advantage. Sure, it’s rarely easy winning from behind, but if you’re a second tier vendor and have to give away your best produce to the market leader when you go to market (like with GPL’d projects) surely it’s so much harder again.

If you had a choice between taking on MySQL (company, pre-takeover) in the MySQL (software) space or EnterpriseDB in the PostgreSQL space, I suspect it would be much easier to take on EnterpriseDB, purely because of that ability to accrue proprietary value. Surely that has an adverse effect on the valuation of EnterpriseDB as opposed to MySQL?

If I’m right, GPL’ing your open source code (rather than BSD’ing it) is the smart move for commercial leaders in an open source project. It advantages you more than your commercial competitors, putting you at a relative advantage. It also helps to create a vital ecosystem around your technology by forcing enhancements out in the open. Maybe EnterpriseDB isn’t such a good acquisition target after all.  Despite backing an open source product, perhaps it looks more like a proprietary company in terms of market valuation and being able to leverage that open source connection. In which case, it’s one more reason I wish PostgreSQL was GPL’d. A strong leading vendor adds a lot to a good open source project.

The nzopengis group recently discussed the possible use of the University of Tasmania’s Bluenet Metadata Entry and Search Tool by the New Zealand Geospatial Office. Bluenet is a version of the GeoNetwork tool that supports the ANZLIG metadata profile. It got me thinking about what it takes to make a successful metadata system. I knew I’d bitten off more than I intended when I ran into Aristotle in my search to put a foundation under the metadata system problem.

The IT landscape is littered with metadata wrecks. The fundamental business problem with most metadata systems is the cost and value centers are too far apart. Those who pay the cost of collecting and maintaining metadata aren’t the ones who benefit from it. Those who benefit are usually far removed from those who pay. They can be removed by intent (how they classify and use the data), location (or more accurately relationship) and worst of all temporally. For a sobering read on why metadata systems don’t work, see Cory Doctorow’s Metacrap piece.

The secret to making a geospatial metadata system work is bringing cost and value closer together. That’s obviously not an easy problem to solve (or else we wouldn’t have so many metadata wrecks). Lowering the cost of collecting and managing it (which is where Bluenet GeoNetwork comes in) is a start, but it doesn’t move the two closer together. Solve that problem, and you’ve got yourself a fighting chance at a working metadata system. The alternative is to add 1 to the count of existing metadata wrecks.

Before we can solve that problem, we need to understand the nature of it. If we look at the cost and value propositions of metadata, we see it often comes last in the chain of data creation events. That’s because it’s not central to the immediate needs of those collecting the data. They know the constraints they’re operating under when they collect the data. They know the accuracies of their data collection methods. They understand the attributes of the data. They’ve got their own informal metadata system happening; “Hey Bob, how good are the boundaries we digitized for suburb X? Aw, about 10m shifted NW with a 3 degree rotation at control pt 1, but fine for the power retic schedules you’re doing”. Yes, that’s very informal, but it’s metadata and it’s valuable. Recording those attributes formally for others comes way down in the list of priorities. Typically it adds cost without adding much value to those collecting (and supposedly) documenting the data. No matter how enthusiastic they may start out, they end up doing a poor job of the formal documentation bit.  That’s because there’s insufficient value in it for them.

When we look from the point of view of removed consumers of the data, it’s a different story. They’re screaming out for metadata. We’ve invented all sorts of fancy schemas and what-not for managing it. Problem is, once the grand schemas are written and systems developed, data consumers have no say in the metadata story. They have a vested interest in the quality and quantity of the metadata, but no (realistic) means to amend or add to it. They’re not the custodians. There goes their chance to spend resources (time and effort mostly) in recording or validating the metadata that’s important to them. Right there is a serious mis-alignment of the value and cost propositions in the metadata story.

So what is this metadata that creates so much supposed value? The trite answer is that metadata is data about data. At that point we usually move on, content that we have a self understood term. Instead, I’d like to use an alternate definition used by David Weinberger, noted internet information systems expert and author of recently published book “Everything is miscellaneous” (and here for the google video). David gives us “metadata is what we know, data is what we want to know”. Using that definition, we get a whole new slant on how we approach the metadata value problem. No longer do we have a simple divide between data (shapefiles, imagery, map sheets, OGC web services etc) and our ISO 19115 ANZLIG metadata profile. Sometimes the data will be the metadata and the metadata will be the data. Unraveling that requires we understand the principles on which we classify information. At that point, we come face to face with Aristotle, and his model of information classification that we have used ever since. That’s a big topic in its own right, so I’ll save it for part 2, along with how the nature of the separation between metadata producers and consumers has an affect on our solution.

FOSS4G will come to Sydney, Australia, in November 2009. It’s the world wide conference of OSGeo, the home of the open source spatial communities. It’s the gathering of the tribes if you like. FOSS4G is a great chance to show the world what’s been happening down under in the GeoFOSS space. Details in the press release. Being one of the bods on the ‘09 organising committee I can promise we’ve got some great ideas for ramping up the conference. That’s not to say it’s at all lacking thus far. On the contrary, It’s a fabulous conference and already gets rave reviews. We’d like to take it to another level though.

There’s a lot happening in the open source GIS space in this part of the world. From high profile initiatives such as SLIP in Western Australia and the BlueNet GeoNetwork web portal and catalogue project at the University of Tasmania, to the rampant under the radar use of tools such as PostGIS, gdal/ogr, grass, mapserver, openlayers or geoserver throughout government and industry there’s some fabulous things happening. I’mlooking forward to being able to show the world what we can do down under. First though, we have FOSS4G’08 in South Africa, which is shaping up to be a real cracker of an event. See you there.