Technology can be protocol-based or a point solution

With information technology a designer always has to choose between building a data integration application as a hardwired “point solution” or as a more exible “protocol-based solution”.

The Data Commons is a protocol-based solution because this is a universalisable solution based on standards.

The argument for the latter is rather like the reasoning for having standards around power plugs. Yes, everyone can build their own unique plug type, but there are real advantages to the community adopting a standard design. New Zealanders can buy any appliance (fridge, toaster) and plug it into anyhousehold power source because we know they fit. The opposite occurs where technology companies cannot agree on standards for computer connections – which is why we have endless numbers of adapter types that fit USB, USB-C, and any number of other Samsung-only or iPhone-only ports.

HTML and TCP/IP protocols (the backbone of the Internet) are successful protocols and show what can happen when technologists get it right.

The same choice is there for data sharing solutions. Code can be written for any particular solution. We could build one big integrated data store (on the cloud perhaps) that lapped up every specific kind of data people threw at it without any standards. But it is not hard to see how costly and fragile that would be. There are advantages to a standards-based approach: solutions then become more scalable, cheaper to implement, interoperable.

The Integrated Data Infrastructure (IDI) solution run by Statistics New Zealand is costly because there are no metadata standards across the social and productive sectors that would allow Statistics NZ easy and low-cost integration of that data. When things change at source, this can break the integration point solution. The same problem arose for a budget bid to integrate education sector data. Not much thought was given to first developing metadata and API standards which would make the solution less costly by tens of millions, and less fragile once built.

Emerging practice for smarter companies and institutions is the use of metadata standards and other protocol-based approaches to reduce cost and increase value. Xero uses “APIs” (Application Program Interfaces) that allow a thriving ecosystem of third party-providers to develop bespoke solutions for specific accounting needs. There are “Farming Accounting” add-ons, for example, that hook into Xero’s data sharing ecosystem. IRD in turn is now publishing APIs (rather than a hardwired point solution) to allow third parties like Xero and MYOB to hook into tax data to develop integrated tax-accounting solutions.