Are There Just Too Many [Open Data Publishing] Cooks in the Kitchen?
Before companies can use Open Data (defined as government data that is deliberately shared in a machine-readable format for free reuse by others), it has to earn a reputation for reliability. No company wants to use inconsistent data sets full of errors.
Today, many governments provide Open Data sets which are typically published by many different entities and agents. Unfortunately there are wide variations in this publishing output, in terms of both quality and consistency. Does this mean there are just too many cooks in the kitchen…or is the recipe at fault?
Imagine going to a restaurant where several cooks prepare your meal, each in their own way and using their own styles and flavors. Unfortunately, every time you order a steak with pepper sauce, your meal is likely to look and taste quite different. That’s exactly what is happening with Open Data! Too many departments are publishing their data using their own ‘styles and flavors’, without agreeing on common formats, standards, and vocabularies. Too often, this leaves the user to figure out what the right data model is and, eventually, translate items of data into a single format in order to combine them.
When companies see variation like this, they won’t use the data. When governments adhere to the same recipe at all times, their Open Data is consistent and therefore becomes extremely valuable. As I mentioned in a previous blog, governments must demonstrably recalibrate their standards and processes before Open Data becomes a trusted input source.
Does this sound familiar? Yes, it’s very similar to the process of creating data warehouse and business intelligence solutions. These are processes that have existed for many years. In fact the same tools and techniques – including master data management (MDM) – can be used for Open Data. So governments don’t need a whole new kitchen and, so long as they agree on a recipe and all follow it, there can be multiple cooks.
The ultimate recipe for Open Data success is very simple – the consistent application of agreed publishing structure and rules.
Yves Vanderbeken is an Hewlett Packard Enterprise account chief technologist and lead enterprise architect for the Flemish government and local governments (Flanders). Since 2011, Vanderbeken has been a core team member of the Flemish government’s Open Data team. He defined the technical strategy for setting up the Open Data platform and extracting, transforming, and publishing information in a consistent manner. He is the co-author of the Open Data Handbook published by the Flemish Government in February 2014. Vanderbeken coaches and advises various departments on their Open Data Master Plan and how to disclose data from their source systems to the Open Data platform, ensuring data quality and consistency across data sets.