11 Horror Stories of Open Data
A cathartic session of data ranting, where Open Data Camp attendees shared their data horrors under the Chatham House rule:
Horror: A PDF full of screenshots
Looking for the location of fire hydrants? If you make FOI requests, you’ll be told they’re national security, or private data or… One council did send the info – but as a PDF. And in the PDF? Screenshots of Excel spreadsheets.
Lesson: Ask for a particular format…
Horror: Paved with good intentions
A government ministry was asked for its spending data, but had to be fought all the way to the Information Commissioner, because they argued that they had intended to publish, and that was enough to give them leeway not to publish. he Information Commissioner disagreed.
Lesson: Just saying “intent” does not let them off the hook
Horror: Customer Disservice
An angry Twitter user asking about his broadband speed was sent a huge dataset of broadband speeds by postcode, as a zipped CSV. And was a bit cross when he realised he couldn’t use it. So a member of the organisation helped out by creating way of reading it – and got told off by his manager for helping the public.
Lesson: No good deed goes unpunished.
Horror: The art of the inobvious search
Googling a list of GP locations, they found an NHS search service – no place to download it. ONS? 2006 data. It took her getting angry, walking away from the computer, and coming back and making a ridiculous search to find it. If you aren’t make it accessible, why bother?
Lesson: Just creating data isn’t enough.
Horror: the paper-eating monster
A government department transformed a big CSV into a bigger XML file. They went to a meeting to discuss it – someone hit the print button and when the 45 minute meeting was done, it was still printing. It might still be printing today.
Lesson: Give unto digital what is digital’s…
Horror: Electoral art boundaries
Looking for polling stations data? Good luck. Some authorities hold it as maps. With hand-drawn boundaries. For the last election. So useful.
Lesson: Elections are an art, not a science
Horror: Bus overflow
Want to know how much a bus company charges? One considers that commercially sensitive information. But… it’s a public service. So they should release it. But then, says the bus company, local councillors might challenge them on it. So they won’t do it. They’re now having to crowd-source that information.
Lesson: Keep your bus tickets
Horror: Don’t publish and be damned
Journalists are making many, many FOI requests – and then not releasing what they get as open data. So, in fact, they’re wasting public money. Nice work, journalists.
Lesson: Never trust a hack… to understand open data.
Horror: Proprietary timetables
While building a bus timetable application for a website, they discovered that individual councils had their timetable data, but not matched when they cross county boundaries. And there’s a standard format – for three fields. All the rest is proprietary.
Lesson: Bus companies are evil. Just accept it.
Horror: Passive aggressive publishing
Some people publish the bare minimum data in a hostile format (PDF) without useful fields (dates).
Lesson: FOI them until they break.
Horror: The Hansard Recursion
A couple of year ago, on a Hansard-based project, they they started working with TheyWorkForYou. They got a unique reference for that debate. They used that on the API – and got a single paragraph back. Their resident Hansard expert gently, but sadly, explained that because it was based on the printed edition, you had to recurse through the paragraphs, until you hit the next debate.
Lesson: Quill and Ink do not good data formats make