October / November ’15 Report from the Data Infrastructure Team

Hi Folks,

Welcome to our new regular report which will capture a broader work program including: data.gov.au; the NationalMap; the work we are doing on the Open Data Network; data skills and capability development across the APS; and any other news from the rest of the Public Data Branch.

Our team has a new name (Data Infrastructure and Government Engagement) within the Public Data Branch– external site at PM&C. We also have several new members and some new responsibilities (like theNationalMap– external site). You’ll be seeing more about the NationalMap in the future as we integrate it into our regular reporting.  We’ve already enabled it as the primary visualisation tool for spatial data on data.gov.au. Below you’ll find an example of the Native Title Determinations Outcomes dataset– external site.

an image demonstrating the native title determinations outcomes dataset visualized on the nationalmap

 

The data publishers amongst our readers will be keen to read up on csv-geo-au– external site. A specification for displaying Australian spatial data captured in tabular datasets.

Speaking of previews, we’ve just introduced a new plugin for data.gov.au that lets you peek at the content of uploaded zip files. This includes those not physically hosted on data.gov.au. We think you’ll find this a helpful way to quickly identify interesting and useful data within zip files.

a preview of how the zip viewer functionality works

The view will be automatically added to any new zip file uploaded to data.gov.au. The data custodian can add the view to existing resources by:

  1. Browsing to the zipped resource
  2. Clicking the Manage button
  3. Clicking on the Views tab link
  4. Clicking on the New view button and select ZIP viewer
  5. Entering an appropriate Title (we suggest Zip view)
  6. Clicking the Add button

We are also developing a plugin for data.gov.au that automatically extracts and hosts machine readable data from zip files making them API enabled.

Unpublished Datasets

You may have noticed unpublished datasets– external site being added to data.gov.au in the last few weeks.  These records are pointers to Australian Government data holdings that may have not been released yet or have additional usage caveats. The listing of this data on data.gov.au makes it easier to discover this data and improves user’s knowledge about data holdings. Like, for example, did you know that the Attorney General’s Department has a register of marriage celebrants– external site?  To access these datasets users should get in touch directly with the data custodian in the first instance. The contact point for each dataset is listed as part of its metadata (located under the Additional Info heading).

You may have noticed a drop in the dataset count in the last few weeks. This is down to a number of datasets being duplicated because of a harvesting issue. We’re exploring ways of mitigating this duplication going forward.

Skills Calendar and Data Roles

We are also pulling together a list of data related courses (e.g. data analysis, statistics, R, python) and putting them into a training calendar for 2016. Send us an email if you have or know of any short courses that you think the wider APS should know about.

We are also working to define some agreed data roles and competencies. What we have found is that a good data analyst is someone with a composite of many skill sets. Working closely with a number of agencies we are hoping to provide clarity around the data roles required to support the functions of government.

NationalMap Forum

We held a NationalMap Forum on 24 November to facilitate an understanding of the NationalMap as a tool to visualise and embed geospatial data and for users to exchange information and experiences. Presentations covered the current version of the NationalMap, upcoming modifications, data related matters and other maps built on the NationalMap software. The forum also provided an opportunity for attendees to voice their current imperatives and future needs.

We will be holding a similar workshop for the data.gov.au platform on Wednesday, 16 December. There will be two sessions, both intended to set an environment for informal dialogue.

Data.gov.au Forum

Session 1 – Presentations: 9.00am – 11.00am

  • Presentations will be from the Department of the Prime Minister and Cabinet (Public Data Branch) and guest agencies will cover:
    • The Public Data Agenda and strategic approach to data
    • Introduction to and the future of data.gov.au
    • Using Data to Inform, Analyse and Advise
    • Publishing for Efficiency
    • Automating Data
    • Value of Data
  • Opportunity for those involved in policy / project management roles to provide feedback and discuss future needs.

Session 2 – Workshop: 11.00am – 1.30pm

  • Interactive small group technical discussions with the data.gov.au team and Data61 to provide information / assistance on formatting and infrastructure issues.
  • Divided into small group workshops:
    • Using data.gov.au and publishing data for beginners
    • Advanced technical questions on data.gov.au including automating data publishing
    • Feature requests and building a roadmap for data.gov.au
  • There will be a concurrent session being run in a different room with members of the general public that use government data to identify data user needs.
  • At 12:30 the workshops will wrap up and there will be an informal networking lunch where government and non-government participants can meet and greet to discuss data. If you do not wish to attend the lunch session, all formal workshop activities will be finalised by 12:30.

If you’d like to come along please send us an email to data@pmc.gov.au.

Cheers,

Allan
Data Infrastructure and Government Engagement team, Public Data Branch

October November
Total Webpage Visits 33,447 33,915
Total Pageviews 136,963 141,936
Total Discoverable Datasets 7,163 7,209
Total Organisations 169 170
Total Data Resources / Files 34,525 34,744
Total Machine Readable / Data API Resources 3,777 3,793

 

October
Datasets
October
Resources
October
datastore_sql_search
Energy Rating Data for household appliances – Labelled Products– external site Air Conditioners– external site 1,795
ASIC – Company Register– external site Company Register – Current– external site 963
Energy Rating Data for household appliances – Labelled Products– external site Fridges and Freezers– external site 608
Energy Rating Data for household appliances – Labelled Products– external site Televisions– external site 558
Energy Rating Data for household appliances – Labelled Products– external site Clothes Dryers– external site 454
Energy Rating Data for household appliances – Labelled Products– external site Dishwashers– external site 402
Energy Rating Data for household appliances – Labelled Products– external site Computer Monitors– external site 363
Australian Public Holidays Dates Machine Readable Dataset– external site Most Recent Update– external site 85
Historical Australian Government Contract Data– external site 2014-2015 Commonwealth Government Contract Data– external site 53
Commercial Building Disclosure– external site CBD Downloadable Data Set– external site 51
November
Datasets
November
Resources
November
datastore_sql_search
Energy Rating Data for household appliances – Labelled Products– external site Air Conditioners– external site 990
Energy Rating Data for household appliances – Labelled Products– external site Fridges and Freezers– external site 506
Energy Rating Data for household appliances – Labelled Products– external site Televisions– external site 485
Energy Rating Data for household appliances – Labelled Products– external site Dishwashers– external site 389
Energy Rating Data for household appliances – Labelled Products– external site Clothes Dryers– external site 351
Energy Rating Data for household appliances – Labelled Products– external site Computer Monitors– external site 350
Energy Rating Data for household appliances – Labelled Products– external site Company Register – Current– external site 265
ASIC – Company Register– external site Company Register – Current– external site 151
School Locations (Victoria)– external site all schools list.csv– external site 32
Historical Australian Government Contract Data– external site 2014-2015 Commonwealth Government Contract Data– external site 27

 

5 Most Active Organisations

Top 5 Organisations by Total Datasets

 5 Most Recent Published Datasets

5 Most Recently Updated Datasets

 

Most Highly Requested Data Sets Votes Status
Free the G-NAF Address Dataset– external site 156 In Review
Free Postcode and Postal Address Data– external site 101 In Review
NBN Datasets– external site 53 In Review
DCDB Cadastre– external site 45 In Review
Australians Schools– external site 43 In Review
Posted in Benefits of open data, Data, Informing Decision-making, Open Government, Posts from feeds, Transparency, Visualising data Tagged with: , , , , , ,