October / November ’15 Report from the Data Infrastructure Team
Hi Folks,
Welcome to our new regular report which will capture a broader work program including: data.gov.au; the NationalMap; the work we are doing on the Open Data Network; data skills and capability development across the APS; and any other news from the rest of the Public Data Branch.
Our team has a new name (Data Infrastructure and Government Engagement) within the Public Data Branch– external site at PM&C. We also have several new members and some new responsibilities (like theNationalMap– external site). You’ll be seeing more about the NationalMap in the future as we integrate it into our regular reporting. We’ve already enabled it as the primary visualisation tool for spatial data on data.gov.au. Below you’ll find an example of the Native Title Determinations Outcomes dataset– external site.
The data publishers amongst our readers will be keen to read up on csv-geo-au– external site. A specification for displaying Australian spatial data captured in tabular datasets.
Speaking of previews, we’ve just introduced a new plugin for data.gov.au that lets you peek at the content of uploaded zip files. This includes those not physically hosted on data.gov.au. We think you’ll find this a helpful way to quickly identify interesting and useful data within zip files.
The view will be automatically added to any new zip file uploaded to data.gov.au. The data custodian can add the view to existing resources by:
- Browsing to the zipped resource
- Clicking the Manage button
- Clicking on the Views tab link
- Clicking on the New view button and select ZIP viewer
- Entering an appropriate Title (we suggest Zip view)
- Clicking the Add button
We are also developing a plugin for data.gov.au that automatically extracts and hosts machine readable data from zip files making them API enabled.
Unpublished Datasets
You may have noticed unpublished datasets– external site being added to data.gov.au in the last few weeks. These records are pointers to Australian Government data holdings that may have not been released yet or have additional usage caveats. The listing of this data on data.gov.au makes it easier to discover this data and improves user’s knowledge about data holdings. Like, for example, did you know that the Attorney General’s Department has a register of marriage celebrants– external site? To access these datasets users should get in touch directly with the data custodian in the first instance. The contact point for each dataset is listed as part of its metadata (located under the Additional Info heading).
You may have noticed a drop in the dataset count in the last few weeks. This is down to a number of datasets being duplicated because of a harvesting issue. We’re exploring ways of mitigating this duplication going forward.
Skills Calendar and Data Roles
We are also pulling together a list of data related courses (e.g. data analysis, statistics, R, python) and putting them into a training calendar for 2016. Send us an email if you have or know of any short courses that you think the wider APS should know about.
We are also working to define some agreed data roles and competencies. What we have found is that a good data analyst is someone with a composite of many skill sets. Working closely with a number of agencies we are hoping to provide clarity around the data roles required to support the functions of government.
NationalMap Forum
We held a NationalMap Forum on 24 November to facilitate an understanding of the NationalMap as a tool to visualise and embed geospatial data and for users to exchange information and experiences. Presentations covered the current version of the NationalMap, upcoming modifications, data related matters and other maps built on the NationalMap software. The forum also provided an opportunity for attendees to voice their current imperatives and future needs.
We will be holding a similar workshop for the data.gov.au platform on Wednesday, 16 December. There will be two sessions, both intended to set an environment for informal dialogue.
Data.gov.au Forum
Session 1 – Presentations: 9.00am – 11.00am
- Presentations will be from the Department of the Prime Minister and Cabinet (Public Data Branch) and guest agencies will cover:
- The Public Data Agenda and strategic approach to data
- Introduction to and the future of data.gov.au
- Using Data to Inform, Analyse and Advise
- Publishing for Efficiency
- Automating Data
- Value of Data
- Opportunity for those involved in policy / project management roles to provide feedback and discuss future needs.
Session 2 – Workshop: 11.00am – 1.30pm
- Interactive small group technical discussions with the data.gov.au team and Data61 to provide information / assistance on formatting and infrastructure issues.
- Divided into small group workshops:
- Using data.gov.au and publishing data for beginners
- Advanced technical questions on data.gov.au including automating data publishing
- Feature requests and building a roadmap for data.gov.au
- There will be a concurrent session being run in a different room with members of the general public that use government data to identify data user needs.
- At 12:30 the workshops will wrap up and there will be an informal networking lunch where government and non-government participants can meet and greet to discuss data. If you do not wish to attend the lunch session, all formal workshop activities will be finalised by 12:30.
If you’d like to come along please send us an email to data@pmc.gov.au.
Cheers,
Allan
Data Infrastructure and Government Engagement team, Public Data Branch
| October | November | |
|---|---|---|
| Total Webpage Visits | 33,447 | 33,915 |
| Total Pageviews | 136,963 | 141,936 |
| Total Discoverable Datasets | 7,163 | 7,209 |
| Total Organisations | 169 | 170 |
| Total Data Resources / Files | 34,525 | 34,744 |
| Total Machine Readable / Data API Resources | 3,777 | 3,793 |
5 Most Active Organisations
- Geoscience Australia– external site
- Australian Institute of Marine Science– external site
- City of Melbourne– external site
- Brisbane City Council– external site
- City of Launceston– external site
Top 5 Organisations by Total Datasets
- Geoscience Australia– external site
- Australian Institute of Marine Science– external site
- Australian Antarctic Division– external site
- City of Melbourne– external site
- City of Greater Geelong– external site
5 Most Recent Published Datasets
- Social Security, Health and Related Information– external site
- ABN Bulk Extract– external site
- Telecommunications in New Developments– external site
- Indigenous Land Use Agreements (Registered or in Notification)– external site
- Income Management Summary by Measure inc BasicsCard– external site
5 Most Recently Updated Datasets
- Leading Indicator of Employment– external site
- Small Area Labour Markets– external site
- Internet Vacancy Index– external site
- LCC Primary Property Address– external site
- Sea Water Temperature Logger Data at Boult Reef, From 13 Sep 2013 To 13 Oct 2015– external site
| Most Highly Requested Data Sets | Votes | Status |
|---|---|---|
| Free the G-NAF Address Dataset– external site | 156 | In Review |
| Free Postcode and Postal Address Data– external site | 101 | In Review |
| NBN Datasets– external site | 53 | In Review |
| DCDB Cadastre– external site | 45 | In Review |
| Australians Schools– external site | 43 | In Review |