Big data, issues and policies
I’ve been working this week on a report on data. I am part of a small team and the bit they have asked me to do is the use of big data, and particularly geo-spatial data, for governments. I am surprised by how much use is already being made of data, although patterns seem very uneven. We did a quick brainstorm in the office of potential areas where data could impact on government services and came up with the following areas:
- Transport
– infrastructure and maintenance
- Council Services
– planning
– Markets/Commerce
– Licenses
- Environmental Services
– Waste and Recycling
– Protection
– Climate
– Woodlands
– Power monitoring
– Real – time monitoring
- Health Services
- Planning
- Employment
- Education
- Social Services
- Tourism
- Heritage Services
- Recreational Services
- Disaster response
- Disease analysis
- Location tracking
- Risk management/ modelling
- Crime prevention
- Service Management
- Target achievements
- Predictive maintenance
There seems little doubt that using more data could allow national, regional and local governments both to design more effective, efficient and personalised services. However there remain considerable issues and barriers to this development. These include:
- Lack of skills and knowledge in government staff. There are already predictions of skills shortages for data programmers and analysts. With the rapid expansion in the use of big data in the private sector, the relatively lower levels of local government remuneration may make it difficult to recruit staff with the necessary knowledge and skills.
- Pressure on public sector budgets. Although there are considerable potential cost savings through the use of big data in planning and providing services, this may require considerable up front investment in research and development. With the present pressure on public sector budgets there is a challenge in securing sufficient resources in this area. Lack of time to develop new systems and services
- Lock-in to proprietary systems. Although many of the applications being developed are based on Open Source Software, there is a danger that in contracting through the private sector, government organisations and agencies will be locked into proprietary approaches and systems.
- Privacy and Security. There is a general societal issue over data privacy and security. Obviously the more data available, the grater the potential for developing better and cost effective services. At the same time the deeper the linking of data, the more likely is it that data will be disclosive.
- Data Quality and Compatibility. There would appear to be a wide variety in the quality of the different data sets presently available. Furthermore, the format of much published government data renders its use problematic. There is a need for open standards to ensure compatibility.
- Data ownership. Even in the limited field of GIS data there are a wide range of different organisations who own or supply data. This may include public agencies, but also for instance utility and telecoms companies. They may not wish to share data or may wish to charge for this data.
- Procurement regulations. Whilst much of the innovation in the use of data comes from Small and Medium Enterprises, procurement regulations and Framework Contracts tend to exclude these organisations from tendering for contracts.
