ConDatos Talk notes: Open data as strategy

[Summary: Notes from a conference talk]

Last week I was in Colombia for AbreLatam and ConDatos, Latin America’s open data conference. Thanks to a kind invitation from [Fabrizio Scrollini](http://www.twitter.com/Fscrollini], I had the opportunity to share a few thoughts in one of the closing keynotes. Here is a lightly edited version of my speaker notes, and full slides are available here.

Open Data as strategy

In a few months, Barak Obama will leave the White House. As one of his first acts as US President, was to issue a memorandum on Transparency and Open Government, framed in terms of Transparency, Participation and Collaboration.

This memo has often been cited as the starting gun for a substantial open data movement around the world, although the roots of the open data movement go deeper, and in many countries adopting open data policies, they have blended with long-standing political priorities and agendas.

For myself, I started studying the open data field in 2009: exploring the interaction between open data and democracy, and I’ve been interested ever since in exploring the opportunities and challenges of using open data as a tool for social change.

So, it seems like a good time to be looking back and asking where have we got to eight years on from Obama’s memo, and nearly ten years since the Sebastapol Open Government Data Principles?

We’ve got an increasing number of datasets published by governments. Data portals abound. And there are many people now working in roles that involve creating, mediating, or using open data. But we’ve still got an impact gap. Many of the anticipated gains from open data, in terms of both innovation and accountability, appear not to have been realised. And as studies such as the Open Data Barometer have shown, many open data policies have a narrow scope, trying to focus on data for innovation, without touching upon data for transparency, participation or collaboration.

Eight years on – many are questioning the open data hype. We increasingly hear the question: with millions of datasets out there, who is using the data?

My argument is that we’ve spent too much time thinking about open data as an object, and not enough thinking about it as an approach: a strategy for problem solving.

Open data as an approach

What do I mean by this?

Well, if you think of open data as an object, it has a technical definition. It is a dataset which is machine-readable, published online, and free to re-use.

The trouble with thinking of open data in this way is that it ignores the contents of the data. It imagines that geodata maps of an entire country, when published as open data, are the same sort of thing as hospital statistics, or meta-data on public contracts. It strips these datasets from their context, and centralises them as static resources uploaded onto open data portals.

But, if we think of open data as an approach, we can get towards a much clearer picture of the kinds of strategies needed to secure an impact from each and every dataset.

What is the open data approach?

Well – it is about transparency, participation and collaboration.

So many of the policy or business problems we face today need access to data. A closed approach goes out and gathers that data inside the organisation. It uses the data to address the local problem, and all other potential uses of that data are ignored.

An open approach considers the opportunities to build shared open data infrastructure. An infrastructure is not just something technical: it involves processes of governance, of data quality assurance, and community building.

Building an open infrastructure involves thinking about your needs, and then also considering the needs of other potential data users – and working together to create and maintain datasets that meet shared goals.

Ultimately, it recognises open data as a public good.

Let me give an example

In the United States, ‘211 Community Directory services play an important role in helping refer people to sources of support for health or social welfare issues. Local 211 providers need to gather and keep up-to-date information on the services available in their area. This can be expensive and time consuming, and often involves groups collecting overlapping information – duplicating efforts.

The Open Referral initiative is working to encourage directory providers to publish their directories as open data, and to adopt a common standard for publishing the data. The lead organiser of the initiative, Greg Bloom, has invested time in working with existing system vendors and information providers, to understand how an open approach can strengthen, rather than undermine their business models.

In the early stages, and over the short-term, for any individual referal provider, getting involved in a collaborative open data effort, might involve more costs than benefits. But the more data that is provided, the more network effects kick in, and the greater the public good, and private value, that is generated.

This demonstrates open data as an approach. There isn’t an open referral dataset to begin with: just issolated proprietary directories. But through participation and collaboration, groups can come together to build shared open data that enables them all to do their job better.

It’s not just about governments pubishing data

It is important to note that an open data approach is not just about governent data. It can also be about data from the voluntary sector and the private sector.

With targetted transparency policies governemnts can mandate private sector data sharing to support consumer choice, and create a level playing field amongst firms.

As in the Open Referral example, voluntary sector and government organisations can share data together to enable better cross-organisation collaboration.

One of the most interesting findings from work of the Open Data in Developing Countries research network in Brazil, was that work on open data created a space for conversations between government and civil society about processes of data collection and use. The impact of an open data approach was not just on the datasets made available, but also on the business processes inside government. By engaging with external data re-users, government had the opportunity to rethink the data it collected, with potential impacts on the data available for internal decision making, as well as external re-use. We are seeing the same thing happen in our work on Open Contracting, which I will discuss more shortly.

The falacy of more data now

Before I move on, however, I want to caution against the common ‘falacy of more data now’.

There are many people who got into working with open data because they care about a particular problem: from local transport or environmental sustainability, to accountable politics, or better education. In exploring those problem, they have identified a need for data and have allied with the open data movement to get hold of datasets. But it is easy at this point to lose sight of the original problem – and to focus on getting access to more data. Just like research papers that conclude calling for more research, an open data approach can get stuck in always looking for more data.

It is important to regularly loop back to problem solving: using the data we do have available to address problems. Checking what role data really plays in the solution, and thinking about the other elements it sits alongside. Any only with a practical understanding, developed from trying to use data, of the remaining gaps, iterating back to further advocacy and action to improve data supply.

Being strategic

So, if open data is, as I’ve argued, an approach, how do we approach it strategically? And how do we get beyond local pilots, to impacts at scale?

Firstly, ‘open by default’ is a good starting point. Strategically speaking. If the default when a dataset is created is to share it, and only restrict access when there is a privacy case, or strong business case, for doing so – then it is much easier for initiatives that might use data for problem solving to get started.

But, ‘open by default’ is not enough. We need to think about standards, governance, and the ecosystem of different actors involved in creating, using, providing access to, and adding value on top of open data. And we need to recognise that each dataset involves it’s own politics and power dynamics.

Let’s use a case study of Open Contracting to explore this more. Colombia has been an Open Contracting leader, one of the founder members of the Open Contracting Partnership, and part of the C5 along with Mexico, France, Ukraine and the UK. In fact, it’s worth noting that Latin America has been a real leader in Open Contracting – with leading work also in Paraguay, and emerging activities in Argentina.

Open Contracting in focus

Public contracting is a major problem space. $9.5tn a year are spent through public contracts, yet some estimates find as much as 30% of that might leak out of the system without leading to public benefit. Not only can poorly managed contracts lead to the loss of taxpayers money, but corruption and mismanagement can be a source of conflict and instability. For countries experiencing political change, or engaged in post-conflict reconstruction, this issue is in even sharper relief. In part this explains why Ukraine has been such an Open Contracting leader, seeking to challenge a political history of corruption through new transparent systems.

Open Contracting aims to bring about better public contracting through transparency and participation.

Standards

To support implementation of open contracting principles, the Open Contracting Partnership (OCP) led the development of OCDS – the Open Contracting Data Standard (OCDS). When working with the Web Foundation I was involved in the design of the standard, and now my team at Open Data Services Co-operative continue to develop and support the standard for OCP.

OCDS sets out a common technical specification for providing data on all stages of a contracting process, and for linking out to contracting documents. It describes what to publish, and how to publish it.

The standard helps Open Contracting scale in two ways:

• Firstly, it makes it easier for data publishers to follow good practices in making their data re-usable. The standard itself is produced through an open and collaborative process, and so someone adopting the standard can take advantage of all the thinking that has gone into how to model contracting processes, and manage complex issues like changes over time, or uniquely identifying organisations.

• Secondly, the standard is built around a number the needs of a number of different users: from the SME looking for business opportunities, to the government official looking to understand their own spend, and the civil society procurement monitor tracking contract delivery. By acting as a resource for all these different stakeholders, they can jointly advocate for OCDS data, rather than working separately on securing separate access to the particular data points they individually care about.

Importantly though, the standard is responsive to local needs. In Mexico, where both the federal government and Mexico City have been leading adopters, work has taken place to translate the standard, and then to look at how it can be extended and localised to fit with national law, whilst also maintaining connections with data shared in other countries.

Governance & dialogue

When it comes to enabling collaboration through open data, governance becomes vitally important. No-one is going to build their business or organisation on top of a government dataset if they don’t trust that the dataset will be available next year, and the year after that.

And governments are not going to be able to guarantee that they will provide a dataset year after year unless they have good information governance in place. We’ve seen a number of cases where data publishers have had to withdraw datasets because they did not think carefully about privacy issues when preparing the data for release.

For Open Contracting, the standard itself has an open governance process. And in Open Contracting Partnership ‘Showcase and Learning Projects’ there is a strong emphasis on building local partnerships, making sure there is dialogue between data publishers and users – creating the feedback loops needed to build a data infrastructure that can be relied upon.

In the UK, adopting of OCDS will soon give the government a better view of how far different departments and agencies are meeting their obligations to publish contracting information. By being transparent with the data, and being transparent about data quality, civil society and the private sector can get more involved in pushing for policies to be properly implemented: combining top-down and bottom-up pressure for change.

Support and community

One of the most important lessons for us from Open Contracting has been that scaling up open data initiatives is not just about standards and technical specs, but it is also about relationships, community and providing the right support at the right time.

The Open Contracting Partnership invest in bringing together champions of open contracting from across the world to get inspired and to share learning. Because they are working with common standards, ideas and tools are more easily transferable. And as I mentioned earlier, thinking about how to improve their open data also creates opportunities for groups to think about improving their internal systems and processes.

In addition, my team at Open Data Services Co-operative provide the ‘technical helpdesk’ for OCDS. We offer e-mail, phone and workshop support to governments working to publish their data, and to groups seeking to use open contracting data. Our goal is to make sure that when data is published, it is easy-to-use, and that all the small barriers to data re-use that exist for so many other datasets are not there when you come to an open contracting dataset.

We do this because data standards are only as strong as their early implementations. But we’re not aiming to be the only support provider for OCDS. In fact, we’re aiming to stimulate an ecosystem of support and data re-use.

Ecosystem

A strategic approach to problem solving with open data needs us to recognise the different roles in a data value chain. And to think about what elements need to be kept open for a vibrant ecosystem, and where to create space for proprietary business models.

If governments need consultancy support to improve their systems to produce OCDS data, and a marketplace of expert consultants develops, this is a good thing for scaling adoption. If private firms build value-added data analysis tools on top of contracting data, this is something to welcome that can scale use.

But if the originally published data is messy, and firms have to spend lots of money cleaning up the raw data before they use it, then barriers to market entry are created. This stiffles innovation, and leads to services only accessible to wealthy private sector, excluding civil society data users.

That’s why there is a need for a range of different actors, public, civil society and private, involved in a data ecosytem – and space for a range of business models.

Business models

I’ve been asked to touch in particular on business models in this talk – not least because the company I’m part of, Open Data Services Co-operative, has been exploring a different model to scale up support for open data.

We’re set up as a Workers Co-operative: a model where Latin America has a strong history. In a workers co-op, the staff own the business: and make decisions about it’s future. This might not sound that significant, but it has a number of distinction against other models:

(1) It’s a business, not a charity. This can give us the flexibility to innovate, and the drive to find sustainable models for our work. Right now, we work through a mix of contracts for technology research and development, and through providing ongoing support for data standards, often ultimately funded by donors who believe in investing in public good open data infrastructure.

(2) Organic instead of investment growth. A lot of the examples used when talking about tech businesses are born out of massive-scale silicon valley investments. Most co-operative models are based on growing through member contributions and revenue, rather than selling equity. Although we are set up as a workers co-op, there are growing discussions around ‘platform co-operatives’ and ‘data co-operatives’, in which those who could benefit from shared data infrastructure collectively support its development through a co-op model.

(3) Social mission focus. We want to provide jobs for people – particularly growing the community of professionals working on open data, as we recognise there are limited opportunities for stable open-data and social change focussed jobs. But we also want to have an impact on the world, through enabling open data-approaches to problem solving. As a worker owned business, we’re not focussed on profit for shareholders or an external owner, but on developing effective projects, and contributing to the wider community and issues we care about.

When it comes to scale, for a co-operative the question is about reaching the right scale, not about unlimited growth. That’s why as demand has been growing for support on the Open Contracting Data Standard in Latin America, we’ve been working with the Open Contracting Partnership to put out a call for a new partner organisation to take on that role – co-operating alongside Open Data Services to provide services across the region.

If anyone would like to find out more about that opportunity – please do check out the details online here.

I’m not here to argue that co-operatives are the only or the best business model for working with open data – but I do encourage you to think about the different models: from supporting individual entrepreneurs, to building open data activities into existing organisations, and supporting the emergence of co-operative groups that can catalyse a wider re-use ecosystem.

Recap

So let’s recap.

• Open data is an approach, not an object

• Open approaches win out over closed approaches when it comes to creating both social value, and opportunities for innovation and growth

• But, we need to be strategic about open data: using it for problem solving, and making sure data quality and reliability is good enough for ongoing re-use

• An we need sector-specific approaches, with a mix of different organisations involved

I’ve shared case studies of Open Referral and Open Contracting. But in other sectors the right approaches may be different.

My challenge to you is to think about you can apply values of transparency, participation and collaboration to your open data projects, and how you can act strategically to make use of standards, community engagement and support for data publishers and users in order to build vibrant ecosystems.