Deliver a well understood national digital identity program. Web20 offers some opportunities here, solutions like OpenID will work well for a large number of government services but may never quite go far enough when it comes to transactions that require more security or involve the transfer of funds between government and non-government entities.

Over the years many governments have experimented with national identity programs, and many have failed for one reason or another. A gov20 mindset offers the opportunity to try a new approach, federating a secure government identity program with other identity providers that citizens use every day from organizations like Microsoft, Google, Facebook or Twitter, along with the widely recognized and already federated approach of OpenID.

Not all government transactions need parties on either side to identify themselves, and very few need the security that is built into most government developed eID programs. In most cases the user just needs government to remember their preferences, and deliver the experience that they signed up for.

Deliver a SINGLE government experience. In many countries the gov20 experience is being rolled out on a department by department basis.

More traditional eGovernment programs have been very focused on delivering single and joined up government experiences for well over a decade now, and we need to ensure that we don’t lose sight of that focus as a more interactive gov20 experience takes hold.

As national governments reorganize and restructure the onus should not be on the citizen to know which department needs to manage the transaction that they need to complete. The nirvana of a citizen or a business being able to open a conversation with government as a single entity still feels like it is a long way off in a lot of jurisdictions.

Make transactions available externally, not just data. We’re all agreed that open government data is a powerful concept, we have already seen the possibilities as developers have built a wide array of new types of application that were hard to conceive without the data that government holds.

There is another optional step though.

Government don’t just hold data, government also has control over a vast number of transactions that could become integral parts of other applications, truly delivering citizen services in places where citizens expect to find them.

Imagine booking your next vacation. With your permission your favourite travel site should be able to check the validity of your passport, ensure you have a visa for your destination country and verify that you have the right vaccinations for the trip. As you travel home your credit card company should be able to interact with a government service to ensure that all that unnecessary sales tax that you spent is quickly refunded to you.

Internal government systems manage all of these transactions today (and about 10,000 more in any given country), it makes sense to allow others to build on them to provide new types of services that just can’t be offered today.

Semantic enablement of published datasets. My last post talked a little about the role of RDF in ensuring that we get value out of data that is published for external use.

As an international community we need to ensure that the semantic descriptors that are applied to datasets have some degree of harmony, allowing citizens to pull data from different jurisdictions to answer questions that they have.

A single understanding of published data would help consumers of these datasets at both ends of the scale. From a child using the data to complete a project for school, through to complex development of cross border policy.

Close the Digital Divide. This is an issue that never quite seems to go away, although many would argue that progress is being made. What we still rarely see however is clear articulation of how recently published gov20 plans will help close the digital divide, in fact in some cases the increasing numbers of services being offered online are only serving to make the problem worse.

New form factors for computing devices are certainly helping, and delivery of government services through technology that citizens already have access to (i.e. the TV, or the cell phone) is certainly a step in the right direction.

Gov20 plans need to carefully consider this issue in detail and find the right answer for their own national situation.

Conversations about the governance of standards setting organizations will from time to time stray towards the topic of reform, along with the need for simplicity in the standards development process. I would argue that the complexity that we deal with today is both necessary and an important component of a functioning ICT industry.

Today SSOs are very diverse. Almost all of them produce both effective standards and standards that never achieve marketplace acceptance. But the flexibility, competition and choice that this ecosystem provides is healthy for the ICT industry. The processes by which ICT standards are created can vary greatly and are constantly evolving. Formal ICT standards are developed in formal SSOs, industry consortia, professional associations, and other industry groups. Many of these diverse organizations have open and published processes that allow all relevant stakeholders to participate and help to balance conflicting requirements. Other ICT SSOs are more focused and less formal collaborations, which can produce needed standards that are very targeted in nature or which can incubate standards for further standardization at a more formal SSO.

While this diversity and breadth of SSOs can be perceived as overwhelming, the truth is that it has emerged as a result of the market in which we all operate. It provides for flexibility, competition and choice. No one process can guarantee that every standard it produces has some level of immediate intrinsic value. No one standards body or process necessarily produces “better” standards; again the test of success and relevance of a standard is the extent to which it ultimately gets used in the marketplace. (As an example, the IETF TCP/IP standard became much more widely implemented than the ISO OSI standard despite the fact that ISO has produced many other very successful ICT standards.) SSOs routinely review their activities, procedures and policies, and they make improvements and changes as needed.

Standards-setting organizations also have collaborative actions and liaisons between themselves, and with other bodies that support related conformance or interoperability testing, business initiatives and so on. Many standards make references to other standards coming from other SSOs or have ratification processes that they apply to other SSOs’ work.

There is certainly no shortage of people who find the diversity of SSOs confusing and frustrating, and from time to time I find myself in that camp. However, any conversation about making changes to the broad array of standards setting processes that we know today has to dig deeply into why we are where we are, and has to recognize the many successful standards that are already implemented by in hardware and software by many hundreds of developers using every possible language and from every possible discipline.

While the subject of reform is an important discussion point when it comes to SSO governance we risk the word bringing an unnecessary level of drama along with it. What I would personally rather see is a broader array of voices collaborating in the ongoing work to evolve the process that we have today to meet the needs that we know the industry will have in years to come.

We have a proven, solid and working standards setting system in place today and in my view it is important that as a community and as an industry we continue to build upon that

Well described and semantically tagged data is always going to be at the top end of that spectrum, with the additional metadata making it clearer to the developer what the purpose of the data is and how it can be used. 

I’ve talked before about how much work a government agency might want to put into publishing their data, and conversely how much use that data then is to consumers. With additional usefulness comes additional cost, complexity and governance. Individual governments will decide for themselves how far along this path they want to walk.

The Resource Description Framework (or RDF) has been around for the best part of a decade and was designed, in part, on the basis that we would eventually see massive amounts of machine processable data and information published for consumption on the web – exactly as we are beginning to see as governments publish various national and state level datasets.

As additional background reading on the topic, Tim Berners-Lee talks about how RDF might apply directly to Open Government Data in his June 2009 article: “Putting Government Data online”.

To help explain the concept the W3C have published a comprehensive RDF Primer, which begins with the introductory text;

The Resource Description Framework (RDF) is a language for representing information about resources in the World Wide Web. It is particularly intended for representing metadata about Web resources, such as the title, author, and modification date of a Web page, copyright and licensing information about a Web document, or the availability schedule for some shared resource. However, by generalizing the concept of a "Web resource", RDF can also be used to represent information about things that can be identified on the Web, even when they cannot be directly retrieved on the Web. Examples include information about items available from on-line shopping facilities (e.g., information about specifications, prices, and availability), or the description of a Web user’s preferences for information delivery.

RDF is intended for situations in which this information needs to be processed by applications, rather than being only displayed to people. RDF provides a common framework for expressing this information so it can be exchanged between applications without loss of meaning. Since it is a common framework, application designers can leverage the availability of common RDF parsers and processing tools. The ability to exchange information between different applications means that the information may be made available to applications other than those for which it was originally created.

… it then goes on to say;

[..] while English is good for communicating between (English-speaking) humans, RDF is about making machine-processable statements. To make these kinds of statements suitable for processing by machines, two things are needed:

• a system of machine-processable identifiers for identifying a subject, predicate, or object in a statement without any possibility of confusion with a similar-looking identifier that might be used by someone else on the Web.
• a machine-processable language for representing these statements and exchanging them between machines.

Anybody with an interest in this area should read both the full RDF primer and the W3C homepage for RDF.

As one further point I thought it might also be useful (in a slightly self serving way) to dig around and find areas where Microsoft has been working with RDF in either research projects, products or within various developer tools.

With the help of a colleague in Redmond we came up with the list below. It is not exhaustive, but does gives a representative sample of the type of work that the company is doing in this area;

(Portions of this article quote the WC3 Primer on RDF which is held under the Copyright of the W3C, full notice and further information: Copyright © 2004 W3C^® (MIT, ERCIM, Keio), All Rights Reserved. W3C liability, trademark, document use and software licensing rules apply.)

The Canadian City of Edmonton was in the news for a couple of unrelated reasons this morning, first of all as the Olympic torch arrived, and secondly for plans that they have to work with Microsoft’s OGDI platform.

From Peter Galli’s blog on Port25;

The Canadian City of Edmonton has become the first North American city to use Microsoft’s Open Government Data Initiative (OGDI) solution, and is working with the company to develop the region’s first public open data catalogue, an online site that will give citizens and developers easier and more transparent access to information and allow them to develop new solutions and suggest ideas that enhance public infrastructure and services.

More information on OGDI can be found here. From the page;

The Open Government Data Initiative (OGDI) is a cloud-based collection of software assets that enables publicly available government data to be easily accessible. Using open standards and application programming interfaces (API), developers and government agencies can retrieve the data programmatically for use in new and innovative online applications, or mashups that can help:

• Improve citizen services
• Enhance collaboration between government agencies and private organizations
• Increase government transparency
• And more…

OGDI promotes the use of this data by capturing and publishing re-usable software assets, patterns, and practices. The data repository already holds over 60 different government datasets that are readily available for use in new applications, and is continuously updated with additional government datasets.

Edmonton’s Open Data Catalogue is already public and operating in “community preview mode”’ if you want to go take a look.

A few of the items on the list continue from last year, and there are a couple of obvious new ones.

Gov20. This has been a major focus area in Australia and New Zealand over the last twelve months and a great deal of progress has been made. This coming year needs to be a year that sees implementation of many of the Gov20 ideas and proposals. There were some grand and exciting plans discussed during 2009 and it would be great to see them become reality.

Increased focus on the semantic web. I hear you yawning already, but I think that some of the spill over topics from the Gov20 conversation (massive amounts of published government data for example) coupled with advancing semantic search tools, along with semantic tools appearing in products like Bing, have the potential to give this topic a push.

Government Interoperability Frameworks. In a post from March of last year I suggested that it was time for governments to begin to rethink the way that their interoperability frameworks are written. Many of them have devolved into little more than a list of standards, delivering very little by way of interoperability between government systems and people. I’m already seeing some of the Australian state governments taking a more scenario based approach to interoperability, along with work from organizations like CSTransform offering new ideas around government interoperability policy.

The move to IPv6. The move is not going to happen in 2010, but by all accounts the date when we will see IPv4 address space exhausted is getting closer. Couple that with IPv6 dependant features creeping into several market available operating systems and I would expect to be involved in more discussions around IPv6 in 2010 than in previous years.

National broadband networks. Each of the respective national broadband networks in Australia and New Zealand has the potential to have a significant impact on the way that we think about, build and implement software standards. Organizations like the Institute for a Broadband Enabled Society (IBES) at Melbourne University are starting to think about use scenarios for these networks and in 2010 I would expect to see a similar organization form in one of New Zealand’s Universities.

The future role of IT standards. This might sound like an odd one. Standardization as we know it today is a long and arduous process, but for many good reasons. Developing an ISO level standard involves process to consult over one hundred nations at a national level and will sometimes involve many hundreds of contributors and reviewers. As Web20 technologies play a bigger role in government systems delivery I’m expecting to participate in conversations about how we speed up the standards development process to accommodate rapidly changing public API and data definitions. (think Twitter and Facebook).

Technology to enable standards development. This final point involves a project that myself and a handful of colleagues have been discussing lately. If we’re going to see broad participation in the standards development process, for IT or other markets, we need to think about ways of reducing the cost and complexity of involvement. Technology itself potentially has a role to play here if we can find low cost ways of enabling cross group collaboration for any standards development community. More on this later…

On a personal note, last year was a challenging one for me. I made a conscious decision to spend more time at home with the family, and took on a new role in Microsoft that would make that possible. So far, five months later, it has been a significant mental shift and has obviously resulted in a very different daily routine.

My previous three years were spent as Regional Technology Officer for our Asia Pacific business and pulled me into discussions about pretty much any technology policy related topic across Asia and the South Pacific. A broad set of topics spread across an even broader set of countries, both in terms of geography and demographics.

This new position revolves around the standards setting communities in Australia and New Zealand with three main areas of focus;

• Increase Microsoft participation in local standards development and testing activity
• Understand standards related requirements and policies in Australia and New Zealand
• Help the product development groups understand how they need to support us

The first five months have been a genuine voyage of discovery for me. I’ve met a long list of people in both countries with a lot of enthusiasm for IT standardization and all that it entails, I’ve been introduced to each of the respective government views on the role of IT standards in systems design and deployment (sometimes expressed in no uncertain terms!) and I’ve crossed the Tasman Sea more times than I care to count.

At the same time I’ve been discovering new parts of Microsoft. For the last fifteen years I have held various senior roles in our Public Sector sales organization, leaving that behind to join a more central corporate function has involved building new networks inside of the company and expanding the number of topics that I need to involve myself with inside of Microsoft. On some levels it amazes me that it is possible to spend almost a decade and a half with a company and still find news areas that drive significant personal growth within the organization.

Finally, the field facing team that I’m now part of is a new component to our corporate standards group. Our corporate standards organization has been around for a few years now but last year was the first time we put dedicated people in the national subsidiaries with a focus on standards related work. The result, as with any new team, is that we have had to find ways to work together. Just getting to know the people that I work directly with has taken up a significant chunk of time over the last few months.

In many ways 2009 was a reboot and restart year for me… it involved massive personal and professional change and I think I’m only just starting to feel like I have some sense of clarity around what I need to be doing from here.

I’m conscious that a number of things I wanted to do in 2009 didn’t get done, my most significant failing over the last few months has probably revolved around communication. This blog is an example, it has been getting very little of my attention in recent months – I’m assuming that readers have little or no interest in the details of my long list meetings involving introductions to new people and organizations.

So I begin 2010 with a long list of new people that I now work with (inside and outside of Microsoft), some ideas around how I think Microsoft can do a better job of working in this area in Australia and New Zealand and a handful of personal projects that I want to get done over the next twelve months that I think will benefit the standards development environment in general.

I’ll find the time over the next few days to write up a few of those in detail… I promise!

There is no debate that standards have always played an important role in the design and delivery of eGovernment systems, since the mid-1990s we have been seeing standards play critical roles in data exchange, authentication and the way that information is ultimately presented back to the user.

Early eGoverment systems represented a small revolution for many governments, providing ways to increase levels of administrative efficiency while at the same time providing services that were much more broadly available than in previous years.

However, if we turn and look specifically at agencies responsible for archiving governments information then the shift to digitally delivered services also brought some new challenges. Archiving paper is well understood, archiving digital records adds complexity that is still being worked out in many jurisdictions. Only now are we starting to see standards emerge for storage and maintenance of these digital records, over ten years after we saw the mass shift to digital by governments all over the world.

The issues will be obvious to many, governments have a decade wide void in the records that they have managed to keep, information has been simply lost as individual computers and email inboxes have been redeployed or the hardware has been recycled.

At this point in time we are witnessing a second iteration of that revolution, governments, citizens and businesses are collaboratively talking about Government 2.0 (or gov20), examining ways that they will use microblogging, social media and the publishing of massive government datasets to find new ways for government to interact with citizens and for developers to deliver a range of tools that could not be developed by government alone.

Within these gov20 conversations we are seeing more than just the digitization of government services, politicians are finding new ways of communicating directly with their electorate and senior departmental officials are finding new ways to more deeply understand the people that their services ultimately serve.

So once again we are seeing a massive shift in the technology that is being used to run the business of government, and once again we don’t yet have the standards to retain the conversations that take place over microblogging services, or the huge amount of inbound information that departments will eventually use as part of their decision making processes that they collect from an array of social networking tools.

As a standards community, in support of the ongoing evolution of eGovernment, now is the time for us to start to think about how we will solve these complex challenges. Work needs to begin on archiving standards that will retain the information that is driving decisions today and as technology plays an increasingly larger role in the business of government archiving standards needs to be a core part of systems design, not a problem that we try and solve after the fact.

Today most of us search in a single dimension, we tap a term into our favourite web search tool and get back a list of links that represent pages that are currently published somewhere on the net. Most of us are not planning on doing any level of analysis on that information, we are just trying to find something, so the list of links are enough for us.

Governments are starting a new trend though, massive amounts of machine readable data that we can use to draw our own conclusions to complex questions about our environment or our society.

In his now infamous TED presentation, “Let my dataset change your mindset”, Hans Rosling gave us a preview of the way that many of us will be using these government datasets in years to come, along with similar datasets that we will eventually see commercial organizations publishing in the same way.

Using available data, developers will continue to build new applications that could never have been funded by government, citizens and businesses will be able to offer complex and well thought out advice to policy makers, economists will be able to build empirical models that demonstrate societal trends and eventually historians will reconstruct the environment that we leave behind.

For all of this to work internet search has to evolve, a list of links won’t meet our needs. Here’s three examples;

First of all, a piece that we’re close today, we need to be able to search by geography. When we begin to break down massive datasets the geography becomes important, any piece of data has a special meaning when we can tie it to a country, a county, a town or a particular street. Most of the government policy makers I meet have had a long term understanding of the role of geographic data in government process, but few tools exist to enable the publishing of that data externally in a way that is useful.

Secondly, we need to be able to search by timeframe. Future analysis of data, either for an economist constructing trends over a limited number of years or long term reviews by historians will require us to find a way to roll datasets back to a point in time that is relevant to the users analysis.

Finally, it is not enough for a single country to solve this, international standards need to evolve to support this type of search.

Very quickly we will find ourselves at a point where it will not be enough for us to look at an issue in the context of a single country. In the short term, policy advice to a given government could be enhanced by the ability to cross analyze that advice with data from similar nations – e.g. to lower the cost of building a kilometre of road in New Zealand, I might also want to look at the costs in the UK, Canada and Australia. – and in centuries to come historians will need a way to show how global society evolved.

We are not far from a point where we are going to see a need to enable a software instigated search for data relating to a particular issue, in a certain place and during a given timeframe.

It is then that we will really begin to experience the power that published data gives us.

For governments, the digital revolution presents some important challenges, including how best to ensure that their digital documents will be accessible and readable essentially forever, regardless of how technology evolves. This is a question I encounter often in my role as a Regional Standards Officer, working with various public agencies in Australia and New Zealand that have responsibility for archiving records and other documents.

Fortunately, the challenge of e-archiving has been addressed over the past decade or so through standards development and other collaborative efforts around the world. International standards offer guidance on how to capture documents and how to use metadata to ensure that they can be located and understood by future generations. Published specifications for the Open Document Format (ODF), Open XML and Portable Document Format (PDF) also ensure that documents in these formats will be readable long after the formats are obsolete.

That said, a lot more work is still to be done.

Although ODF, OpenXML and PDF account for the bulk of documents produced today, other formats need to be documented. Microsoft has put documentation for our Office binary formats, for example, into the hands of the British Library for reference by future generations. We encourage other vendors to take similar steps with their storage formats.

New standards are needed to ensure retention of at least some of the massive volume of data produced in social networking activities, such as the micro-blogging increasingly used by elected officials to interact with constituents. Broad collaboration on enhanced standards and processes would help clarify how much of this data needs to be retained, and how to store it.

Microsoft is committed to working with governments and all other interested parties to meet these challenges and preserve the history of this decade, and the next, and the next.

As I talk to friends in government, participate in various events around open data and watch the work that developers are doing in this area I can’t help but think that there is a massive gap between the expectations that government leaders have for open data, the tools that developers are building and the objectives of the general public.

On one side of the conversation you have government ministers and departmental heads. My conversations with this group often come back to ways that data can be used to make government more efficient. Ministers have looked at the success of crowd sourcing in other sectors and are keen to find ways to apply those ideas to the machinery of government.

In many cases ministers are looking for assistance from the community to analyse the cost of building a kilometre of road, maintaining a hospital bed or operating a prison cell, then helping find ways to reduce those costs and increase governmental efficiency.

As an example, in a recent speech to The Institute of Public Administration in New Zealand, Bill English the countries Deputy Prime Minister said;

A second concept for the future is “inside out government”.

Government holds a wealth of information.  Some of it – quite rightly – is sensitive and access should be strictly controlled – tax records for example. 

But in other areas, I see no reason why we can’t turn government inside out, so to speak, and make the same data and information available to those outside of government. 

Government can tap wider resources in the community to analyse and use government data to help solve problems and produce insights. A ministerial committee is exploring this concept.

Inside out government also requires government to be open to good ideas from business.

We want to see ideas generated in the private sector and NGO sector genuinely considered and appraised – not simply ruled out on the basis that these sectors might not understand all aspects of government.

As I have said in previous articles, governments have a business to run and business leaders will look for ways to improve the way that it works using the assets and tools that they have available.

On the other side of the equation we have the tools and applications that are being built with the data that governments are already starting to publish, the theme of many of these applications appears to be somewhat different.

Early applications and much of the conversation that is being driven outside of government tends to focus on government transparency and public control of departmental activity, delivering applications that will help the public understand ministerial expenses, ensure that bills are read in detail before votes are cast and alerting citizens as a piece of legislation that potentially affects them passes through parliament.

The reality is that there is significant benefit underpinning both of these agendas, more efficient government is certainly a good thing, as is a more transparent and accountable government. The work that needs to be done to accommodate both agendas is probably pretty similar, and some of the foundations that we’re seeing from initiatives like NZGOAL will go a long way towards delivering what both sides need.

To those who have worked around government for some time the risks should start to become evident. While the goals of the two agendas may be similar, the language and the end expectations differ dramatically.

In the example above where Minister English talked about inside out government there was a great deal of cheering for the prospect of the New Zealand government publishng more open data but I saw very little mention of how community led projects would help him meet his objectives.

Both sides need to start to listen very carefully to the other, if they don’t then I worry that we’re on a trajectory that will eventually lead to two very unhappy and dissatisfied groups of people.

I’ve used examples from New Zealand in this article, but I don’t see the debate being dramatically different in the many other countries that I’m also following.