osrin.net

On the flight yesterday I was thinking through the various models that we are seeing emerge for open government data. As with most computer systems you can begin considering a model that should help when it comes to thinking about how we might make raw data available, and then contrast that with the flexibility of what we can do with that data.

For the sake of simplicity I looked at two different examples, one from the United Nations, and a second based upon a proposal that has recently been put forwards by Tim Berners-Lee for the United Kingdom. While both are extremely valuable implementations, it is clear that they present very different options for government.

The United Nations data can be found at http://data.un.org, and at time of writing represents twenty two data sets and over sixty million records. The data can be downloaded as a series of CSV files that can be quickly read into a spreadsheet, database or other tool of the users choosing. Making data available in this form would take less time an effort by the agencies involved, but would obviously still make it possible to build a wide variety of applications using the information available.

The proposal put forward by Dr. Berners-Lee is a little more complex in nature, suggesting that government data should be mapped through RDF and semantically tagged as it is published. In the long run this would represent significant value as individuals and organizations started work to understand and analyze the data but at the same time would represent a much larger project across government to enable the data to be published in the first place.

The overall model might help us evaluate options for implementation plans, end user tools, timelines and milestones and the value that the project would deliver.

In essence it might look a little like the graphic below, of course it would be a lot more useful with a significantly larger set of projects and case studies mapped against it;

As with the early stages of eGovernment I suspect we will see governments take a more pragmatic approach in the short term, implementing systems that fall on the left hand side of this graph, while holding aspirations and a vision that firmly maps to the right hand side.

Sphere: Related Content

The discussion around open data is gathering pace in several countries around the world, and the more I see of that discussion the more convinced I am that we’re watching the second significant revolution within government that has been driven by software.

The first change is one most of us take for granted today.

eGovernment became a buzz word during the late 90s and really pushed Government to consider new ways that services could be delivered. Many of us have quickly forgotten about the complexity of working with government in decades before then when we would have to visit an agency building and queue for hours or use one of the many call centres that appeared during the 1980s.

Today we expect services to be available online and accessible through a range of devices including mobile phones, home PCs or interactive television. For many of us these multi channel services are available equally, regardless of how much bandwidth we have, which societal demographic we belong to or where we might live geographically.

In the work I was engaged with back then the early push for eGovernment was frequently vendor led, with companies like Compaq, DEC, IBM and Microsoft taking a handful of marketing slides to government departments that painted the picture of a bright and rosy online future for the country.

It was the civil service who were able to do the hard work and translate that vision into the change and transition within the government that was eventually responsible for the delivery of the services that we see today.

During this time many countries around the world experimented with different ways of delivering online services. Over time a small number of those experiments became national programs that were then replicated around the world. A great example is the work around the eGovernment Interoperability Frameworks (eGIFs) that was pioneered by countries like New Zealand and the United Kingdom and has since become one of the accepted requirements for national level eGovernment programs.

At the moment it seems we are participating in a second revolution that has some very similar patterns to it, but with different goals and frequently different constituents driving the conversation.

Where eGovernment was about service delivery, open data is much more about the ability to make more informed and inclusive public policy decisions.

In place of the vendors offering the early marketing vision we are seeing civil society groups leading the discussion around the value of making government data available to the masses, and helping paint the picture of the benefits that could potentially follow.

While I am witnessing a growing number of experiments around the world it is not yet clear to me which country will find the right model for selecting and exposing datasets that are of value in the public policy making process, and be able to prove that their efforts in this space are really returning value back to the country.

From the outside Government looks straightforward enough. Direction is set, decisions are made, policy is written and eventually things happen.

Inside government it is often a little different. Thousands of processes across hundreds of departments, affecting millions of lives all have to be taken into account as major change is carefully managed.

With a hat tip of significant respect to everybody who has been involved in kick starting these discussions in their respective countries, at this point I find myself waiting to see who the academics and civil servants are who pick up the idea of open data and begin to drive significant programs within specific national governments and agencies, and then how replicable the policy and programs that they build are for other nations.

It is then that we will have a much clearer view of what “open data” really means, and what the real potential is within the complex machinery of national governance.

Sphere: Related Content

Twitter Comments

Steven Quezada pointed me to this video, originally put together by a cloud computing platform provider called GoGrid, a really straightforward explanation of cloud computing and how to make use of it.

Sphere: Related Content

Mix‘09 was held in Las Vegas last week and as usual there were all sorts of announcements relating to both our platform and our developer tools.

A couple of them were Azure related and worth a mention here.

The first was a step that we’re taking toward providing more flexibility for customers concerned with where their cloud based applications and data are stored.

My colleague James Brown (the eGovernment one, not the other one) talked about the relevance of this issue on his blog. “Cloud Computing and Data Sovereignty”;

One announcement that is great to see is that Azure is going to let developers choose which data centers they want their code and data stored in.  At the moment Azure is running on two data centers in the US, but more locations outside of the US are planned.  This is going to make the Data Sovereignty issue slightly easier in some cases.  The legal ramifications of storing UK Citizen data in the US is very different to storing it in another EU country, for example. Also from a technical point of view, being able to locate your code in the same geographical area as your users makes a lot of sense from a performance standpoint.

The second related to choice for developers working on the Azure platform, Computerworld carried the story “Microsoft opens Azure to PHP developers”;

Specific improvements made this week include expanding beyond managed code to native code support; allowing ‘full trust,’ which is how most applications or services are written; and offering FastCGI support to allow PHP development.

“Basically, the Windows Server team has done a ton of work with FastCGI that allows Windows Server to now support programming languages beyond just .Net and Visual Studio,” Ketkar said. Through the FastCGI interface, developers can take existing PHP skills and PHP applications and services and run them on Azure.

Developers might also be able to run other languages via FastCGI, said Ketkar. Microsoft, though, has done stress-testing for PHP but not for other languages. “There is no reason that Ruby won’t work through that same FastCGI interface,” he said.

So, when Azure moves from CTP to release you’ll not only be able to choose from a range of development languages, you will also be able to decide for yourself where you would like your application and data stored.

Sphere: Related Content

I’ve posted extensively about our various interop related efforts over the last year or so, highlighting many of the big steps that we have made in this area, each one incrementally making us a more interoperable platform than before and a more participatory citizen in the wider software industry.

When I came into the office this morning I knew there was a backlog of such announcements building up that I wanted to talk about, but hadn’t quite grasped how much has gone on over the last couple of weeks.

Below is a round up of news around multi-platform development for Azure, the document interoperability initiative, ODF support in Wordpad (!), working with the Samba project, participating in the Advanced Message Queuing Protocol (AMQP) WG, SAML 2.0, and OpenID.

First of all is news of what we’re doing for developers around the Windows Azure and Azure Services platform that I talked about yesterday. Up on microsoft.com you’ll find pointers to the SDKs that are already under development, not only for .NET but also for Java and Ruby, and a whitepaper that explains the objectives of these projects.

From the whitepaper;

The Azure Services Platform has been built from the ground up with interoperability in mind. With its standards-based and interoperable approach, the services platform supports multiple Internet protocols, including HTTP, REST, SOAP, and “plain old XML” (POX). This interoperability opens up opportunities to build new or enhanced applications using existing skills with the Microsoft Visual Studio development environment and the .NET Framework, or with other developments environments such as Java, or Ruby.

For a product that is still only a community technology preview (CTP) I think we’re off to a good start.

Next is news of the latest Document Interoperability Initiative workshop that was held in Redmond last week, this time focusing on plans for Open XML. The DII workshops are an important component of our commitment to engage more openly and more thoroughly with the community, this is the second workshop to be held in Redmond, the first one looked predominately at our plans for ODF support in Microsoft Office 2007 SP2.

Jesper Lund Stocholm has his hands on the beta for the SP2 code, and has posted about his experiences with it earlier this week.

Doug Mahugh has a comprehensive write-up of the latest DII workshop;

The latest DII workshop took place in Redmond over the last two days. There were presentations at this event from a variety of people, including members of the Office product groups at Microsoft and developers and consultants from several other companies. Topics covered included planning an IS29500 document test library, server-side document assembly strategies, various approaches to document validation, use of content controls in e-courseware, and goals for future DII events. John Head already blogged a few thoughts from the workshop, and I expect we’ll see more in the days ahead from John, his colleague Andrew Schwantes, or other blogger attendees such as Dennis Hamilton and Alex Brown. The DII we site has an event summary, and you can find downloadable copies of most of the presentations here.

Doug’s post is lengthy and gives really good insight into the discussions that took place at the event, I’d encourage you to follow the link and read the whole report. There are also a couple more posts on the topic that I’ve seen, John Head of PSC has shared his presentation, and Alex Brown has declared Open XML boring, in the nicest possible way of course!

Finally on DII, for those of you who are really interested in digging into the depths of the topics discussed, all of the Microsoft led presentations have been posted on the Document Interoperability Initiative site.

In other brief document format related news, you might have caught Stephen Sinofsky earlier today mentioning that the version of WordPad in Windows 7 will support both ODF and Open XML, Stephen McGibbon has promised some screen shots.

Applets. “We’ve also decided that once every 15 years or so we’re going to update the applets in Windows,” Sinofsky said, showing off his bone-dry wit. That means updates to Calculator, Paint, WordPad, which will now support open document formats including Open XML and ODF.

Next up is a post from Samba’s Andrew Bartlett in which he discusses the significant progress that has been made between the Samba Project and Microsoft’s Server devlelopment team over the last year, it was Brett Roberts who brought this to my attention, as a tangential point Brett also has some discussion about a recent event he ran alongside Google discussing our respective approaches to Software + Online Services.

Back to Samba, Andrew’s post is here;

This has been an amazing year of changes for those of us with an interest in interoperability with Microsoft, and these two events are an excellent example of the change in practice.

In short, Microsoft organised an industry plug fest for CIFS and AD technologies and then invited the Samba Team to it’s home campus for a week of hands on testing with their engineers. This follows up on documentation of over 100 protocols delivered, well over 100 requests for clarification answered, Samba code debugged and fortnightly conference calls held.

There can be little disagreement that this is night and day from the position we had with the Samba team three or four years ago. I think we probably have a way to go before the company as a whole pervasively understands the value of this type of relationship, but there are certainly a growing and significant number of individuals in the company who “get it”.

Fourth on the list is our recent decision to get involved in the Advanced Message Queuing Protocol Working Group.

From Sam Ramji’s blog;

Messaging (and I mean enterprise messaging, rather than email) is an area that is of keen interest to customers like JP Morgan Chase and Credit Suisse. As they run their businesses on real-time messaging, they need to be deep experts, and drive changes in their messaging platforms to fit their business. Along with companies like Cisco, Novell, iMatix, RabbitMQ, WSO2, and Red Hat, these industry leaders have built a standard for ubiquitous messaging: AMQP. 

The Advanced Message Queueing Protocol is an open specification supported by open source communities and currently implemented by Apache QPID, RabbitMQ, and OpenAMQ.  The contributors established the AMQP Working Group as a body to manage the process of developing the specification.

It’s my pleasure to announce that Microsoft has been invited to join the AMQP working group  by the six founding members. We have committed to participate in the development of the specification and are keenly interested in the developing need for interoperability in enterprise messaging.

This is a great standards story, and a great interoperability story.

Fifth on the list is support for the Web SSO profile for SAML 2.0 in “Geneva”, a code name for our upcoming claims based access platform.

Don Schmidt has the following;

At the Professional Developers Conference this week Microsoft is announcing the beta release of “Geneva”, the codename for its new claims based access platform. This platform helps developers and IT professionals simplify user access to applications and other systems with an open claims-based model. “Geneva” helps developers to externalize user authentication and identity processing from application code by using claims that are obtained with pre-built security logic that is integrated with .NET tools. “Geneva” helps IT professionals to efficiently deploy and manage new applications by reducing user account management, promoting a consistent security model, and facilitating seamless collaboration across departmental, organizational and vendor boundaries. User access benefits include shortened provisioning lead times, reduced accounts, passwords and logins, and enhanced privacy support. “Geneva” implements the Identity Metasystem vision for open and interoperable identity, and includes built-in support for standard federated identity protocols.

SAML 2.0 support by Microsoft has been a big discussion point here in the region, especially in countries such as New Zealand where SAML plays key roles in the government’s national authentication program.

Finally, and equally important, an announcement from the LiveID team at PDC yesterday that they will be supporting OpenID.

TechCrunch has some info on this announcement;

Login standard OpenID has gotten a huge boost today from Microsoft, as the company has announced that users will soon be able to login to any OpenID site using their Windows Live IDs. With over 400 million Windows Live accounts (many of which see frequent use on the Live’s Mail and Messenger services), the announcement is a massive win for OpenID. And Microsoft isn’t just supporting OpenID – the announcement goes as far as to call it the de facto login standard.

…as does Microosft’s Peter Galli;

This means that Windows Live ID accounts will be able to be used to sign in to any OpenID Web site. The Windows Live ID OpenID Provider (OP) enables anyone with a Windows Live ID account to set up an OpenID alias and to use that alias for identification at an increasing number of OpenID 2.0 relying party sites-for example: Plaxo, Pibb, StackOverflow.com and Wikispaces.

Putting around half a billion LiveID users to one side. this is good news for me personally, next time I want to post a comment on Harish Pillay’s blog I won’t need to do it anonymously!

That should be more than enough for one post, as I’ve said before it is evidence that we’re delivering on the public promises we have made around interoperability that will eventually make a difference to the way that developers, ISVs, customers and other partners perceive us.

This week feels like it has delivered a pretty strong set of evidence to support those public statements… and it is still only the second day of PDC!

Sphere: Related Content

We’re holding our Professional Developers Conference in LA this week, and along the way there are a few really significant announcements coming from our executive staff.

Monday saw the first public unvieling of the CTP for “Azure”, our cloud services platform.

There is coverage pretty much all of the computing and general press, the story in the NZ Herald is linked here.

The man who replaced Bill Gates as Microsoft’s top technical thinker said that Microsoft will compete with Amazon.com, IBM and other rivals in selling information storage space and computing power “in the cloud,” distributed across massive data centres worldwide. The system, Windows Azure, will let companies and hobbyists alike build web-based programs without having to invest in their own server farms.

Ozzie’s remarks at a Los Angeles conference for software developers indicated that after several years of disparate experiments, Microsoft is closer to a companywide strategy for coping with an upheaval in the software industry – the shift from powerful desktop programs to more lightweight, inexpensive ones that run over the internet.

… and from the BBC here.

The platform was described by Microsoft’s chief software architect Ray Ozzie as “Windows for the cloud”.

The framework will be offered alongside the next Windows release, Windows 7.

The move sees Microsoft taking on established players like Google and Amazon in the rapidly growing business of online software.

The aim is to allow developers to build new applications which will live on the internet, rather than on their own computers.

Looking at some of the work taking place in the technology industry here in the region, this is a significant step for us. As Ray points out, you no longer need to be a multi-national corporation to draw on the levels of compute and storage power that is needed to build applications to serve the wider base of customers that exist on the internet. For Asia especially, where small and medium size businesses are a significant piece of the economies in the region, this represents a huge amount of opportunity.

If you’re looking for a little more reading on the shift that is happening in the industry at the moment then my friend and colleague Stephen McGibbon pointed to an extensive article that was posted in The Economist earlier this week.

The Economist just published a leader “Clouds and Judgment” together with a special report, “Let it rise” all about cloud computing.

IT’s global “cloud”
The evolution of data centres
Software as a service
Connecting to the cloud
The economics of the cloud
The long nimbus
Computers without borders

More soon…

Sphere: Related Content