osrin.net

The current trend of governments releasing massive and diverse datasets will demand something different from internet search tools in the future, something that we might consider a little extraordinary today.

Today most of us search in a single dimension, we tap a term into our favourite web search tool and get back a list of links that represent pages that are currently published somewhere on the net. Most of us are not planning on doing any level of analysis on that information, we are just trying to find something, so the list of links are enough for us.

Governments are starting a new trend though, massive amounts of machine readable data that we can use to draw our own conclusions to complex questions about our environment or our society.

In his now infamous TED presentation, “Let my dataset change your mindset”, Hans Rosling gave us a preview of the way that many of us will be using these government datasets in years to come, along with similar datasets that we will eventually see commercial organizations publishing in the same way.

Using available data, developers will continue to build new applications that could never have been funded by government, citizens and businesses will be able to offer complex and well thought out advice to policy makers, economists will be able to build empirical models that demonstrate societal trends and eventually historians will reconstruct the environment that we leave behind.

For all of this to work internet search has to evolve, a list of links won’t meet our needs. Here’s three examples;

First of all, a piece that we’re close today, we need to be able to search by geography. When we begin to break down massive datasets the geography becomes important, any piece of data has a special meaning when we can tie it to a country, a county, a town or a particular street. Most of the government policy makers I meet have had a long term understanding of the role of geographic data in government process, but few tools exist to enable the publishing of that data externally in a way that is useful.

Secondly, we need to be able to search by timeframe. Future analysis of data, either for an economist constructing trends over a limited number of years or long term reviews by historians will require us to find a way to roll datasets back to a point in time that is relevant to the users analysis.

Finally, it is not enough for a single country to solve this, international standards need to evolve to support this type of search.

Very quickly we will find ourselves at a point where it will not be enough for us to look at an issue in the context of a single country. In the short term, policy advice to a given government could be enhanced by the ability to cross analyze that advice with data from similar nations – e.g. to lower the cost of building a kilometre of road in New Zealand, I might also want to look at the costs in the UK, Canada and Australia. – and in centuries to come historians will need a way to show how global society evolved.

We are not far from a point where we are going to see a need to enable a software instigated search for data relating to a particular issue, in a certain place and during a given timeframe.

It is then that we will really begin to experience the power that published data gives us.

Twitter Comments

	wendywhite RT @osrin: Blog: Searching Open Government Data in four dimensions http://osrin.net/88h #gov20 #opendata
	Re-Tweet | Reply | View Tweet

	hansbos RT @osrin: Blog: Searching Open Government Data in four dimensions http://osrin.net/88h #gov20 #opendata
	Re-Tweet | Reply | View Tweet

	osrin Blog: Searching Open Government Data in four dimensions http://osrin.net/88h #gov20 #opendata (clumsy blunder fixed!)
	Re-Tweet | Reply | View Tweet

	Bridge_Design RT @osrin: Searching Open Government Data in four dimensions http://osrin.net/88h
	Re-Tweet | Reply | View Tweet

	allisonhornery RT @osrin: Searching Open Government Data in four dimensions http://osrin.net/88h #gov20 #gov2au #opendata
	Re-Tweet | Reply | View Tweet

	You2Gov RT @stevequezada: RT @osrin: Searching Open Government Data in four dimensions http://osrin.net/88h #gov20 #gov2au #opendata
	Re-Tweet | Reply | View Tweet

	stevequezada RT @osrin: Searching Open Government Data in four dimensions http://osrin.net/88h #gov20 #gov2au #opendata
	Re-Tweet | Reply | View Tweet

	osrin Searching Open Government Data in four dimensions http://osrin.net/88h #gov20 #gov2au #opendata
	Re-Tweet | Reply | View Tweet

Earlier this week Google’s Public Policy Blog carried a post titled “Why the next generation Internet Protocol matters” which pointed towards a meeting that the company held on the topic of IP deployment at their headquarters earlier this month.

As usual (check out the comments) the post sparked the usual set of debates around address exhaustion and the role of NATs in sustaining IPv4 technology over the longer term.

Those who read my blog on a regular basis will know that this is a topic that I have a keen interest in. IPv6 is one of the next logical steps for the internet and has the potential to drive the next significant wave of innovation around the network.

One of the lessons that we have learned as a company is that the users of technology are always far smarter than the developers or the product planners, give those users an increased level of functionality in the platform and you will never cease to be amazed at the way that functionality gets used to build new and interesting solutions. I’m convinced that this argument applies as well to the role of IPv6 as it does to the Windows operating system.

If you’re looking for some more reading on this debate I’ll offer you one more piece of data from either side.

Triggered by work that was prepared for Google’s summit, Networkworld is carrying a story titled “No business case for IPv6, survey finds”, which points to a survey undertaken by the Internet Society.

Business incentives are completely lacking today for upgrading to IPv6, the next generation Internet protocol, according to a survey of network operators conducted by the Internet Society (ISOC).

In a new report, ISOC says that ISPs, enterprises and network equipment vendors report that there are no concrete business drivers for IPv6.

However, survey respondents said customer demand for IPv6 is on the rise and that they are planning or deploying IPv6 because they feel it is the next major development in the evolution of the Internet.

The full copy of the report from the Internet Society (ISOC) can be found here and reaches the following conclusion;

The respondents stated that a key missing element in IPv6 is the lack of support for IPv6 in applications. Although no specific applications were mentioned, many applications embed knowledge of the IP version number into the implementation (specifically the address encoding) that makes it difficult to get them to run on the new protocol. Others mentioned that many support tools that have been developed for IPv4 have the same limitation. In particular, network management systems were singled out as being a bit behind the curve in their support of IPv6.

Meanwhile, on the other side of the debate David Green over at Command Information Blog talks about a significant piece of work that has been undertaken by the US federal CIO council to draw up a plan to complete the deployment of IPv6 across the US Federal Government;

The growth and innovation of our economy is becoming more and more dependent on our communications infrastructure and its main component – the Internet. The remaining pool of unallocated Internet address space will be exhausted within the next year and the depletion will cause operational issues with the Internet. Recognizing this dependence, the US Government and leading Internet authorities have determined that change to the next-generation Internet is necessary to ensure the continuity and growth of ecommerce, egovernment, and netcentric operations into the next decade.

You can find a copy of that draft business case here (copy), on first reading it appears to be a pretty comprehensive document, mapping out the current state of IPv6 deployment in the US government and a milestone based plan for what has to happen from here.

Mix‘09 was held in Las Vegas last week and as usual there were all sorts of announcements relating to both our platform and our developer tools.

A couple of them were Azure related and worth a mention here.

The first was a step that we’re taking toward providing more flexibility for customers concerned with where their cloud based applications and data are stored.

My colleague James Brown (the eGovernment one, not the other one) talked about the relevance of this issue on his blog. “Cloud Computing and Data Sovereignty”;

One announcement that is great to see is that Azure is going to let developers choose which data centers they want their code and data stored in.  At the moment Azure is running on two data centers in the US, but more locations outside of the US are planned.  This is going to make the Data Sovereignty issue slightly easier in some cases.  The legal ramifications of storing UK Citizen data in the US is very different to storing it in another EU country, for example. Also from a technical point of view, being able to locate your code in the same geographical area as your users makes a lot of sense from a performance standpoint.

The second related to choice for developers working on the Azure platform, Computerworld carried the story “Microsoft opens Azure to PHP developers”;

Specific improvements made this week include expanding beyond managed code to native code support; allowing ‘full trust,’ which is how most applications or services are written; and offering FastCGI support to allow PHP development.

“Basically, the Windows Server team has done a ton of work with FastCGI that allows Windows Server to now support programming languages beyond just .Net and Visual Studio,” Ketkar said. Through the FastCGI interface, developers can take existing PHP skills and PHP applications and services and run them on Azure.

Developers might also be able to run other languages via FastCGI, said Ketkar. Microsoft, though, has done stress-testing for PHP but not for other languages. “There is no reason that Ruby won’t work through that same FastCGI interface,” he said.

So, when Azure moves from CTP to release you’ll not only be able to choose from a range of development languages, you will also be able to decide for yourself where you would like your application and data stored.

I’ve just spent the last hour reading through the review that Sir Peter Gershon undertook of the Australian Government’s Use of ICT. There is plenty useful guidance in that document for any government around the world that is looking for ways to get more value out of their ICT assets.

One of the sections that stood out for me is a recommendation that can be found in section 5.4 and highlights the need for the Australian government to create and manage IT career paths for employees working in this field.

“5.4.1 Recognise that there is a national and global competition for talent and the APS needs to recruit, develop and retain key ICT skills”;

To address the issues around ICT skills shortages, the talent attraction and retention of ICT skills within the APS needs to be improved. I recommend two key measures.

Firstly, there needs to be the creation of a whole-of-government ICT career structure, developed by AGIMO in consultation with the Australian Public Service Commission (APSC) and PM&C. This will include training and development programs for ICT professionals in key skills areas (for example, project management and systems architecture) and should utilise existing taxpayer funded initiatives wherever possible (for example, the Defence Materiel Organisation project management training).

Secondly, a whole-of-government strategic ICT workforce plan needs to be developed and maintained. The plan will be developed and supported by the APSC and AGIMO, and will leverage the work done by other agencies, such as the ATO, on a workforce planning tool. AGIMO should construct the plan annually based on inputs from agencies and recommend to the SIGB options for dealing with identified skills issues, including recruitment, training, and development in areas of skills shortages.

With the combination of a common career structure and a workforce plan, it should become much easier to plan to smooth peaks and troughs of demand for ICT skills in individual agencies.

This is a move that I have been advocating to many of our customers for several years now.

Well formed career structures within the system of government would provide a framework that allows the many contractors who circle around agencies to move into full time roles with little or no impact to their income, training programs or the tools that they need to get their job done.

In return government would be in a position to leverage the institutional knowledge that full time employees collect over time, which I am confident would help deliver projects at lower cost and more efficiently.

At a political and policy level you would over time see governments build a strong leadership capability based upon their real needs rather than, as you sometimes find, being based upon the many views of external third parties. These leaders would understand the impact of the massive scale requirements that government systems face, and how to deal with them.

As ICT has become more critical to government service delivery we have seen similar trends play out around the world, first of all with large scale outsourcing to help obtain the skills needed, then more recently consolidating strong ICT skills at the center of government allowing the setting of national policy and common guidance to agencies and departments.

Both of those strategies have served government well over the last decade, the next logical step does appear to be – as Sir Peter Gershon recommends – to distribute strong ICT skills across departments using full time civil service employees in a manner that is competitive to industry.

Internet Explorer 8 is now shipping and can be downloaded from here. For the curious among you the video below talks a little about the performance of the latest release of Microsoft’s browser and compares it directly to Firefox and Chrome.

IE8 Performance

Science Commons  is working with the scientific community around the world to help increase the level of sharing and value that comes out of scientific research.

Their website describes three key goals ;

Making scientific research “re-useful” — We help people and organizations open and mark their research and data for reuse. Learn more.

Enabling “one-click” access to research materials — We help streamline the materials-transfer process so researchers can easily replicate, verify and extend research. Learn more.

Integrating fragmented information sources — We help researchers find, analyze and use data from disparate sources by marking and integrating the information with a common, computer-readable language. Learn more.

To further that goal, Microsoft has this week announced a project to build an add-in for Word 2007 that will provide functionality allowing scientists to embed the ontologies being defined on Neurocommons directly into documents produced using Microsoft Office.

From our press release;

The nuggets of information necessary for science to progress are often hard to find, submerged deep within the Web, or within databases that can’t be easily accessed or integrated. As a result, many scientists today work in relative isolation, follow blind alleys and unnecessarily duplicate existing research.

Addressing this critical challenge for researchers, Microsoft Corp. and Creative Commons announced today, before an industry panel at the O’Reilly Emerging Technology Conference (ETech 2009, http://en.oreilly.com/et2009), the release of the Ontology Add-in for Microsoft Office Word 2007 that will enable authors to easily add scientific hyperlinks as semantic annotations, drawn from ontologies, to their documents and research papers. Ontologies are shared vocabularies created and maintained by different academic domains to model their fields of study.

As with an increasing number of our projects these days the source code for the add-in is available on Codeplex for anybody who wants to make use of it;

Microsoft is making the source code for both the Ontology Add-in for Office Word 2007 and the Creative Commons Add-in for Office Word 2007 tool available under the Open Source Initiative (OSI)-approved Microsoft Public License (Ms-PL) on CodePlex, Microsoft’s Web site for hosting open source projects, at http://ucsdbiolit.codeplex.com and http://ccaddin2007.codeplex.com respectively.

A little more on Neurocommons from their home page;

The NeuroCommons project seeks to make all scientific research materials – research articles, annotations, data, physical materials – as available and as useable as they can be. We do this by both fostering practices that render information in a form that promotes uniform access by computational agents – sometimes called “interoperability”. We want knowledge sources to combine meaningfully, enabling semantically precise queries that span multiple information sources.

Our work covers general data and knowledge sources used in computational biology as well as sources specific to neuroscience and neuromedicine. The practices that we develop and promote are designed to play well on the Semantic Web. We view our technical work not as creating a new service or content library, although we do both, but rather as helping to promote the growth of semantically linked scientific information.

It is 6:15am here in London and pretty close to freezing outside, a great time to catch up with the vast amounts of email that has been backing up in my inbox over the last couple of weeks.

First off the block; Matthew Aslett over at the 451 Group has made a gallant attempt to categorize the various open source business strategies that are out there today.

How does IBM’s open source strategy compare to Sun’s? Or Microsoft’s? What’s the difference between MySQL’s strategy and JasperSoft’s? Are some strategies better suited to engaging with organic open source communities, rather than inorganic? What on earth is the Open Core model?

These are some the questions we hoped to try and address with our Open Source is Not a Business Model report, published in October last year. As I mentioned yesterday, however, without an agreed set of definitions and a common vocabulary it is difficult for a broader understanding the implications of the various models to develop.

Matthew goes on to categorize Development Models, Vendor Licensing Strategies and Revenue Triggers.

There are an increasingly large group of people and companies, of which Microsoft is one, who believe that things have moved on from the simple scenario of Proprietary Software vs. Open Source to an environments where developers are drawing upon code that helps them solve their problems regardless of origin.

This post from 451 really helps outline how far some of those models have evolved, and how diverse the options are in today’s software industry.

Intel’s co-founder, Gordon E. Moore, shared a statement with us a long time ago that has become known as Moore’s Law.

Since the invention of the integrated circuit in 1958, the number of transistors that can be placed inexpensively on an integrated circuit has increased exponentially, doubling approximately every two years.

Around the world the category of notebooks we have come to know as netbooks are providing users with low end requirements with a route to enjoy the benefits that computing brings. Essentially these netbooks are providing consumers with a low cost, low functionality notebook. The price point has driven revolutions in some markets, the number of education departments around the world looking at issuing laptops to their K12 students today is a good example of that.

Michael Horowitz talks about what a netbook actually is in a blog entry from last October.

A Netbook is a new type of laptop computer, defined by size, price, horsepower, and operating system. They are small, cheap, under-powered, and run either an old or unfamiliar operating system.

Moore’s Law is already hard at work on the netbook. The Lenovo S10 that I picked up on my way through Singapore two weeks ago is more than capable of running Windows 7 along with many of my usual tools, Michael’s accurate observations from last October are already becoming out of date.

People choose netbooks today for a variety of different reasons. For schools they are cheap, for professionals they are lightweight, for others they’re just a fun toy. Whatever the reason for choosing one there will be compromise in their purchasing decision – reduced battery life, lower processing capacity, less storage or less available RAM.

The dramatic evolution of these devices over the last six months, combined with the three decades of lessons that Moore’s Law has taught us gives us a clear picture of the future.

Six months ago you might have had to make a choice that would involve “an old or unfamiliar operating system”, soon you will be able to choose Windows 7 on a netbook size device. Netbooks are also already shipping with 160Gb hard drives rather than the 10Gb or 20Gb of SSD that we saw in early devices.

Some of the remaining decision driving criteria such as lower capacity processing, reduced battery life or minimal graphics ability will disappear just as quickly.

Two years from now you won’t be deciding between a netbook or a notebook, you will just be deciding what size of device you want based upon where and how you want to use it. The netbook category will have all but disappeared.

This is the first role of it’s kind that I have seen advertised by a sovereign government, and I think a very commendable move by my home country.

The UK is seeking a “Director of Digital Engagement”, a senior leader who will have responsibility for an evolution of what we used to call eParticipation in the country through an array of digital media tools.

Judgement will be crucial in this role. It leads on the future of Government engagement with citizens through digital means. This means that the post will be breaking new ground on a daily basis, across Government. The agenda is politically very high profile and full of complex issues between and within departments that you will have to exercise very sensitive judgement on how to manage and resolve. You will have a level of professional expertise that is likely to mean that you will be unique in your ability to exercise judgement and provide advice to Ministers and Permanent Secretaries/ senior officials on matters within your remit. [..snip..]

This is not a role for a generalist. The professional skills required are formidable. Engagement in the digital space is a young ‘profession’ and the job requires someone who would be acknowledged by their peer group to be a leader in this field. The successful candidate will have a CV that creates instant credibility and confidence with Ministers, senior officials and digital communicators in Whitehall.

It is pretty evident that a lot of work has already taken place on this front in the UK, starting from the very top with the twittering, polling and video work that has been emanating from 10 Downing Street for a while now.

A short while ago I shared a report from HIMMS that outlined the progress that is being made around the world to deploy electronic health records to our various healthcare systems.

Sometimes when we look at reports like that it feels like we’re making substantial progress towards computerizing complex processes like EHRs.

This morning somebody kindly sent me the video that is linked below, which just goes to show how slow we really at at using technology to solve some of these important social issues.

The video outlines IBM’s attempts in 1961 to deploy electronic healthcare records into Johns Hopkins. The presenter promises a paper free system, a comprehensive view of patients healthcare needs and the ability to reduce errors in diagnosis

Sound familiar?