Most government financial and economic data is free; but that is the last of the good news. I thought when the Obama administration took over that some of its Web 2.0 savvy and great UI presentation would be carried over to the delivery of public data Unfortunately, no sign of improvements. Here is the table from the BEA- Bureau of Economic Analysis on Industry Profits by sector [trust me, this is typical of hundreds of data series]:
Check the data out here
Another example is the SEC Financial data also known as EDGAR. For twenty years, I have tried to access this data about corporate financials in a fast and efficient manner. But effectively the data is locked away in hundredss of separate files. It is a nightmare getting the data together. Even worse is when you are trying to compare several comapnies, even from the same industry sector. Despite SEC and Financial Accounting rules, companies manage to report Profit and Loss and Balance Sheet Information in unique and non comparable ways. Users must do a whole lot of extra work. No wonder a whole cottage industry of for- profit companies offer the free SEC data, each with their own interetation of the data.
Here is the gateway to the SEC data:
I challenge readers to get their data quickly and free through this gateway to financial data hell.
And so is it any surprise that 16 agencies monitoring terrorists could not act in any semblance of co-ordinated fashion. 16 isolated silos of information, complex interactions/display plus data-turf fights almost allowed for a calamitous terrorist attack. Likewise prior to and during the financial crisis tracking the risk exposure of the banks and investment houses was already tough with systems like EDGAR as the gateway – but this was made ten times worse by the fact that most derivative and other risk instruments were in shadow markets where transparency and reporting were largely ignored.
In sum, a major sore is that government information and data, already hard to get access to and easily display, is now becoming further entangled by special interests both within and external to the government. Given the rapid and large change pressures confronting the US[think climate policy, energy, health, education, jobs, etc], tenuous access to relevant government data is not a good sign for intelligent problem solving and policy making.