Derek Slater is a Policy Manager at Google.

Last year, we hired a respected consultancy, Communications Chambers, to produce an international dataset of retail broadband Internet connectivity prices. The dataset can be used to make international comparisons and evaluate the efficacy of particular public policies—e.g., direct regulation and oversight of Internet peering and termination charges—on consumer prices.

We received a lot of positive feedback and suggestions—thank you!—and have now made available an updated dataset.

• A Fusion Table containing the price observations for 1,523 fixed broadband plans can be found here.
• A Fusion Table containing 2,167 mobile broadband prices can be found here.
• Explanatory notes here, and ancillary data is here.

Steven Weber is a professor in the School of Information and Political Science department at UC Berkeley.

It's commonly said that most people overestimate the impact of technology in the short term, and underestimate its impact over the longer term.

Where is Big Data in 2013? Starting to get very real, in our view, and right on the cusp of underestimation in the long term. The short term hype cycle is (thankfully) burning itself out, and the profound changes that data science can and will bring to human life are just now coming into focus. It may be that Data Science is right now about where the Internet itself was in 1993 or so. That's roughly when it became clear that the World Wide Web was a wind that would blow across just about every sector of the modern economy while transforming foundational things we thought were locked in about human relationships, politics, and social change. It's becoming a reasonable bet that Data Science is set to do the same—again, and perhaps even more profoundly—over the next decade. Just possibly, more quickly than that.

There are important differences which have equally come into focus. Let's face it: Data Science is just plain hard to do, in a way that the Web was not. Data is technically harder, from a hardware and a software perspective. It's intellectually harder, because the expertise and disciplines needed to work with this kind of data span (at a minimum) computer science, statistics, mathematics, and—controversially—domain expertise in the area of application. And it will be harder to manage issues of ethics, privacy, and access, precisely because the data revolution is, well, really a revolution.

Can data, no matter how big, change the world for the better? It may be the case that in some fields of human endeavor and behavior, the scientific analysis of big data by itself will create such powerful insights that change will simply have to happen, that businesses will deftly re-organize, that health care will remake itself for efficiency and better outcomes, that people will adopt new behaviors that make them happier, healthier, more prosperous and peaceful. Maybe. But almost everything we know about technology and society across human history argues that it won't be so straightforward.

Data Science is becoming mature enough to grapple confidently and creatively with humans, with organizations, with the power of archaic conventions that societies are stuck following. The field is broadening to a place where data science is becoming as much a social scientific endeavor as a technical one. The next generation of world class data scientists will need the technical skills to work with huge amounts of data, the analytical skills to understand how it is embedded in business and society, and the design and storytelling skills to pull these insights together and use them to motivate change.

What skills, knowledge, and experience do you and your organization need to thrive in a data-intensive economy? Come join senior industry and academic leaders at DataEDGE at UC Berkeley on May 30-31 to engage in what will be a lively and important conversation aimed at answering today's questions about the data science revolution—and formulating tomorrow's.

Harvard University's Center for Geographic Analysis has released an incredible map-based visualization of global tweets, called TweetMap ALPHA (via Information Aesthetics). From the project site:

TweetMap is an instance of MapD, a massively parallel database platform being developed through a collaboration between Todd Mostak, (currently a researcher at MIT), and the Harvard Center for Geographic Analysis (CGA).

The tweet database presented here starts on 12/10/2012 and ends 12/31/2012. Currently 95 million tweets are available to be queried by time, space, and keyword. This could increase to billions and we are working on real time streaming from tweet-tweeted to tweet-on-the-map in under a second.

MapD is a general purpose SQL database that can be used to provide real-time visualization and analysis of just about any very large data set. MapD makes use of commodity Graphic Processing Units (GPUs) to parallelize hard compute jobs such as that of querying and rendering very large data sets on-the-fly.

Back in March, Rachel Burstein of the New America Foundation wrote about some of the findings from their research on civic innovation in California. The full report is now available for download, and Rachel has given us permission to repost her announcement from govloop.com.

Whether you work at the Department of Agriculture, the California State Treasury Office or the Planning Division of the City of San Jose, you have probably encountered the following scenario. You are tasked with solving a problem—say, how to encourage those eligible for food stamps to take advantage of the program, or how to eliminate a sizeable part of the public safety budget without also reducing costs—and you want to investigate possible solutions systematically. But you don’t know what approaches have been tried, the effectiveness of such approaches, or the applicability of those solutions to the specific situation your department and constituencies are facing.

What do you do? Perhaps you begin with a basic Google search. You find some examples that seem like they might relevant. Perhaps you read an article about a town government in another state that consolidated its police department with that of another community, thereby saving millions of dollars a year. The city manager and members of the City Council have good things to say about the arrangement, but you have trouble finding information about obstacles the town leadership faced in implementing the consolidation. Plus, given the difficulties you’ve had collaborating with a neighboring town on a recycling program and what you know about a nearby city’s approach to policing, you’re not sure if consolidation of departments is a good option for your town.

What’s your next step? If you're ambitious, maybe you find contact information for the city manager in the city that tried the consolidation strategy and ask him about difficulties he faced in the project. Or maybe you send a query to a professional association list-serv asking if anyone can direct you to resources on similar local projects. Or perhaps you bring up the topic at the next meeting of the city managers group to which you belong.

The problem with any of these scenarios is that you have gleaned only limited, generic, or second-hand information from either unverifiable sources or from sources with limited understanding of how the solution will operate in the circumstances you face. For certain types of information—say, creating a new form for renting your agency's facilities, or determining what icons to use to designate recycling containers—this may not be a problem. But when it comes to government solving tough problems through innovative approaches, strong personal networks are key.

This finding is one of many found in a new report released by the New America Foundation’s California Civic Innovation Project. The report summarizes survey and interview data on perceptions of, obstacles to, and motivations for innovation in local government. It assesses how knowledge sharing between locales promotes innovation, and the particular importance of personal networks in facilitating effective knowledge sharing around innovation.

Among the report’s major findings are the following:

• Internal organizational or managerial changes to improve service delivery while reducing costs—not e-government, public-private partnerships, or civic engagement projects—are the most important innovations adopted in cities and counties, according to those who work in local government.
• Resource constraints both motivate innovation and serve as an obstacle to effective knowledge sharing and the potential for innovation diffusion in local government.
• Pressure from elected officials and legislative mandates are more significant drivers than community input for city managers and county administrators when it comes to adopting new approaches.
• By far, personal contacts—especially those in geographically proximate communities—are the most valuable source of knowledge for city and county administrators investigating and implementing new approaches.
• Professional associations are more valuable as knowledge sources for innovation than the individual tools and services (e.g. list-servs, professional development opportunities, webinars, etc.) that such groups offer. Personal channels are the most typical way that local government staffers share knowledge about innovation with colleagues in other communities.
• There are wide divides between urban and rural communities when it comes to perceptions of civic innovation and the ways in which knowledge is acquired and shared.

We hope to start a conversation among various stakeholders at all levels of government in order to develop specific recommendations deriving from this research. What can professional associations do to enlarge and strengthen the personal networks of their members? What can government managers do to communicate their strategies—successes, failures, and aborted projects—to others faced with similar problems? What types of institutional support need to be in place to facilitate such changes? These are the questions that we hope to begin to answer in the coming months. We hope that you will be part of the conversation!

In the meantime, you can download the full report here. We look forward to hearing from you.

Sylwia Giepmans-Stepien is a Public Policy and Government Relations Analyst for Google in Brussels.

We now create as much information every two days as we did from the dawn of civilization up until 2003. And this rich flow is destined to accelerate. McKinsey projects 40% growth annually in global data generated. To showcase the potential of data for Europe’s economy and society, we recently teamed up with the European Innovation and Technology Foundation, the Bavarian Representation to the European Union and Euronews.

The forum, Data-Driven Innovation: The New Imperative for Growth, debated how data can improve the delivery of public services, provide accurate healthcare diagnosis, and generate higher business productivity. Androulla Vassiliou, European commissioner for education, culture and multilingualism, and Neelie Kroes, European commissioner in charge of the digital agenda, both called for unleashing a Big Data revolution in Europe. "This is the new frontier of the information age," Vassiliou said. "In the current path to stimulate European growth and jobs, there has never been a more critical time to harness the potential of data."

Androulla Vassilou

Alfred Spector

Senior representatives of the education, research, policy and business communities presented compelling evidence of how data could address big societal challenges. Computer-powered DNA sequencing open the possibility of accelerating medical diagnoses. Online college courses could revolutionize education. Google's own Vice President for Research Alfred Spector showed how we use data for products such as Google Translate.

Data also is powering entrepreneurs. New online business models make sense out of data include social media power startups such as news organiser Storify. Its founder Xavier Damman explained how established organisations and top politicians such as BBC, the White House or UK Prime Minister David Cameron use his company’s services to share knowledge from different online data sources, including Twitter, Google+, and traditional media websites.

The concluding panel looked at the ethical aspects of collecting, sharing and using data. Among other examples, they discussed how organizations such as DataKind are bringing together data scientists and NGOs to address social problems ranging from dirty water to urban sprawl. While speakers stressed that data-driven innovation is not based exclusively on data about people, they acknowledge, that all data regardless the source and type requires making tough ethical choices.

The Innovation Forum aims to inject data-driven innovation on the Brussels policy agenda. As well as focusing on privacy and data protection, we also need to encourage the unprecedented economic potential of data.

Adrienne St. Aubin is a Policy Analyst at Google

Google is excited to sponsor this year’s international AppMyCity! Prize from the New Cities Foundation, celebrating mobile applications that improve the urban experience, connect people, and make cities more fun, vibrant, sustainable places.

We're bullish on the value of open public data to inspire innovation and improve citizens' daily lives. Last year Francisca Rojas of Harvard Kennedy School’s Transparency Policy Project highlighted the positive impact of open transit data on the number of transit apps developed—and the indication that more people are likely to utilize public transportation systems when apps help improve the experience via real-time information. Imagine the possibilities for other kinds of public data like health, employment, education, environmental, demographic and cultural info.

The first step toward generating value from public data is for governments to make data available in machine-readable formats, not just PDFs or image files, and ensure it stays up to date. No one wants to build or use an app that shows out-of-date schedules or last year’s parking zones. But governments aren’t the only ones who have a responsibility here, even though they are the generators and keepers of the data. Developers and citizens have a role to play too, by using what’s out there, giving feedback about how it can be improved, and growing the demand side of the market.

Of course, the value of open data isn’t just about apps. But creating and using apps is one of the most concrete ways we can engage with the public information around us. Imagine together how it can make our communities—and the world—a better place.

About the AppMyCity! Prize

Entries are now being accepted at www.appmycity.org and the submission deadline is April 26, 2013. The New Cities Foundation will announce ten semi-finalists on April 30, 2013. This list will be assessed by a panel of expert judges, who will select the three finalists. The finalists will be announced on May 7, 2013.

Three AppMyCity! Prize finalists will be invited to attend the New Cities Foundation’s New Cities Summit in São Paulo June 4-6 to present their project to an international audience of urban leaders, thinkers and innovators, and the winner will receive 5,000 USD to support further development of the app.

In January, Atlantic editor Richard Florida kicked off a series of posts called the "Class-divided Cities." Each post includes an analysis and map visualizations of socio-economic polarization within different areas of US cities.

This divide is seen most clearly in where members of each class live. A recent report from the Pew Research Center found that residential segregation between upper- and lower- income households has risen in 27 of America's 30 largest metros over the past several decades. Compounding this polarization between rich and poor neighborhoods, the share of middle-income neighborhoods has declined substantially.

[...]

To get a better sense of the scale of the divide in American cities, my research team at the Martin Prosperity Institute — relying on data from the U.S. Census Bureau's American Community Survey — plotted and mapped the residential locations of today's three major classes: the shrinking middle of blue-collar workers in manufacturing, transportation, and maintenance; the rising numbers of highly paid knowledge, professional, and creative workers in the creative class; and the even larger and faster-growing ranks of lower-paid, lower-skill service workers. For the next few weeks, I'll be exploring the various divides in some of America's largest cities and metros.

The series began with New York, and yesterday, San Francisco became the 11th.

List of city analyses, in the order in which they were posted:

• New York
• Los Angeles
• Chicago
• Washington, DC
• Atlanta
• Miami
• Dallas
• Houston
• Philadelphia
• Boston
• San Francisco