Opinion: this year's SXSW technology conference may have had Barack Obama as its headliner, but the tone was still surprisingly sombre, says Kieran Long. Are the tech evangelists as worried about the future as the rest of us?

What is the future according to the world's leading technology conference? Well, in short, it's the future you've probably already heard about: driverless cars and autonomous vehicles; drones (Farming! Delivering stuff!); all kinds of robots (Playing the marimba! Helping you find stuff in a DIY store! Hoodwinking presidential candidates!); AI and natural language speech recognition; 3D printing; big data and ever more creative ways to use it to make money.

The more exotic stuff is there too: prostheses and neural interfaces; gene therapy; the gamification of healthcare; fannotation; virtual supermarkets in Baltimore food deserts and so on.

It's the lineup of things that Silicon Valley believes will disrupt and improve our world. SXSW Interactive is the definitive gathering of these people: 25,000 of them who fly to Austin, Texas to meet, to learn, to show off, to pitch and be pitched to, and to party.

It's everything you'd expect it to be. Outlandishness and experimentation, techno positivism and unintentional irony that stops just short of charlatanism. Robots walk the streets, but the queue for Gus' fried chicken restaurant is so out of control that crowds of people with lanyards are nearly coming to blows.

This year, the BBC was there showing some really interesting experiments in interactive television, but they couldn't get a decent wireless connection on their trade show stand.

People pitch their startups in the corridors. Big companies spend thousands sending their staff to Austin, but they'll all turn up to any party with free tacos and booze. It's the best and worst of technology.

But if you've always imagined SXSW Interactive as a kind of living Wired magazine, this year's edition put paid to that narrow notion. Barack Obama, the first sitting president to speak at SXSW, was just the most high-profile example of a cast of mayors, senators, international business people, bankers, motivational speakers and so on, who headed to Austin to commune with the leaders of the digital community, who they and we assume has the future in its hands.

And they came not to command or commission them, but to ask politely, at times obsequiously, for help in tackling some of the biggest challenges America has to face. The dark shadows cast by Ferguson, the rise of Donald Trump, increasing inequality in American cities, ISIL and the radicalisation of young people online, the Snowden revelations and the fight between Apple and the FBI over iPhone encryption and privacy were detectable in almost every talk I attended. Even self-professed optimist Jimmy Wales sounded wary and talked about his own attempts to secure Wikipedia – the online encyclopaedia he co-founded – against government snooping.

Obama was speaking in a Republican state and to a Silicon Valley community that is stereotypically neoliberal, with pronounced libertarian tendencies. Texans and techies are two groups that pathologically object to bureaucracy of any kind and tend to believe that government is incapable of delivering any useful or efficient service. So Obama was tentative about saying precisely what he meant.

But his speech was an appeal: for help from websites on tackling ISIL online, for big tech companies to divert their resources of talent towards national problems and services, and for moderation in attempts to make all personal data inaccessible to government agencies. (On whether phones should be entirely closed to the authorities, he said: "If you can't crack that at all, government can't get in, then everybody is walking around with a Swiss bank account in their pocket." Which, to some in the audience, may not be the self-evident evil he clearly intended it to sound like.)

Obama set the scene for a conference that was shot through with anxiety about big topics, coupled with a sense that the tech community must embrace civic responsibility and their role as enablers of so many of our social interactions today.

One of the most powerful talks was by the sitting mayor of Baltimore, Stephanie Rawlings Blake. Her attitude seemed to suggest that there were few challenges her city faces that she felt could be solved by an app. Her list of long-term and widespread heroine addiction, rising poverty, what she called a "lack of civility" in American politics and decreasing participation in the public realm was sobering to listen to and outlined the scale of the challenge that politics faces. What that talk was doing at a technology conference is an interesting question. It can only be the belief that the people at the forefront of interaction and digital design, of all fields, might just be the group who can make the internet a place of civic participation.

Even the fields of design that everyone agrees are the future were beset by anxiety. One member of the audience asked the inventor of Siri (who is currently working on a major new AI project entitled Viv) whether he was concerned about the ethics of creating a legion of sentient robot slaves.

Even Jennifer Haroon, one of the top bosses of Google's self-driving car project, was wary of sounding too boosterish, and talked about her struggles campaigning against legislation in the state of California that would require all autonomous cars to have a qualified driver behind the wheel. At least there was, in that session, some sense that technology will lead to a better world. 32,000 people die in American traffic accidents each year. The prospect of improving that statistic, alongside the manifold environmental, social and cityscape benefits of automated cars underlie a project that will face years of struggling with capricious and ill-informed legislators to become a reality.

But perhaps the sober tone should be reassuring. It is difficult these days for many of us to feel unreservedly optimistic about the future, given the pace of technological change and the debasement of politics. But the tech community clearly feels the same and are grinding through the same, very real and tangible world as the rest of us.

If the experts are in retreat from blind techno positivism, perhaps we can allow ourselves a note of optimism. Maybe now they might begin to transform from an industry that uses its considerable brainpower to sell adverts online to one that tackles some of the challenges of the future.

Kieran Long is keeper of the design, architecture and digital department of the Victoria & Albert Museum. He presents Restoration Home and the series The £100,000 House for the BBC.