Safe Bets

Issue 3: Despite what many are keen to tell you, developing a strategy for generative AI is difficult

The Civic AI Observatory (civicai.uk) is a new initiative from Nesta and Newspeak House to support organisations working for the public good as they plan and adapt to the rapidly evolving field of generative artificial intelligence. Resources, case studies, events, online spaces, and more.

• Join our whatsapp community to chat with us & other readers:

  • #CivicAI: Safe Bets (to discuss this issue)

  • #CivicAI: Productivity (to discuss issue 2)

  • #CivicAI: Organisation Policies (to discuss issue 1)

  • #CivicAI: Link Drop & General Discussion

• Join us at these upcoming events:

  • 24 April: Civic AI Unconference

  • 25 April: AI for Grantmakers

  • 25 April: Prompt Jam (free ticket with code CIVICAI)

• Doing anything with generative AI? Seen something useful? Tell us about it: hello@civicai.uk

---

Safe Bets

Despite what many are keen to tell you, developing a strategy for generative AI is difficult. It is such a general purpose technology that it can be applied to almost any process in your organisation. Outside of generic use cases like the ones covered in the last issue, experimentation requires expensive skills. The technology also remains frustratingly unpredictable, a significant concern for civic organisations that must be extra careful to mitigate any risks to users, and also to maintain public trust. Despite these challenges, the impressive capabilities and rapid improvement of generative AI makes it hard to ignore. So if you’re a larger organisation that has some capacity to start exploring, how should you decide where to invest?

In this issue we are highlighting the work of the UK Cabinet Office’s new Incubator for Artificial Intelligence. While it’s easy to be cynical about such government initiatives, the team is impressive, and the three projects they have announced so far are widely applicable and align closely with our assessment of the current most promising areas for custom applications. We believe these projects are a good indication of realistic civic use cases we’ll be seeing a lot over the next year.

Following and learning from how these projects develop will help other civic organisations develop confidence for trying similar initiatives of their own. As a bonus, the incubator’s emphasis on open source will make it much easier and cheaper to replicate them, and means your organisation is less likely to get stuck maintaining an orphan codebase.

Finally, the fact that central government is actively experimenting with these projects at scale sends a strong signal to the civic sector. If they deem these initiatives as acceptable and worthwhile, it provides a level of assurance and legitimacy for smaller organisations to explore similar applications, to some extent alleviating concerns about the risks and uncertainties associated with this emerging technology.

The three projects they’ve announced so far are the Consultation Analyser, Redbox Copilot, and Caddy, with more promised soon. Here is info about each:

Consultation Analyser

The UK government conducts around 800 public consultations annually, with popular topics getting hundreds of thousands of written responses. Transforming unstructured text into structured data is one of the more conventional and labour-saving applications of large language models (LLMs), and this is an obvious use case.

While more traditional natural language processing techniques remain surprisingly relevant in the age of LLMs, the latter are rapidly closing the gap in terms of performance and are much more versatile, making it possible to tackle a wide range of text analysis tasks with a single, powerful tool.

The Consultation Analyser extracts and summarises themes from consultation responses, and presents them clustered on a dashboard where analysts can drill down to see the raw submissions. It’s been piloted it with the Department of Health and Social Care running in parallel with their existing consultation process, and more tests are now being rolled out to different departments.

For your organisation, this same approach could be used to help analyse any kind of large unstructured text dataset almost instantly and much more cheaply than any manual analysis. This opens up the possibility of, for example, running much larger and more frequent surveys that include more free text fields. It could also be used for inputs gathered via any other channel, such as Whatsapp, email, social media, or even phone transcripts. If your organisation has a large and active membership that have a lot they want to say – to you or to each other – understanding and engaging with them is becoming much more feasible.

Redbox Copilot

After a few months of being trialled in ministers’ offices, Redbox Copilot is now being rolled out to the wider civil service. Named for the iconic red boxes used to transport state papers, this project uses an AI technique called retrieval augment generation (RAG) to search and collate information from the many thousands of documents held across government, allowing civil servants to chat with a clued up ChatGPT-style assistant, or rapidly produce a summary of a set of internal documents complete with citations.

The project has had to tackle all of the compliance constraints that come with public sector deployment such as data privacy and retention, as well as come up with acceptable standards for the AI specific challenges: safety, bias, and hallucination. The project itself, or parts of it, may be useful if your organisation has growing piles of (potentially sensitive) internal knowledge that nobody has the time to think about.

Caddy

After Klarna’s recent announcement that it saved $40m by switching to AI powered customer service, there’s been a lot of interest in how replicable this is and whether it could safely be deployed in the public sector. Caddy, a collaboration between the Incubator for AI and Citizens Advice, is a tool that uses a mix of internal and external information sources to draft responses for human advisors, thereby avoiding most of the risks of unsupervised AI.

Caddy is built to be interoperable and slot into a range of different systems. The prototype is built on Google Chat, with an integration for Microsoft Teams coming soon. It is designed to make it easy to use data from many different sources; the current prototype combines internal Citizens Advice training material, the contents of the public Citizens Advice website, as well as all of GOV.UK.

A randomised controlled trial of Caddy at Citizens Advice centres will start in the coming weeks, looking to understand how it's impacting both advisors and advisees. Recent research suggests this kind of AI support will give a modest improvement in productivity with no loss of quality, and is particularly useful for helping new advisors get up to speed. If the results look good, the tool could be used in many places across government and beyond, and the incubator is actively seeking partners interested in exploring applications.

---

Next steps

• The Incubator for Artificial Intelligence hasn’t set up anything to make it easy to follow their projects yet - just a twitter account @i_dot_ai - but they promise that they’ll start publishing a newsletter soon. In the meantime, we’ll keep you updated on relevant updates in this issue’s Whatsapp Group #CivicAI: Safe Bets, and you can always reach out to them directly via i-dot-ai-enquiries@cabinetoffice.gov.uk
  

• Do you think you can use any of these projects? Any important use cases conspicuously missing from this list? Tell us! hello@civicai.uk

• Join our whatsapp community to chat with us & other readers:

  • #CivicAI: Safe Bets (to discuss this issue)

  • #CivicAI: Productivity (to discuss issue 2)

  • #CivicAI: Organisation Policies (to discuss issue 1)

  • #CivicAI: Link Drop & General Discussion
    

• Join us at these upcoming events:

  • 24 April - Civic AI Unconference

  • 25 April - AI for Grantmakers

  • 25 April - Prompt Jam (free ticket with code CIVICAI)