Artificial intelligence (AI) is increasingly being implemented in the public sector to enhance operational efficiency, reduce costs, and improve service delivery to citizens, with notable examples from the UK and US where AI is employed to tackle issues like fraud reduction and asylum processing. While AI holds immense potential for productivity gains and economic benefits, such as the projected £200 billion annual savings in the UK and $519 billion productivity boost in the US by 2033, its adoption faces hurdles due to privacy concerns and outdated IT infrastructure. To overcome these barriers, strategies like the "data mesh" approach and retrieval augmented generation (RAG) workflows are being utilized to integrate proprietary data with generative AI securely, enhancing the accuracy of AI outputs without compromising data privacy. Successful implementations, such as Georgia State University's use of generative AI to streamline financial aid processes, highlight the potential of AI to transform public services, though it requires a strong data foundation and a privacy-first approach to ensure responsible use.