Paul Scanlon discusses the drawbacks of using Google Fonts in Gatsby sites, emphasizing the impact on web performance due to additional HTTP requests required to fetch fonts from Google's servers, which can negatively affect the Largest Contentful Paint (LCP) metric. He suggests a more efficient alternative by loading font files locally, which reduces external requests and improves overall performance metrics as shown by Lighthouse reports. The process involves uninstalling the gatsby-plugin-google-fonts, adding font files to the static directory, and using the @font-face CSS rule with the rel="preload" attribute to prioritize font loading, ultimately demonstrating that a more manual approach can yield significant performance benefits for web developers.