September 2022 Summaries

Nicholas Laing, a Conversation Designer at JPMorgan Chase, is a great example of someone who didn't wait for an invitation to join the field. He built his own path by launching a smart-home consulting business and realizing that the key to success lay in virtual assistants. People come from diverse backgrounds, including chemistry, art direction, and content writing, and often round out their skills by learning to build or write. To be successful, one needs to be unusually curious and follow that curiosity, with a love for problems, puzzles, and challenges. Building a portfolio is essential, whether starting at a small startup or large enterprise, as it showcases one's work and helps network in the space. The field of conversation design has a wonderful community that wants to see newcomers find their way, offering resources such as courses, job descriptions, and templates, and encourages people to ask for help and just "do it."

A homeowner's analogy of water and sewer pipes illustrates the importance of understanding and diagnosing issues in data systems, comparing water flow from a reservoir to tap with data flow from a database to end-user applications. The article highlights the utility of binary search techniques for systematic troubleshooting, where issues are narrowed down by checking segments incrementally, but also emphasizes the value of informed guesses based on experience, such as recognizing old or problematic sections that might be the source of contamination. This experience-driven approach allows for faster problem resolution by focusing efforts on the most likely trouble spots, akin to how understanding the intricacies and history of a plumbing system aids in efficiently resolving water quality complaints. The broader message is that while building new systems is exciting, maintaining and understanding existing systems is crucial for designing better ones, as it offers insights into potential pitfalls and improvements.

In the conversation design portfolio, the content is more important than the medium, with a focus on showcasing case studies and demonstrating understanding of the impact on users and businesses. The best approach to presenting a CxD project involves explaining the "why" behind the design, highlighting the solutions considered, and providing a video presentation or prototype that allows for interactive exploration. Curating a portfolio requires selecting projects that demonstrate responsibilities, measuring quality, and showcasing mistakes as opportunities for growth. Conducting a portfolio review involves learning from interviews, avoiding presenting ideas versus finished products, and seeking feedback from others in the industry. Finally, success in creating a conversation design portfolio depends on focusing on the big picture of experience, self-advocating during job interviews, and highlighting strengths and dream projects.

To set up tests for a conversational AI, it's essential to define clear goals and priorities, such as improving release times and testing user order phrasing. The next step is to create and prioritize use cases, focusing on intent testing and entity testing. Test cases can be sourced from team brainstorming, user testing results, production data, or machine learning generation, with a focus on brainstorming and user testing data for this project. Once test cases are created, they need to be embedded into the code or configuration files, depending on how tests are being run. After that, it's time to run the tests, which may require writing custom wrapper code for NLU models without built-in testing functionality. Finally, the tests can be integrated with the conversational AI workflow, running them whenever major changes are made to the NLU, and potentially integrating them with CI/CD pipelines as the team matures.