October 2023 Summaries

A full-stack conversation designer has a blended set of technical and creative skills, mastering natural language understanding (NLU) design, dialogue design, and response design. This person should be well-versed in all three areas to understand how they work together, enabling them to craft engaging experiences. Full-stack designers can analyze smaller pieces of data, extrapolate them across conversations, and update responses accordingly. In a multidisciplinary team, having one person with full-stack skills is advantageous, as it provides a holistic view of the conversation design process, making them a more effective collaborator and teammate.

This stage involves translating a vision into a tangible plan, creating a blueprint that outlines potential interactions and the expertise necessary to bring them to fruition. The blueprint is created through collaboration with stakeholders, product owners, and natural language understanding (NLU) designers, involving steps such as gathering requirements, creating a brief, holding a briefing session, drafting the design, conducting user testing, iterating on feedback, getting team and stakeholder signoff, and refining the prototype. Once the blueprint is complete, the next stage involves filling in the details by crafting each conversational turn, mirroring overarching objectives while leaving room for real-world complexities. This stage requires meticulous collaboration between designers and developers to create a strong data foundation and refine the conversation's structure. The design process also involves error handling, response design, development, QA testing, refinement, final test, and launch. After the design is live, it's up to the designers to monitor how users interact with the agent, identifying weak points and areas of friction through iterative updates and refinements. Measuring performance helps determine what's working and what's not, using metrics such as intent coverage, entity accuracy, confidence, and other NLP metrics for NLU design, turns to complete, dead ends, backtracks, task completion for dialog design, fallback count, repeat responses, response length, and user feedback for response design. Ultimately, the process of conversational design (CxD) is ongoing and iterative, requiring continuous refinement and improvement to ensure users receive a seamless and helpful experience.