How are your LLM Products Used?

Generative AI is transforming the way users interact with software tools and data by introducing natural language as a new interface layer, expanding the input possibilities beyond traditional front-end applications and APIs. This shift allows for a broader range of potential applications but also presents challenges in developing products based on large language models (LLMs), as it becomes nearly impossible to anticipate all user interactions before deployment. The complexity arises from numerous edge cases, the impact of small input variations on outputs, and the diverse ways the same request can be articulated. To address this, an analysis of over 10,000 real-world interactions with LLM-based products led to the creation of a taxonomy that categorizes user inputs into three main groups with further subcategories.