System Cards Go Hard
Blog post from Promptfoo
System cards are detailed documents accompanying the release of large language models (LLMs) that provide system-level information about the model's deployment, distinct from model cards, which focus solely on model details. Originating from Meta researchers and popularized by OpenAI in March 2023 with its GPT-4 release, these cards include comprehensive data on training, security policies, limitations, risk assessments, and model behaviors, serving as a critical resource for those concerned about application security and responsible AI practices. Existing examples from OpenAI and Anthropic highlight both the potential risks and the mitigations in place, such as red-teaming exercises and evaluations of cybersecurity and biological threats, while underscoring the importance of accountability and evolving standards as technology advances. The cards are seen as invaluable for educating users and informing deployment practices, suggesting that as system cards evolve, their format should maintain essential content while adapting to emerging technological concepts to enhance their utility and interest.