How we fixed prompt injection for all models on Fireworks

Fireworks Training introduces a feature called "safe_tokenization" to address the vulnerability of prompt injection in machine learning models, where user input can inadvertently be encoded as control tokens, altering system behavior. This issue arises from the tokenization process used by many open models, where user text and control tokens are not adequately separated, leading to potential exploitation. Safe_tokenization ensures that user content is tokenized separately from control tokens, maintaining the intended structure of prompts and preventing unauthorized modifications. This feature, available in the Fireworks Chat Completions API, enables the separation of user and system content at the token level, providing a defense against adversarial inputs without altering the user content itself. The implementation involves pre-processing the chat template and encoding user content to break any control-token bytes into subword pieces, ensuring the integrity of the system prompt. Fireworks plans to make this feature the default for new integrations, emphasizing its importance for maintaining product integrity in production environments.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.