FiftyOne Agent Skills: Teach the Agent to Use Your Plugin

Voxel51’s Gemini Vision plugin integrates Google Gemini Omni’s video generation and video-understanding capabilities into FiftyOne, while bundled Markdown-based Agent skills instruct the FiftyOne Agent on when, how, and under what cost and safety constraints to use its operators. The plugin supports text-, image-, reference-, edit-, and extension-based video generation, stores generated clips as dataset samples with provenance metadata, and preserves Gemini interaction IDs so generated videos can be iteratively edited, though imported footage cannot be edited this way. For understanding tasks, Gemini returns structured timestamped events that FiftyOne stores as TemporalDetections, making results seekable, filterable, and usable as dataset labels rather than unstructured chat responses. Skills are discovered automatically through a plugin manifest when enabled, can require user approval before billable operations, and guide the agent in selecting tasks, setting parameters, tagging generated media as synthetic, and reporting failures. In a 30-minute repetitive-video test, static sampling at 0.5 frames per second found a target accurately at about one ninth the token cost of high-thinking agentic analysis, while low-thinking agentic mode missed it, illustrating that agentic processing is not always the most efficient option.

Use this post, company, and trend context to find content marketing opportunities, perform competitive analysis, or address product feature gaps via the Plushcap MCP server or the Plushcap API.