October 2026 Summaries
32 posts from Deepinfra
Filter
Month:
Year:
Post Summaries
Back to Blog
New open AI models often become substantially faster and cheaper to serve in the weeks after launch because initial inference-engine support prioritizes correctness over optimization. Using Qwen3.8-Flash-Next as an example, the discussion explains that its hybrid recurrent-and-sparse-attention design, mixture-of-experts routing, wide residual streams, and large n-gram embedding table require specialized memory management, GPU kernels, caching strategies, and scheduling logic rather than simple configuration updates. Early engine implementations from SGLang, vLLM, and llama.cpp could run the model, but later work added sparse-attention indexer kernels, fused operations, host-memory embedding offload, lower-precision checkpoints, and speculative decoding, improving throughput, cache capacity, and long-context performance. These technical gains can lower providers’ cost per token, although pricing changes and performance improvements vary by hardware, context length, workload, engine maturity, and provider configuration. The piece advises users to evaluate newly released models based on capabilities while retesting speed, latency, quality, and pricing over time, particularly at realistic context lengths, and to maintain portable integrations so they can benefit from evolving serving options.
Oct 31, 2026
2,877 words in the original blog post.
DeepInfra’s comparison of Jev alternatives describes three main approaches to structured AI decisions: hosted decision-specific APIs, self-hosted open-source decision models, and standard LLM endpoints constrained with schemas to select only from fixed options. Jev is presented as a strong baseline because of its low latency, low input-token price, support for choices, scores, and yes/no judgments, and especially well-calibrated confidence estimates. In tests on 144 three-option decisions, Jev and DeepSeek-V4.1-Flash using DeepInfra structured outputs each achieved 96.5% accuracy, though Jev was faster at a 245 ms median latency versus 568 ms and had lower calibration error; a local Strands Decider 2B model was fastest at 154 ms but less well calibrated. DeepInfra’s enum-based structured outputs offer a relatively inexpensive way for teams already using open models to enforce valid choices, while echo-based log-probability scoring can provide probabilities for every option at higher cost. The comparison recommends Jev for low-latency hosted decisions, DeepInfra for integration with existing LLM workflows, local models for privacy-sensitive or request-path use cases, and specialized alternatives such as Clef or OpenAI’s Decisions API for image-based tasks.
Oct 30, 2026
3,394 words in the original blog post.
DeepInfra has added NVIDIA Nemotron 3.5 Transcribe, an open-weight offline speech model, to its OpenAI-compatible speech-to-text API on the day of its release. The model combines multilingual transcription, English translation, contextual prompting, timestamps, code-switching, rich formatting, audio understanding, and planned speaker attribution in a single 30B-A4B Mixture-of-Experts architecture, reducing the need to combine separate ASR, diarization, translation, and language models. It supports recordings up to 60 minutes, transcription across 40 languages, translation between English and 31 languages, and contextual terms for improved recognition of specialized names. NVIDIA and Cekura benchmark results cited by DeepInfra indicate competitive word-error rates against proprietary services, while its open weights allow inspection, fine-tuning, and self-hosting. Intended uses include contact-center analysis, meeting notes, media captioning, and searchable audio archives, with word, segment, SRT, and VTT timestamp outputs available. DeepInfra prices the service at $0.002 per audio minute, states that requests have zero data retention, and provides API access for transcription and English translation.
Oct 28, 2026
1,109 words in the original blog post.
Wan3.0 is Alibaba’s multimodal video-generation model, capable of using text, images, video, audio, documents, and web pages to create synchronized-audio videos up to 30 seconds in 1080p, though providers differ in supported controls, pricing, compliance, and job delivery. DeepInfra offers a unified API across several Wan generations and other video models, zero data retention, and flat $0.20-per-second Wan3.0 pricing, while Alibaba Cloud provides first-party access, editing capabilities, regional rates ranging from roughly $0.041 to $0.20 per second for standard generation, and faster Prime tiers. Atlas Cloud emphasizes SOC 2 Type II certification and HIPAA-aligned hosting with discounted rates, Runware focuses on explicit validation and detailed first-frame and reference-media controls but does not publish public pricing, and fal provides a typed JavaScript client with queue progress updates, prompt expansion, reasoning, and safety settings. PoYo uses non-expiring prepaid credits rather than per-second billing, while MuAPI supports polling or webhooks, Standard, Prime, and Spicy variants, multimodal references, and ComfyUI integration. The comparison recommends selecting a provider based on organizational compliance requirements, first-party feature access, workflow controls, client tooling, billing preferences, and the intended balance between lower-cost drafts and final 1080p renders.
Oct 23, 2026
2,143 words in the original blog post.
DeepInfra’s guide compares API providers, gateways, and self-hosting options for IBM’s Apache 2.0-licensed Granite-4.2-8B reasoning model, which supports a 131,072-token context window and is intended for enterprise and developer workloads. It identifies DeepInfra as its overall recommendation based on stated low blended pricing, cached-input discounts, fast measured output speed, function calling, JSON support, and private deployment options, while CoreWeave is positioned for output-heavy workloads due to lower output-token pricing and low first-chunk latency. OpenRouter, TrustedRouter, and LLM Gateway offer OpenAI-compatible gateway access, published routing or pricing options, and varying failover and privacy features; IBM emphasizes enterprise protections, training-data transparency, guardrails, and on-premises deployment. For organizations wanting direct infrastructure control, Vultr provides GPU-hosted self-deployment options, while Hugging Face supplies downloadable weights and multiple quantized formats for local use or fine-tuning. The guide concludes that provider selection should be based on actual input-output token ratios, speed requirements, privacy and compliance needs, reliability, and whether teams prefer managed APIs or self-hosted deployments.
Oct 22, 2026
1,592 words in the original blog post.
DeepInfra’s Wan3.0-Video is an all-in-one, omni-modal video generation model from Wan-AI that accepts text, images, video, audio, documents, and web links to create or edit videos with synchronized audio, character consistency, and support for style transfer and clip extension. The model generates MP4 videos at up to 1080p and 30 fps for durations from two to 30 seconds, supports up to 20 reference assets, and ranks first in Artificial Analysis’s text-to-video leaderboards for both audio and silent generation, though leading models are closely ranked. Compared with Wan2.7, Wan3.0 consolidates separate workflows into one model, expands input types and maximum duration from 15 to 30 seconds, and costs $0.20 per generated second rather than $0.10. Developers can access it through DeepInfra’s POST API using an account and API key, configure options including resolution, aspect ratio, audio, prompt enhancement, watermarking, and reproducibility seeds, and receive a video URL alongside request status, runtime, and cost information. DeepInfra states that it uses zero retention for inputs and outputs, while pricing is straightforward at $2 for a 10-second clip and $6 for a 30-second clip.
Oct 22, 2026
1,312 words in the original blog post.
DeepInfra’s overview introduces IBM’s Apache 2.0-licensed Granite 4.2 8B, an 8-billion-parameter reasoning model designed for coding, tool calling, structured output, and agentic workflows. The model provides full-thinking, low-effort, and non-thinking modes to balance reasoning depth against latency, supports a 128K-token native context window expandable to 512K, and officially handles 12 languages. Reported benchmark results include 86.67% on AIME 2025, 64.14% on GPQA, 74.04% on MMLU-Pro, and 47.67% on SWE-Bench Verified, positioning it between the smaller 3B and more resource-intensive 30B Granite variants. Available through DeepInfra’s OpenAI-compatible chat-completions API, it supports common generation controls, JSON schemas, function calling, and prompt caching, while responses can include internal reasoning in think tags followed by final answers. Pricing is listed at $0.06 per million input tokens, $0.25 per million output tokens, and $0.015 per million cached tokens, with private enterprise deployments also offered.
Oct 21, 2026
1,261 words in the original blog post.
DeepInfra has added NVIDIA’s Nemotron 3.5 Embed 1B, Embed 8B, and Rerank 8B models to its API, offering pay-per-token access without deployment setup. Part of the Nemotron Retriever collection, the models are designed to improve agentic retrieval by embedding and searching not only text but also tables, charts, images, diagrams, and complete document pages, while the reranker prioritizes the strongest evidence for a query. The 1B embedding model targets efficient large-scale use, the 8B version emphasizes retrieval quality, and the Rerank 8B model is intended to support iterative, multi-source agent workflows; NVIDIA reports that reranking can reduce agent inference costs by up to 14%. DeepInfra presents the models as suitable for enterprise document search, multi-step research, agent memory, and code retrieval, and provides Python examples for embedding documents, selecting similar results, reranking candidates, and searching page images. Pricing per million input tokens ranges from $0.015 to $0.070 for embedding models depending on model size and modality, while Rerank 8B costs $0.050.
Oct 21, 2026
1,271 words in the original blog post.
DeepInfra’s comparison of Wan 3.0 video-generation API providers examines pricing, input capabilities, operational transparency, and data handling rather than model quality, since Wan 3.0 leads the Artificial Analysis audio-enabled text-to-video leaderboard at 1,173 Elo. Wan 3.0, Alibaba’s managed model, supports native audio, clips up to 30 seconds, resolutions from 480P to 1080P, and document or web-page inputs, while Wan2.2 remains available as Apache 2.0 open weights. DeepInfra charges a flat $0.20 per generated second across resolutions and positions its unified API for Wan2.2 through Wan3.0, zero-retention policy, and SOC 2 and ISO 27001 certifications as advantages, though its lower-resolution prices exceed competing rates. Alibaba Cloud offers first-party access with regional and resolution-based pricing, OpenRouter provides discounted Alibaba-backed access alongside published latency and availability data, fal emphasizes web-oriented tooling and configurable generation options, and Atlas Cloud advertises discounted low rates and compliance-oriented features. The comparison concludes that provider selection depends chiefly on a team’s target resolution, required inputs, compliance needs, preference for consolidated APIs, and need for visible reliability metrics.
Oct 21, 2026
1,743 words in the original blog post.
DeepInfra’s comparison of Wan 3.0 video-generation pricing finds that Alibaba Cloud offers the lowest listed per-second rates outside Singapore, ranging from $0.041256 at 480p to $0.165025 at 1080p, while OpenRouter begins at $0.0425 per second for 480p and provides published latency and availability information. DeepInfra charges a flat $0.20 per generated second regardless of resolution, positioning its service around broader multimodal inputs and production controls rather than the lowest rendering cost. Video billing is driven primarily by output duration, resolution, region, and costly regenerations, with no separate listed charges for supplied inputs. Alibaba Cloud is presented as most economical for straightforward workloads, OpenRouter as a lower-cost gateway for iterative generation, and DeepInfra as better suited to workflows requiring reference images, video, audio, files, web pages, seeds, watermarking, and other controls through one endpoint. The examples show that DeepInfra generally costs more for equivalent output volume, but may reduce total workflow complexity for media-rich applications, while a split approach using inexpensive 480p drafts on OpenRouter and final renders on DeepInfra can substantially lower costs.
Oct 20, 2026
2,219 words in the original blog post.
IBM’s Granite 4.2 8B is an Apache 2.0-licensed, open-weight text reasoning model with 8 billion parameters, a 131K-token context window extendable to 512K, support for 12 languages, and configurable full-thinking, low-effort, and non-thinking modes. It is positioned for enterprise use cases such as structured API calls, document processing, classification, internal assistants, coding, and agentic workflows, with native reasoning-assisted tool calling and reported strengths in mathematics, scientific reasoning, and selected software-engineering benchmarks. Artificial Analysis rates it 11 on its Intelligence Index, above the comparable open-weight median of 8, while measured performance shows 98.4 output tokens per second and 0.85-second time to first token, although reasoning can add substantial waiting time before answers begin. DeepInfra is presented as the preferred API provider due to its low blended price and fastest output speed, while CoreWeave offers lower output-token pricing for output-heavy workloads and OpenRouter provides OpenAI-compatible routing and failover. The review also notes limitations including unusually verbose outputs, reasoning latency, growing memory needs at long contexts, uneven multilingual quality, possible inaccurate or unsafe results, text-only operation, and weaker performance on more difficult agentic coding tasks, making human review advisable for high-stakes or client-facing work.
Oct 19, 2026
1,690 words in the original blog post.
DeepInfra has introduced Alibaba’s Wan3.0, a unified video-generation model that combines text-to-video, image-to-video, reference-based generation, video editing, and video extension through a single API endpoint. The model accepts prompts alongside up to 20 reference assets—including images, video clips, audio, documents, or public web pages—and can produce videos up to 30 seconds long at 1080p and 30 fps with synchronized dialogue, music, and sound effects. Wan3.0 is ranked first on Artificial Analysis’s text-to-video-with-audio leaderboard at 1,173 Elo and expands on earlier Wan models by consolidating their separate workflows and doubling maximum generation length. Available through DeepInfra as Wan-AI/Wan3.0-Video, it costs $0.20 per generated second, supports parameters for duration, resolution, aspect ratio, audio, prompt rewriting, watermarking, and reproducible seeds, and operates under DeepInfra’s zero-retention policy and SOC 2 and ISO 27001 certifications.
Oct 19, 2026
1,217 words in the original blog post.
IBM’s Granite-4.2-8B is an 8-billion-parameter, Apache 2.0-licensed open-weight reasoning model released in August 2026, offering a 131,072-token context window, tool calling, structured JSON output, and configurable reasoning modes. DeepInfra’s pricing guide positions it as a cost-conscious option for production use, with direct pricing of $0.06 per million input tokens, $0.25 per million output tokens, and $0.015 per million cached tokens, while CoreWeave charges more for input and cache but less for output at $0.15 per million tokens. The comparison emphasizes that DeepInfra is generally better suited to prompt-heavy, RAG, document-processing, multi-turn, and cached agent workloads, whereas CoreWeave may be more economical for applications generating lengthy responses. OpenRouter provides access through routing, failover, and compatibility features but may yield variable costs depending on the selected provider. Although Granite-4.2-8B has competitive latency and throughput figures, its reported verbosity means output-token usage can substantially affect actual spending, making input length, output length, and cache-hit rates the main factors in estimating deployment costs.
Oct 18, 2026
2,613 words in the original blog post.
DeepInfra announced the availability of IBM Granite 4.2 8B, an Apache 2.0-licensed dense reasoning model designed for enterprise workloads, coding agents, tool use, and workflow automation. The 8B-parameter model offers selectable per-query reasoning modes—full chain-of-thought, low-effort reasoning, and direct answers—allowing users to balance quality, latency, and compute costs without changing model weights. Post-trained through supervised fine-tuning, GRPO-based reinforcement learning, and preference alignment, it supports reasoning-augmented OpenAI-schema function calling, 12 languages, JSON output, and a native 128K-token context window, with a 512K extension available. IBM reports scores including 86.67 on AIME 2025 and 47.67 on SWE-Bench Verified, while noting that these self-reported results should be independently validated. Available through DeepInfra’s OpenAI-compatible API in bfloat16 precision, the model costs $0.06 per million input tokens, $0.25 per million output tokens, and $0.015 per million cached tokens, with the platform emphasizing zero data retention and SOC 2 and ISO 27001 certifications.
Oct 17, 2026
1,269 words in the original blog post.
Open-weight AI models are increasingly used in production as organizations seek lower-cost alternatives to frontier systems, but access to model weights does not automatically grant unrestricted rights to modify, distribute, host, or commercially deploy them. The article explains that while many recent models, including Qwen3, Mistral 3, Gemma 4, gpt-oss, and DeepSeek V4, use permissive MIT or Apache 2.0 licenses, other models retain custom terms involving revenue thresholds, user limits, separate agreements for model-as-a-service providers, security reviews, attribution rules, and prohibited-use policies. It recommends that enterprises evaluate five areas before deployment: commercial-use limits, hosted-inference restrictions, fine-tuning rights, redistribution and notice requirements, and use-case prohibitions. It also notes broader risks involving intellectual-property uncertainty, warranty limitations, data handling, contractual protections, and evolving AI and data-transfer regulations. DeepInfra presents its managed inference platform, GPU infrastructure, data-retention terms, security certifications, and data-processing agreements as enterprise-oriented features, while emphasizing that these services do not eliminate the need for independent legal review of each model license.
Oct 16, 2026
2,039 words in the original blog post.
TypeSafe’s Jev is a hosted decision model that evaluates text or JSON against predefined choice, score, and yes/no questions rather than generating prose, offering low input costs, rapid responses, and the ability to process multiple questions over the same state in one request. DeepInfra’s review tested Jev for prompt-injection screening, support-ticket routing, LLM model routing, agent tool-call approvals, and lead scoring, finding particularly promising results for routing simple versus difficult or private prompts to appropriate models and for escalating risky financial or account actions, while estimating a 55% reduction in non-private LLM costs through model routing. However, small hand-written test sets, apparent overconfidence on ambiguous inputs, sensitivity to phrasing in injection detection, and ineffective probability-margin escalation in ticket routing limited the conclusions. The review recommends using Jev as a narrow, layered decision component rather than a standalone safety or reasoning system, with fail-closed handling for screening and approvals, and compares it with newer open alternatives including Cloudflare Clef and AWS Strands Decider, which offer different performance, pricing, and self-hosting tradeoffs.
Oct 15, 2026
3,290 words in the original blog post.
DeepInfra has launched NVIDIA Nemotron 3.5 Super VL through its OpenAI-compatible API, positioning the open multimodal model for agents that combine visual understanding, reasoning, and tool use across screenshots, documents, images, and text. The model is designed to reduce the need for separate OCR, vision, and reasoning systems by interpreting visual inputs and deciding which external tools to call within a single context. Its mixture-of-experts architecture contains roughly 121 billion parameters with 12.7 billion active per token, while a Mamba-2-based backbone and limited attention layers aim to support efficient long-context agent workflows up to 262,000 tokens. DeepInfra reports generation speeds exceeding 400 tokens per second in a tested configuration using NVFP4 weights on NVIDIA Blackwell GPUs, and the model supports 10 languages, open weights, and NVIDIA fine-tuning and LoRA recipes. Suggested uses include document extraction, customer support, GUI automation, and multimodal research, with an example showing the model extracting an order number from a screenshot and calling a shipping-status lookup tool rather than relying on potentially outdated screen information. Pricing starts at $0.20 per million input tokens and $1.00 per million output tokens, while DeepInfra states it is SOC 2 and ISO 27001 certified and does not retain prompts or outputs.
Oct 15, 2026
1,429 words in the original blog post.
Growing enterprise interest in open-weight AI models reflects a shift from experimentation toward managing AI as an operational cost, with companies increasingly routing high-volume, bounded tasks to lower-cost models while retaining frontier systems for complex or high-risk work. The analysis recommends auditing request-level logs to identify expensive features, classifying workloads by how structured and verifiable they are, and comparing real cost per completed task rather than token prices alone. It argues that models such as DeepSeek V4.1 Flash and Qwen3.8 Flash can be dramatically cheaper than Claude Opus 5.5 or Sonnet 5.5 for tasks like classification, extraction, summarization, and retrieval-grounded question answering, although savings decline when outputs are verbose or requests require frontier-model fallbacks. Benchmark results suggest open models are relatively competitive on bounded professional tasks but lag substantially on long-horizon agentic work, advanced reasoning, and sensitive decisions. Before deployment, teams are advised to evaluate candidates on representative production traffic, set quality thresholds in advance, inspect failure cases, and use shadow testing. A gradual rollout with validation-based escalation, canary traffic, version pinning, monitoring, and a rapid rollback mechanism can help organizations reduce costs while maintaining reliability, with the broader goal of creating a reusable routing and evaluation system rather than committing permanently to any one model.
Oct 15, 2026
2,705 words in the original blog post.
DeepInfra compares two newly announced Western open-weight AI models, Reflection AI’s 501-billion-parameter Beam and Mistral’s 1.05-trillion-parameter Large 4, known as “Le Chonk,” with leading Chinese alternatives such as GLM-5.3, Kimi K3, DeepSeek V4.1 Flash, and MiMo-V2.6-Pro. It concludes that neither Western model yet matches the Chinese leaders on independently measured capability or price per capability, while Beam’s claimed 3–4× compute efficiency remains unverified because it relies partly on potentially shorter generated responses and excludes important costs such as context-prefill compute. Beam emphasizes promised Apache 2.0 licensing and efficient reasoning, whereas Large 4 emphasizes European operation, data sovereignty, multilingual and multimodal inputs, and potential private-cloud or on-premises deployment, though its license and full availability were still pending. The comparison argues that Western models may be especially relevant for organizations constrained by licensing, security, procurement, or jurisdictional rules, while Chinese models remain compelling where those restrictions do not apply. It also highlights currently available Western options including Inkling, NVIDIA’s Nemotron models, gpt-oss-120b, and Gemma, and recommends evaluating models through workload-specific measurements of cost, token use, throughput, and policy compliance rather than vendor benchmarks alone.
Oct 14, 2026
3,456 words in the original blog post.
Decision models are presented as a new AI infrastructure layer for handling frequent, bounded agent decisions—such as request routing, tool safety checks, task-completion verification, and document relevance—without using costly generative language models. Unlike standard LLMs, these models accept context and a predefined option set, then return a selected option with calibrated confidence scores in a single forward pass, enabling faster, threshold-based automation and escalation of uncertain cases. The category expanded rapidly in late 2026 through proprietary offerings including TypeSafe’s Jev and OpenAI’s Decisions API, alongside open or self-hosted alternatives such as AWS Strands Decider, Laya, Kev, and Ollaya. Their architectures typically replace an LLM’s vocabulary-generation head with a pointer or decision head, reducing latency and eliminating output-token costs, though GPU hardware is generally needed to retain these advantages. The article argues that hybrid systems are currently the most practical design, using decision models for clear, high-confidence choices and generative models for ambiguous or reasoning-intensive cases, while noting limitations in handling difficult edge cases, explaining decisions, generalizing beyond known options, and performing efficiently on CPUs.
Oct 13, 2026
3,319 words in the original blog post.
DeepInfra’s comparison of Qwen3.8-27B inference providers describes the 27-billion-parameter open-weight model as capable of running on a 24GB GPU at 4-bit quantization or an 80GB GPU at full precision, while emphasizing that production choices depend on cost, speed, context length, scaling needs, and operational control. It recommends DeepInfra as a balanced default for low-cost, production-oriented API access with a 262,144-token native context window, multimodal support, prompt caching, function calling, and JSON mode; Alibaba Cloud for official access and a native 1-million-token context window; and Cerebras or Multiverse Computing for low-latency workloads, with Cerebras claiming roughly 1,500 output tokens per second and Multiverse reporting rapid first-token response times. OpenRouter is presented as an aggregation layer with routing and failover across 17 backends, while Akash Network, RunPod, and Northflank target different self-hosting needs through decentralized bare-metal GPUs, scale-to-zero serverless endpoints, and managed Kubernetes orchestration, respectively. Novita AI is positioned for cost-sensitive use cases with substantial cached-input discounts, and the overall recommendation is to align provider selection with workload volume, context requirements, latency targets, budget, and available engineering resources.
Oct 10, 2026
1,648 words in the original blog post.
DeepInfra announces Qwen3.8-27B, a 27-billion-parameter dense vision-language model designed for advanced coding, research, autonomous-agent, and document-intelligence workloads. Built on the Qwen3.5 architecture, it combines a native vision encoder, hybrid linear and full-attention layers, multi-token prediction, a 262,144-token context window expandable to one million tokens, and support for text, images, and video. Its configurable thinking controls allow users to adjust reasoning depth and preserve reasoning context across extended conversations. DeepInfra reports that the model surpasses Qwen3.6-27B and Muse Glimmer-30B in most cited text and multimodal benchmarks, while exceeding Opus4.6 Max on several coding, computer-use, instruction-following, and visual-reasoning tests. The model is available through DeepInfra’s OpenAI-compatible API, with parameters for structured output, streaming, multimodal messages, and reasoning behavior. Pricing is token-based, with discounted standard rates, lower cached-input costs, and Priority and Flex tiers for different latency requirements.
Oct 09, 2026
1,255 words in the original blog post.
DeepInfra’s review compares API providers for Alibaba’s Qwen3.8-27B, a 27.78-billion-parameter open-weight reasoning and vision-language model released under Apache 2.0 in August 2026. The model supports text, image, and video inputs, produces text output, offers a 262K-token native context window extendable to 1 million tokens, and is positioned for coding, agentic workflows, data analysis, and visual reasoning. Citing Artificial Analysis benchmarks, the review presents DeepInfra as a cost-focused option with promotional pricing of $0.15 per million input tokens and $1.875 per million output tokens, though its approximately 29-token-per-second throughput is slower than competitors. Multiverse Computing and Crusoe are identified as the fastest options at roughly 191–192 output tokens per second, while Multiverse also has the lowest reported time to first token at 0.66 seconds; OpenRouter emphasizes multi-provider routing and extended context support, and Alibaba Cloud offers first-party access with slower performance and higher pricing. The review attributes Qwen3.8-27B’s strong benchmark results to its hybrid-attention dense architecture and adjustable reasoning effort, while noting that verbose reasoning can increase output costs and latency.
Oct 07, 2026
2,641 words in the original blog post.
DeepInfra’s announcement compares hosting options for Alibaba’s Apache 2.0–licensed Qwen3.8-27B, a 27-billion-parameter open-weight reasoning model with roughly a 256K–262K native context window, extensibility to 1 million tokens, image and video inputs, and text outputs supporting JSON and function calling. The model is presented as strong for coding, agents, and multimodal tasks, with reported results including 73.0 on Terminal Bench 2.1, 61.7 on SWE-bench Pro, 84.3 on OSWorld-Verified, and 64.8 on WebArena-Verified, although it is also characterized as relatively slow and unusually verbose. The central comparison argues that DeepInfra’s listed prices of $0.15 per million input tokens, $1.875 per million output tokens, and $0.038 for cached tokens are substantially lower than Alibaba API’s $0.50 input and $3.00 output rates, making cost differences especially important for long-context retrieval, repeated agent prompts, tool-heavy workflows, and large code or reasoning outputs. Through several projected workload examples, the post estimates lower monthly costs on DeepInfra for document assistants, coding copilots, multimodal browser agents, and high-cache-reuse applications, while describing Alibaba as the first-party option for users who prioritize official managed access, contractual alignment, or compliance considerations.
Oct 06, 2026
2,835 words in the original blog post.
DeepInfra’s comparison of nine GLM-5.3 API providers evaluates pricing, throughput, latency, context capacity, deployment options, and platform features for Z.ai’s reasoning model, which supports a roughly one-million-token context window and is aimed at coding and agentic workloads. Based largely on Artificial Analysis benchmarks using 10,000-token prompts, it identifies DeepInfra as the lowest-cost option, with a $0.72-per-million-token blended price, discounted cached-input rates, JSON mode, function calling, and private endpoints, while Inco’s FAST tier delivers the fastest output and lowest time to first answer token but at the highest blended price. Fireworks AI and FriendliAI are positioned as high-speed alternatives, while Z.ai provides first-party access and coding-plan subscriptions, Together AI offers dedicated deployments and a 99.9% SLA, and GMI Cloud combines inference with bare-metal GPU rentals. Telnyx and DigitalOcean are presented as suitable for organizations already using their broader infrastructure, although Telnyx lacks independent benchmark data and DigitalOcean has the slowest measured output speed among the compared providers. The comparison emphasizes that provider choice depends primarily on the trade-off among cost, response speed, contractual reliability, and existing platform integration, and notes that time to first answer token is especially important because GLM-5.3 performs reasoning before producing its final response.
Oct 06, 2026
1,704 words in the original blog post.
DeepInfra’s integration guide presents Z.ai’s GLM-5.3 as an open-weights reasoning model for software engineering, agentic workflows, and cybersecurity tasks, offering text input and output with a 1,048,576-token context window and served by DeepInfra in fp4 under a zero-retention policy. Built on the same base as GLM-5.2, the model’s reported improvements stem from post-training, with Z.ai claiming substantial gains in coding, cyber evaluations, and token efficiency; its published benchmark results show broad improvement over GLM-5.2, while closed models such as GPT-5.6 Sol remain ahead on several tests. The guide cautions that benchmark results use Z.ai’s own evaluation setup and that cited throughput figures do not represent DeepInfra service performance. Pricing currently includes a 25% promotional discount, with a lower-cost Flex tier for retryable, non-real-time work that may queue for up to ten minutes. GLM-5.3 is accessible through an OpenAI-compatible chat-completions API, supports reasoning controls, structured output, streaming, function calling, and standard sampling parameters, although most outputs are capped at 16,384 tokens despite the much larger total context window.
Oct 05, 2026
1,663 words in the original blog post.
DeepInfra has introduced Qwen3.8-27B, a 27-billion-parameter open-weight vision-language model designed for software engineering, agentic workflows, long-context reasoning, and multimodal inputs including images, documents, and video. Its hybrid 64-layer architecture combines Gated DeltaNet linear attention with standard gated attention, supporting a native 262,144-token context window that can extend to one million tokens through YaRN scaling. DeepInfra reports substantial benchmark improvements over earlier Qwen models, including scores of 61.7 on SWE-bench Pro, 79.0 on QwenSWEBench, and 84.3 on OSWorld-Verified, with results presented as competitive with or exceeding some larger closed models on coding, visual, and GUI-agent tasks. The model provides adjustable reasoning controls, allowing users to toggle thinking mode, set reasoning effort, and preserve prior reasoning context to balance response quality and latency. Available through DeepInfra’s OpenAI-compatible API, it supports function calling, JSON output, multimodal inputs, and private deployments, with token-based pricing, zero data retention, and SOC 2 and ISO 27001 certifications.
Oct 05, 2026
1,320 words in the original blog post.
DeepInfra has made Z.ai’s GLM-5.3 reasoning model available through its platform, positioning it for complex software engineering, cybersecurity, and long-horizon agentic workflows. Built on the same base model as GLM-5.2, GLM-5.3’s reported improvements come solely from expanded reinforcement-learning post-training, using long-context processing, long-horizon optimization, and asynchronous training infrastructure. Z.ai reports substantial gains in coding and security benchmarks, including stronger results on Terminal Bench, DeepSWE, CyberGym, and ExploitGym, though many figures originate from internal or company-run evaluations and are not independently reproducible. The model supports a roughly one-million-token context window, function calling, JSON output, and three always-on reasoning settings, with maximum effort recommended for coding tasks. Z.ai also reports that its models have identified thousands of vulnerabilities in real codebases, although most findings remain under embargo. On DeepInfra, GLM-5.3 is offered through an OpenAI-compatible API with promotional, usage-based token pricing, optional Flex discounts, fp4 quantization, and platform security features including zero data retention and SOC 2 and ISO 27001 certifications.
Oct 03, 2026
1,854 words in the original blog post.
GLM-5.3 is an open-weights Mixture-of-Experts reasoning model from Z.ai with 753 billion total parameters, 40 billion active parameters per token, a context window of roughly one million tokens, and support for text generation, structured outputs, and function calling through hosted providers. Positioned as a capable but relatively expensive, slow, and verbose model, it is aimed at long-context coding, tool use, agent workflows, and document-heavy retrieval applications rather than low-cost general-purpose use. Pricing varies substantially by provider, with list rates cited around $1.40 per million input tokens and $4.40 per million output tokens, while DeepInfra advertises standard rates of $0.90 and $3.00 respectively and a lower-priced Flex tier at $0.72 and $2.40; cached-input pricing can further affect costs for repeated-context workloads. The discussion emphasizes that output volume, reasoning settings, caching, latency, and deployment features such as private endpoints, zero data retention, and OpenAI-compatible routing may matter as much as headline input prices. DeepInfra is presented as particularly competitive for managed high-volume or batch deployments, while OpenRouter is described as offering easier multi-provider access, failover, and potentially faster routes, and self-hosting remains an option for organizations able to manage the associated infrastructure and operational costs.
Oct 03, 2026
3,474 words in the original blog post.
DeepInfra’s guide compares API providers and related tooling for deploying DeepSeek-V4.1-Flash, a model presented as having a 1M-plus-token context window and Engram conditional memory, with selection driven by latency, throughput, context caching, cost, availability, and compliance requirements. It positions DeepInfra as a balanced option for low-latency, cost-conscious production deployments, while the official DeepSeek API is recommended for repeated large-context workloads because of claimed cache-hit discounts. Canopy Wave and Atlas Cloud are highlighted for privacy and regulated-use requirements, OpenRouter and EvoLink.AI for routing, fallbacks, and avoiding lock-in, Fireworks AI and Novita AI for high-speed agentic or generation-heavy applications, Together AI for globally distributed enterprise reliability, and Clarifai for managed deployment with prompt-testing tools. Apidog is described not as a hosting provider but as a testing platform for comparing model outputs, latency, and token use before production releases.
Oct 02, 2026
1,936 words in the original blog post.
DeepInfra’s review describes GLM-5.3, Z AI’s August 2026 open-weights flagship model for coding, agentic reasoning, and cybersecurity, as a 753-billion-parameter mixture-of-experts model with 40 billion active parameters, a one-million-token context window, and an always-on reasoning engine. It reports an Artificial Analysis Intelligence Index score of 45 versus a comparable median of 18, alongside major benchmark gains over GLM-5.2, including Terminal-Bench 3.0 and CyberGym, but notes that the model is verbose and has relatively modest native output speed of 63.4 tokens per second. The native Z AI API is presented as the baseline, with a 3.43-second time to first token and blended pricing of $0.90 per million tokens, while the comparison of 14 providers identifies DeepInfra as the lowest-cost option at $0.72 per million blended tokens, Inco (FAST) as the throughput leader at 409.5 tokens per second, Databricks as the lowest-latency third-party provider at 9.38 seconds, and Fireworks as a high-throughput alternative for large-context workloads. The review advises teams to choose providers according to cost, output speed, latency, context requirements, capacity guarantees, and differing API behavior, including the removal of support for disabling GLM-5.3’s reasoning process.
Oct 02, 2026
1,738 words in the original blog post.
DeepInfra’s overview introduces DeepSeek-V4.1-Flash, a multimodal Mixture-of-Experts model with a 552-billion-parameter backbone, text and image inputs, text outputs, and a native context window of 1,048,576 tokens. Its Causal Encoder-Decoder architecture activates 8 billion parameters during prompt prefill and 16 billion during decoding, while Compressed Sparse Attention 2 and bounded replay are presented as reducing KV-cache memory use to 890 bytes per token. The model also offers adjustable reasoning effort from 1 to 100, allowing users to trade speed and cost against reasoning depth. Internal benchmark results indicate strengths in coding, mathematics, automation, and selected agent evaluations, though it trails competing models on some difficult terminal, general-question-answering, and factual benchmarks, and results can vary with the evaluation scaffold. DeepInfra serves the model through an OpenAI-compatible API with function calling, JSON output, streaming, image-message support, and standard, priority, and flex service tiers. Standard pricing is listed at $0.20 per million input tokens, $0.60 per million output tokens, and $0.006 per million cached input tokens, with a typical output cap of 16,384 tokens despite the much larger total context capacity.
Oct 01, 2026
1,574 words in the original blog post.