Modal review: Serverless GPU and CPU cloud for running AI inference, training and agent sandboxes

What Modal does

Runs AI workloads on serverless GPUs and CPUs: serving models, fine-tuning them, and hosting isolated sandboxes for agent code.

Modal is a cloud infrastructure platform for teams that build, train and serve AI applications. Its main products are Inference, for serving language, image, video and audio models; Sandboxes, which are isolated environments for coding agents, reinforcement learning rollouts and untrusted code; and Training, for fine-tuning and RL on GPUs. Functions let developers turn code into scalable compute, including HTTPS endpoints that scale to zero when idle.

Pricing is usage-based. Compute is billed by the second for GPUs, CPU cores and memory, and the pricing page says nothing is charged for idle time. There are three plans. Starter costs $0 plus compute per month and includes $30 per month in free credits, 3 workspace seats, 100 containers and 10 GPU concurrency. Team costs $250 plus compute per month and includes $100 per month in credits, unlimited seats, 5000 containers and 50 GPU concurrency. Enterprise pricing is custom and adds audit logs, Okta SSO, HIPAA and support through a private Slack.

Some costs sit outside the plan allowance. Shared Endpoints are billed per token rather than per second, and included compute does not cover them. Non-preemptible execution is priced at 3x base prices, and Sandboxes are billed at that rate. Region selection carries a multiplier on base prices. Observability covers logs and metrics, and data can be exported to OpenTelemetry providers. The site states that applications must be containerized and use Python.

• Inference for LLM, image, video and audio models
• Isolated Sandboxes for agents and untrusted code
• Fine-tuning and reinforcement learning on GPUs
• Functions that scale to zero when idle
• OpenAI-compatible endpoints
• Logs and metrics with OpenTelemetry export
• Per-second GPU and CPU billing

Our take

Modal fits engineering teams that want to run GPU-heavy AI workloads, such as model serving, fine-tuning or large numbers of agent sandboxes, without managing servers. Billing by the second with no idle charge suits spiky or unpredictable traffic, and the free $30 monthly credit on Starter allows a low-cost start. The trade-offs are visible on the pricing page: Starter is capped at 3 seats and 100 containers, token billing for Shared Endpoints sits outside the credit, and sandboxes and region selection cost more than base rates. The pages describe deployment as Python-based and containerized, so teams working in other setups should check fit first. Enterprise price is not published.