Groq review: Inference cloud with dedicated infrastructure, inference stack and enterprise controls

What Groq does

Groq runs AI inference as a cloud service. According to its homepage, the company designed the LPU chip and now pairs its newer LPX hardware with NVIDIA's next-generation GPUs. The vendor says its goal is making inference work at scale, and that millions of developers run trillions of tokens on it every week.

The platform page splits the offer into three layers that build on each other. GroqMetal is dedicated bare-metal infrastructure. GroqCore adds a tested inference stack on top of dedicated capacity, aimed at teams without infrastructure expertise. GroqAssured adds governance, auditability and control for enterprises. Each layer includes the ones below it.

The page words this as "when released", so these racks are presented as a coming product. Groq says it operates 13 data centers across four continents, with sites in the USA, Canada, Finland, Australia, the UK and Saudi Arabia.

Supported platforms and API details are not stated either.

• Dedicated bare-metal infrastructure (GroqMetal)
• Tested inference stack (GroqCore)
• Enterprise governance and auditability (GroqAssured)
• LPU inference chip
• LPX racks working with NVIDIA GPUs
• 13 data centers across four continents

Our take

Groq fits teams and companies that want to run inference at large scale and are interested in dedicated infrastructure, a managed inference stack or enterprise governance, chosen as separate layers. The rack specifications also refer to hardware that is not yet released, so buyers should confirm current availability, pricing and API access directly with the vendor before committing.