DeepSeek has released DeepInfer, the software it says powers its large-scale AI services, as a free download. The code went up on GitHub and drew 420 points and 139 comments on Hacker News. It is aimed at engineers, not everyday users.
What it is
Inference means running a finished AI model to produce answers. DeepInfer is the engine that does that work for DeepSeek models. It is free under the Apache 2.0 license. DeepSeek says it was built primarily on vLLM, with ideas from SGLang and TensorRT-LLM. The default model it runs is DeepSeek-V4.1-Flash-1017.
What it needs
- Linux, Python 3.12 and GCC 13 or newer
- NVIDIA chips with CUDA 13, or Huawei Ascend chips with CANN 9.2
- Building the software from source code
This release covers offline use only. The parts needed for a complete online service are not included, so it is not a full copy of what DeepSeek runs.
What is not known
The material does not say how much money DeepInfer could save, or how it compares with other tools. DeepSeek lists its own speed tests, measured on 4 NVIDIA-class chips for one stage and 64 for another. For most businesses, the practical effect would come later, if free tools like this lower the cost of AI features built into the software they use.
The takeawayDeepInfer is a free but partial release for engineers, and any lower AI costs for ordinary businesses would come later, if at all.