DeepSeek is looking to make one of the core components of its AI models more open and accessible to other developers.
The Chinese AI startup said it will be sharing technical details about its internal inference engine with the open-source community. Inferencing is one of the many stages of building a large language model (LLM). It involves the trained AI model generating new data, which shows the patterns that the model has learned based on its parameters.
DeepSeek said that its internal inference engine and training framework have been instrumental in accelerating the training and deployment of its AI models. While its training framework is built upon the PyTorch platform, the startup’s inference engine is a modified version of vLLM, an open-source library for LLM inferencing that has been developed by researchers at UC Berkeley, United States.
“Given the growing demand for deploying models like DeepSeek-V3 and DeepSeek-R1, we want to give back to the community as much as we can. We are deeply grateful for the open-source ecosystem, without which our progress toward AGI [artificial general intelligence] would not be possible,” a DeepSeek researcher’s note posted on Hugging Face, an online repository for open-source AI models.
However, the company is not making its internal inference engine fully open-source and accessible. Instead, DeepSeek said it will share the design improvements it made to the vLLM inference engine as well as details about its implementation, with existing open-source projects. It also committed to pulling out useful features and sharing them as standalone, reusable libraries with the open-source community.
DeepSeek identified certain stumbling blocks to making its inference engine fully open-source such as lack of maintenance bandwidth, infrastructural restrictions, and a heavily customised codebase. In February this year, DeepSeek made portions of its AI models such as code repositories open-source as part of its ‘open-source week’ initiative.
Beyond cost and compute efficiency, DeepSeek’s breakthrough was celebrated by AI researchers and tech executives for being open-source. However, its models do not fit the widely accepted definition of an open-source AI system provided by the Open Source Initiative (OSI). The data used to train its flagship R1 model as well as the training framework and training code have not been released under the permissive MIT licence.
© IE Online Media Services Pvt Ltd
Expand
