Five months after its open source release, Baidu has now added Kubernetes support to its PaddlePaddle. The deep learning framework will get flexibility through the Kubernetes compatibility and reach new web projects.
Baidu released PaddlePaddle in last September to make it easy for developers to build their own deep learning applications easily. The addition of Kubernetes would expand this focus by providing end-to-end deep learning process.
Kubernetes has the ability to scale on demand and allocate on the basis of available resources. Both these features will help PaddlePaddle to ease the management and utilisation of hardware resources efficiently. Also, there is fault-tolerance that would enable the framework to leverage high throughput and robustness through redundancy.
“We want to run all jobs — online and offline, production and experiments — on the same cluster so we could make full utilisation of the cluster, as different kinds of jobs require different hardware resource,” said Yi Wang of Baidu Research and Xiang Li of CoreOS write in a joint blog post, adding, “Based on our research of different container based solutions, Kubernetes fits our requirement the best.”
Going forward, Baidu’s team is planning to improve certain areas to smooth the support for Kubernetes. The engineers are set to enhance the current trainer scheduling by adding a PaddlePaddle master that works in tandem with Kubernetes API. Uplifting PaddlePaddle job configuration is also in the pipeline.
Furthermore, there are plans to improve GPU support on Kubernetes to make it the best solution for PaddlePaddle.
[…] Though Google initially promoted the open source container cluster manager, companies like Baidu, Canonical and Intel recently added its support to grow with container applications. And now, it is […]