Google has released semantic image technology as open source, which means developers can use the same technology in their own apps. Portrait Mode is just one example of how this technology can be used. Developers can do even more cool stuff.
This open sourced release includes DeepLab-v3+ models built on top of a powerful convolutional neural network (CNN) backbone architecture [2, 3] for the most accurate results, intended for server-side deployment.
Google actually uses AI to pull that off – and now, they have open-sourced a similar technology that developers can use in their third-party camera apps for high-quality Portrait Mode style shots like the one above.
As part of this release, Google is additionally sharing the Tensorflow model training and evaluation code, as well as models already pre-trained on the Pascal VOC 2012 and Cityscapes benchmark semantic segmentation tasks.
Undoubtedly, Pixel 2 family has the best cameras on any smartphone right now. But most of the magic is happening on the software side. The camera uses semantic image segmentation. And with Google announcing it open sourced, the plethora of creativity can be explored.
