The Milvus open source project was founded by the creators of Zilliz, who today revealed significant contributions to the Milvus 2.1 release. The new functionality addresses the most frequent worries of developers by bridging the gap across data pools, destroying data silos, and improving performance and availability. One of the most cutting-edge vector databases in the world, Milvus can handle vast amounts of both organised and unstructured data to hasten the creation of next-generation data fabric.
The LF AI & Data Foundation’s graduated-stage open source project Milvus, which is utilised by numerous businesses in various industries, is designed for scalable similarity search. As data volumes and workloads rise, it can readily scale and embrace distributed design. Milvus allows DML operations (adding, deleting, and updating) and near real-time search of vectors on a trillion-byte scale. It is also highly scalable, dependable, and incredibly quick.
By lowering search latency on million-scale datasets to five milliseconds with this 2.1 release, Milvus sees a considerable boost in its performance. Deployment and ops procedures are also made even simpler. Every day, enormous amounts of scalar and vector data are produced by machine learning. Milvus 2.1 is bridging this crucial gap between data pools with the addition of more scalar data types.
Zilliz has explicitly made the following contributions to the 2.1 release:
– A significant improvement in throughput for small-NQ application scenarios, such as reverse image search and intelligent chatbots; support for multiple memory replicates for small tables to increase throughput; and a 2x improvement in search performance.
– Improved scalar data processing that permits index creation for scalar data and adds Varchar to the list of supported data types, bringing hybrid search to a new level of usability.
– Production-grade improvements and increased availability, with more transparent monitoring metrics for observability, simpler and more flexible deployment options like embedded Milvus for quick deployment and Ansible for offline deployment, integration that supports Kafka as log storage, and improved security supporting password protection and TLS connection.
– A developer-friendly ecosystem is being created that will connect Milvus with the open source vector data ETL framework Towhee and add Feder, an open source tool that aids Milvus users in choosing the index that is most appropriate for their application scenario by visualising the vector similarity search process.
Milvus will offer additional functionality necessary to contemporary security mechanisms, such as ACL (Access Control Lists) and sophisticated encryption techniques, in addition to the integration and security elements already mentioned.
