Introduction to Milvus (Vector Database) Using Python

Explore Milvus architecture with a decoupled storage and compute design, detailing the access layer, route, data, index, and query coordinators, stateless worker nodes, and the log broker and storage roles.

Explore Milvus storage concepts, including collections, partitions, and segments, and learn to define collection and field schemas, including primary key allocation, while understanding segment-based data storage.

Explore Milvus installation options, including standalone and Milvus lite, plus cluster deployments, using docker compose, helm, or Kubernetes, and learn hardware requirements.

Install and configure the PyMilvus Python SDK and Milvus packages in a Python virtual environment. Connect to a Milvus server via gRPC port 19530 using Docker and Jupyter Lab.

Explore Milvus collections, fields, and collection schemas, including primary keys, auto id, dynamic data model and fixed schema model, partitions, and vector, json, and array data types.

Explore how Milvus uses partitions and segments within a collection like Album One, and how to create_partition and drop_partition partitions such as disk one and disk two.

Learn to manage Milvus data in Python by inserting and deleting entities in a collection. Prepare five random entities, delete by song ID, and apply segment compaction.

Index data in Milvus to speed vector similarity search and scalar queries, using vector indexes with euclidean, cosine, inner product metrics and in-memory, on-disk, or gpu options.

Learn to perform vector, scalar, and hybrid searches in Milvus by loading indexed collections, configuring search parameters, and retrieving results with primary keys, distances, and fields.

Explore dynamic schema in Milvus collections, enabling insertion of new fields without altering the schema by setting enable_dynamic_field to true and building a vector index with L2 distance (IVF).

Learn how Milvus uses partition keys to create and search partitions automatically, define a partition key field, and use language as an example to limit searches.

Demonstrate end-to-end workflow for generating sentence embeddings with the universal sentence encoder in TensorFlow, storing them in Milvus, and enabling vector similarity search for SMS messages.

Explore role based access control in Milvus, creating roles, assigning privileges, and managing user permissions and authentication to secure database resources.

Explore A2, the open source vector database admin tool for Milvus, and learn to connect, manage collections, create indexes, perform vector search, and administer users with Docker.

Milvus Lite is a lightweight Milvus version with embedded storage and a single Python binary for quick evaluation without Docker or Kubernetes, not for production, usable in Colab or notebooks.

Explore Milvus vector indexes for binary and floating point vectors, learning how to configure index type, distance metric, and the parameters for creating and searching similar vectors.

Explore Milvus flat index, simplest vector index that computes distances using Euclidean distance to all vectors to return top k nearest neighbors; accurate, with no data compression and not scalable.

Use Milvus inverted file indexes (IVF) to cluster vectors by centroids with k-means, then search across multiple clusters to return the top K vectors.

Milvus applies scalar quantization in the IVF index, compressing vectors to 8-bit integers with roughly 75% storage reduction, while incurring some accuracy loss.

Apply product quantization in Milvus by splitting vectors into sub-vectors, clustering subspaces, and using centroids to build a fast, memory-efficient approximate search with lookup tables.

Explore hierarchical navigable small world (hnsw) graphs as a graph-based Milvus index for vector data, enabling fast nearest neighbor search across multiple layers from low-degree to high-degree vertices.

Demonstrate image similarity search with Milvus and PyTorch by embedding Kaggle animal strain dataset images with Inception v3, storing vectors in Milvus, and performing L2-based queries on a flat index.

Demonstrates building a Linkin Park song search with Milvus and OpenAI embeddings in Python, using a Spotify Kaggle dataset, creating a Milvus collection, and performing similarity searches.

Build a q&a pipeline by integrating Milvus with LangChain and OpenAI embeddings in Python. Load and chunk text, store embeddings in Milvus, then use a MapReduce chain with zero temperature.

Celebrate completing the Milvus (vector database) using Python course and prepare to showcase your Milvus skills on real-time projects while sharing feedback to improve future offerings.