Inside Oracle AI Vector Search: Indexes, Metrics, and Best Practices
00:00
Welcome to the Oracle University Podcast, the first stop on your cloud journey. During this series of informative podcasts, we'll bring you foundational training on the most popular Oracle technologies. Let's get started!
00:26
Nikita: Welcome to the Oracle University Podcast! I'm Nikita Abraham, Team Lead: Editorial Services with Oracle University, and joining me is Lois Houston, Director of Communications and Adoption Programs with Customer Success Services.
Lois: Hi everyone! Thanks for joining us again as we continue our exploration into the exciting world of Oracle AI Vector Search. In today's episode, we're taking you inside the technology powering vector search in Oracle Database 23ai. We'll break down core concepts like vector indices, how vectors are stored and managed, and how you can use similarity metrics to unlock new possibilities with your data.
01:09
Nikita: We'll also dig into best practices for handling vectors, everything from memory requirements and table creation to the nuts and bolts of running both exact and approximate similarity searches. Back with us today is Senior Principal APEX & Apps Dev Instructor Brent Dayley. Hi Brent! What exactly are vector indexes?
Brent: Now, vector indexes are specialized indexing data structures that can make your queries more efficient against your vectors. They use techniques such as clustering, and partitioning, and neighbor graphs. Now, they greatly reduce the search space, which means that your queries happen quicker. They're also extremely efficient. They do require that you enable the vector pool in the SGA.
02:06
Lois: And are there different types of vector indices supported?
Brent: So, Oracle AI Vector Search supports two types of indexes, in-memory neighbor graph vector index. HNSW is the only type of in-memory neighbor graph vector index that is supported. These are very efficient indexes for vector approximate similarity search. HNSW graphs are structured using principles from small world networks along with layered hierarchical organization.
And neighbor partition vector index. Neighbor partition vector index, inverted file flat index, is the only type of neighbor partition index supported. It is a partition-based index which balances high search quality with reasonable speed.
In order for you to be able to use vector indexes, you do need to enable the vector pool area. And in order to do that, what you need to do is set the vector memory size parameter.
You can set it at the container database level. And the PDB inherits it from the CDB. Now bear in mind that the database does have to be balanced when you set the vector pool.
Other considerations, vector indexes are stored in this pool, and vector metadata is also stored here. You do need to restart the database. So large vector indexes do need lots of RAM, and RAM constrains the vector index size. You should use IVF indexes when there is not enough RAM. IVF index is used both the buffer cache as well as disk.
04:05
Lois: Now, memory is definitely a key consideration, right? Can you share more about the memory requirements and considerations for working with vectors?
Brent: So to remind you, a vector is a numerical representation of text, images, audio, or video that encodes the features or semantic meaning of the data, instead of the actual contents, such as the words or pixels of an image. So the vector is a list of numerical values known as dimensions with a specified format.
Now, Oracle does support the int8 format, the float32 format, and the float64 format. Depending on the format depends on the number of bytes. For instance, int8 is one byte, float32 is four bytes.
04:56
Nikita: And how do you calculate the size of a vector?
Brent: Now, that's going to depend upon the embedding model that you use to create those embeddings. Oracle AI Vector Search supports vectors with up to 65,535 dimensions. As a reminder, vectors are stored in tables and table data is stored on disk.
05:19
Nikita: Let's talk about working with vectors in tables. Can you walk us through how Oracle Database 23ai supports creating tables with vector columns?
Brent: Now, Oracle Database 23ai does have a new VECTOR data type. The new data type was created in order to support vector search.
The definition can include the number of dimensions and can include the format. Bear in mind that either one of those are optional when you define your column. The possible dimension formats are Int, float 32, and float 64. Float 32 and float 64 are IEEE standards, and Oracle Database will automatically cast the value if needed.
Let's take a look at some of the declaration examples. Now, if we just do a vector type, then the vectors can have any arbitrary number of dimensions and formats. If we describe the vector type as vector * , *, then that means that vectors can have an arbitrary number of dimensions and formats. Vector and vector * , * are equivalent. Vector with the number of dimensions specified, followed by a comma, and then an asterisk, is equivalent to vector number of dimensions.
Vectors must all have the specified number of dimensions, or an error will be thrown. Every vector will have its dimension stored without format modification. And if we do vector asterisk common dimension element format, what that means is that vectors can have an arbitrary number of dimensions, but their format will be up-converted or down-converted to the specified dimension element format, either INT8, float 32, or float 64.
07:25
Lois: Are there any operations or configurations that are prohibited with the VECTOR data type?
Brent: You cannot define vector columns in or as external tables, index-organized tables, neither as the primary key nor as non-key columns, in clusters or cluster tables, global temporary tables, subpartitioning key, primary key, foreign key, or unique constraint.
Additionally, you cannot define vector columns in or as check constraints, default value, modify column, manually segment space manage tablespaces. Only the SYS user can create vectors as basic files in manually segment space manage tablespaces. For continuous query notification queries, or for non-vector indexes such as B-tree, bitmap, reverse key, text, or spatial indexes. Also, bear in mind that Oracle does not support distinct, count distinct, order by, group by, join condition, or comparison operators such as less than, greater than, or equal to with vector columns.
08:46
Have you already nailed the basics of AI? Then it's time to level up. Explore advanced AI with our OCI AI Professional courses and certifications covering Data Science, Generative AI, and AI Vector Search. Are you ready to take the next step? Head over to mylearn.oracle.com and learn more!
09:12
Nikita: Welcome back!! Now, let's shift gears and discuss vector search itself. How does one create a vector "on the fly" for testing or learning purposes?
Brent: Now, the vector constructor is a function that allows us to create vectors without having to store those in a column in a table. These are useful for learning purposes. You use these usually with a smaller number of dimensions. Bear in mind that most embedding models can contain thousands of different dimensions. You get to specify the vector values, and they usually represent two-dimensional like xy coordinates. The dimensions are optional, and the format is optional as well.
10:01
Lois: Once we have vectors, how do we compare them or measure how "close" they are to each other?
Brent: Now vector distance uses the function VECTOR_DISTANCE as the main function. This allows you to calculate distances between two vectors and therefore takes two vectors as parameters. Optionally, you can specify a metric. If you do not specify a metric, then the default metric, COSINE, would be used.
You can optionally use other shorthand functions, too. These include L1 distance, L2 distance, cosine distance, and inner product. All of these functions also take two vectors as input and return the distance between them. Now the VECTOR_DISTANCE function can be used to perform a similarity search. And bear in mind these caveats. If a similarity search query does not specify a distance metric, then the default cosine metric will be used for both exact and approximate searches.
If a similarity search does specify a distance metric in the VECTOR_DISTANCE function, then an exact search with that distance metric is used if it conflicts with the distance metric specified in a vector index. If the two distance metrics are the same, then this will be used for both exact as well as approximate searches.
11:44
Nikita: Can you break down the distance metrics we use in Oracle AI Vector Search?
Brent: We have Euclidean and Euclidean squared distances. We have cosine similarity, dot product similarity, Manhattan distance, and Hamming similarity. Now let's take a closer look at the first of these metrics, Euclidean and Euclidean squared distances. This gives us the straight-line distance between two vectors. It does use the Pythagorean theorem. And notice that it is sensitive to both the vector size as well as the direction.
With Euclidean distances, comparing squared distances is equivalent to comparing distances. So when ordering is more important than the distance values themselves, the squared Euclidean distance is very useful as it is faster to calculate than the Euclidean distance, which avoids the square root calculation.
12:54
Lois: Cosine similarity is a term I hear often. How does it work exactly?
Brent: It is one of the most widely used similarity metrics, especially in natural language processing. The smaller the angle means they are more similar. While cosine distance measures how different two vectors are, cosine similarity measures how similar two vectors are.
13:20
Nikita: Dot product similarity comes up a lot, too. What's its role?
Brent: Dot product similarity allows us to multiply the size of each vector by the cosine of their angle. The corresponding geometrical interpretation of this definition is equivalent to multiplying the size of one of the vectors by the size of the projection of the second vector onto the first one or vice versa. Larger means that they are more similar. Smaller means that they are less similar.
13:58
Lois: How does Manhattan distance differ from other metrics, and when is it used?
Brent: This is useful for describing uniform grids. You can imagine yourself walking from point A to point B in a city such as Manhattan. Now, since there are buildings in the way, maybe we need to walk down one street and then turn and walk down the next street in order to get to our result. As you can imagine, this metric is most useful for vectors describing objects on a uniform grid such as city blocks, power grids, or perhaps a chessboard. Now these are faster than the Euclidean metric.
14:48
Nikita: And how is Hamming similarity different from the others?
Brent: This describes where vector dimensions differ. They are binary vectors, and it tells us the number of bits that require change to match. It compares the position of each bit in the sequence. Now, these are usually used in order to detect network errors.
15:17
Nikita: Now that we've covered the foundations, how do we actually search for the "closest" vectors in our data? What's an exact similarity search?
Brent: An exact similarity search allows you to calculate the query vector distance to all other vectors. This is also called a flat search or an exact search. This does give you the most accurate results. It gives you perfect search quality. However, you might have potentially long search times.
Now, this comparison is done using a particular distance metric. But what is important is the result set of your top closest vectors not the distance between them.
Let's take a look at one of the metrics. This one is Euclidean. The Euclidean similarity search retrieves the top k nearest vectors in your space relative to the Euclidean distance metric and a query vector.
Now let's take a look at Euclidean squared distance. In the case of Euclidean distances, comparing squared distances is equivalent to comparing distances. So when ordering is more important than the distance values themselves, the Euclidean squared distance is very useful, as it is faster to calculate than the Euclidean distance, avoiding the square-root calculation.
16:46
Lois: How does that compare to approximate searches, which are usually faster, using vector indices?
Brent: Approximate similarity search is a type of vector search that uses vector indexes. In order to use vector indexes, you have to ensure that you have enabled the vector pool in the SGA. For a vector search to be useful, it needs to be fast and accurate.
These types of searches can be more efficient. However, the trade off is that they can be less accurate. Now, approximate searches use vector indexes, and there are many types of approximate searches that you can perform using vector indexes. Vector indexes can be less accurate, but they can consume less resources. Because 100% accuracy cannot be guaranteed by the heuristics, vector index searches use target accuracy.
Internally, the algorithms used for both the index creation and index search are doing their best to be as accurate as possible. You do have the option to influence those algorithms by specifying a target accuracy.
Let's take a look at vector indexes a little closer. We have two types of vector indexes. We have HNSW indexes, which stand for Hierarchical Navigable Small World index, and we have Inverted File Flat index, or IVF.
18:23
Nikita: And for more complex requirements, how does Oracle handle multi-vector similarity search?
Brent: Multi-vector similarity search is usually used for multi-document search. The documents would be split into chunks. The chunks would be embedded individually into vectors. It does use the concept of groupings called partitions. A multi-vector search consists of retrieving the top K vector matches, using the partitions based on the document's characteristics.
The ability to score documents based on the similarity of their chunks to a query vector being searched is facilitated in SQL using the partitioned row-limiting clause.
Now, the partition row-limiting clause extension is a generic extension of the SQL language. It does not have to apply to just vector searches. Multi-vector search with the partitioning row limit clause does not use vector indexes.
19:32
Lois: We covered quite a lot today! Thanks for that, Brent! If you want to learn more about the topics we discussed today, go to mylearn.oracle.com and search for the Oracle AI Vector Search Fundamentals course. Until next time, this is Lois Houston…
Nikita: And Nikita Abraham, signing off!
19:52
That's all for this episode of the Oracle University Podcast. If you enjoyed listening, please click Subscribe to get all the latest episodes. We'd also love it if you would take a moment to rate and review us on your podcast app. See you again on the next episode of the Oracle University Podcast.
Więcej odcinków z kanału "Oracle University Podcast"
Nie przegap odcinka z kanału “Oracle University Podcast”! Subskrybuj bezpłatnie w aplikacji GetPodcast.
