Retrieval Augmented Generation (RAG)

Join hosts Lois Houston and Nikita Abraham as they explore one of the most exciting innovations in enterprise AI: Retrieval Augmented Generation (RAG) powered by Oracle AI Vector Search. In this episode, Senior Principal APEX & Apps Dev Instructor Brent Dayley walks through the fundamentals of RAG, explaining how it combines Oracle Database 23ai, vector embeddings, and large language models to deliver accurate, context-rich answers from both business and unstructured data. Discover the typical RAG workflow, practical setup steps on Oracle Cloud Infrastructure, and how to work with embedding models for real-world applications.   Oracle AI Vector Search Deep Dive: https://mylearn.oracle.com/ou/course/oracle-ai-vector-search-deep-dive/144706/ Oracle University Learning Community: https://education.oracle.com/ou-community LinkedIn: https://www.linkedin.com/showcase/oracle-university/ X: https://x.com/Oracle_Edu   Special thanks to Arijit Ghosh, Anna Hulkower, Kris-Ann Nansen, and the OU Studio Team for helping us create this episode.   Please note, this episode was recorded before Oracle AI Database 26ai replaced Oracle Database 23ai. However, all concepts and features discussed remain fully relevant to the latest release.   ----------------------------------------------   Episode Transcript 00:00 Welcome to the Oracle University Podcast, the first stop on your cloud journey. During this series of informative podcasts, we'll bring you foundational training on the most popular Oracle technologies. Let's get started! 00:26 Nikita: Welcome to the Oracle University Podcast! I'm Nikita Abraham, Team Lead: Editorial Services with Oracle University, and joining me is Lois Houston, Director of Communications and Adoption Programs with Customer Success Services. Lois: Hi everyone! If you've been with us this season, you'll know we've already covered a lot about Oracle AI Vector Search. In Episode 1, we introduced the core concepts—how vectors let you search by meaning, not just keywords, and how embedding models translate your unstructured data into a searchable format inside Oracle Database 23ai.  Nikita: Then, in Episode 2, we took a deeper dive into how these vectors are actually stored and managed. We explored the different types of vector indexes, similarity metrics, and best practices for designing and optimizing your database for semantic search.  Lois: Right. Today, we're shifting gears into one of the most exciting real-world applications: Retrieval Augmented Generation, or RAG. You'll learn how RAG combines the power of Oracle AI Vector Search with large language models to answer natural language questions using both business and unstructured data.  01:39 Nikita: We'll walk through the workflow, highlight why Oracle Database is uniquely suited for RAG, and give you the essential steps to get started. Back again is Senior Principal APEX & Apps Dev Instructor Brent Dayley. Hi Brent! Could you explain what RAG is, and why it's important for working with AI and large language models? Brent: Well, RAG stands for Retrieval Augmented Generation. And this is a technique that allows us to enhance the capabilities of large language models, also known as LLMs, and this provides them with relevant context from external knowledge sources. This will allow the LLMs to generate more accurate, informative, and context-aware responses. Real world applications include answering questions, chatbot development, content summarization, and knowledge discovery.  02:35 Lois: Brent, what makes Oracle Database 23ai a good platform for implementing RAG workflows? Brent: Now, there are some key advantages of using Oracle Database 23ai as a RAG platform. These include native functionality, allowing built-in tools and packages specifically designed for RAG pipeline development.  Also, if you are a PL/SQL developer, then this will allow you to develop within a familiar and robust database environment. Also, Oracle has a plethora of security and performance tools. And this ensures enhanced security and optimized performance.  03:18 Nikita: What does a typical RAG workflow look like in Oracle Database 23ai? What are the main steps involved? Brent: Now, the primary workflow steps are going to be to generate vector embeddings from your unstructured data. You do this using vector embedding models. And you can generate those embeddings either inside or outside of the database.  Next, you need to store the vector embeddings, the unstructured data, and the relational business data, and you can store all of that in the Oracle Database. You might want to also create vector indexes that can allow you to run similarity searches over huge vector spaces with really good performance.  Finally, you need to query data with similarity searches. You can use Oracle AI Vector Search native SQL operations to combine similarity with relational searches to retrieve relevant data. And optionally, you can generate a prompt and send it to a large language model for full RAG inference.  04:30 Lois: Can you give us an example of how this workflow operates in practice? Brent: A user's natural language question is encoded as a vector and sent to AI Vector Search. Next, AI vector search finds private content, such as documents, that are stored in the database, and those will match the user's question. The content is then sent to Oracle's GenAI service to help answer the user's question. And then GenAI uses the content plus general knowledge to provide an informed answer back to the user.  05:14 Nikita: What does the overall user experience look like when interacting with RAG? How does Oracle ensure the answers are both accurate and up to date? Brent: In this case, we have a chatbot. This is the interface that we usually use to enable dialogue with the large language model. Now, in order to improve the quality of the answers, we want to search your private business data, and that allows us to pass the most relevant facts back to the LLM.  Next, we want to format the similarity search results as a prompt and context for the large language model. Now, this will allow us to use up to date facts as input to LLMs. And that will minimize the probability of the LLM hallucinating. And those high-quality responses are then returned back to the chatbot.  06:12 Lois: Brent, what does the setup process look like for getting RAG up and running with Oracle AI Vector Search on OCI? Can you take us through the main steps? Brent: First, you will log into OCI. Provide your cloud account name and click Next. There are also interfaces for signing in using a traditional cloud account. And if you're not an Oracle Cloud customer yet, you can also sign up using this page.  Next, after signing in, you will create a compute instance. And you will use Oracle Infrastructure Cloud Console in order to do this. And you will wind up with the user called OPC. You'll notice that you're using SSH in order to connect to your compute instance, and you're running a script in order to set up the Oracle Database.  After that, you will set up the Python environment, again using SSH to connect as an OPC user to your compute instance.  07:22 Do you want to optimize your implementation strategies? Check out the Oracle Fusion Cloud Applications Process Essentials training and certifications for insight into key processes and efficiencies across every phase of your Fusion Cloud Apps journey. Learn more at mylearn.oracle.com.  07:43 Nikita: Welcome back! So far, we've seen how Oracle AI Vector Search powers RAG, letting you surface relevant business knowledge for large language models and enhance their answers. At the heart of all this is the process of transforming unstructured data, like text or documents, into mathematical representations called embeddings.  Lois: Those embeddings are what make meaningful, semantic search possible. But have you wondered how those embeddings actually get created, or what goes on behind the scenes when you choose an embedding model?  Nikita: Up next, we'll take a closer look at embedding models themselves: what they are, how to use them inside Oracle Database 23ai, and how you can experiment with different models to get the results that best fit your business needs.  Lois: We'll walk through importing models, generating embeddings, and even how you can swap out embedding models to compare results. But before we get into the nitty-gritty details, let's quickly recap embedding models, since we've mentioned them in our previous episodes.  08:47 Nikita: Brent, for listeners who might need a refresher, can you explain what embedding models are and why they're so central to AI Vector Search?  Brent: AI Vector Search is based on similarity properties. You can search data by semantic similarity rather than by the actual values. Vector embeddings are created by embedding models to represent the unstructured data. So we have input data.  What we'll want to do is to use an embedding model to generate vector embeddings. And then the vector embeddings would be stored inside of a vector column in a table. We would then compare those vectors to each other using vector distance function.  And we would get the relevant content back based on the number of returns that we describe. For instance, maybe we want to bring back the five closest pieces of data compared to the input data.  There is a new function that allows you to generate vector embeddings that is called the vector embedding function. It allows you to generate vectors within the database.  10:08 Lois: Can you walk us through the practical steps for using embedding models with Oracle AI Vector Search? Brent: In order to create and set up a table, we might use the Python program called create_schema.py. And that will allow us to create a table.  We would ensure that the table was successfully created with the data. As an example, I would create a table called MY_DATA. Next, we would use a sentence transformers embedding model in order to vectorize the table. We can use the Python program, vectorize_table_SentenceTransformers.py. We would then query the MY_DATA table in the Oracle Database to verify that the data has been updated.  And then we would use sentence transformers in order to perform the similarity search. The Python program is called similarity_search_SentenceTransformers.py And what that would do is create the table and then perform a similarity search using the sentence transformers. Now what if you decide that you want to maybe change embedding models? Maybe you want to compare the results by using one particular model as compared to a different model.  So you can change the embedding model. And in order to do that, you would change the embedding model in both of the programs and re-vectorize the table using the vectorize_table_SentenceTransformers.py program. You would then use the new model with different words, possibly, and then compare and review the results, and then choose which one gets you back the data that you're looking for that is most similar.  12:02 Nikita: Well, that's a wrap on this episode. A big thank you, Brent, for sharing your expertise with us.  Lois: If you want to learn more about the topics we discussed today, visit to mylearn.oracle.com and search for the Oracle AI Vector Search Deep Dive course. Until next time, this is Lois Houston… Nikita: And Nikita Abraham, signing off! 12:25 That's all for this episode of the Oracle University Podcast. If you enjoyed listening, please click Subscribe to get all the latest episodes. We'd also love it if you would take a moment to rate and review us on your podcast app. See you again on the next episode of the Oracle University Podcast.