The goal of this project is to create a chatbot that runs locally and accesses a cloud-based database to provide intelligent answers based on organizational data. It functions as a "ChatGPT" connected to the company's database, allowing users to interactively and dynamically query strategic information.
The main advantage of this model is the integration of generative AI with internal data, making the analysis more contextualized and relevant to the organization. It combines several techniques, and to better explain, I will briefly present some of the concepts used:
LLM (Large Language Model) - Ollama
An LLM (Large Language Model) is an artificial intelligence model trained on large volumes of textual data to understand, generate, and process natural language in an advanced way. These models are the foundation of chatbots, virtual assistants, and generative AI tools, such as ChatGPT, Claude, and Gemini. For this project, I used the generative AI model Llama 3.2. It can be obtained for free through the link (https://ollama.com/library/llama3.2), and it is also necessary to install the platform for the model to run (https://ollama.com/).
RAG (Retrieval-Augmented Generation)
A technique that improves the generation of responses by retrieving relevant information from a database before generating the final response. This ensures that the AI uses updated and company-specific data. For example, imagine a customer support chatbot for an online store.
Without RAG: If we ask "What is the status of my order?", the AI might answer generically: "Orders are typically delivered in 5 to 7 business days."
With RAG: The AI queries the store's database and responds with personalized information: "Your order #12345 was shipped yesterday and is expected to arrive tomorrow."
Google BigQuery
For this project, I used the cloud database Google BigQuery, which stores the reports and provides fast responses to the AI queries. This process could also be performed by querying a local database directly. In the case of GBQ, the challenge was to make the local machine access data in the cloud. In this step, a Python script was developed to feed data into GBQ, thus automating the process that updates the database.
Embeddings and Vectorization
When a user asks the chatbot a question, how does it know which report contains the most relevant answer? This is where vectorization comes in, a process that transforms text into numbers so the computer can understand and compare it. Here’s how it works:
a) Each report is converted into a set of numbers (called a vector) using a model called sentence-transformers/all-MiniLM-L6-v2.
b) These vectors represent the meaning of the text and allow the chatbot to quickly compare stored information with the user’s question.
c) When someone asks a question, the chatbot searches for the most similar vectors and finds the most relevant report.
Imagine a digital library:
Without vectorization: You would need to read each book to find the answer.
With vectorization: The system already knows which books are most similar to your question and shows the most relevant ones in seconds.
Streamlit.
A framework that creates the interactive interface for the chatbot, allowing users to send questions and receive answers in real-time.
By combining these concepts, it was possible to develop this chatbot adapted from the Data Science Academy, which, unlike a generic chatbot, uses internal data from Google BigQuery. Some advantages include:
Instead of manually searching through reports, users can ask questions and get direct answers.
The AI retrieves the most relevant and updated data from the database.
The database can be constantly updated, ensuring that the AI always works with the most recent information.
It uses accessible technologies, such as Ollama (LLM), RAG, embeddings, and Google BigQuery.
As the data is stored within the company’s environment (BigQuery), there is greater control over the privacy and security of the information.
This chatbot can have various applications, such as customer support for quick answers to product and service inquiries, in management/finance for queries about billing, expenses, and profitability, and internally for employees to access content and procedures.
And in your company, what would be the application? Contact us for more details on how this application can add value to your business.
Comments