Skip to content

Building a RAG System with Google Drive Integration and PostgreSQL Vector Search

I recently completed a project that tackles document search challenges by combining Retrieval-Augmented Generation (RAG), Google Drive API, FastAPI, and PostgreSQL with vector capabilities.

What This Project Does

This application lets you search and query your Google Drive documents using natural language. It connects to your Google Drive, downloads your documents, processes them into vector embeddings, and stores them in a PostgreSQL database. You can then ask questions about your documents and receive AI-generated answers based on their content.

Check out the project repository: https://github.com/nofilamer/RAG_GDRIVE_PGVector_FASTAPI.git

The Tech Stack Breakdown

1. PostgreSQL with Timescale Vector

Instead of using specialized vector databases, I leveraged PostgreSQL with pgvector and Timescale Vector extensions. This approach offers several advantages:

  • Cost-effective: According to Timescale, their implementation is 75% cheaper than dedicated vector databases
  • Single database solution: Manage both relational and vector data in one place
  • Advanced indexing: Uses DiskANN-inspired indexes for significantly faster similarity searches
  • Rich ecosystem: Benefits from PostgreSQL's robust tools and community support

2. Retrieval-Augmented Generation (RAG)

RAG combines retrieval systems with generative AI for more accurate, contextual responses. My implementation:

  • Chunks document text into manageable segments with overlap to preserve context
  • Generates vector embeddings using OpenAI's text-embedding-3-small model
  • Retrieves relevant context chunks based on query similarity
  • Uses a structured system prompt to guide the LLM in generating coherent answers
  • Returns not just the answer, but also the thought process and context sufficiency

3. Google Drive Integration

The GoogleDriveService provides seamless access to your Google Drive documents:

  • Handles OAuth2 authentication with secure token management
  • Searches for files in your Drive by name
  • Downloads files and extracts text from multiple formats (Google Docs, Sheets, PDFs, Word)
  • Converts Google's proprietary formats to text for processing

4. FastAPI Web Application

The application is built using FastAPI, providing:

  • High-performance REST API endpoints for integration with other systems
  • Simple web UI using Jinja2 templates for direct interaction
  • Strong typing with Pydantic models for request/response validation
  • Easy deployment with Docker containerization

Why This Architecture Works

  1. Modularity: Clean separation between components makes the codebase maintainable
  2. Performance: Timescale Vector provides efficient similarity search for large document collections
  3. Extensibility: The LLMFactory allows easy switching between OpenAI, Anthropic, or other LLM providers
  4. User Experience: Both API and web interface options for different use cases
  5. Cost-Effective: Using PostgreSQL as a vector database saves on infrastructure costs

The project is containerized with Docker and ready to use with your Google Drive API credentials and OpenAI API key.