RAG with Laravel: Building a Document Q&A System Using Vector Search and AI

Traditional search looks for keywords. Modern users want answers.

Retrieval-Augmented Generation (RAG) bridges this gap. It combines your private data with the reasoning power of Large Language Models (LLMs). Instead of training a model, you feed it relevant context on the fly.

Laravel provides the perfect foundation for this. With its elegant syntax and robust ecosystem, you can ship a production-ready RAG system quickly. You don't need to reinvent the wheel.

The RAG Architecture: A Three-Step Loop

RAG systems operate in three distinct phases: Ingestion, Retrieval, and Generation.

First, you turn documents into numbers. These numbers are called embeddings. They represent the "meaning" of your text in a high-dimensional space. You store these in a vector database.

Second, when a user asks a question, you convert that question into a vector. You search your database for the most similar document chunks. This is semantic search.

Third, you pass those chunks to an LLM like GPT-4. You ask the model to answer the question using only the provided context. This eliminates hallucinations. It ensures the model speaks from your data.

Setting the Foundation: Vector Storage

To build a RAG system, your php web framework needs a place to store vectors. PostgreSQL with the pgvector extension is the gold standard for most teams.

Start by creating a documents table. You need a column to hold the vector data.

Schema::create('documents', function (Blueprint $table) {
    $table->id();
    $table->text('content');
    $table->jsonb('metadata')->nullable();
    $table->timestamps();
});

// Add the vector column via raw SQL for specific dimensions
DB::statement('ALTER TABLE documents ADD COLUMN embedding vector(1536);');

The dimension 1536 matches OpenAI's text-embedding-3-small model. If you use different models, adjust this number.

Generating Embeddings: Turning Text into Math

You need to communicate with AI providers to generate vectors. The Laravel AI SDK simplifies this process. It acts as a bridge between your application and providers like OpenAI or Anthropic.

Create a service to handle the heavy lifting. This keeps your build rest api with php logic clean.

namespace App\Services;

use Laravel\AI\Facades\AI;

class EmbeddingService
{
    public function createVector(string $text): array
    {
        return AI::embeddings()->create([
            'model' => 'text-embedding-3-small',
            'input' => $text,
        ]);
    }
}

This service returns an array of floats. This array is the mathematical fingerprint of your text.

Indexing with Laravel Scout

Laravel Scout is one of the most powerful php developer tools for search. While traditionally used for Algolia or Meilisearch, it now supports vector-capable drivers.

Add the Searchable trait to your Document model. Map your content to the vector column.

namespace App\Models;

use Illuminate\Database\Eloquent\Model;
use Laravel\Scout\Searchable;

class Document extends Model
{
    use Searchable;

    public function toSearchableArray(): array
    {
        return [
            'id' => $this->id,
            'content' => $this->content,
            'embedding' => $this->embedding,
        ];
    }
}

Now, every time you save a document, Scout ensures it is indexed. This makes your search logic reactive and maintainable.

Implementing Semantic Search

When a user submits a query, you don't look for exact words. You look for similar concepts.

Embed the user's question first. Then, use the <-> (distance) operator in PostgreSQL to find the nearest neighbors.

public function retrieveContext(string $query)
{
    $queryVector = $this->embeddingService->createVector($query);

    return Document::query()
        ->orderByRaw('embedding <-> ?', [json_encode($queryVector)])
        ->limit(5)
        ->get();
}

Lower distance means higher similarity. This query finds the five most relevant chunks of information in your database.

The Generation Phase: Bringing it All Together

Now you have the context. You need to give the LLM clear instructions. This is called "grounding" the prompt.

Build a controller to handle the incoming request. Use the mvc framework php structure to keep the response fast and reliable.

public function ask(Request $request)
{
    $question = $request->input('question');
    
    // Step 1: Retrieve context
    $context = $this->retrieveContext($question);
    
    $contextString = $context->pluck('content')->implode("\n\n");

    // Step 2: Generate answer
    $response = AI::chat()->create([
        'model' => 'gpt-4o',
        'messages' => [
            ['role' => 'system', 'content' => 'Answer only using the provided context.'],
            ['role' => 'user', 'content' => "Context: $contextString \n\n Question: $question"],
        ],
    ]);

    return response()->json([
        'answer' => $response->content(),
        'sources' => $context->pluck('id'),
    ]);
}

This ensures the AI doesn't make up facts. If the information isn't in your documents table, the model will say it doesn't know.

Deploying Your AI System

Performance is critical in AI applications. Vector searches can be slow on large datasets.

Use Laravel Forge to manage your production servers. It simplifies the installation of PostgreSQL and extensions like pgvector.

Monitor your LLM calls with Nightwatch. It provides the logs you need to debug failed requests or slow embeddings. If you're building a massive index, consider moving your search to Laravel Cloud for managed infrastructure that scales with your data.

Moving Forward

RAG transforms static documentation into interactive knowledge bases. It allows your php framework applications to feel intelligent.

Start with a small set of documents. Build the ingestion pipeline first. Once your embeddings are accurate, refine your prompts.

We'd love to see what you build. Share your RAG implementations with the community on GitHub or X. Your story belongs here.