> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/pfrankov/obsidian-local-gpt/llms.txt
> Use this file to discover all available pages before exploring further.

# RAG System Architecture

> Deep dive into the Retrieval-Augmented Generation implementation in Local GPT

# RAG System Architecture

Local GPT uses a sophisticated Retrieval-Augmented Generation (RAG) system to enhance AI responses with context from your Obsidian vault. This page explains the technical implementation details.

## Overview

The RAG system processes linked documents, extracts relevant content, and provides context to the AI model. This is implemented primarily in `src/rag.ts` with support for both Markdown and PDF files.

<Info>
  **Enhanced Actions** automatically extract context from documents linked in your selection to provide richer AI responses.
</Info>

## Document Processing Pipeline

### 1. Link Discovery

The system identifies linked files through the `getLinkedFiles()` function:

```typescript theme={null}
// Supports both wiki-links and markdown links
const WIKI_LINK_REGEX = /\[\[([^\]|#]+)(?:#[^\]|]+)?(?:\|[^\]]+)?\]\]/g;
const MARKDOWN_LINK_REGEX = /\[[^\]]+\]\(([^)]+)\)/g;
```

**Supported file types:**

* Markdown files (`.md`)
* PDF documents (`.pdf`)

<Note>
  The system sanitizes content by removing code blocks, HTML comments, and inline code before extracting links.
</Note>

### 2. Graph Traversal

The RAG system traverses your vault's knowledge graph with a depth limit:

```typescript theme={null}
const MAX_DEPTH = 10;
```

**Processing flow:**

<Steps>
  <Step title="Start from active file">
    The system begins with the currently active file and extracts all linked documents.
  </Step>

  <Step title="Process forward links">
    Recursively processes documents linked FROM each file up to MAX\_DEPTH levels.
  </Step>

  <Step title="Process backlinks">
    Includes documents that link TO each file (reverse references).
  </Step>

  <Step title="Prevent duplicates">
    Each file is processed only once, even if linked multiple times.
  </Step>
</Steps>

### 3. Content Extraction

#### Markdown Files

Markdown content is read using Obsidian's cached read API for optimal performance:

```typescript theme={null}
return vault.cachedRead(file);
```

#### PDF Files

PDF processing uses `pdfjs-dist` with intelligent caching:

```typescript theme={null}
// Check cache first
const cachedContent = await fileCache.getContent(file.path);
if (cachedContent?.mtime === file.stat.mtime) {
  return cachedContent.content;
}

// Extract if not cached or outdated
const arrayBuffer = await vault.readBinary(file);
const pdfContent = await extractTextFromPDF(arrayBuffer);
```

<Warning>
  PDF extraction maintains layout by detecting line breaks based on text position (`transform[5]` values). Large PDFs may take longer to process on first use.
</Warning>

**PDF processing details** (from `src/processors/pdf.ts`):

* Uses a Web Worker for background processing
* Extracts text with line break detection
* Preserves page structure with double newlines between pages
* Caches extracted text in IndexedDB for performance

### 4. Document Structure

Each processed document is stored with metadata:

```typescript theme={null}
interface IAIDocument {
  content: string;
  meta: {
    source: string;      // File path
    basename: string;    // File name without extension
    stat: FileStat;      // Creation/modification times
    depth: number;       // Graph traversal depth
    isBacklink: boolean; // Whether this is a reverse reference
  };
}
```

## Vector Search & Embedding

### Embedding Process

The system uses your configured embedding provider to create vector representations:

```typescript theme={null}
const results = await aiProviders.retrieve({
  query,
  documents,
  embeddingProvider,
  onProgress: (progress) => {
    // Updates progress bar with chunk processing status
  },
  abortController,
});
```

<Tip>
  Configure your embedding provider in **Settings → AI Providers → Embedding AI Provider**. Popular choices include `nomic-embed-text` for Ollama or OpenAI's `text-embedding-3-small`.
</Tip>

### Chunking Strategy

Documents are automatically split into chunks by the AI Providers SDK:

* Chunks are sized based on the embedding model's context window
* Progress tracking reports `processedChunks` and `totalChunks`
* Each chunk is embedded independently for granular matching

### Similarity Search

The `searchDocuments()` function:

1. Embeds your query (selected text)
2. Embeds all document chunks
3. Computes similarity scores (cosine similarity)
4. Returns ranked results

## Context Formatting

### Result Ranking

Results are organized intelligently:

<CardGroup cols={2}>
  <Card title="Group by File" icon="folder">
    Results from the same file are grouped together under `[[filename]]`.
  </Card>

  <Card title="Sort by Creation Time" icon="clock">
    File groups are sorted by creation time (newest first).
  </Card>

  <Card title="Rank by Relevance" icon="star">
    Within each group, chunks are sorted by similarity score.
  </Card>

  <Card title="Respect Limits" icon="gauge">
    Context is truncated at the configured limit (see [Context Limits](/advanced/context-limits)).
  </Card>
</CardGroup>

### Output Format

The formatted context follows this structure:

```markdown theme={null}
[[Document 1]]
Relevant chunk from Document 1 with highest score...

Another relevant chunk from Document 1...

[[Document 2]]
Relevant chunk from Document 2...
```

<Info>
  The total character count is logged in development mode: `"Total length of context"`
</Info>

## Performance Optimizations

### Caching Strategies

**PDF Content Cache** (`src/indexedDB.ts`):

* Stores extracted text by file path
* Includes modification time (`mtime`) for invalidation
* Prevents re-extraction on every request

**Metadata Cache**:

* Uses Obsidian's `MetadataCache` for link resolution
* Leverages `resolvedLinks` for graph traversal
* Avoids manual parsing of markdown files

### Parallel Processing

```typescript theme={null}
await Promise.all(
  linkedFiles.map(async (file) => {
    await processDocumentForRAG(file, context, processedDocs, 0, false);
    updateCompletedSteps?.(1);
  }),
);
```

All linked files at the same depth level are processed concurrently for maximum speed.

### Progress Tracking

The system provides real-time progress updates:

<Steps>
  <Step title="Initialize">
    `totalProgressSteps` starts at the number of linked files
  </Step>

  <Step title="Dynamic Updates">
    As chunking begins, `addTotalProgressSteps()` increases the total
  </Step>

  <Step title="Completion Tracking">
    `updateCompletedSteps()` increments as each chunk is processed
  </Step>

  <Step title="Status Bar">
    Displays percentage in the Obsidian status bar: "✨ Enhancing 45%"
  </Step>
</Steps>

## Technical Reference

### Key Functions

<AccordionGroup>
  <Accordion title="getLinkedFiles(content, vault, metadataCache, currentFilePath)">
    Extracts all wiki-links and markdown links from content.

    **Parameters:**

    * `content`: Text to parse for links
    * `vault`: Obsidian vault instance
    * `metadataCache`: Metadata cache for link resolution
    * `currentFilePath`: Current file path for relative resolution

    **Returns:** Array of `TFile` objects (md and pdf only)
  </Accordion>

  <Accordion title="processDocumentForRAG(file, context, processedDocs, depth, isBacklink)">
    Recursively processes a document and its linked files.

    **Parameters:**

    * `file`: File to process
    * `context`: Processing context (vault, metadataCache, activeFile)
    * `processedDocs`: Map to store processed documents
    * `depth`: Current traversal depth
    * `isBacklink`: Whether this is a backlink reference

    **Returns:** Updated `processedDocs` map
  </Accordion>

  <Accordion title="searchDocuments(query, documents, aiProviders, embeddingProvider, abortController, ...)">
    Performs vector similarity search across documents.

    **Parameters:**

    * `query`: Search query (usually selected text)
    * `documents`: Array of processed documents
    * `aiProviders`: AI providers service instance
    * `embeddingProvider`: Configured embedding provider
    * `abortController`: For cancellation support
    * `updateCompletedSteps`: Progress callback
    * `addTotalProgressSteps`: Dynamic total adjustment callback
    * `contextLimit`: Maximum context length in characters

    **Returns:** Formatted context string
  </Accordion>
</AccordionGroup>

## Source Code References

<CardGroup cols={2}>
  <Card title="Core RAG Logic" icon="code">
    `src/rag.ts` - Main RAG implementation
  </Card>

  <Card title="PDF Processing" icon="file-pdf">
    `src/processors/pdf.ts` - PDF text extraction
  </Card>

  <Card title="File Caching" icon="database">
    `src/indexedDB.ts` - IndexedDB cache layer
  </Card>

  <Card title="Integration" icon="puzzle-piece">
    `src/main.ts:688-747` - `enhanceWithContext()` method
  </Card>
</CardGroup>

## Next Steps

<CardGroup cols={2}>
  <Card title="Context Limits" icon="gauge" href="/advanced/context-limits">
    Learn how to configure context limits for different models
  </Card>

  <Card title="Troubleshooting" icon="wrench" href="/advanced/troubleshooting">
    Debug common RAG and embedding issues
  </Card>
</CardGroup>