> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/pfrankov/obsidian-local-gpt/llms.txt
> Use this file to discover all available pages before exploring further.

# Enhanced Actions (RAG)

> Use Retrieval-Augmented Generation to enhance AI actions with context from links, backlinks, and PDFs

Enhanced Actions leverage **Retrieval-Augmented Generation (RAG)** to provide AI with rich context from your vault's linked notes and PDF files. This makes responses more accurate and contextually aware.

<img width="450" alt="Enhanced Actions" src="https://github.com/user-attachments/assets/5fa2ed36-0ef5-43b0-8f16-07588f76d780" />

## What is RAG?

RAG (Retrieval-Augmented Generation) enhances AI responses by:

1. **Analyzing** your selected text and linked documents
2. **Retrieving** relevant content from connected notes and PDFs
3. **Augmenting** the AI prompt with this contextual information
4. **Generating** more informed and accurate responses

<Note>
  RAG automatically processes links (`[[like this]]`), backlinks, and even PDF file references in your notes.
</Note>

## How It Works

### Automatic Context Detection

When you run an action, Local GPT automatically:

<Steps>
  <Step title="Scans for Links">
    Detects wiki-style links `[[note]]` and markdown links `[text](note.md)` in your selected text
  </Step>

  <Step title="Follows Backlinks">
    Finds notes that link back to the current document
  </Step>

  <Step title="Processes PDFs">
    Extracts text from linked PDF files
  </Step>

  <Step title="Retrieves Relevant Chunks">
    Uses embedding models to find the most relevant content
  </Step>

  <Step title="Enhances Prompt">
    Includes this context in the AI request
  </Step>
</Steps>

<CodeGroup>
  ```typescript src/rag.ts theme={null}
  export async function startProcessing(
    linkedFiles: TFile[],
    vault: Vault,
    metadataCache: MetadataCache,
    activeFile: TFile,
    updateCompletedSteps?: (steps: number) => void,
  ): Promise<Map<string, IAIDocument>> {
    const processedDocs = new Map<string, IAIDocument>();
    const context: ProcessingContext = { vault, metadataCache, activeFile };

    await Promise.all(
      linkedFiles.map(async (file) => {
        await processDocumentForRAG(file, context, processedDocs, 0, false);
        updateCompletedSteps?.(1);
      }),
    );

    return processedDocs;
  }
  ```
</CodeGroup>

## Setup

### 1. Install an Embedding Model

You need an embedding model to enable RAG. For **Ollama** users:

<Tabs>
  <Tab title="English Only">
    ```bash theme={null}
    ollama pull nomic-embed-text
    ```

    Fastest option, optimized for English text.
  </Tab>

  <Tab title="Multilingual">
    ```bash theme={null}
    ollama pull bge-m3
    ```

    Slower but more accurate for multiple languages.
  </Tab>
</Tabs>

### 2. Configure Embedding Provider

1. Open Local GPT settings
2. Find **Embedding Provider**
3. Select your embedding model provider
4. Choose a model with a large context window for best results

<img width="479" alt="Settings" src="https://github.com/user-attachments/assets/5337e74c-864b-45cb-82e0-2c32bbbfa3ed" />

<Tip>
  Use the largest model with the largest context window your system can handle for better results.
</Tip>

## Supported File Types

<CardGroup cols={2}>
  <Card title="Markdown Files" icon="markdown">
    `.md` files are processed for text content and links
  </Card>

  <Card title="PDF Files" icon="file-pdf">
    `.pdf` files are processed to extract text content
  </Card>
</CardGroup>

<CodeGroup>
  ```typescript src/rag.ts theme={null}
  const SUPPORTED_RAG_EXTENSIONS = new Set(["md", "pdf"]);

  export async function getFileContent(
    file: TFile,
    vault: Vault,
  ): Promise<string> {
    if (file.extension === "pdf") {
      const cachedContent = await fileCache.getContent(file.path);
      if (cachedContent?.mtime === file.stat.mtime) {
        return cachedContent.content;
      }

      const arrayBuffer = await vault.readBinary(file);
      const pdfContent = await extractTextFromPDF(arrayBuffer);
      await fileCache.setContent(file.path, {
        mtime: file.stat.mtime,
        content: pdfContent,
      });
      return pdfContent;
    }

    return vault.cachedRead(file);
  }
  ```
</CodeGroup>

## Context Limits

You can configure how much context to include based on your model's capabilities:

<ParamField path="Context Limit" type="select" default="local">
  <Expandable title="Context Limit Options">
    * **Local** (10,000 chars): For local models with smaller context windows
    * **Cloud** (32,000 chars): For cloud-based models
    * **Advanced** (100,000 chars): For advanced models with large context windows
    * **Max** (3,000,000 chars): For cutting-edge models with massive context support
  </Expandable>
</ParamField>

<CodeGroup>
  ```typescript src/main.ts theme={null}
  private resolveContextLimit(): number {
    const preset = this.settings?.defaults?.contextLimit as
      | "local"
      | "cloud"
      | "advanced"
      | "max";
    const map: Record<string, number> = {
      local: 10_000,
      cloud: 32_000,
      advanced: 100_000,
      max: 3_000_000,
    };
    return map[preset];
  }
  ```
</CodeGroup>

## Link Processing

### Wiki-Style Links

```markdown theme={null}
This note references [[Another Note]] and [[Research Paper]].
```

Local GPT will retrieve content from both "Another Note" and "Research Paper".

### Markdown Links

```markdown theme={null}
See [this reference](notes/reference.md) for more details.
```

### PDF References

```markdown theme={null}
According to [[research-paper.pdf]], the findings show...
```

<CodeGroup>
  ```typescript src/rag.ts theme={null}
  export function getLinkedFiles(
    content: string,
    vault: Vault,
    metadataCache: MetadataCache,
    currentFilePath: string,
    includeAllMarkdownLinks = false,
  ): TFile[] {
    const sanitizedContent = sanitizeMarkdownForLinks(content);
    const wikiLinks = Array.from(
      sanitizedContent.matchAll(WIKI_LINK_REGEX),
      (match) => match[1],
    );
    const markdownLinks = Array.from(
      sanitizedContent.matchAll(MARKDOWN_LINK_REGEX),
      (match) => normalizeMarkdownLink(match[1]),
    ).filter((link): link is string => Boolean(link));

    return [...wikiLinks, ...markdownLinks]
      .map((linkText) => {
        const linkPath = metadataCache.getFirstLinkpathDest(
          linkText,
          currentFilePath,
        );
        return linkPath ? vault.getAbstractFileByPath(linkPath.path) : null;
      })
      .filter(isSupportedRagFile);
  }
  ```
</CodeGroup>

## Backlinks

Local GPT also processes **backlinks** — notes that reference the current document:

<CodeGroup>
  ```typescript src/rag.ts theme={null}
  export function getBacklinkFiles(
    file: TFile,
    context: ProcessingContext,
    processedDocs: Map<string, IAIDocument>,
  ): TFile[] {
    const resolvedLinks = context.metadataCache.resolvedLinks || {};
    const backlinks: TFile[] = [];

    for (const [sourcePath, links] of Object.entries(resolvedLinks)) {
      if (processedDocs.has(sourcePath) || !links?.[file.path]) {
        continue;
      }
      const backlinkFile = context.vault.getAbstractFileByPath(
        sourcePath,
      ) as TFile | null;
      if (backlinkFile?.extension === "md") {
        backlinks.push(backlinkFile);
      }
    }

    return backlinks;
  }
  ```
</CodeGroup>

## Performance

Local GPT includes several optimizations:

<AccordionGroup>
  <Accordion title="PDF Caching">
    PDF content is cached to avoid re-processing. The cache is invalidated when the PDF file is modified.
  </Accordion>

  <Accordion title="Depth Limiting">
    Link traversal is limited to a maximum depth of 10 levels to prevent infinite loops and excessive processing.
  </Accordion>

  <Accordion title="Progress Tracking">
    A status bar shows processing progress when RAG is active, so you know the system is working.
  </Accordion>
</AccordionGroup>

<CodeGroup>
  ```typescript src/rag.ts theme={null}
  const MAX_DEPTH = 10;

  export async function processDocumentForRAG(
    file: TFile,
    context: ProcessingContext,
    processedDocs: Map<string, IAIDocument>,
    depth: number,
    isBacklink: boolean,
  ): Promise<Map<string, IAIDocument>> {
    if (depth > MAX_DEPTH || processedDocs.has(file.path)) {
      return processedDocs;
    }
    // ... process document
  }
  ```
</CodeGroup>

## Status Bar Indicator

When RAG is processing context, you'll see a status bar indicator:

```
Enhancing with context: 45%
```

This shows the progress of embedding generation and document retrieval.

<Warning>
  Press `Escape` to cancel RAG processing at any time.
</Warning>

## Example

Given this note:

```markdown theme={null}
Based on [[Project Goals]] and the findings in [[research.pdf]], 
we should focus on user experience.
```

When you select this text and run an action like "Summarize", Local GPT will:

1. Read the content of "Project Goals"
2. Extract text from "research.pdf"
3. Find relevant sections using embeddings
4. Include this context when generating the summary

The result is a summary that's informed by all linked documents, not just the selected text.

## Next Steps

<CardGroup cols={2}>
  <Card title="Vision Support" icon="image" href="/features/vision-support">
    Learn how to analyze images with vision models
  </Card>

  <Card title="Community Actions" icon="users" href="/features/community-actions">
    Explore and install community-contributed actions
  </Card>
</CardGroup>