> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/pfrankov/obsidian-local-gpt/llms.txt
> Use this file to discover all available pages before exploring further.

# Context Limits

> Configure context window limits for optimal RAG performance

# Context Limits

Context limits control how much information from linked documents is included when using Enhanced Actions (RAG). Choosing the right limit is crucial for balancing answer quality, performance, and cost.

## Available Presets

Local GPT offers four context limit presets, each designed for different use cases:

<CardGroup cols={2}>
  <Card title="Local models" icon="computer">
    **10,000 characters**

    Optimized for local models with smaller context windows (e.g., 8K-32K tokens).
  </Card>

  <Card title="Cloud models" icon="cloud">
    **32,000 characters**

    Suitable for standard cloud models with medium context windows (32K-64K tokens).
  </Card>

  <Card title="Top: GPT, Claude, Gemini" icon="sparkles">
    **100,000 characters**

    For advanced models with large context windows (100K+ tokens).
  </Card>

  <Card title="No limits (danger)" icon="triangle-exclamation">
    **3,000,000 characters**

    Effectively unlimited. Use with extreme caution.
  </Card>
</CardGroup>

## Configuration

Context limits are configured globally in the plugin settings:

<Steps>
  <Step title="Open Settings">
    Navigate to **Settings → Local GPT → Advanced settings**
  </Step>

  <Step title="Locate RAG Context">
    Find the **Enhanced Actions** section and look for **RAG context**
  </Step>

  <Step title="Select Preset">
    Choose the appropriate preset for your primary AI model
  </Step>
</Steps>

<Note>
  The setting is stored in `settings.defaults.contextLimit` with values: `"local"`, `"cloud"`, `"advanced"`, or `"max"`
</Note>

## Implementation Details

The context limit is resolved in `src/main.ts:783-796`:

```typescript theme={null}
private resolveContextLimit(): number {
  const preset = this.settings?.defaults?.contextLimit as
    | "local"
    | "cloud" 
    | "advanced"
    | "max";
  const map: Record<string, number> = {
    local: 10_000,
    cloud: 32_000,
    advanced: 100_000,
    max: 3_000_000,
  };
  return map[preset];
}
```

The limit is enforced during context formatting in `src/rag.ts:368-390`:

```typescript theme={null}
for (const [basename, groupResults] of groups) {
  if (totalLength >= contextLimit) break;
  
  formattedResults += `[[${basename}]]\n`;
  // Add chunks until limit reached
}
```

## How to Choose the Right Preset

<AccordionGroup>
  <Accordion title="Local Models (10K characters)">
    **Best for:**

    * Ollama models (Llama 3, Mistral, Gemma, etc.)
    * LM Studio
    * Other local inference servers
    * Models with 8K-32K token context windows

    **Why this limit:**

    * Most local models have limited context windows
    * Prevents out-of-memory errors
    * Maintains fast inference speed
    * Focuses on only the most relevant chunks

    **Approximate token count:** \~2,500-3,000 tokens (assuming 4 chars/token)
  </Accordion>

  <Accordion title="Cloud Models (32K characters)">
    **Best for:**

    * GPT-3.5-turbo
    * Claude 3 Haiku
    * Gemini 1.5 Flash
    * Standard API models

    **Why this limit:**

    * Balances context richness with cost
    * Fits comfortably in most cloud model windows
    * Good performance/quality trade-off

    **Approximate token count:** \~8,000-10,000 tokens

    **Cost consideration:** At this limit, each request uses moderate tokens, keeping costs reasonable for pay-per-token services.
  </Accordion>

  <Accordion title="Advanced Models (100K characters)">
    **Best for:**

    * GPT-4 Turbo (128K context)
    * Claude 3 Opus/Sonnet (200K context)
    * Gemini 1.5 Pro (1M+ context)
    * Specialized long-context models

    **Why this limit:**

    * Leverages extended context capabilities
    * Provides rich, comprehensive context
    * Enables complex reasoning across many documents

    **Approximate token count:** \~25,000-30,000 tokens

    <Warning>
      This setting significantly increases:

      * API costs (3x+ compared to "Cloud models")
      * Request latency
      * Token usage

      Only use with models that have proven long-context performance.
    </Warning>
  </Accordion>

  <Accordion title="No Limits / Max (3M characters)">
    **Best for:**

    * Extreme edge cases
    * Testing and development
    * Models with multi-million token contexts

    **Why this limit:**

    * Essentially removes the limit
    * Includes all retrieved chunks

    <Warning>
      **Danger Zone:** This setting can:

      * Cause requests to fail due to token limits
      * Result in extremely high API costs
      * Slow down or crash local models
      * Provide too much context, reducing quality

      **Not recommended for production use.**
    </Warning>

    **Approximate token count:** Up to 750,000+ tokens
  </Accordion>
</AccordionGroup>

## Impact on Performance

### Quality

<Tabs>
  <Tab title="Too Small">
    **Symptoms:**

    * AI lacks necessary context
    * Answers are generic or incomplete
    * Important linked information is missed

    **Solution:** Increase to the next preset level
  </Tab>

  <Tab title="Optimal">
    **Characteristics:**

    * AI has relevant context
    * Answers are specific and accurate
    * All critical information is included
    * Model can focus on what matters
  </Tab>

  <Tab title="Too Large">
    **Symptoms:**

    * AI is overwhelmed with information
    * Answers become unfocused or verbose
    * Response quality decreases ("lost in the middle" problem)
    * Slower generation

    **Solution:** Reduce to a lower preset
  </Tab>
</Tabs>

### Speed

**Processing time increases with context size:**

```
Local (10K):     Fast    ████░░░░░░ 40%
Cloud (32K):     Medium  ██████░░░░ 60%
Advanced (100K): Slow    █████████░ 90%
Max (3M):        Slowest ██████████ 100%
```

<Info>
  Processing time includes:

  * Document extraction
  * Chunking
  * Embedding generation
  * Vector search
  * API request time
</Info>

### Cost

For paid APIs, token usage directly impacts cost:

| Preset          | Est. Tokens | Cost Multiplier |
| --------------- | ----------- | --------------- |
| Local (10K)     | \~2.5K      | 1x (baseline)   |
| Cloud (32K)     | \~8K        | \~3x            |
| Advanced (100K) | \~25K       | \~10x           |
| Max (3M)        | Variable    | Up to 300x+     |

<Warning>
  These multipliers apply to **input tokens only**. Output tokens depend on response length and are not affected by context size.
</Warning>

## Monitoring Context Usage

In development mode, Local GPT logs context statistics:

```javascript theme={null}
// Console output
ℹ️ Passed contextLimit for context: 100000
// ...
📊 Total length of context: 87345
```

To enable development mode, set `NODE_ENV=development` in your build.

## Advanced: When Context is Truncated

When the context limit is reached:

1. **Graceful Truncation**: The system stops adding chunks mid-file if needed
2. **No Partial Chunks**: Individual chunks are never split
3. **Highest Scores First**: Within each file group, highest-scoring chunks are prioritized
4. **File Order Preserved**: Newer files (by creation time) are processed first

**Example scenario:**

With a 32K limit and 5 linked files:

* File A (created today): 15K characters, all included
* File B (yesterday): 12K characters, all included
* File C (last week): 8K characters, **5K included, 3K truncated**
* File D & E: **Excluded entirely**

## Migration from Previous Versions

If you upgraded from an earlier version, your context limit is set to `"local"` (10K) by default. This migration happens in `src/main.ts:1102-1112`:

```typescript theme={null}
private migrateToVersion8(settings: LocalGPTSettings): boolean {
  if (settings._version && settings._version >= 8) {
    return false;
  }
  
  (settings as any).defaults = (settings as any).defaults || {};
  (settings as any).defaults.contextLimit =
    (settings as any).defaults.contextLimit || "local";
  
  settings._version = 8;
  return true;
}
```

<Tip>
  Review your context limit after upgrading to ensure it matches your AI provider's capabilities.
</Tip>

## Best Practices

<CardGroup cols={2}>
  <Card title="Start Conservative" icon="seedling">
    Begin with "Local models" or "Cloud models" and increase only if needed.
  </Card>

  <Card title="Match Your Model" icon="bullseye">
    Choose the preset that matches your AI model's context window.
  </Card>

  <Card title="Monitor Quality" icon="magnifying-glass">
    If answers lack context, increase the limit. If they're unfocused, decrease it.
  </Card>

  <Card title="Consider Cost" icon="dollar-sign">
    Higher limits = more tokens = higher costs for paid APIs.
  </Card>
</CardGroup>

## Next Steps

<CardGroup cols={2}>
  <Card title="RAG System" icon="gears" href="/advanced/rag-system">
    Learn how the RAG system processes and ranks context
  </Card>

  <Card title="Troubleshooting" icon="wrench" href="/advanced/troubleshooting">
    Fix issues with context limits and embedding
  </Card>
</CardGroup>