Skip to main content

Managing AI Context Properly

AI coding tools have a limitation that most new users don't realize: they can only "see" a limited amount of text at once.

This limit is called the context window. Think of it like a desk. The AI can only work with what's on the desk at any given moment. If your project is larger than the desk, the AI starts forgetting things.

Understanding this limitation — and working around it — is essential for getting good results on anything beyond a small prototype.

What Is a Context Window?​

Every AI model has a maximum amount of text it can process in a single conversation. This includes:

  • Your prompts and instructions
  • The code files you've shared
  • The AI's previous responses
  • Error messages and output you've pasted

When you exceed this limit, the AI starts "forgetting" earlier parts of the conversation. It might:

  • Lose track of your project's architecture
  • Forget requirements you specified earlier
  • Generate code that contradicts previous decisions
  • Miss important context about your tech stack

How Context Windows Work in Practice​

AI ToolTypical ContextWhat This Means
ChatGPT (GPT-4)~8K-32K tokensRoughly 10-50 pages of text
Claude~100K-200K tokensRoughly 150-300 pages of text
CursorVaries by modelCan reference your entire codebase
GitHub CopilotLimited per fileFocuses on the file you're editing

Even with large context windows, more context doesn't mean better results. The AI processes all that information, but it may not prioritize the most important parts correctly.

Common Context Problems​

Problem 1: The AI Forgets Your Architecture​

You start a project and explain your architecture clearly. The AI understands it. But 20 prompts later, it generates code that violates your architecture — because it forgot the earlier conversation.

What this looks like:

Prompt 1: "We're using PostgreSQL. Use parameterized queries."
Prompt 25: "Add a search feature." → AI generates raw SQL injection-vulnerable queries.

Problem 2: Contradictory Code​

The AI generates one approach early in the conversation, then a different approach later. The two pieces don't work together.

What this looks like:

  • Early prompt: Uses bcrypt for password hashing
  • Later prompt: Uses plain SHA-256 for a "password reset" feature
  • Neither the AI nor you notice the inconsistency until something breaks

Problem 3: Lost Requirements​

You specified important rules early on (like "all API endpoints must check authentication"). After many prompts, the AI starts generating endpoints without those checks.

What this looks like:

  • The first 10 API endpoints have proper auth checks
  • Endpoint #11 doesn't — the AI "forgot" the requirement

How to Manage Context Effectively​

1. Start Fresh for Each Major Feature​

Don't keep one long conversation going for your entire project. Instead:

  • Use one conversation for planning and architecture
  • Start a new conversation for each feature or component
  • Include relevant context in each new conversation

This keeps each conversation focused and within the context window.

2. Create a Project Brief File​

Create a file in your project called AI_CONTEXT.md or .aide that contains:

💡 New to Markdown? This example uses Markdown — a simple way to format text with symbols like # for headings and - for bullet lists. If any of this looks unfamiliar, check out our Markdown Crash Course — it covers everything you need in 2 minutes.

# Project Context for AI

## Tech Stack
- Framework: Next.js 14
- Database: PostgreSQL with Prisma ORM
- Auth: NextAuth.js
- Hosting: Vercel

## Architecture
- Monorepo with apps/web and apps/api
- API routes use /api/v1/ prefix
- All API routes require authentication unless marked public

## Key Conventions
- Use TypeScript strict mode
- Use server components by default
- Database queries use Prisma, never raw SQL
- Error responses follow { error: string, code: number } format

## Current State
- Authentication is implemented
- User profiles are working
- Currently working on: payment integration

At the start of each new conversation, paste this file. The AI now has your project's full context without needing to "remember" it from earlier prompts.

3. Reference Files Explicitly​

Instead of saying "remember the database schema," say:

"Look at prisma/schema.prisma for the database schema. The User model has fields: id, email, name, createdAt."

Or paste the relevant section directly into your prompt.

4. Break Large Files Into Smaller Ones​

AI works better with focused, smaller files. A 500-line file is harder for AI to process than five 100-line files.

Organize your codebase so that:

  • Each file has a single responsibility
  • File names clearly indicate their purpose
  • Related code is grouped together

5. Use "Checkpoints" in Long Conversations​

If you must have a long conversation, periodically ask the AI to summarize:

"Before we continue, summarize what we've built so far, the key decisions we've made, and what's left to do."

This helps you verify the AI still has the right context — and gives you a summary you can paste into a new conversation if needed.

6. Keep Prompts Self-Contained​

Each prompt should include enough context to stand on its own:

Too Dependent on Previous ContextSelf-Contained
"Add validation to that form.""Add validation to the registration form we built in the previous step. It has email, password, and name fields. Validation rules: email must be valid format, password must be 8+ characters, name is required."
"Fix the error.""The login endpoint returns a 500 error when the database is down. Expected behavior: return a 503 with message 'Service temporarily unavailable'."

7. Use a "Running Notes" File​

Keep a file that tracks:

  • What's been built so far
  • Key architectural decisions
  • Known issues and TODOs
  • The current task

Update this file as you go. Paste it into new AI conversations to bring them up to speed instantly.

Context Management Workflow​

Here's a practical workflow for managing context on a real project:

Project Start
    ↓
Create AI_CONTEXT.md with tech stack, architecture, conventions
    ↓
Conversation 1: Plan feature, write spec
    ↓
Conversation 2: Implement feature (paste AI_CONTEXT.md + spec)
    ↓
Conversation 3: Implement next feature (paste updated AI_CONTEXT.md + new spec)
    ↓
Conversation 4: Debug / fix issues (paste relevant code + error messages)
    ↓
Periodically: Update AI_CONTEXT.md with new decisions and state

The Bottom Line​

AI context windows are a hard limit. Work with them, not against them.

The most common reason AI-generated code goes wrong on larger projects isn't that the AI is bad — it's that the AI doesn't have the full picture. By managing context deliberately, you ensure the AI always has the information it needs to make good decisions.

Treat context management as part of your development workflow, not an afterthought. Your AI will produce better code, and you'll spend less time fixing misunderstandings.

Last updated on