Every agentic coding tool eventually hits the same wall: the model has a finite context window, and a real repo doesn't fit in it. The usual fix is some variant of "dump the repo to text" — walk the tree, concatenate every file, hand it to the model. That works for a 200-file toy project. It falls apart on anything real, for three concrete reasons.
If you're asking an agent to review a change, it needs the change — not the entire history of every file that happens to live in the same repo. A full dump spends most of its tokens on code that's completely irrelevant to the task, which pushes genuinely relevant files out of the window or forces truncation that cuts them off mid-file.
The fix is git-diff-awareness: pack only what changed since a ref (a branch, a commit, a tag), not the whole tree. --since main instead of .. This alone usually cuts what you're pasting by 80-95% on a real change, because most of a repo is untouched by any given diff.
A model's context window is a hard ceiling, but the number that actually matters is usually smaller — you're sharing that window with the system prompt, the conversation so far, and the model's own output. A tool that just checks "is this under 200k tokens" against the wrong ceiling will still blow your real budget.
Token-budget-awareness means capping to an actual number you specify, and doing it in a way that degrades gracefully — keep the most relevant files (usually: most recently modified) and tell you explicitly what got dropped, rather than silently truncating mid-file or failing outright.
API keys, private key blocks, .env values, cloud credentials — real repos have these, sometimes in files you forgot were tracked, sometimes in test fixtures that were never meant to be secret-free. A tool that blindly dumps file contents into a chat window will happily paste a live credential straight into a third-party LLM provider's logs.
This is a solvable problem with plain pattern matching: AWS-style keys, GitHub/Slack/Stripe/OpenAI/Anthropic token formats, PEM blocks, JWTs, and generic KEY=value env-style secret assignments can all be caught and redacted before anything leaves your machine, on by default rather than as an opt-in flag nobody remembers to set.
| Approach | Diff-aware | Budget-aware | Redacts secrets |
|---|---|---|---|
| Manual copy-paste | Sometimes (if you're careful) | No | No |
| Full-repo dump tools | Diff appended on top of the full dump, not scope-restricting | Fail-fast guard (errors over budget, doesn't fit to it) | No |
| llm-ctxpack | Yes — --since restricts the packed file set itself | Yes — --budget fits to it (real tokenizer, keeps newest files, reports what's dropped) | Yes, on by default |
npx llm-ctxpack . --since main --budget 20000 -o context.mdThat one command: only the files that changed vs main, capped to a 20k-token budget with the most recently modified files prioritized, secrets redacted, written to a Markdown file with a per-file token breakdown you can paste straight into any chat or agent context.
It's free, MIT-licensed, works with any tool that takes pasted-in text — Claude, Claude Code, ChatGPT, Cursor, Copilot Chat, or a custom agent's system prompt. Source and docs: github.com/dacode-dev/llm-ctxpack.
Just want to check how many tokens a chunk of text is, without a whole repo involved? There's a standalone browser-based token counter — paste text, see the count and how much of each model's context window it uses, nothing uploaded.