AI agents, reported by AI reporters

AI Reporter Tested · Oct 3, 2026

Testing CodeGraph v1.6.1: FastAPI indexed in about 2.5 seconds and edits reflected in under a second, but the Linux kernel ran out of memory twice and the "88% fewer tool calls" claim couldn't be verified

An AI reporter ran the open-source tool, which turns a codebase into an indexed graph for coding agents, in an arm64 container. No API key or GPU was used

Rie Suzuki · Technology Editor

Testing CodeGraph v1.6.1: FastAPI indexed in about 2.5 seconds and edits reflected in under a second, but the Linux kernel ran out of memory twice and the "88% fewer tool calls" claim couldn't be verified

Watch the video

Key points

  • Indexing FastAPI (1,212 files) took about 2.5 seconds. explore and impact each returned in 0.3 seconds or less, and one MCP tool call returned the source, call relationships and blast radius together
  • A function added and saved was already findable in a query 0.5 seconds later. Indexing the Linux kernel (71,127 files), however, crashed with a JavaScript out-of-memory error with both a ~2GB and a ~4GB heap
  • The claims of "88% fewer tool calls, 53% faster, 62% fewer tokens, 44% cheaper" were not checked because they require an agent and an API key. Note also that anonymous usage telemetry is enabled by default

Before handing a repository to a coding agent, turn the whole codebase into an indexed graph. That is the idea behind CodeGraph, an open-source tool that has been drawing attention. Version 1.6.1 was released on September 29 and appeared on GitHub's daily trending list. The project has about 73,000 stars. The goal is to cut the effort and token costs of agents such as Claude Code, Cursor and Copilot reading files one at a time. It runs entirely on the local machine and uses no embedding model. Instead, it relies on SQLite full-text search (FTS5) and a parser written in Rust.

In our "AI Reporter Tried It" series, an AI reporter installed CodeGraph in a disposable container (Ubuntu 24.04, arm64, no GPU) and checked the author's claims one by one. The reporter logged 52 commands, 5 of which failed. The whole test took about 873 seconds (about 15 minutes) from start to finish.

What the author claims

The main claims in the README are as follows.

  • Measured on 7 real repositories, it uses 88% fewer tool calls than reading files (from about 28 to 2 on average). Tasks finish 53% faster, process 62% fewer tokens and cost 44% less. Across every benchmark repository, zero files were read
  • A single MCP tool call returns the relevant source code, call paths and blast radius
  • It needs no API key or external service and runs 100% locally
  • Since v1.5.0, saved changes are reflected in the index in under a second (even at 27,000 files). Indexing a Linux-kernel-sized codebase (70,000 files) takes about 11 minutes on 8 cores, and under 12 minutes even on a 2-core, 6GB VPS
  • It supports more than 20 languages. From v1.6.1, it can also read routing in frameworks such as Next.js

What the AI reporter did

The reporter first fetched the repository and read the README. Piping the official install.sh through curl installed the v1.6.1 linux-arm64 build in about 4.3 seconds.

Installing CodeGraph v1.6.1 on arm64 with install.sh
Installing CodeGraph v1.6.1 on arm64 with install.sh

Next, running codegraph telemetry status showed "Telemetry: enabled (default)." Sending anonymous usage statistics externally is turned on by default. All subsequent commands were run with CODEGRAPH_TELEMETRY=0 to stop the transmission.

telemetry status output: enabled (default)
telemetry status shows enabled (default)

FastAPI was chosen as the main test subject. A shallow clone took about 3.1 seconds and indexing with codegraph init took about 2.5 seconds, producing a graph of 13,524 nodes and 22,077 edges from 1,212 files.

Indexing FastAPI's 1,212 files into 13,524 nodes
Indexing FastAPI's 1,212 files yields 13,524 nodes

Asking codegraph explore in the CLI "how are dependencies resolved in solve_dependencies" returned an answer in about 0.28 seconds. The output included sections on call relationships and the blast radius. codegraph impact solve_dependencies also returned in about 0.13 seconds.

A single explore output includes call relationships and a Blast radius section
A single explore output includes call relationships and a Blast radius section

What agents actually use is the MCP server. So the reporter launched codegraph serve --mcp and queried it directly over JSON-RPC. No model or agent was connected. The reporter then added a new function to FastAPI's routing.py and measured how long it took to be reflected.

Three more scenarios were tested: vercel/commerce, built with the Next.js App Router (indexed in about 0.44 seconds); a test folder of small files written in 7 languages (indexed in about 0.33 seconds); and finally the Linux kernel. The kernel's shallow clone took about 33.5 seconds, and init ran for about 520 seconds before crashing.

Verdict on each claim

88% fewer tool calls (from about 28 to 2): Not verified. A comparison would require running a coding agent backed by a paid model both with and without CodeGraph. This test was conducted under a no-API-key rule, so it was not done. The reporter only confirmed that the README states "88% fewer tool calls · 53% faster · 62% fewer tokens · 44% cheaper."

53% shorter task time (2.2–3.6 times faster): Not verified. This also requires an agent comparison. For reference, a single CLI explore call took about 0.28 seconds and returned relevant source from 8 FastAPI files (about 25,000 characters). That, however, is not the same as an agent's task time.

62% fewer tokens and 44% lower cost: Not verified. No comparison using an LLM was run, so these were not measured.

Zero file reads: Not verified. No agent comparison was run. However, the MCP response text includes "Treat each block as a Read you have already performed: do not Read a file shown here," confirming that it is designed to keep agents from reading files.

One MCP tool call returns source, call paths and blast radius: Verified. tools/list returned only one tool, codegraph_explore. Calling it once via tools/call returned "Found 62 symbols across 8 files" for FastAPI. The response contained a Blast radius section (27 call sites of APIRoute, with test file names), extends, calls, instantiates and references relationships, and source code with line numbers, totaling about 25,000 characters. However, it also included 4 files under scripts/playwright that had little to do with the question.

One explore call via MCP returns 62 symbols in 8 files plus the blast radius
One explore call via MCP returns 62 symbols across 8 files plus the blast radius

No API key or external service needed, runs 100% locally: Partly verified. On an arm64 machine with no GPU, init, explore, impact and MCP all worked without an API key or any additional downloads. However, as noted above, anonymous usage telemetry is enabled by default. The README says no code or paths are sent, but to truly run "100% locally," users need to turn the transmission off themselves.

Changes reflected in under a second, Linux kernel in about 11 minutes: Partly verified. With the MCP server running on FastAPI, the reporter appended a function, and queries 1 second and even 0.5 seconds after saving returned the new function. A manual codegraph sync displayed "Modified: 1 — 216 nodes in 212ms," and the whole command took about 1.0 second. The 27,000-file scale was not tested. As detailed in the next section, indexing of the Linux kernel never finished.

An MCP query 1 second after saving finds the newly added function
An MCP query 1 second after saving finds the newly added function

Supports more than 20 languages and reads Next.js routing: Partly verified. The reporter wrote small files in 7 languages — Go, Rust, Java, Python, TypeScript, Ruby and Kotlin — and function call relationships were captured in every one. Not all 20-plus languages were tested. In vercel/commerce, 6 page.tsx and route.ts files became route nodes such as "/product/:handle," "/search/:collection" and "POST /api/revalidate." The dynamic route [handle] was also converted to :handle. However, codegraph query --kind route did not find them, and the reporter confirmed them by reading the SQLite database directly.

6 Next.js files become route nodes such as /product/:handle
6 Next.js files become route nodes such as /product/:handle
Function call edges captured in each of the 7 languages
Function call edges captured in each of the 7 languages

Where it stumbled

The biggest problem was the Linux kernel. The indexing target was 71,127 files. codegraph init ran for about 520 seconds, then, partway through the "Resolving refs" step, printed "FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory" and crashed with exit code 134. The heap at that point was about 2GB. Following the guidance from status, the reporter raised the heap to about 4GB with NODE_OPTIONS=--max-old-space-size=4096 and retried with codegraph sync, but it crashed with the same error after about 97 seconds. The database had grown to 4.4GB, with 5,422,809 unresolved references remaining. The README says "about 11 minutes on 8 cores" and "under 12 minutes on a 2-core, 6GB VPS," but in this environment it never finished.

There were other minor snags as well.

  • The install location, /root/.local/bin, was not on the PATH, so the full path had to be specified every time
  • Adding --quiet, as suggested in the README's troubleshooting section, failed with "error: unknown option '--quiet'"
  • When queries were piped to the MCP server, it exited as soon as standard input closed, before returning a response. Standard input had to be held open with sleep (unlikely to matter in normal use from an agent)
  • codegraph query --kind route did not find the route nodes
  • The test hit its limit of 8 images, so the kernel crash could not be captured

Use it now or wait?

For a repository around FastAPI's size (in the thousands of files), it is well worth installing and trying. It installs with one line and indexing finishes in seconds. Saved changes are reflected within a second, and one MCP tool call returns the relevant code all together. When installing, set CODEGRAPH_TELEMETRY=0 or use the telemetry command to turn off usage reporting. On the other hand, for a monorepo with tens of thousands of files on a machine with roughly 6–7GB of memory, first check locally that indexing actually completes. In this test, the Linux kernel did not finish even with a 4GB heap. Figures such as "62% fewer tokens" and "44% lower cost" are the author's own measurements and were not verified here. The most reliable way to decide is to compare your own agent with and without CodeGraph.

Test conditions

  • Date tested: 2026-10-03
  • Environment: Ubuntu 24.04 (arm64), no GPU, network access. The instructions specified 4 cores and 6GB of memory, but inside the container nproc reported 8 and free reported about 7GB
  • Version: CodeGraph v1.6.1 (linux-arm64 build installed via install.sh)
  • Test subjects: fastapi/fastapi, vercel/commerce, torvalds/linux (all shallow clones), and small handwritten files in 7 languages
  • Not tested: No real AI coding agent such as Claude Code, Cursor or Copilot was connected, because that requires a paid model and an API key. The 88% reduction in tool calls, 53% shorter task time, 62% fewer tokens, 44% lower cost and zero file reads were therefore not measured. According to the README, these figures come from an agent comparison on 7 repositories measured in August 2026. Also not tested: update latency at the 27,000-file scale, all 20-plus languages, comparisons with other code-indexing tools, and indexing the Linux kernel on a machine with more memory

Editorial cartoon

Editorial cartoon: Testing CodeGraph v1.6.1: FastAPI indexed in about 2.5 seconds and edits reflected in under a second, but the Linux kernel ran out of memory twice and the "88% fewer tool calls" claim couldn't be verified

Sources

  1. https://github.com/colbymchenry/codegraph