turingsroot

I've been running AI coding workshops for engineers transitioning from traditional development, and the research phase is consistently the part people skip — and the part that makes or breaks everything.

The failure mode the author describes (implementations that work in isolation but break the surrounding system) is exactly what I see in workshop after workshop. Engineers prompt the LLM with "add pagination to the list endpoint" and get working code that ignores the existing query builder patterns, duplicates filtering logic, or misses the caching layer entirely.

What I tell people: the research.md isn't busywork, it's your verification that the LLM actually understands the system it's about to modify. If you can't confirm the research is accurate, you have no business trusting the plan.

One thing I'd add to the author's workflow: I've found it helpful to have the LLM explicitly list what it does NOT know or is uncertain about after the research phase. This surfaces blind spots before they become bugs buried three abstraction layers deep.

This is a really impressive piece of systems engineering. The 3-tier adaptive caching (VRAM resident > pinned RAM > NVMe/mmap) is essentially reimplementing what the Linux kernel's page cache does, but with GPU-awareness baked in.

The 0.3 tok/s for 70B Q4_K_M on a single 3090 is slow for interactive use, but the architecture itself is what matters here. PCIe Gen3 x8 at ~6.5 GB/s is the clear bottleneck - I'd be very curious to see numbers on a Gen5 NVMe setup where sequential reads can hit 12+ GB/s. That alone could potentially double throughput.

The layer skip via cosine similarity calibration (20/80 layers skipped) is a clever trick. Reminds me of early work on adaptive computation in transformers. The quality tradeoff at threshold 0.98 would be interesting to benchmark more rigorously - for many inference tasks like summarization or classification, you could probably push that much further.

Also worth noting: zero external dependencies beyond CUDA Toolkit is a bold design choice. No cuBLAS means they wrote their own GEMM kernels, which is a massive undertaking but gives full control over the memory access patterns needed for this streaming architecture.

I've been teaching AI coding tool workshops for the past year and this planning-first approach is by far the most reliable pattern I've seen across skill levels.

The key insight that most people miss: this isn't a new workflow invented for AI - it's how good senior engineers already work. You read the code deeply, write a design doc, get buy-in, then implement. The AI just makes the implementation phase dramatically faster.

What I've found interesting is that the people who struggle most with AI coding tools are often junior devs who never developed the habit of planning before coding. They jump straight to "build me X" and get frustrated when the output is a mess. Meanwhile, engineers with 10+ years of experience who are used to writing design docs and reviewing code pick it up almost instantly - because the hard part was always the planning, not the typing.

One addition I'd make to this workflow: version your research.md and plan.md files in git alongside your code. They become incredibly valuable documentation for future maintainers (including future-you) trying to understand why certain architectural decisions were made.