Skip to content
Hannctl.
Field Notes

E12 · Com2us Platform · Feb 2023 – Feb 2026

An AI that writes infrastructure code from a plain request

A prototype with a local LLM and a vector database, and the limits found in testing

  • AI
  • LLM
  • RAG
  • Ollama
  • Chroma
  • Docker
  • Terraform
Make an S3 bucket moduleChroma vector DBSimilar codeSimilar codeSimilar codeLocal LLMGenerated Terraform codeModuleModuleModuleNext designLangChain
A "next design" arrow leads to a LangChain block.

In one line

Built an infrastructure code generation prototype with a local LLM and a vector database, demoed it to the team, and found its limits in testing. It did not reach day-to-day use.

Background

We ran hundreds of Terraform and Terragrunt modules. Every new resource meant looking up the existing structure and writing similar code again, which took a lot of time.

What I did

  1. I set up a local LLM (run with Ollama) and a Chroma vector database in Docker.
  2. I indexed the structure of our production Terraform and Terragrunt modules.
  3. A plain-language request finds similar existing code, and the LLM generates new code to fit that structure.
  4. I presented and demoed it to the team.
Plain explanationWhat is RAG?

RAG (retrieval-augmented generation) means the AI first finds related material and sees it together with the question before it answers.

The AI does not know our code, so we find similar code and pass it along with “use this as a reference”.

To do that, code is split into small pieces and each piece’s meaning becomes a list of numbers (an embedding) stored in a vector database. A request is converted to numbers too, and the closest pieces in meaning are retrieved.

Result

Limits found in testing

Plain RAG did not understand the module structure. It stopped at matching patterns in the retrieved code.

When asked for a combination not in the search results, it produced code that looked plausible but connected modules incorrectly.

Next design

  • Improve embedding-based similarity search
  • Move to a RAG pipeline built on LangChain (a framework connecting retrieval, prompting, and generation)

Team priorities changed, and it did not reach day-to-day use.

What I learned

AI output must always be verified.

A good tool goes unused if it does not match the team’s priorities. Before building, confirm which team problem it solves right now.

Had I shared the direction early instead of polishing alone, I would have found the limits sooner.

Connected stories

Open to work · Cloud infrastructure and security