code-generation

Code Generation

Intro

Codegen produces source files from a higher-level input — a schema, DSL, AST, or template — instead of having humans hand-write the output. It pays off when output spans multiple languages, must be zero-dependency, or comes from a mechanical mapping. It's a liability when a plain library would do.

Overview

Generate vs. abstract

The first decision is whether to generate at all.

Generate when: output spans multiple languages, must be zero-dependency at runtime, or the input-to-output mapping is mechanical and repetitive (OpenAPI clients, protobuf stubs, GraphQL typed clients).

Abstract when: a shared library with configuration solves the problem, runtime flexibility matters, or maintenance of N generated files would exceed maintenance of one library.

If everything is in one language and there is no zero-dep constraint, default to a library.

Template engines

Engine	Language	Best for
Jinja2	Python	Config files, IaC, multi-language output
Handlebars	JS/TS	Logic-less templates, client-side rendering
EJS	JS/TS	Embedded JS logic, quick prototypes
Tera	Rust	Rust-native projects, Jinja2-like
Go templates	Go	Go projects, kubectl/Helm templates

Rules of thumb:

• Keep templates logic-free. Push logic into the data model that feeds the template.
• Use template inheritance/blocks for shared structure across variants.
• Store templates alongside the generator, not in the output directory.
• Emit a header comment on every generated file: // Code generated by <tool> — DO NOT EDIT.

AST manipulation

Use language-specific parsers when transformations depend on syntactic context, must guarantee valid output, or are too complex for regex. Examples: syn (Rust), ast and libcst (Python), ts-morph (TypeScript), javaparser (Java).

The pattern is always the same: parse source to AST, transform the tree, emit back to source. Preserve formatting and comments where the library allows it.

Scaffolding tools

Cookiecutter, Yeoman, cargo-generate, create-* CLIs.

• Define templates with placeholder variables, sensible defaults, and validation.
• Scaffolded projects must compile and run immediately. Include tests, linting, and CI config out of the box.
• Pin the scaffolder version so reruns are deterministic.

Build-step generation

When codegen runs as part of the build:

• Store the source-of-truth (schema, spec, DSL) in version control.
• Fail the build if generated output differs from committed files (run codegen, then git diff --exit-code).
• Pin the generator version. Cache intermediate artifacts.
• Provide a single command to regenerate everything: make generate or just generate.

Keeping generated code in sync

• Mark generated files clearly: header comment, .generated. infix in filename, or a manifest file listing all generated paths.
• One command to regenerate all of it.
• CI check that runs codegen and diffs against committed files.
• Never hand-edit generated files. Fix the template or input instead, then regenerate.

Gotchas

Agent-specific failure modes — provider-neutral pause-and-self-check items:

• Hand-editing generated files. Generated files are overwritten on the next regeneration run. All fixes must go to the template or input schema, then regenerate. Without this discipline, edits silently accumulate until the next full regeneration wipes them.
• Missing DO NOT EDIT header. Without a clear header, future contributors will edit the output, lose work, and curse the system. Every generated file must declare itself generated with the tool name and source.
• No CI check for generator drift. If CI does not run codegen and diff against committed output, the generator and the checked-in files will silently diverge. Add git diff --exit-code after running the generator in CI.
• Logic embedded in the template. Templates with embedded conditionals, loops, and computations are untestable and hard to debug. Push all logic into the data model that feeds the template; keep the template a dumb projection.
• Unpinned generator version. If the generator is invoked without a pinned version, reruns may produce different output. Pin the generator version the same way you pin any other tool dependency.
• Generating when a library would do. The generate-vs-abstract decision defaults to "abstract" when everything is in one language and there is no zero-dependency constraint. Proposing codegen for a same-language, same-repo problem is usually over-engineering.
• No single regeneration command. "Run these eight steps in order" does not survive a year. Every project with codegen needs a single command — make generate, just generate, or equivalent — that regenerates everything from scratch.

Full reference

Macro systems

Rust proc_macro, Lisp macros, the C preprocessor, and Scala macros all sit on the codegen spectrum but operate at compile time within the language.

• Prefer declarative macros (macro_rules!) over procedural macros when the pattern is simple. Procedural macros are powerful but hard to debug and slow to compile.
• Always document macro expansion. Provide cargo expand (or equivalent) so consumers can inspect what the macro produces.
• Keep generated code minimal — delegate to library functions for complex logic. The macro should be a thin shell around library calls, not a place to embed an entire algorithm.
• Avoid macros that generate names by string concatenation; they defeat tooling like jump-to-definition.

When generated code wins over a library

There are real cases where codegen beats abstraction even in a single language:

• Output must be inspectable and debuggable as plain source (e.g. generated SQL migrations).
• Performance demands inlining that a runtime abstraction can't achieve.
• The target language doesn't support the abstraction you'd need (no generics, no reflection, no macros).
• Cross-team contracts: the spec is the API, generated code is just the implementation.

Anti-patterns

• Hand-edited generated files. They will be overwritten. Fix the source.
• Generated files not in version control. Reviewers can't see what changed. Commit them and gate on diff.
• Logic embedded in templates. Push computation into the data model and keep the template a dumb projection.
• No DO NOT EDIT header. Future contributors will edit the output, lose work, and curse you.
• No regeneration command. "Run these eight steps in order" doesn't survive a year.
• Mixing generators. Two tools writing into the same file with different conventions ends in chaos. Pick one tool per file.

Worked scenarios

OpenAPI -> TypeScript + Python clients. Read the spec, evaluate options (openapi-generator for multi-language vs. openapi-typescript for TS-only). Choose openapi-generator for the multi-language requirement. Wire generate.sh producing both clients with DO NOT EDIT headers, plus a CI step that runs the generator and fails on diff.

Same 40-line DB pattern in every service. All services are Python; variation is just table name and columns. Recommend a shared library: single language eliminates the codegen advantage, the library allows runtime changes, and one library is less to maintain than N generated files.

Renaming log_event to emit_event across 200 files. Find/ replace hits false positives in strings and comments. Write a ts-morph (or libcst) codemod that parses each file, finds call expressions where the callee is log_event, renames to emit_event, and writes back preserving formatting. Run it once; commit the diff.