Run AI on hardware you can size.
Infra makes practical e-books and guides for engineers who self-host language models: picking GPUs, building inference stacks, and keeping private data private.
Will it fit on one 80 GB GPU?
140 GB of weights. That does not fit. Weights alone need 2 cards, or a lower precision.
Rough, weights-only arithmetic (parameters × bits ÷ 8). Real deployments need extra memory.
Guides in the works
These are example entries. Titles are working titles, prices aren't set, and none are on sale yet.
Self-hosting open LLMs
Working title. A practical guide to running open-weight language models on hardware you control.
Price TBA (example, not a live product)
GPU sizing for inference
Working title. How to estimate the GPU memory and count you need before you buy or rent.
Price TBA (example, not a live product)
Private RAG on your own stack
Working title. Building retrieval-augmented generation where documents never leave your infrastructure.
Price TBA (example, not a live product)
About Infra
Most writing about AI assumes you call someone else's API. Infra is for the other case: you run the model, so you pick the hardware, the serving stack, and where the data lives.
The guides are meant to be read with a terminal open. Each one aims to answer a concrete question an engineer or tech lead hits when moving a model onto their own infrastructure, such as how much GPU memory a deployment needs or how to keep retrieval data inside your network.
Hear when the first guide ships
One email when there's something to read. Sign-ups open soon.