The complete architecture behind Nyeera.
Hardware, operating system, inference engine, RAG orchestration and the languages behind every layer — all in one place, with the repository open for inspection.
The forgotten machine
The only component that was added was the SSD drive. The goal of the study was to measure what this specific configuration can actually sustain.
Software layers chosen by weight, not by trend
Every component was evaluated by what it consumes at rest and under load. Wherever there was a choice between "more complete" and "lighter", the lighter one won by default.
Linux, minimalist distribution
A lean install, with no mandatory graphical environment, cutting the system's own idle RAM usage before the pipeline even starts.
llama.cpp (via Python bindings)
A C/C++ runtime specialized in CPU inference for quantized LLMs, with no dependency on CUDA or GPU drivers.
Local index (FAISS / Chroma)
Storage and similarity search running embedded in the process, with no external database server.
Local API in FastAPI
A lightweight HTTP service exposing the RAG pipeline, letting the model be integrated into other applications without rewriting the inference logic.
From the HTTP request to the generated token
The stack in layers, top to bottom — from the interface the user touches to the hardware that actually runs the model.
HTML/CSS/JS pages — like this one — for documentation, demos and direct consumption of the service.
Query and document-ingestion endpoints, with input validation and streamed responses.
Chunking, query routing, prompt assembly with retrieved context, and the call to the inference engine.
A compact embedding model that turns documents and queries into comparable vectors.
Local storage of vectors with approximate nearest-neighbor retrieval.
Loading and running the LLM in GGUF format, with explicit control over CPU threads.
Ubuntu Server. 7.0.0-29-generic #29-Ubuntu
CPU Intel Core i5-3210M (dual-core, 4 threads). Memory RAM 16GB PC3-12800 DDR3. Integrated GPU Intel HD Graphics 4000 (not being used). Storage: SATA HD 5400-rpm. Kingston NV3 NVMe M.2 on external case
| Technology | Category | Role in the project |
|---|---|---|
| Python 3.x | Main language | Pipeline orchestration, the API, and indexing scripts. |
| C / C++ | Low-level language | Core of the inference engine (llama.cpp), critical for CPU performance. |
| llama.cpp | Inference engine | Runs the quantized LLM (GGUF) without depending on a GPU. |
| LangChain / LlamaIndex | RAG orchestration | Chains context retrieval and response generation. |
| sentence-transformers | Embeddings | Generates the vectors used in similarity search. |
| FAISS / Chroma | Vector store | Local indexing and search of the context vectors. |
| FastAPI + Uvicorn | API | Exposes the pipeline as a local HTTP service. |
| HTML / CSS / JavaScript | Frontend | Documentation and demo interface for the project. |
The repository is open for inspection
The entire pipeline — indexing scripts, inference-engine configuration, and the API — is versioned publicly.
github.com/elmironeto/nyeera/
Case study and reference implementation for local RAG on constrained hardware: no dedicated GPU, 16GB of RAM, legacy CPU.
An architecture built to be audited, not just demoed.
Every layer of this stack exists for a reason tied to cost — memory, latency, or network dependency. The code in the repository is the proof.