Ανοιχτός κώδικας, έτοιμες εφαρμογές.

A load balancer that routes by KV-cache pressure instead of round-robin. When you serve an LLM across several GPU instances, naive routing sends a request to a worker that has to re-read the whole prompt, throwing away prefill work another worker already did. This routes to the instance that already holds the relevant cache. Almost nobody has heard of the problem, and it is worth real latency.

Compiling a model to a TensorRT-LLM engine and serving it on Triton, written down step by step because the official path has sharp edges and the error messages do not help. This is the work behind "we can run it on your own hardware" — it either works at 2am or it does not.

Turns any sentence-transformer into a TensorRT engine on Triton. Retrieval quality lives or dies on embeddings, and embedding throughput is usually what makes self-hosted search feel slow. This is how you stop paying an API per document.

Attaching a small draft model to a large one so the big model verifies several tokens at once instead of generating them one by one. Experiments with measured speedups, including the cases where it does not pay off — which is the part most write-ups leave out.

A Greek language model running usefully on a 12GB consumer GPU. Greek is badly served by most tooling and the good models are large; getting one to answer well on hardware a small business can actually buy is the difference between a demo and something a Greek company can own.

Live speech transcription and translation across many languages, served from a single consumer machine. Whisper for the audio, a language model for the translation, and no per-minute API bill. Proof that "real-time AI" does not require a cloud contract.

A dashboard for reading back long AI coding sessions — what was changed, what was tried, and what it cost. Built because we run agents on real work and needed to audit them, which is the same instinct we bring to any AI system we ship for a client: if you cannot inspect it, you cannot trust it.