spiega deploy reveal config

Everything here is containerized:

safe

I exactly know what the agents can see

scale

I can reproduce easily the same configuration on other hardware

performant

I decide how many resources I want to allocate

admin

I can kill a container anytime

storage

I sacrifice a bit of storage but gain a lot in control

image

it took a lot of time to find the best image to serve multiple services on nvidia, amd and arm

We want to use a simple configuration, we have models hosted on a simple gaming laptop and an IoT device jetson. What we use should work with AMD and ARM. ARM is particularly tricky since the compatibility between cuda-toolkit and pytorch requires a lot of trials between versions and compiling from source. ARM lacks a lot of deb packages and I tried to install blender 5 from source for example but didn’t succeeded.

The host has few tools, the idea is that the client side is minimal and the heavy calculation will be performed on the containers which can be easily moved. Decreasing the context and tools for the coding assistants increases security and improves productivity.

---
title: knowledge parsing
fontSize: 10
darkMode: True
theme: neo-dark
---
flowchart LR
KN["`
source code
knowledge
personal
`"]
DOC@{ shape: docs, label: "Knowledge"}
LLM@{ shape: procs, label: "LLMs"}
EM@{ shape: lin-cyl, label: "emacs" }

EM -- writes --> DOC
LLM -- reads --> DOC
LLM -- writes --> KN

What we have tested so far ¹ :

ollama

fully featured and integrated it is really easy to bind it to software like editors or coding assistants but performances are pretty poor and it’s the token/s are not enough

unsloth studio

really performant (llama_cpp level), fully featured but still in beta. It needs a lot of work around with endpoints

[ ] llama_cpp

performant but not user friendly. It requires a lot of configuration, even finding and downloading a useful model is painful

[ ] vllm

provided a configuration but I still couldn’t test it

[ ] lm studio

good performances but I want to simplify the stack

We test jev as well

cd ~/Downloads/
git clone https://github.com/browser-use/jev-ultrafast.git
cd jev-ultrafast
uv sync
cp .env.example .env
uv run jev

In order for LLM to be helpful we need the functionalities of coding assistants so they can indeed test what they claim and reduce hallucinations.

IDE tools

they include IDE tools like syntax checks, linters, unit tests so the model can test the output of the produced code before exiting

search

web search for specific API documentation (which I cache)

Coding assistants can be really dangerous

rules

nothing grants they get followed, coding agents often overcome them

filesystem

they can access the entire filesystem and they often start looking at files outside of the context

sensitive information

they can access passwords, keys and sensitive data

delete

they can delete a single file or the entire project without backuping the data

execute

they execute commands outside of the context and rules

super user

access to docker is really dangerous

computing

they get stuck in loops and keep consuming resources

We worked and tested different coding assistants

pi coding

pi coding, nicely integrated with ollama. Less tools and more productive

opencode

over featured, not too efficient

[ ] aider

UI not really useful

[ ] cursor

not a real assistant, produces code which you can’t edit without explaining what you did and why and hoping it understands the context

After many many trials I tested different images to be able to select a good configuration for LLM serving and coding agent. Ollama has a simple configuration but is not performant so I had to create new containers to link the other services. It is pretty straightforward to launch a standalone service but to integrate them it gets pretty complicated. I tried agentic supports but they were always find non integrated solutions which make portability hard.

The tricky part is to let an user inside docker modify the local files on the host without changing the permissions. This costed me a lot of time to configure and I still find the setup not optimal. Docker decided to create a non root user as ubuntu which makes difficult to replicate the same permissions. The workaround I found uses a conf in Dockerfile and to link the ubuntu user to username to match my host user and add a bunch of groups and permissions. The /home directory is still mounted as root so the user struggles to create conf files in home. I don’t want to mount the entire home directory but only the folder containing the source code.

RUN usermod -l username ubuntu
RUN usermod -d /home/username/ username
RUN usermod -s /bin/bash username
RUN groupadd username
RUN usermod -aG ubuntu username
RUN usermod -aG username username

export DOCKER_USER="$(id -u):$(id -g)"

and in docker-compose.yml to mount the folder as current user

user: "${UID-1000}:${GID-1000}"

In case we don’t want to type, we can use dictation software to transpose our voice into text. Here we test different dictation tools.

Voxtype

sudo apt install cargo libasound2-dev libclang-dev cmake pkg-config
cargo build --release
sudo apt install wtype
./target/release/voxtype setup --download
./target/release/voxtype

Hyprland (~/.config/hypr/hyprland.conf):
bind = SUPER, V, exec, voxtype record start
bindr = SUPER, V, exec, voxtype record stop

speedofsound

#sudo apt install voxtype
#flatpak install flathub net.mkiol.SpeechNote
sudo snap install speedofsound

(keymap-global-set "s-z" #'speedofsound)

CONTAINER ID   IMAGE                            COMMAND                  CREATED          STATUS                  PORTS                                                                          NAMES
ca8d2804a6e5   ghcr.io/k3d-io/k3d-proxy:5.9.0   "/bin/sh -c nginx-pr…"   13 minutes ago   Up 13 minutes           80/tcp, 0.0.0.0:37877->6443/tcp                                                k3d-spiega-serverlb
056c497f5a54   rancher/k3s:v1.35.5-k3s1         "/bin/k3d-entrypoint…"   14 minutes ago   Up 13 minutes                                                                                          k3d-spiega-server-0
4a16eed24c3a   websearch-api                    "/entrypoint.sh"         2 hours ago      Up 2 hours              0.0.0.0:8092->8092/tcp, [::]:8092->8092/tcp                                    orio-search-api
d07ec672c3e3   redis:7-alpine                   "docker-entrypoint.s…"   2 hours ago      Up 2 hours (healthy)    0.0.0.0:6379->6379/tcp, [::]:6379->6379/tcp                                    orio-search-redis
578c40f81b15   searxng/searxng:latest           "/usr/local/searxng/…"   2 hours ago      Up 2 hours (healthy)    0.0.0.0:8091->8080/tcp, [::]:8091->8080/tcp                                    orio-search-searxng
49b9a2e4dbff   alpine/socat:latest              "socat TCP-LISTEN:91…"   3 days ago       Up 3 days                                                                                              blender-mcp-bridge
60962c6c4fb6   alpine/socat:latest              "socat TCP-LISTEN:13…"   3 days ago       Up 3 days                                                                                              lemonade-mcp-bridge
4f5e8494e64a   mcp_server-mcp_server            "bash /app/run.sh"       13 days ago      Up 4 days (unhealthy)   127.0.0.1:8171-8172->8171-8172/tcp                                             mcp_server
17e093fb3202   mariadb                          "docker-entrypoint.s…"   2 months ago     Up 4 days               3306/tcp                                                                       webserver-mysql-1
d0bfb6f96a58   webserver-nginx                  "nginx -g 'daemon of…"   2 months ago     Up 4 days               0.0.0.0:80->80/tcp, [::]:80->80/tcp, 0.0.0.0:443->443/tcp, [::]:443->443/tcp   nginx

nerd dictation

pip3 install vosk
git clone https://github.com/ideasman42/nerd-dictation.git
cd nerd-dictation
wget https://alphacephei.com/kaldi/models/vosk-model-small-en-us-0.15.zip
unzip vosk-model-small-en-us-0.15.zip
mv vosk-model-small-en-us-0.15 model

whisper model

pip install -U openai-whisper

Unsloth was originally developed as tool for training. Here we want

custom

a custom model that learns from my way of programming while I use a coding agent

curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo add grafana https://grafana.github.io/helm-charts
helm repo update
helm install prometheus prometheus-community/kube-prometheus-stack
helm install grafana grafana/grafana

"grafana" has been added to your repositories
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "grafana" chart repository
Update Complete. ⎈Happy Helming!⎈
NAME: grafana
LAST DEPLOYED: Wed Sep 23 16:42:00 2026
NAMESPACE: default
STATUS: deployed
REVISION: 1
NOTES:
1. Get your 'admin' user password by running:

   kubectl get secret --namespace default grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo


2. The Grafana server can be accessed via port 80 on the following DNS name from within your cluster:

   grafana.default.svc.cluster.local

   Get the Grafana URL to visit by running these commands in the same shell:
     export POD_NAME=$(kubectl get pods --namespace default -l "app.kubernetes.io/name=grafana,app.kubernetes.io/instance=grafana" -o jsonpath="{.items[0].metadata.name}")
     kubectl --namespace default port-forward $POD_NAME 3000

3. Login with the password from step 1 and the username: admin
#################################################################################
######   WARNING: Persistence is disabled!!! You will lose your data when   #####
######            the Grafana pod is terminated.                            #####
#################################################################################

# Install K3s (works on ARM, AMD, NVIDIA)
curl -sfL https://get.k3s.io | sudo INSTALL_MODE=1 sh -
# Configure GPU support in containerd (requires NVIDIA container runtime)
# Add to /etc/k3s/agent.env for worker nodes
# K3S_EXEC="--node-label gpu=true --node-role=worker"

• Server Configuration (k3s.yaml)

# This is for the server startup args, usually passed to the server process
# server:
#   service-port: 6443
#   agent-port: 10000
#   cluster-init: false
#   token: "mytoken"
#   data-store-endpoint: /var/lib/rancher/k3s/server/db
#   disable:
#     traefik: false  # Only if you handle ingress manually
#
# To enable GPU support at runtime, ensure the 'nvidia-container-toolkit'
# is installed on the host and the runtime is set in containerd.

• Deployment (Ollama + HostPath)

#+BEGIN_EXAMPLE

This defines the LLM service with hostPath for source editing.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ollama
  namespace: llm
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ollama
  template:
    metadata:
      labels:
        app: ollama
    spec:
      containers:
        - name: ollama
          image: ollama/ollama:latest
          command:
            - ollama
            - serve
          ports:
            - containerPort: 11434
          securityContext:
            runAsUser: 1000  # Match host UID
            fsGroup: 1000     # Match host GID for hostPath
          volumeMounts:
            - name: models
              mountPath: /root/.ollama
            - name: workspace
              mountPath: /workspace
      volumes:
        - name: models
          hostPath:
            path: /root/.ollama
            type: DirectoryOrCreate
        - name: workspace
          hostPath:
            path: ~/projects/spiega  # Adjust to your host path
            type: DirectoryOrCreate