agent calling

The “yeast” start from a ubuntu image and installs basic libraries

FROM ubuntu:latest
RUN apt-get update && apt-get install -y locales
RUN apt install -y curl libatomic1 python3-pip bash git  
#&& rm -rf /var/lib/apt/lists/* 
#RUN localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8
ENV LANG en_US.utf8

We copy the app files into the container

COPY ./app /app
RUN chmod +x  /app/run.sh
WORKDIR /app

We need then to allow the container user to edit the files keeping the same permission as the host user to allow coding assistants to edit the files keeping the same consistency. Unfortunately this step is not easy in docker for reasons I don’t understand but I found a workaround ref1.

ARG UID=1000
ARG GID=1000
RUN usermod -l $USER ubuntu
RUN usermod -aG ubuntu $USER
RUN usermod -aG $USER $USER
RUN usermod -d /home/$USER/ $USER
RUN usermod -s /bin/bash $USER
RUN groupadd $USER

We grant sudo and no password:

RUN passwd --delete sabeiro
RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
RUN adduser sabeiro sudo
RUN mkdir -p /home/sabeiro/ && chown -R ${UID}:${GID} /home/sabeiro/
USER sabeiro
RUN sudo ls

We need to add to docker-compose.yml to mount the volume as non-root

user: "${UID}:${GID}" 

On top of this image we install some applications

RUN curl -fsSL https://openclaw.ai/install.sh | bash -s -- --no-prompt --no-onboard
RUN curl -fsSL https://pi.dev/install.sh |  bash -s -- --no-prompt --no-onboard
RUN curl -fsSL https://opencode.ai/install | bash -s -- --no-prompt --no-onboard
RUN curl -LsSf https://aider.chat/install.sh | bash -s --
RUN curl -fsSL https://unsloth.ai/install.sh | sh

And we build the image

docker compose up --build

We then test the configuration using bash inside the container

docker compose exec -it ubuntu bash

Docker compose here starts the container built in the previous step

services:
  ubuntu_base:
    build:
      context: ./
      dockerfile: Dockerfile
    container_name: ubuntu_base
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    runtime: "nvidia"
    ports:
      - "8083:8083"
    restart: unless-stopped
    stdin_open: true
    tty: true
    working_dir: /app/
    entrypoint: ["bash","/app/run.sh"]
    healthcheck:
      test: ["CMD", "cat", "/app/run.sh"]
      interval: 30s
      timeout: 30s
      retries: 3
    volumes:
      - ${HOME}/Downloads/llm_model/:${HOME}/models/
      - ${HOME}/lav/src/:${HOME}/lav/src/
      - ${HOME}/log/ubuntu/:/var/log/
      - ./app/:/app/
      - /var/run/docker.sock:/var/run/docker.sock
    networks:
      - webserver-net

networks:
  webserver-net:
    name: webserver_webserver-net
    driver: bridge
    external: true

Network and webserver are on a separate container

services:
  nginx:
    container_name: nginx
    build:
      context: nginx/
    restart: unless-stopped
    tty: true
    cpus: 0.2
    environment:
      SERVICE_NAME: app
      SERVICE_TAGS: dev
    volumes:
      - ./nginx/conf.d:/etc/nginx/conf.d
      - ${HOME}/lav/siti:/var/www/html/
      - ${HOME}/log:/var/log/
    ports:
      - "80:80"
      - "443:443"
    networks:
      - webserver-net

All the LLM services like ollama, llama.cpp, unsloth, lmstudio… have docker versions we can use but those implementations can be barely used outside of testing since the real added value of LLM is the combination with a coding assistant. Ollama is by far the most easy to integrate tool, their APIs are included in many packages and you can integrate

We have different backends to explore:

ROCm

Open software stack that includes programming models, tools, compilers, libraries, and runtimes for AI and HPC solution development on AMD GPUs

CPU

cuda

allows software to use certain types of graphics processing units for accelerated general-purpose processing

vulkan

next generation graphics and compute API that provides high-efficiency, cross-platform access to modern GPUs used in PCs, consoles, mobile phones and embedded platforms.

Generally the GPU is a better option for LLM but the CPU can access 32GB of memory.

To use the nvidia graphic cards I need to install many packages

From linuxvox.

sudo apt update && sudo apt upgrade
sudo mkdir --parents --mode=0755 /etc/apt/keyrings
wget https://repo.amd.com/rocm/packages-multi-arch/gpg/rocm.gpg -O - | \
    gpg --dearmor | sudo tee /etc/apt/keyrings/amdrocm.gpg > /dev/null
 
sudo tee /etc/apt/sources.list.d/rocm.list << EOF
deb [arch=amd64 signed-by=/etc/apt/keyrings/amdrocm.gpg] https://repo.amd.com/rocm/packages-multi-arch/ubuntu2404 stable main
EOF
sudo apt update
sudo apt install rocm
sudo usermod -a -G render,video $LOGNAME
pip install torch torchvision --extra-index-url https://download.pytorch.org/whl/rocm7.2/

Nowadays laptops have NPU (neural processing units) but it is generally difficult to use them the same way they are used in cell phones. The recommended size for LLM application is 40 tops but I should be able to use my 16 tops for light tasks like autocompletion. The idea is to use fast flow lm with this implementation Let’s try to follow this guide and the official documentation. Need to download the drivers.

Ubuntu 26 has the package lemonade: run LLMs and Whisper on the AMD XDNA 2 NPU. This solution is made up of:

• Upstream NPU driver in the Linux 7.0+ kernel (with backports for 6.xx kernels).
• AMD IRON compiler for XDNA NPUs.
• FastFlowLM, a lightweight LLM runtime for AMD NPUs, which today has added Linux support.
• Lemonade, ties everything together with a streamlined user experience.

sudo apt update
sudo apt install python3.12 python3.12-venv libboost-filesystem1.74.0 dkms
sudo apt update && sudo apt install linux-oem-24.04c
cd /home/sabeiro/Elŝutujo
unzip RAI_1.8_Linux_NPU_XRT.zip
sudo apt install --fix-broken -y ./xrt_202620.2.25.37_24.04-amd64-base.deb
sudo apt install --fix-broken -y ./xrt_202620.2.25.37_24.04-amd64-base-dev.deb
sudo apt install --fix-broken -y ./xrt_202620.2.25.37_24.04-amd64-npu.deb
sudo apt install --fix-broken -y ./xrt_plugin.2.25.260102.56.release_24.04-amd64-amdxdna.deb

source /opt/xilinx/xrt/setup.sh
xrt-smi examine

Need to download ryzen ai.

cd /home/sabeiro/Elŝutujo
mkdir ryzen_ai-1.8.0
cp ryzen_ai-1.8.0.tgz ryzen_ai-1.8.0
cd ryzen_ai-1.8.0
tar -xvzf ryzen_ai-1.8.0.tgz

Dependencies for docker git clone https://github.com/hpenedones/fastflowlm-docker.git cd fastflowlm-docker docker build -t fastflowlm .

sudo add-apt-repository ppa:amd-team/xrt
sudo apt update && sudo apt install libxrt-npu2
# Set memlock to unlimited (needs reboot)
echo -e "* soft memlock unlimited\n* hard memlock unlimited" | sudo tee -a /etc/security/limits.conf
sudo reboot

git clone https://github.com/hpenedones/fastflowlm-docker.git
cd fastflowlm-docker
docker build -t fastflowlm .

Between ollama, llama.cpp, unsloth, vllm the best experience was with lemonade. Lemonade has a simple UI able to manage:

llama.cpp

for serving LLM

vllm

for serving LLM

whisper

talk2text

stable diffusion

image generation

ROCm

for amd

cuda

nvidia drivers

onnx

custom models

vulkan

cross platform API for 3D (like openGL)

Based on the system monitoring, here are the findings:

A Gantt representation of the project

---
displayMode: compact
title: Ignored if specified in chart
config:
  gantt:
      useWidth: 800
      rightPadding: 0
      topAxis: true  #false
      numberSectionStyles: 2
---
gantt
    dateFormat  <YYYY-MM-DD>
    title  Knowledge base action plan
    excludes  weekends

    review : vert, v1, <2026-06-22>, 1d

    section local models
        deploy LLMs          :done,  deploy, <2026-05-20>, 7d
        coding agent         :done,  deploy, <2026-05-27>, 14d

    section list & summarize
        parse knowledge         :done,  dev, <2026-06-01>, 5d
        create graphs           :done,  dev, <2026-06-07>, 7d

    section hierarchy
        hierarchy     :done, distil, <2026-06-16>, 9d
        visualization :crit, distil, <2026-06-20>, 4d

    section video
        sceencast    :active, create, <2026-06-12>, 5d
        animate text :active, create, <2026-06-17>, 9d

    section publish
        content     :active, share, <2026-06-22>, 10d
        publication :milestone, share, <2026-06-27>, 5d

Chat providers are built with make-llm-ollama (default http://127.0.0.1:11434). Unlike gptel, it does not route through the MCP server (the bottino/mcp FastAPI agent), so it has no access to the MCP tools (ollama_list_models, camera_describe_scene, etc.) or the MCP tool-call loop.

ellama_setup.el discovers providers dynamically at startup. The codepath used is my-unsloth-list-models — it queries Unsloth’s {UNSLOTH_URL}/api/inference/models and requires UNSLOTH_TOKEN (sent as a Bearer token; Unsloth rejects unauthenticated model-list requests). Set UNSLOTH_URL (default http://127.0.0.1:8889) to point at your Unsloth server. my-ollama-list-models (Ollama’s /api/tags via OLLAMA_URL, default http://127.0.0.1:11434) is present but currently not used — the refresh helper was switched to the Unsloth source.

Listed model names appear in M-x ellama-switch-provider automatically after a 3-second idle delay at startup (or instantly via M-x my-ellama-refresh-providers), with a qwen2.5:7b fallback provider if discovery fails.

Note: discovery lists Unsloth model names but builds make-llm-ollama providers — the resolved names must be reachable through your chosen backend.

Ellama starts via the global keymap defined in ellama_setup.el:

C-x e is deliberately left unbound by ellama so it keeps its stock meaning kmacro-end-and-call-macro (macro execute); the ellama prefix uses C-c e.

Defined in =~/.emacs.d/ellama_setup.el:

Default chat provider (ellama-provider)

• Model: qwen3.5:9b
• Embedding: nomic-embed-text
• Context: 8192 tokens (num_ctx)
• Used for: general conversation, questions

Coding provider (ellama-coding-provider)

• Model: qwen2.5-coder:7b
• Embedding: nomic-embed-text
• Context: 32768 tokens
• Used for: code generation, debugging, refactoring
• Activated via M-x ellama-code (there is no dedicated prefix for it, but the coding provider can also be selected with M-x ellama-switch-provider)

Summarization provider (ellama-summarization-provider)

• Model: llama3.2:latest
• Embedding: nomic-embed-text
• Context: 32768 tokens
• Used for: summarizing regions, buffers, or files

Translation provider (ellama-translation-provider)

• Model: qwen2.5:7b
• Embedding: nomic-embed-text
• Context: 32768 tokens

Extraction provider (ellama-extraction-provider)

• Model: qwen2.5-coder:7b-instruct-q8_0
• Embedding: nomic-embed-text
• Context: 32768 tokens
• Used for: structured data extraction from text

Naming provider (ellama-naming-provider)

• Model: qwen2.5:7b
• Used for: auto-naming new chat sessions (via ellama-generate-name-by-llm)
• Uses a stop non-standard param of a newline

Dynamic providers for interactive switching

ellama-providers is (re)built by my-ellama-refresh-providers from the Unsloth model list at startup (my-unsloth-list-models), falling back to a single qwen2.5:7b provider when discovery fails. Each entry embeds qwen2.5:7b-style make-llm-ollama settings with nomic-embed-text and 32768-token context, so newly pulled models appear automatically in M-x ellama-switch-provider.

Switch interactively with M-x ellama-switch-provider.

Start a chat

C-c e or M-x ellama

This opens a chat buffer in full frame. Type your message, then C-c C-c to send.

Coding assistant

M-x ellama-code or select the coding provider interactively.

Ask for code, refactoring, or debugging help. The coding provider uses qwen2.5-coder:7b with 32K context – suitable for whole functions or small files.

Summarize text

Select a region, then M-x ellama-summarize or M-x ellama-summarize-region. Ellama will summarize the selected text using the summarization provider.

Translate text

M-x ellama-translate – prompts for target language, then translates the current region or buffer.

Extract structured data

M-x ellama-extract – extracts structured information from text (e.g. names, dates, quantities from free-form text).

Use a different model

M-x ellama-switch-provider – pick from the dynamically-discovered providers, or any other provider you define.

Display behaviour

• Chat buffer opens in full frame (display-buffer-full-frame) — note that ellama_setup.el overrides the chat/instant display action to display-buffer-no-window at load (lines 5-6), so the interactive prompts stay unobtrusive.
• Header line shows ellama context and session ID in all buffers (ellama-context-header-line-global-mode, ellama-session-header-line-global-mode).

Requirements

• Emacs ≥ 30
• An LLM backend reachable by ellama_setup.el — either:
  • Unsloth, for dynamic model discovery (UNSLOTH_URL, default http://127.0.0.1:8889; needs UNSLOTH_TOKEN); or
  • Ollama (OLLAMA_URL, default http://127.0.0.1:11434) serving the chat models below
• Models expected (check with ollama list):
  • qwen3.5:9b (default chat)
  • qwen2.5-coder:7b (coding)
  • llama3.2:latest (summarization)
  • qwen2.5:7b (naming, translation, fallback)
  • qwen2.5-coder:7b-instruct-q8_0 (extraction)
  • nomic-embed-text (embeddings)

Ellama now discovers models dynamically at startup (with a 3-second idle delay so init isn’t blocked). When you pull a new model in Ollama, it appears automatically after restarting Emacs, or you can refresh on demand:

M-x my-ellama-refresh-providers

This queries Ollama’s /api/tags endpoint, filters out embedding-only models (nomic-embed-text, all-minilm), and rebuilds the ellama-providers alist with a make-llm-ollama entry per model.

ollama list

or from Emacs:

M-x shell-command ollama list

By default, discovery hits http://127.0.0.1:11434/api/tags. To route through the MCP server’s nginx instead, set:

MY_OLLAMA_URL=http://bottino:11434

or in Emacs:

M-x set-variable my-ollama-url "http://bottino:11434" M-x my-ellama-refresh-providers

ollama pull deepseek-coder:6.7b

Then in Emacs:

M-x my-ellama-refresh-providers M-x ellama-switch-provider → select deepseek-coder:6.7b