emacs and LLMs

The purpose of this integration is to remove dependencies from external software and work with minimal org-files. Emacs already has multiple utils to make such a setup fit for managing big projects with the help of LLMs. We first of all work at a local single-user setup and think then how to make it work for a more complex environment. LLMs can be helpful but they have limitations and risks associated. We need to keep control of the project and its quality and let LLMs support with tasks which can be easily over-viewed. The things I want to improve in working with LLMs:

maintainability

LLMs often write code too hard to maintain and difficult to debug

tools

too often LLMs don’t use useful tools but write brand new code not using at best more efficient commands

responsibility

making sure we know how reliable is the product

resilient

too often LLMs cover a single use case

ineffective

often the code runs but doesn’t work, the tests are passed by data are not manipulated. They tend to put too many conditions and let the code run

It is tricky to have good performances with local models, there is a lot you need to balance. The best is to find the perfect combination of models and memory depending on the task. We want to make the best use of a gaming laptop and an nvidia jettison. A really cheap setup for local AI.

We have multiple LLM packages in emacs and we need to wisely distribute loads across such a tiny system. Usually the maximum size of a model we can run locally is 9B but for some tasks we should rather use smaller systems.

Let’s list first what we might need:

autocompletion

code or text completion based on the project most important files

knowledge

obtain information about softwares, tools, etc

debug

help in debugging code

describe

describe documents, images

translate

translate text from one language to another

document

support in writing documentation

manage

out of text file update all the project management tools

agenda

current tasks, dates, status

tasks

manage tasks and subtasks, and deadlines

projects

manage multiple tasks and their relationships

workforce

organize workforce and keep track of progresses

link

build links between ideas and concepts

Of course we know that nowadays this is possible with frontier LLMs but let’s say we want to manage everything with some constraints:

emacs

emacs is the interface for everything, this means LLMs are proficient with org-mode and lisp and can read package information

local

local LLMs only for privacy, cost, and sustainability

reproducible

no chat, everything should be managed by persistent files

persistent

files contains the information needed to replicate the project

minimal

the source of truth is always text (the images should be generated by text like in mermaid)

updated

the agent should scan the files and make sure the files contain the updated project information with no contradictions

history

history should be managed in a different format to not collide with the current status

First of all you need to serve an LLM on your machine. I recently tested different LLM providers:

ollama

easy to install and configure but poor in performance

unsloth

pretty advanced tool for fine-tuning but I struggled reaching good performances

llama.cpp

it should be the most performant but I struggled to have good resource control on my machine llama_cpp

vllm

tricky to configure

lmstudio

the basic UI worked at the beginning but the API integration failed

lemonade

my current choice:

lightweight

small package

hardware

it makes the best use of CPU, NPU and GPU

models

highlights the best fitting models for the scope

UI

the user interface is simple but extensive

The list of local models we can use is limited and we need to get the best out of it:

coding

Llama-3.2-3B-Instruct-GGUF

it looks weak

Qwen-3.5-9B-GGUF

usually my preferred option

Qwen-3.0-0.6B-GGUF

easy quick tasks

Mistral-7B-Instruct-v0.3-GGUF

a bit slow and sometimes

deepseek

weak

Gemma-4-12B-it-GGUF

to test for writing

writing

llama3.2

image generation

SD-Turbo

melancholic style, not really sticking to the prompt

stable-diffusion-xl-base-1.0-GGUF-Q8_0

text2speech

Moonshine-Medium-Streaming

Whisper-Tiny

object detection

YOLOv4-12-1.1

audio analysis

DeepVoice-12-1.1

audio synthesis

I used and tested different coding assistants. Since they have far too many access to the system I restrict their access putting them in a container where I can restrict the access to sensitive information and reduce the context.

aider

a lot of interactions, tiring aider

opencode

solid but not the most efficient

pi-coding

pretty efficient but sometimes too autonomous

I used and tested different LLM packages and I want to find the best combination among those and the models they call.

Create a todo list out of this description

ellama

interface easy to integrate in buffer, it messes up the org files

gptel

nicely integrated in org-mode. Still need to have it proficient with emacs knowledge base emacs_ollama

helm-LLM

useful for using multiple models in one interface

ollama-buddy

pretty extensive, I need to explore it more

minuet

autosuggestions as you go, it has frequent interactions with the models making it pretty heavy and needs nevertheless good quality results to be useful. Most of the models just provide generic autocompletions

We want to be able to use all the resources:

NPU

Minuet NPU Configuration Guide

GPU

CPU

Agents have to search the web since they often hallucinate. We have a websearch docker to locally perform websearch.

curl -X POST http://localhost:8091/search \
  -H "Content-Type: application/json" \
  -d '{"query":"what is intertino.it","max_results":1,"format":"json"}'

<!DOCTYPE html>
<html class="no-js theme-auto center-alignment-no" lang="en-EN" >
<head>
  <meta charset="UTF-8">
  <meta name="endpoint" content="search">
  <meta name="description" content="SearXNG — a privacy-respecting, open metasearch engine">

We use the same service for generating answers

curl -X POST http://localhost:8092/search \
  -H "Content-Type: application/json" \
  -d '{"query": "python async programming","search_depth": "basic","topic": "general","max_results": 5,"include_answer": true,"include_images": false,"include_raw_content": false,"time_range": "week","include_domains": ["docs.python.org"],"exclude_domains": ["pinterest.com"]}'

{"query":"python async programming","answer":null,"results":[{"title":"Developing with asyncio — Python 3.14.7 documentation","url":"https://docs.python.org/3/library/asyncio-dev.html","content":"Asynchronous programming is different from classic “sequential” programming. This page lists common mistakes and traps and explains how to avoid them.","score":1.0,"raw_content":null},{"title":"Novedades de Python 3.12 — documentación de Python - 3.12.14","url":"https://docs.python.org/es/3.12/whatsnew/3.12.html","content":"Se ha mejorado significativamente el rendimiento de escritura en sockets en asyncio. asyncio ahora evita copias innecesarias al escribir en sockets y usa sendmsg() si la plataforma lo admite.","score":1.0,"raw_content":null},{"title":"asyncio and free-threaded Python — Python 3.14.7 documentation","url":"https://docs.python.org/3/library/asyncio-threading.html","content":"vor 2 Tagen ... asyncio uses an event loop as a scheduler to enable highly efficient concurrency by switching between tasks to allow non-blocking I/O ...","score":0.5,"raw_content":null},{"title":"Event loop — Python 3.14.7 documentation","url":"https://docs.python.org/3/library/asyncio-eventloop.html","content":"The event loop is the core of every asyncio application. Event loops run asynchronous tasks and callbacks, perform network IO operations, and run subprocesses.","score":0.3333,"raw_content":null},{"title":"Built-in Functions — Python 3.14.7 documentation","url":"https://docs.python.org/3/builtins/functions.html","content":"This calls the __anext__() method of async_iterator, returning an awaitable. Awaiting this returns the next value of the iterator. If default is given, it is ...","score":0.25,"raw_content":null}],"images":[],"response_time":2.713}

In ollama-buddy the chat is not really helpful but we can add few interesting functions thanks to dyerdwelling.

The init file is gptel_tools.el where we need to launch elisp:(gptel-load-and-call) to open a new buffer to connect with the backend defined in . In this file we open different MCP servers as well.

This folder contains the description of the project, the executive summarize and important information about its scope and implementation. The description shouldn’t be too much detailed

The knowledge base should be constantly improved and enriched with new data and insights as the project goes on. The content here can be quite detailed if needed. LLMs should periodically check the consistency of the information

LLMs should exploit the capabilities of org-files to automate project management. org-agenda already has a lot of useful tools and agents should support and review making sure background processes work as expected.

agenda.org

update the calendar with tasks important dates

diary

update the diary with important events

project_viz.org

Gantt, kanban and timeline

check-in.org

the weekly report, the current status of the project

We need to sync bidirectionally the local folder with a webdav server so we can edit the files on mobile and sync the latest version on the local laptop. Services:

syncthing

unison

What is the best option?