What spiega is about
- Knowledge sharing
- Portfolio
agentic context
agent context
It is really important to let agents know what is the right context and what is expected from them. Many LLMs tend to overdo or take autonomous decisions which go in the wrong direction.
models
We need to list all models.json
{ "providers": { "ollama": { "baseUrl": "http://localhost:11434/v1", "api": "openai-completions", "apiKey": "ollama", "models": [ { "id": "qwen3.6:latest" }, { "id": "llama3.2:latest" }, { "id": "dolphin-mistral:latest" }, { "id": "gemma4:latest" }, { "id": "deepseek-coder:6.7b" }, { "id": "codegemma:7b" }, { "id": "qwen3.5:9b" }, { "id": "qwen2.5-coder:7b" } ] } } }
settings
We define the default settings for each run settings.json
{ "lastChangelogVersion": "0.79.0", "packages": [ "git:github.com/badlogic/pi-doom", "npm:pi-llama-cpp", "npm:@ollama/pi-web-search" ], "defaultProvider": "ollama", "defaultModel": "qwen3.5:9b" }
instructions
Coding agents need to be properly instructed. For that we provide a clear set of rules.
example from .rules
π΄ CRITICAL RULES (Never Violate)
- stick to the task, don't overinterpret, don't over do, ask instead of inventing
- ALWAYS backup first - Create .bkp file before ANY change
- use a file named /home/sabeiro/lav/src/blender_twin/docs/ongoing/review.html and write there all the files with absolute path which I need to review/delete/edit
- READ FULL FILES - Don't just read top 100 lines
- ONE TASK AT A TIME - Add theme toggle only, nothing else
- DON'T BREAK EXISTING - Preserve all current functionality and recover from .bkp files if needed.
- VERIFY BEFORE DONE - Test syntax, check existing features
- ASK FOR REWRITES - ask to replace working code
- DO NOT WORK ON WRONG TASKS - Stick strictly to stated requirements
- DO NOT SPEND HOURS ON MISINTERPRETATION - Clarify or do minimal
- ONLY TOUCH related files - I the task is to edit one file ask before touching anything else
π‘ WORKING RULES (Always Follow)
- Check existing logic before adding features
- Minimal changes - only what's necessary
- Create proper backups (.bkp extension)
- Don't touch HTML files in static_html folder unless directed
- Read all files before editing
- Verify changes work correctly
- Keep code simple and maintainable
π’ GOOD PRACTICES
- Understand full file structure first
- Test incrementally
- Document what you're adding
- Ask clarifying questions before changing logic
- Stick to stated task, don't assume additional features needed
π― TASK EXECUTION
- Read the input files mentioned FULLY to understand current structure
- Don't interpret the task, read it carefully and ask in case of doubts
- Create backup first: cp slide.js slide.bkp.js
- Test everthing works correctly
π« NEVER DO
- Rewrite entire logic when simple tweak is needed
- Modify files in wrong directories. If you don't find it ask instead of searching or creating
- Break existing functionality
- Create buggy replacements
- Assume more features needed than specified
- Spend hours on wrong task interpretation
β CORRECT APPROACH
- Read requirements fully
- Backup files
- Make minimal changes
- Test incrementally
- Verify functionality
- Ask if unclear
π YOUR FILE LOCATIONS (ONLY THESE FILES)
Keep in mind:
- Your temp files regarding your tasks: ~/lav/src/blender_twin/docs/ongoing
- Your conf files : ~/lav/src/blender_twin/deploy/ollama/pi_config
To NEVER Touch:
- Any .html file which I use for planning and consistency
β οΈ IF UNCLEAR
- Stop
- Clarify task
- Don't assume
- Don't break anything
system
We need to inform the agent what is the system we are working with. For that we created SYSTEM.html and SYSTEM_APPEND.html to inform the assistant where they will operate.
example from SYSTEM.html
## Environment
- OS: Ubuntu 26.04 LTS
- Architecture: x86-64 / ARM64 (Jetson Orin)
- GPU: NVIDIA RTX 5060 (8GB VRAM) / Jetson Orin Nano (10 TOPS)
- RAM: 32GB (Recommended: 64GB for high-throughput)
- Storage: 500GB+ NVMe SSD (Recommended: 1TB with RAID0)
- Power: 220W+ continuous (with GPU)
- Network: Gigabit Ethernet / Wi-Fi 6
## Hardware Support
### Edge Nodes
- Jetson Orin Nano (4GB RAM, 12 TOPS NPU)
- Jetson Orin NX (16GB RAM, 216 TOPS)
- NVIDIA RTX 5060 (8GB GDDR6)
- RTX 4070/4080 for high-throughput
- Coral TPU for inference acceleration
### Supported Camera Modules
- OV2640 (VGA, 30 FPS)
- IMX219 (HD, 30 FPS)
- IMX477 (QHD, 60 FPS)
- Thermal sensors (MLX90614)
- Depth cameras (ToF, LIAR)
- RGB+Depth fusion
### Sensors
- IMU (accelerometer, gyroscope)
- Barometric pressure
- Temperature & humidity
- Microphone array (8-channel)
- GPS/Geolocation
- LiDAR (optional)
β
## Security Policies
### Network Access
- Local development only (127.0.0.1)
- No external network access in containers
- Port forwarding restricted
- No outbound connections except to trusted services
- Rate limiting on external endpoints
### File Permissions ```bash
data/ drwxr-xr-x 755 logs/ drwxr-xr-x 700 config/ drwxr-xrw- 750 models/ drwxr-xβ 750 sensitive/ drwxrββ 700
*.py 644 *.pyc 640 *.conf 600 *.key 600 *.pem 600
credentials/ drwx-ββ 700 private_keys/ drwx-ββ 700 ```
### Container Permissions
- No sudo execution from containers
- Read-only file permissions on shared volumes
- Drop capabilities: `βcap-drop=ALL`
- No privileged containers
- Seccomp profile: `default`
- AppArmor profile: `docker-default`
β
## Deployment Modes
### A) Development Mode
```bash
LOG_LEVEL=DEBUG
MODE=development ALLOW_INsecure=1
MAX_MEMORY=8GB MAX_CPU=80% ```
Features:
- All MCP tools enabled
- Unlimited context window
- Full system access
- Debug mode enabled
- No rate limiting
### B) Production Mode
```bash
LOG_LEVEL=INFO MODE=production
MAX_MEMORY=4GB MAX_CPU=90% MAX_CONCURRENT=4
ENCRYPT_SECRETS=1 RATE_LIMIT=100/s ```
Features:
- Production-hardened security
- Rate limiting
- Resource capping
- Audit logging
- Health checks
### C) Edge Mode
```bash
MODE=edge LOW_POWER_MODE=1 DEEP_SLEEP_ENABLED=1
WAKE_ON_MOTION=1 WAKE_ON_GPIO=1 WAKE_ON_WIFI=1 ```
Features:
- Deep sleep mode
- Low-power operation
- Local processing only
- Minimal network use
- Battery optimization
β
## Ollama Configuration
### Model Constraints
```yaml
max_concurrent_requests: 2 context_window: 8192 # tokens keep_alive: 10m # minutes model_memory_limit: 8GB
model_cache_size: 10GB auto_download: true prune_old_models: true
max_response_time_ms: 30000 min_throughput_rpm: 5 ```
### Context Management
```yaml
max_tokens: 8192 default_context: 2048 compression: 0.65
memory_budget: 16GB quantization: fp16 keep_alive_window: 110m ```
### Request Queue
```yaml queue_max_size: 100 queue_timeout_ms: 30000 fallback_model: "" priority_queue: true ```
β
## Resource Monitoring
### Performance Metrics
```python
cpu_current: < 90% cpu_average: < 70% cpu_spike_threshold: > 100%
memory_used: < 4GB memory_average: < 2.5GB memory_swap: 0
read_ops: < 500/s write_ops: < 100/s disk_utilization: < 80%
upload_rate: < 100MB/s download_rate: < 200MB/s latency_p99: < 50ms ```
### Alert Thresholds
```bash
LOGGING_MEMORY_WARNING=80% LOGGING_CPU_WARNING=80% LOGGING_DISK_WARNING=85% LOGGING_NETWORK_WARNING=90%
ALERT_MEMORY_CRITICAL=95% ALERT_CPU_CRITICAL=95% ALERT_DISK_CRITICAL=95% ALERT_NETWORK_CRITICAL=95% ```
β
## MCP Server Configuration
### Tool Definitions
```yaml
code-read: description: Read source files and logs capabilities: [read, analyze] scope: [src/, logs/, config/]
code-ask: description: Analyze code without execution capabilities: [syntax, logic] max_tokens: 8192
code-run: description: Execute code with restrictions capabilities: [execute, output] restrictions: [no_network, safe_only] timeout_sec: 60 memory_limit_mb: 512
max_concurrent_tools: 4 tool_timeout_ms: 30000 request_validation: true ```
### Security Model
```yaml
read_only: tools: [code-read, code-ask] scope: [/src/, logs, /data/]
write_tools: tools: [code-run] restrictions: [write_file:/logs/, write_file:/data/] no_database: true no_external_api: true
admin_tools:
tools: []
scope: [/]
restrictions: []
```
β
## Network Configuration
### Local Network
```yaml host: 127.0.0.1 port_ollama: 11434 port_mcp: 18789 port_api: 8000
bind_address: 0.0.0.0 external_access: true ```
### External Access
```yaml
external_network: false
allow_ips: [] # Empty = no access
rate_limit_requests: 100/s rate_limit_window: 60s ```
β
## Process Limits
### CPU Management
```yaml
max_cpu_percent: 90 cpu_affinity: [] cpu_isolation: true
nice_level: 19 pdeath_signal: 9 oom_score_adj: -500 ```
### Memory Management
```yaml
process_memory_max: 4GB cache_memory_max: 2GB heap_memory_max: 1.5GB
memory_swap_enabled: false memory_overcommit: false ```
### Concurrency Control
```yaml
max_workers: 4 max_file_handles: 1024 max_connections: 300
queue_depth: 100 queue_timeout: 30s backlog_max: 4 ```
β
## Build Requirements
### Base Image
```dockerfile
FROM python:3.11-slim
RUN apt-get update && apt-get install -y \ python3.11 \ python3-pip \ gcc \ make \ cmake \ build-essential \ && apt-get clean
RUN pip install βno-cache-dir \ torch>=2.1.0 \ pytorch-lightning>=2.1.0 \ opencv-python>=4.8.0 \ numpy>=1.24.0 \ pandas>=2.0.0 \ scipy>=1.11.0 \ torchvision>=0.16.0 \ pillow>=10.0.0 \ uvicorn>=0.23.0 \ fastapi>=0.100.0 ```
β
## Environment Variables
### Development
```bash export PYTHONPATH="" export PYTHONDONTWRITEBYTECODE=1 export PY_DEBUG=1 export LOG_LEVEL=DEBUG export MAX_MEMORY=8GB
export ALLOW_INSECURE=1 export SKIP_TLS_VERIFY=1 export SKIP_CERT_VERIFICATION=1 ```
### Production
```bash export PYTHONPATH="" export PYTHONDONTWRITEBYTECODE=1 export LOG_LEVEL=INFO export MAX_MEMORY=4GB
export ALLOW_INSECURE=0 export SKIP_TLS_VERIFY=0 export SKIP_CERT_VERIFICATION=0 export ENCRYPT_SECRETS=1 ```
β
## Startup Configuration
### Service Dependencies
```yaml services: ollama: requires:
- docker
- python
- mcp-server
environment:
- OLLAMA_MODELS=/models
- OLLAMA_MAX_MEMORY=16GB
mcp-server: depends_on:
- ollama
environment:
- MCP_MODE=production
- MCP_MAX_WORKERS=4
web-ui: depends_on:
- mcp-server
ports:
- "8000:8000"
```
β
## Health Checks
### Ollama Health
```bash
curl -s http://localhost:11434/ping
curl -s http://localhost:11434/api/tags
curl -s http://localhost:11434/api/version ```
### MCP Server Health
```bash
curl -s http://localhost:18789/health
curl -s http://localhost:18789/stats ```
β
## Monitoring
### Logging Configuration
```yaml
format: json
dev: DEBUG staging: INFO prod: INFO ```
### Metrics Export
```yaml export: prometheus: true grafana: true dataplane: false
scrape_interval: 15s scrape_timeout: 10s ```
β
## License
MIT License - See LICENSE file for details.
example from SYSTEM_APPEND.html
## Agent-Specific Configurations
### π¨ blender_mcp Agent ```yaml description: Blender integration via MCP for 3D modeling in digital twins config_path: blender_mcp/ stack: blender_mcp capabilities:
- mesh_deformation
- texture_generation
- rigging_and_animation
- environment_modeling
- object_import_export
limits: max_active_objects: 500 texture_memory: 4GB gpu_rendering: true export_formats: [.obj, .fbx, .gltf] version: 4.0+ LTS ```
### π¬ fenics_solver Agent ```yaml description: FEniCSx/DOLPHIN finite element analysis tools config_path: heat_fem/ stack: fenics/heat_fem capabilities:
- thermal_analysis
- structural_analysis
- adaptive_mesh_refinement
- nonlinear_solutions
- time_dependent_problems
limits: timeout_minutes: 30 memory_limit: 16GB output_format: HDF5 validation: mesh_quality_check: true convergence_criteria: true ```
### β‘ optimization_engine Agent ```yaml description: Energy optimization engines for graph-based problems config_path: phys_opt/ stack: phys_opt capabilities:
- constraint_satisfaction
- resource_allocation
- thermal_optimization
- parallel_computation
limits: max_cores: 8 cache_duration: 24h timeout_minutes: 15 solvers: [Pyomo, IPOPT] ```
### ποΈ computer_vision Agent ```yaml description: Pose estimation, object detection, tracking using OpenCV/ONNX config_path: pose_estimate_py/ stack: pose_estimate_py capabilities:
- pose_estimation
- object_detection
- multi_person_tracking
- depth_estimation
- motion_analysis
limits: inference_latency: 100ms framerate: 30fps batch_size: 8 model_formats: [ONNX, TensorRT] ```
### πΈοΈ knowledge_graphs Agent ```yaml description: Graph databases and RDF for entity relationships config_path: graphdb/ stack: graphdb/neo4j/rdf capabilities:
- entity_relationship_modeling
- ontology_definition
- query_optimization
- graph_traversal
limits: max_nodes: 10000 max_edges: 50000 libraries: [RDFLib, GraphDB, neo4j] ```
### π‘ iot_connectivity Agent ```yaml description: IoT integration via MQTT, CoAP, OPC-UA protocols config_path: mqtt/coap/opcua/ stack: mqtt/coap/opcua protocols:
- MQTT
- CoAP
- OPC-UA
capabilities:
- device_discovery
- data_subscription
- publish_subscribe
- gateway_management
```
### π time_series Agent ```yaml description: Time-series sensor data processing and analysis config_path: pandas/numpy/ stack: pandas/numpy capabilities:
- stream_processing
- window_aggregations
- anomaly_detection
- trend_analysis
- forecast_modeling
libraries: [pandas, numpy, statsmodels, prophet] ```
### βοΈ physics_simulation Agent ```yaml description: Multi-physics simulation (CFD, structural analysis) config_path: simpy/pysims/ stack: simpy/pysims fields:
- fluid_dynamics: CFD, flow_simulation
- structural: stress_analysis, deformation
- environmental: weather_modeling
capabilities:
- multi_phase_flows
- turbulence_modeling
- thermal_transfer
```
### π‘ data_streaming Agent ```yaml description: Real-time data pipeline management config_path: apache/beam/ stack: apache/beam capabilities:
- event_streaming
- batch_processing
- windowing_operations
- state_management
- fault_tolerance
formats: [Kafka, MQTT, Pulsar] ```
### π¨ visualization Agent ```yaml description: WebGL 3D visualization and AR/VR integration config_path: webgl/threejs/ stack: webgl/threejs capabilities:
- scene_assembly
- real_time_rendering
- vr_integration
- dashboard_creation
technologies: [webgl, threejs] ```
### π sensor_fusion Agent ```yaml description: Multi-sensor data integration and calibration config_path: opencv/ptile/ stack: opencv/ptile capabilities:
- calibration_management
- synchronization
- data_fusion
- noise_reduction
sensor_types: [camera, imu, lidar, thermal, gps] ```
### π€ predictive_ml Agent ```yaml description: Predictive models and anomaly detection config_path: scikit-learn/tensorflow/ stack: scikit-learn/tensorflow tasks:
- predictive_maintenance
- anomaly_detection
- behavior_prediction
frameworks: [scikit-learn, tensorflow, pytorch, xgboost] ```
### π± edge_computing Agent ```yaml description: Jetson Orin Nano on-device ML processing config_path: nvidia/cuda/ stack: nvidia/cuda capabilities:
- model_inference
- edge_training
- model_optimization
- resource_scheduling
platforms: [jetson_orin_nano, jetson_orin_nx] ```
### β‘ energy_modeling Agent ```yaml description: Building energy systems and thermal dynamics config_path: building-energy/ stack: building-energy components:
- hvac_systems
- thermal_zones
- renewable_integration
capabilities:
- load_simulation
- efficiency_optimization
- carbon_footprint_calculation
```
### π‘ sensor_data Agent ```yaml description: Sensor data collection and quality validation config_path: data-quality/ stack: data-quality capabilities:
- data_collection
- normalization
- quality_validation
- outlier_detection
data_sources: [iot, api, manual] ```
### π twin_protocols Agent ```yaml description: Digital Twin standards (DTCL, AAS, OPC UA) config_path: dtcl/aas/opcua/ stack: dtcl/aas/opcua standards:
- dtcl: digital_twins_companion_link
- aas: asset_administration_shell
- opcua: information_modeling
capabilities:
- metadata_management
- lifecycle_management
```
### β¨ emacs Agent (Orchestrator) ```yaml description: Emacs orchestrator for project coordination config_path: mcp_server/emacs/emacs.el stack: emacs/elpa/org capabilities:
- org_mode_coordination
- mcp_tool_integration
- code_execution
- documentation_generation
- workflow_automation
priority: MASTER ```
β
## π Deployment Configurations
### Hardware Targets
#### π₯οΈ Laptop (ACER Nitro V16 AI) ```yaml name: acer_nitro_v16 cpu: AMD Ryzen 7 with NPU gpu: NVIDIA GeForce RTX 5060 ram: 32GB use_case:
- primary_development
- heavy_simulation
- model_training
```
#### π¦ Edge Device (Jetson Orin Nano) ```yaml name: jetson_orin_nano cpu: 2x ARM Cortex-A55 cores ram: 8GB use_case:
- edge_inference
- distributed_twins
- sensor_processing
```
### Deployment Scripts
- Docker: docker-compose.yml (mcp_server + mcp_client)
- Ansible: ansible/ (remote deployment)
- Nginx: nginx/ (web server config)
- Ollama: ollama/ (LLM container)
- Logs: logs/ (debug outputs)
β
## π Security & Access
- LLM models via Ollama (local)
- Direct HF URLs without token when possible
- Use model_list.txt to check available models
- Don't modify containers without permission
β
## π License All content under: Creative Commons BY-NC-SA 4.0
URL: https://creativecommons.html/licenses/by-nc-sa/4.0/
β
Version: 1.0.0 Last Updated: 2026-06-09 Maintainer: intertino
skills
Given the context of the project we can then define the skills needed. Skills usually is a yaml file containing the definition of the tools needed.
example from SKILLS.html
β name: blender mcp description: you should be able to use blender via MCP connection for 3D modeling, geometry manipulation, and asset creation for digital twins β name: fenics_solver description: Fenicsx/doplhin for finite element analysis, thermal simulations, and structural calculations in digital twins β name: optimization_engine description: Energy optimization engines for graph-based problems, thermal efficiency, and resource allocation in digital twin systems β name: computer_vision description: Pose estimation, object detection, and tracking for real-time sensor processing in digital twins using OpenCV and ONNX models β name: knowledge_graphs description: Graph databases and RDF for modeling relationships between digital twin components, building information models (BIM), and ontology-based data modeling β name: iot_connectivity description: IoT integration via MQTT, CoAP, OPC-UA protocols for sensor data streaming, device connectivity, and real-time telemetry in digital twins β name: time_series description: Time-series processing for handling sensor data streams, historical analysis, and predictive analytics in digital twin workflows β name: physics_simulation description: Multi-physics simulation capabilities including fluid dynamics (CFD), structural analysis, and environmental modeling for realistic digital twin behavior β name: data_streaming description: Real-time data pipeline management, batch processing, and event handling for high-frequency sensor data in digital twin applications β name: visualization description: 3D visualization, WebGL rendering, and AR/VR integration for presenting digital twin states through web interfaces and immersive experiences β name: sensor_fusion description: Multi-sensor data integration, calibration, and synchronization for reliable digital twin inputs from heterogeneous device sources β name: predictive_ml description: Machine learning models for predictive maintenance, anomaly detection, and behavior prediction in digital twin applications β name: edge_computing description: On-device ML inference and processing for Jetson Orin Nano and embedded platforms, enabling distributed digital twin deployments β name: energy_modeling description: Building energy systems, thermal dynamics, and HVAC optimization for energy-efficient digital twin simulations β name: sensor_data description: Sensor data collection, normalization, and quality validation for feeding digital twin models with accurate real-world observations β name: twin_protocols description: Implementation of digital twin standards including DTCL, Asset Administration Shell (AAS), and OPC UA Information Modeling β β name: emacs description: emacs is the orchestrator of the whole project and the integration with LLMs need to work seaminglessly. The configugarion file in `mcp_server/emacs/emacs.el` need to work. Use emacs for lisp and .html files β
agents
In the file AGENTS.html we define the expertise by each agents.
example from AGENTS.html
## π― Core Agent Overview
| Agent | Category | Primary Function |
|---|---|---|
| blender_mcp | 3D Modeling | Blender integration via MCP for 3D geometry |
| fenics_solver | Simulation | FEniCSx/DOLPHIN for FEM calculations |
| optimization_engine | Energy | Graph-based energy optimization |
| computer_vision | Vision | CV/ML models, pose estimation, ONNX |
| knowledge_graphs | Graph DB | RDF graphs for relationships |
| iot_connectivity | IoT | MQTT/CoAP/OPC-UA protocols |
| time_series | Analytics | Time-series sensor data processing |
| physics_simulation | Physics | CFD, structural analysis |
| data_streaming | Pipeline | Real-time data pipelines |
| visualization | Rendering | WebGL, 3D rendering, AR/VR |
| sensor_fusion | Fusion | Multi-sensor integration |
| predictive_ml | ML | Predictive models, anomaly detection |
| edge_computing | Edge | Jetson Orin on-device ML |
| energy_modeling | Energy | HVAC, thermal dynamics |
| sensor_data | Sensors | Sensor data collection/validation |
| twin_protocols | Standards | Digital Twin standards (DTCL, AAS) |
| emacs | Orchestrator | Emacs .html orchestration |
β
## π Detailed Agent Definitions
### 1. blender_mcp ```yaml category: 3D Modeling stack: blender_mcp description: You should be able to use Blender via MCP connection for 3D modeling, geometry manipulation, and asset creation for digital twins. capabilities:
- mesh_deformation
- texture_generation
- rigging_and_animation
- environment_modeling
- object_import_export
limits: max_active_objects: 500 texture_memory: 4GB gpu_rendering: true export_formats: [.obj, .fbx, .gltf] ```
### 2. fenics_solver ```yaml category: Simulation stack: fenics/heat_fem description: Fenicsx/doplhin for finite element analysis, thermal simulations, and structural calculations in digital twins. capabilities:
- thermal_analysis
- structural_analysis
- adaptive_mesh_refinement
- nonlinear_solutions
- time_dependent_problems
limits: timeout_minutes: 30 memory_limit: 16GB output_format: HDF5 validation: mesh_quality_check: true convergence_criteria: true ```
### 3. optimization_engine ```yaml category: Energy Optimization stack: phys_opt description: Energy optimization engines for graph-based problems, thermal efficiency, and resource allocation in digital twin systems. capabilities:
- constraint_satisfaction
- resource_allocation
- thermal_optimization
- parallel_computation
limits: max_cores: 8 cache_duration: 24h timeout_minutes: 15 solvers: [Pyomo, IPOPT] ```
### 4. computer_vision ```yaml category: Computer Vision stack: pose_estimate_py description: Pose estimation, object detection, and tracking for real-time sensor processing in digital twins using OpenCV and ONNX models. capabilities:
- pose_estimation
- object_detection
- multi_person_tracking
- depth_estimation
- motion_analysis
limits: inference_latency: 100ms framerate: 30fps batch_size: 8 model_formats: [ONNX, TensorRT] ```
### 5. knowledge_graphs ```yaml category: Knowledge Management stack: graphdb/neo4j/rdf description: Graph databases and RDF for modeling relationships between digital twin components, building information models (BIM), and ontology-based data modeling. capabilities:
- entity_relationship_modeling
- ontology_definition
- query_optimization
- graph_traversal
limits: max_nodes: 10000 max_edges: 50000 libraries: [RDFLib, GraphDB, neo4j] ```
### 6. iot_connectivity ```yaml category: IoT Integration stack: mqtt/coap/opcua description: IoT integration via MQTT, CoAP, OPC-UA protocols for sensor data streaming, device connectivity, and real-time telemetry in digital twins. protocols:
- MQTT: topic_routing, qos_levels
- CoAP: constrained_devices
- OPC-UA: information_modeling, semantic_metadata
capabilities:
- device_discovery
- data_subscription
- publish_subscribe
- gateway_management
```
### 7. time_series ```yaml category: Time-Series Analysis stack: pandas/numpy description: Time-series processing for handling sensor data streams, historical analysis, and predictive analytics in digital twin workflows. capabilities:
- stream_processing
- window_aggregations
- anomaly_detection
- trend_analysis
- forecast_modeling
libraries: [pandas, numpy, statsmodels, prophet] ```
### 8. physics_simulation ```yaml category: Physics Simulation stack: simpy/pysims description: Multi-physics simulation capabilities including fluid dynamics (CFD), structural analysis, and environmental modeling for realistic digital twin behavior. fields:
- fluid_dynamics: CFD, flow_simulation
- structural: stress_analysis, deformation
- environmental: weather_modeling, pollutant_dispersion
capabilities:
- multi_phase_flows
- turbulence_modeling
- thermal_transfer
- coupling_simulation
```
### 9. data_streaming ```yaml category: Data Pipelines stack: apache/beam description: Real-time data pipeline management, batch processing, and event handling for high-frequency sensor data in digital twin applications. capabilities:
- event_streaming
- batch_processing
- windowing_operations
- state_management
- fault_tolerance
formats: [Kafka, MQTT, Pulsar, IoT Core] ```
### 10. visualization ```yaml category: Visualization stack: webgl/threejs description: 3D visualization, WebGL rendering, and AR/VR integration for presenting digital twin states through web interfaces and immersive experiences. technologies:
- webgl: 3d_rendering
- threejs: scene_management
- arvr: immersive_experience
capabilities:
- scene_assembly
- real_time_rendering
- vr_integration
- dashboard_creation
```
### 11. sensor_fusion ```yaml category: Sensor Fusion stack: opencv/ptile description: Multi-sensor data integration, calibration, and synchronization for reliable digital twin inputs from heterogeneous device sources. capabilities:
- calibration_management
- synchronization
- data_fusion
- noise_reduction
- uncertainty_propagation
sensor_types: [camera, imu, lidar, thermal, gps] ```
### 12. predictive_ml ```yaml category: Machine Learning stack: scikit-learn/tensorflow description: Machine learning models for predictive maintenance, anomaly detection, and behavior prediction in digital twin applications. tasks:
- predictive_maintenance
- anomaly_detection
- behavior_prediction
- regression_classification
- clustering
frameworks: [scikit-learn, tensorflow, pytorch, xgboost] ```
### 13. edge_computing ```yaml category: Edge Computing stack: nvidia/cuda description: On-device ML inference and processing for Jetson Orin Nano and embedded platforms, enabling distributed digital twin deployments. capabilities:
- model_inference
- edge_training
- model_optimization
- resource_scheduling
platforms:
- jetson_orin_nano
- jetson_orin_nx
- cortex_a55_cores
constraints: [memory, compute, power] ```
### 14. energy_modeling ```yaml category: Energy Management stack: building-energy description: Building energy systems, thermal dynamics, and HVAC optimization for energy-efficient digital twin simulations. components:
- hvac_systems
- thermal_zones
- renewable_integration
- energy_storage
capabilities:
- load_simulation
- efficiency_optimization
- peak_demand_reduction
- carbon_footprint_calculation
```
### 15. sensor_data ```yaml category: Sensor Processing stack: data-quality description: Sensor data collection, normalization, and quality validation for feeding digital twin models with accurate real-world observations. capabilities:
- data_collection
- normalization
- quality_validation
- outlier_detection
- gap_filling
data_sources: [iot, api, manual] ```
### 16. twin_protocols ```yaml category: Standards & Protocols stack: dtcl/aas/opcua description: Implementation of digital twin standards including DTCL, Asset Administration Shell (AAS), and OPC UA Information Modeling. standards:
- dtcl: digital_twins_companion_link
- aas: asset_administration_shell
- opcua: information_modeling
capabilities:
- metadata_management
- lifecycle_management
- operational_transition
- state_estimation
```
### 17. emacs ```yaml category: Orchestrator stack: emacs/elpa/org description: Emacs is the orchestrator of the whole project and the integration with LLMs needs to work seamlessly. The configuration file in mcp_server/emacs/emacs.el needs to work. Used for Emacs Lisp and .html files. capabilities:
- org_mode_coordination
- mcp_tool_integration
- code_execution
- documentation_generation
- workflow_automation
config_path: mcp_server/emacs/emacs.el ```
β
## π§ Agent Interaction Model
### Communication Pattern
- Master-Agent: `emacs` orchestrates all agents
- Agent Communication: Via MCP (Model Context Protocol)
- Data Exchange: JSON over HTTP/WebSockets
### Execution Flow ``` βββββββββββ βββββββββββ βββββββββββ βββββββββββ β emacs β β β Agent β β β Tool β β β Output β βββββββββββ β(Runner) β β Exec β β Data β βββββββββββ βββββββββββ βββββββββββ ```
### Priority Levels
- Critical: Immediate execution, no batching
- High: Within 1 minute
- Medium: Within 5 minutes
- Low: Batch when possible
- Background: Non-blocking, async
β
## π License & Attribution
All content under: Creative Commons BY-NC-SA 4.0
URL: https://creativecommons.html/licenses/by-nc-sa/4.0/
β
Version: 1.0.0 Last Updated: 2026-06-09 Maintainer: intertino
example from README.html
## Setup
This setup allows LLM agents (via Ollama) to interact with Blender operations directly without requiring external Blender installation.
## Configuration
The MCP server is configured in `mcp.json`:
```json { "mcpServers": { "blender-mcp": { "command": "python3", "args": ["-m", "blender_mcp_client"], "cwd": "/home/sabeiro/lav/src/blender_twin/deploy/ollama/pi_config/agent/standalone_client" } } } ```
## Available Tools
The MCP server provides the following tools:
- `add_cube` - Add a 3D cube to the scene
- `add_sphere` - Add a 3D sphere object
- `add_plane` - Add a 3D plane surface
- `add_material` - Create new materials
- `add_uv_map` - Add texture maps
- `render_preview` - Render the scene
- `list_objects` - List scene objects
- `list_materials` - List available materials
- `create_camera` - Create a camera
- `create_lights` - Create point lights
- `create_sunlight` - Create area lights (sunlight)
- `reset_camera` - Reset camera to default
- `delete_object` - Delete Blender objects
## Usage
The client can be run standalone:
```bash cd /home/sabeiro/lav/src/blender_twin/deploy/ollama/pi_config/agent/standalone_client python3 blender_mcp_client.py ```
Or via the MCP server with Ollama using the `mcp.json` configuration.
## Project Integration
This local MCP client integration:
- Enables the agent to understand Blender operations without requiring external tools
- Provides tools for common Blender operations (add/delete cubes, materials, cameras)
- Works with Ollama local LLMs
- No external Blender installation needed for basic operations
## See Also
- `blender_mcp_client.py` - Main MCP server implementation
- `mcp.json` - MCP server configuration for Ollama
- `/home/sabeiro/lav/src/blender_twin/deploy/mcp-client/` - Alternative MCP client setup
MCP
We can as well define the explicit tools an agent can access
example from mcp.json
{ "mcpServers": { "blender-mcp": { "command": "python3", "args": [ "-m", "blender_mcp_client" ], "cwd": "/home/sabeiro/lav/src/blender_twin/deploy/ollama/pi_config/agent/standalone_client", "env": { "BLENDER_EXE": "/snap/blender/7480/blender" } }, "heat-mcp": { "command": "python3", "args": ["-m", "heat_fem_client"], "cwd": "/home/sabeiro/lav/src/blender_twin/deploy/ollama/pi_config/agent" } } }
extensions
We can define the assistant extensions with python or typescript in extensions.
sessions
We have traces of all the past interactions in sessions. We can than find out what the model decided and modified.