Spiega documentation
- Knowledge sharing
- Portfolio
spiega tech reveal config
shared documentation across projects
:INSTRUCTION: You are a language model to assist different IT projects helping with coding, project planning and documentation. Read the instructions in ~/lav/src/spiega/article/agent_context.org, avoid to write in the chat and let’s communicate via this org files in the folder ~/lav/src/spiega/knowledge/ :ID: reaveal_config
spiega tech
Spiega is a collection of articles/post written during 17y of career spanning over 3k3 source code files and 2k images. The main areas are around data science and the impact of using machine learning in operations and business. Some content is presented as slides or as automated generated summary in the overview page and connected in sitemap. For a more business oriented description of those activities refer to spiega business or the former blog index.
Figure 1: module library
During this time I tried to make the organize the diverse topics into some main areas which are:
management and client projects management
Here is a collection of experiences regarding project scoping, execution and reporting
- governance
- manage and design data platforms
- team management
- how to manage and kick off a team
- project tools
- what makes the execution of a project efficient
- professional growth
- how to see a career overall
Figure 2: module library
consultancy and service provider consultancy
- dauvi
- ERP tools and firm consultancy, mainly focused on setting up and implementing ERP tools and connecting different company components
- intertino
- web services. Offer segmentation strategies, agent compensation models, lagged metrics, customer lifetime value calculation, contact channels main and control metrics, scooter movement analysis, auto sensors assessment, shared bike usage evaluation, and weather prediction models.
Figure 3: module library
optimization experience data_science
Collection of projects for optimization challenges
- geomadi
- articles on geo features, routing applications, location intelligence, motorway stoppers analysis, routing comparisons, and sensor triangulation.
- antani
- optimization engine (reinforcement learning, simulations).
- mallink
- simulation engine in python.
- allink
- C/C++ simulation code for biophysical membranes
Figure 4: module library
data science competences data_science
Collection of libraries used as support to data projects
- kotoba
- generative ML. ML painting, text generation, music composition, agent naming environment for redundancy in images.
- albio
- time series analysis.
- lernia
- machine learning library used across multiple projects. Regressions without neural networks
- deep lernia
- machine learning library used across multiple projects. Neural network models
- ndoe
- Equations of motion, ride behavior, mobility concepts, POI capture rate, activation potential, data quality assessment, telemetry data types, prediction anomalies, and spatial analysis.
Figure 5: data science competences
programming and devops dev
Collection of work to develops products and manage data platforms
- sawmill
- big data processing. Data platform basics, data storage applications, compliance with access rules for sensitive data, modeling concepts, security practices, webserver and networking principles, messaging systems, middleware to protect requests, cloud provider impressions, CI/CD processes, testing methods, scheduling jobs, log processing, UI and data visualization techniques, and programming documentation.
- intertino
- web services. Offer segmentation strategies, agent compensation models, lagged metrics, customer lifetime value calculation, contact channels main and control metrics, scooter movement analysis, auto sensors assessment, shared bike usage evaluation, and weather prediction models.
- malastro
- useful scripts to manage a laptop, a server, embedded devices, media… Mainly using bash, ffmpeg…
Figure 6: data pipe example
art and science science
Non commercial projects around art, music and science
- kotoba
- generative ML. ML painting, text generation, music composition, agent naming environment for redundancy in images.
- science
- scientific contributions. PhD defense/paper on membrane inclusion objects, master defense/thesis on nanoparticle stability, bachelor defense/thesis on ion diffusion in lattice, Fokker-Planck equations, Fluttuazione theorem, physics papers, computer science lectures, PLOS One paper on influenza fusion, PR Letters paper on string method, and Sciencedirect paper on pore formation.
- algorithmics
- computer science features mainly used in scientific computation
- viudi
- music tech including theoretical discovery of composition, electronics.
Figure 7: module library
dev
- sciame
- IoT applications. Coil for pickups/sustainers, synth collection, DSP, effects, mechanical amplifier, no-keyboard keyboard (touch surface), MIDI router player, audio mixer with op amps, piezo buffer for impedance matching and noise reduction, and creative coding techniques including openCV, openFrameworks, and processing.
Figure 8: sciame library
management
This section collects the relevant experiences in project and team management. The whole list of the 40 projects is in portfolio.
Figure 9: management experience
project tools management
Being efficient in running projects is essential and (especially in an agentic world) we need to prepare a good background for those agents to operate
- agent workflow
- local agents for your productivity
- org files
- productivity with org files
- agent context
- instruct agents how to behave and be productive
- agentic interview
- an interview run by agents
- documentation present
- create professional documentation out of your notes
- spiega deploy
- how to replicate this project
- agent call
- how we use agents with MCP tools
- knowledge graph
- create a knowledge graph from your notes
- multimedia utils
- ease the creation of multimedia content for presentations
- productivity_tools
- for teams
- knowledge_create
- create the knowledge base for your project
Figure 10: module library
professional growth
Career is a long run, is made out of many investments in time and money. It needs planning and occasions, promotion and networking
- 13-2?] bio
- overview on the main achievements
- 26-2?] personal finance
- manage your savings and investments
- 26-2?] documentation present
- learn how to create presentations
- 26-2?] portfolio_creation
- create your portfolio
- 26-2?] video animations
- to build a portfolio
- 26-2?] hobby profession
- when your hobbies turn into real jobs
- 26-2?] second hand
- when your career change make sense of your old tools
- 26-2?] youtube channel viudi_channel_short
- communicate with videos
- 13-2?] personal_projects
- professional growth aside job
Figure 11: manage your income
governance management
Data platform management
- 26-2?] LLM governance
- how to manage agentic systems
- 26-2?] storage governance
- avoid resource waste in storing and processing data
- 13-2?] management
- management experience
- 26-2?] data_sovereignty
- on-prem infra
- 26-2?] knowledge_graph
- auto generated locally
- 26-2?] product design
- features to build a product
Figure 12: module library
team management management
Team management
- 13-2?] team composition
- how to set up a team, SoW
- 13-2?] company tools
- productivity tools
- 13-2?] team productivity
- what makes a team productive
- 13-2?] product design
- the important questions for building a product
- 13-2?] presales
- make sure you can deliver
- 13-2?] client relationship
- promise, deliver, tell
- 13-2?] corporate organization
- managers, politics, career progression
- 13-2?] well being
- body health for productivity
- 13-2?] experience
- overview on professional experience
- 13-2?] competences
- bot generated content of competences from these articles
Figure 13: module library
optimization
Libraries mainly focused on optimization engine (operation, mobility…)
Figure 14: module library
geomadi data_science
Geomadi was used to perform complex operations on a large network. The mongo database contained all the information regarding maps and POIs in Germany and we were analyzing spatial data regarding specific subset of a complex network. The library uses spatial queries and geometrical transformations.
- 17-19] graph creation
- for routing applications
- 17-19] location intelligence
- to enrich predictions
- 17-19] motorway stoppers
- analysis on motorway traffic and capture rate
- 17-19] routing comparison
- comparing routing solutions
- 20-21] mini hub
- improve logistics by placing mini hubs for temporal storage in city centers
- 17-19] triangulation of sensor data
- locate mobile users
- 17-20] spatial utils
- for GIS
- 17-20] spatial_data
- structure of spatial data
Figure 15: geomadi library
antani data_science
Antani is an optimization engine for logistics building efficient routes inspired by ants. The routes grow as polymers and variate to maximize opportunity/cost. This is a mixture of MonteCarlo as built in my PhD thesis and reinforcement learning. Each move has different polymer expansion options and the reinLearn decides what action to take to improve overall optimization time.
- 19-20] antani concept overview
- 19-20] antani infrastructure design filatto infra
- 19-20] antani microservice integration
- 19-20] antani kpi comparison
- 19-20] antani overview page
- 19-20] antani compared with routific
Figure 16: antani library
mallink data_science
ndoe data_science
Ndoe analyzes motion behaviors and builds mobility reports
- 17-20] equations of motion
- improve gps data
- 17-20] ride behaviour
- what users do
- 17-20] mobility concepts
- how people move around
- 17-19] capture rate for restaurants
- how people buy
- 17-20] activation potential
- how likely is a spot to convert
- 17-20] geo admin
- spatial utilities and GIS
- 20-21] quality on telemetry
- process and clean telemetry data
- 20-21] data types for telemetry
- manage and stream telemetry data
- 20-21] feature in telemetry
- feature engineering for telemetry
- 20-21] forecast anomalies
- forecast disruptions
- 20-21] prediction on telemetry
- predict disruptions with telemetry data
- 20-21] spatial
- analysis of geo referenced data
Figure 18: ndoe motion patterns
data science
Machine learning and statistics used in different projects
Figure 19: module library
lernia data_science
Lernia is a library used across project to use ML. It divides into lernia and deep_lernia where the second one is built on keras
- 15-20] lernia feature selection
- avoid useless features
- 15-20] lernia library overview
- what the library does
- 15-20] lernia blind test on series forecast
- why blind test is important
- 17-19] convNet on time series
- time series as images
- 17-19] sensor filtering
- increase resolution
- 17-19] skewness in predictions
- find the skewness in your data
- 15-2?] neural networks
- overview
- 15-2?] machine_learning
- models
- 15-25] AI in industry
- 2025 update
- 15-25] causality
- from patterns to knowledge
- 15-2?] ml_eng
- machine learning engineer
- 15-2?] llm_provider
- serve llm
- 15-2?] machine_learning
- relevant concepts
Figure 20: lernia library
albio data_science
Albio is a library to work with time series. During my career I used a lot of series decomposition and forecasts.
- 15-20] albio forecast
- with exogen variables
- 15-20] time series
- forecast in production
- 17-20] prediction_telemetry
- prediction with telemetry data
- 17-20] telemetry_quality
- quality of telemetry data
Figure 21: albio library
deep lernia data_science
agentic
Integrate agents into your workflow requires particular needs:
- llm_provider
- serve llm
- agent_instruction
- define how your agents should behave
- agent_call
- configure the system
- emacs_llm
- connect llm to emacs
programming and devops
Knowledge used across the projects. How we take big logs and process them to get the most important information out of the big volume.
Figure 23: sawmill library
sawmill dev
Big data, data engineering, data pipelines, machine learning ops
- 13-2?] data platform basics
- example designs and principles
- 13-2?] data storage
- applications and principles
- 14-2?] data compliance
- access, sensitive data, anonymization, retention
- 15-2?] data modeling
- concepts and applications
- 15-2?] security
- practices and examples
- 12-2?] webserver and networking
- for websites and traffic to containers
- 12-2?] web_admin
- manage your digital presence
- 20-2?] messaging system
- for stream and IoT
- 17-2?] middleware
- to protect and speed up external requests
- 15-2?] cloud providers
- impressions and experiences
- 15-2?] CI/CD
- praxis and options
- 06-2?] test and checks
- types and necessity
- 17-2?] scheduling jobs
- monitor and plan batch processes
- 15-2?] log processing
- from web, Iot, applications
- 06-2?] UI and data visualization
- for the business and client’s insights
- 12-2?] admin tools
- collection of script to admin a system
- 12-2?] tech_stack
- criteria for creating a tech stack
- 12-2?] sysadmin
- manage your systems
- 12-2?] backup_sync
- manage your storage
- 12-2?] queries
- example of useful queries
- 12-2?] python_tricks
- some commands I keep forgetting about
- 12-2?] documentation_praxis
- how to document projects
- 12-2?] tech_reference
- link to selected source code
- 12-22] business_intelligence
- BI and data visualization
malastro dev
Development, sysadmin, prototyping
- 06-2?] programming praxis
- documentation, conventions, processes
- 02-2?] emacs
- mighty editor
- 06-13] c++
- simulations, native visual applications, openGL
- 13-2?] python
- data exploration, visualization, ETL, python tricks
- 05-17] R
- data exploration, visualization, ETL
- 22-2?] go
- middelware to interact with relational databases
- 06-2?] SQL
- manage databases, create tables, join, format
- 13-2?] app
- native, angular, react, nodejs, cordova
- 06-2?] websites
- html5, css, bootstrap, php
- 06-2?] javascript
- openlayer, d3.js, maps
- 06-2?] node
- bot, automation, server
- 17-20] spark
- batch processing of logs
Figure 24: malastro library
assessments data_science
Coding interviews
- 19] scooter movement
- a portal for visualizing geo data
- 20] auto sensors
- analyze connected cars data
- 19] shared bike usage
- user usage of ride hail
- 17] ufo sightnings
- geospatial information
- 2?] weather prediction
- basics around data science
Figure 25: assessment library
consultancy and service provider
How to advice companies and clients to develop a digital/data strategy tech_consultancy.
Figure 26: management experience
intertino consultancy
Web marketing and digital services, customer retention and communication strategies
- 13-17] offer segmentation
- customer profiling
- 20-23] agent compensation
- align your workers to the company goals
- 20-23] lagged metrics
- optimize on long lasting benefits
- 20-23] customer lifetime value
- understand your customer value
- 14-23] contact channels
- principal and control metrics
- 14-2?] imprint
- legal digital stuff
- 14-2?] privacy
- legal digital stuff
Figure 27: intertino library
dauvi consultancy
art and science
Non commercial projects for art and science
Figure 29: art and science
kotoba art
Language and perception
- 19-2?] painting with ML
- generative ai
- 17-2?] text generation
- generate text with simple ML
- 04-2?] music composition
- generate music following your style
- 23-2?] agent naming environment
- how agents perceive and name the environment
- 11-12] redundancy in natural images
- how the brain
- 99-2?] phonosymbolic language
- can we create a phonosymbolic language?
- 26-2?] org_creator
- how to organize your ideas and work
- 26-2?] chess_as_language
- name your ideas
- 26-2?] music_as_language
- name your progressions
Figure 30: kotoba
viudi art
Music development
- 13] entropy in music
- how to recognize the author of a song
- 23] music chart valuation
- how people judge the quality of songs
- 06] music composition
- generative melody generator
- 10] video production
- create songs with musical gears
- 10] video presentation
- description of the channel
- 10] video detail
- video detail
- 17] flute
- flute size calculator
Figure 31: viudi
allink dev
Software created during the PhD thesis
- 08-12] documentation
- for the project allink in C++
Figure 32: allink library
science science
Documents during the physics research period
- 09-13] PhD defense
- membrane inclusion objects
- 09-13] PhD thesis
- membrane inclusion objects
- 08-09] master defense
- nanoparticle stability
- 08-09] master thesis
- nanoparticle stability
- 05-06] bachelor defense
- ion diffusion in lattice
- 05-06] bachelor thesis
- ion diffusion in lattice
- 11-11] Fokker-Plank
- and master equations
- 07-07] Teorema della fluttuazione
- traiettorie a entropia negativa
- 02-09] Fizikaj Eroj
- klassika kaj kvuantuma mekaniko
- 13-13] Lezioni di informatica
- per piccole imprese
- 12-12] Paper
- Influenza Fusion, String method, Pore formation
Figure 33: fizikaj eroj library
algorithmics dev
Development of algorithms for better performances
- 11-12] configurational bias
- 06-12] Monte Carlo and molecular dynamics
Figure 34: algo library
media production art
Collection of media projects
- 13-15] creative coding
- openCV, openFrameworks, processing
- 26-2?] blender animation
- language models and 3d modeling
- 26-2?] blender tricks
- best settings for blender
- 15-2?] youtube channel
- manage a channel#+RESULTS: video-summary
- 15-2?] viudi channel
- current channel
- 10-2?] video production
- create songs with musical gears
Figure 35: media library
hardware sciame
Sciame is a collection of projects for embedded and IoT devices
Figure 36: electronics and IoT
sciame dev
- 18-2?] coiler
- coiler for pickups and sustainers
- 21-2?] synth
- collection of synths for cheap microdevices
- 21-2?] dsp
- dsp and effects
- 15-17] McAmp
- mechanical amplifier
- 15-17] no-key-board
- keyboard with no keys (touch surface
- 22-2?] midi-hub
- midi router
- 97-2?] audio mixer
- with op amps
- 19-2?] piezo buffer
- for impedance match and noise cleaning
- 25-2?] robotics
- tools for robotics development
Figure 37: sciame library
table of content
| ITEM | EFFORT | TODO | RATING | Tags |
|---|---|---|---|---|
| spiega tech | ||||
| [[management]] and client projects | :management: | |||
| [[consultancy and service provider]] | :consultancy: | |||
| [[optimization]] experience | :data_science: | |||
| [[data science]] competences | :data_science: | |||
| [[programming and devops]] | :dev: | |||
| [[art and science]] | :science: | |||
| [[hardware]] | :dev: | |||
| management | ||||
| project tools | :management: | |||
| professional growth | ||||
| governance | :management: | |||
| team management | :management: | |||
| optimization | ||||
| geomadi | :data_science: | |||
| antani | :data_science: | |||
| mallink | :data_science: | |||
| ndoe | :data_science: | |||
| data science | ||||
| lernia | :data_science: | |||
| albio | :data_science: | |||
| deep lernia | :data_science: | |||
| agentic | ||||
| programming and devops | ||||
| sawmill | :dev: | |||
| malastro | :dev: | |||
| assessments | :data_science: | |||
| consultancy and service provider | ||||
| intertino | :consultancy: | |||
| dauvi | :consultancy: | |||
| art and science | ||||
| kotoba | :art: | |||
| viudi | :art: | |||
| allink | :dev: | |||
| science | :science: | |||
| algorithmics | :dev: | |||
| media production | :art: | |||
| hardware sciame | ||||
| sciame | :dev: | |||
| table of content |