Work in progress

[Granny]

Quick summary of my Cognitive stack:

The robot has a Raspberry Pi 5 in the head. For now, the Pi is mainly the eyes, ears, speaker, and pipe: camera stream, microphone input, audio output, and basic voice I/O. The heavier work runs offboard on a Mac Mini M4 that was already part of my home setup. The core idea is that the LLM is not the brain. The brain is the orchestration layer: memory, beliefs, goals, decisions, agents, and traceability. The LLM is only used to formulate the final answer from a context built by the system.

Current stack:

1. • Pi 5 in the head for capture and I/O
   • Mac Mini M4 for perception and cognition
   • PostgreSQL + pgvector for memory
   • Redis Streams as event bus
   • MLXserve for local LLM inference
   • FastAPI services for vision, dialogue, and cognition

   What works now:

   • persistent memory
   • beliefs with confidence scores
   • context built from the DB
   • vision + memory orchestration
   • multi-agent routing
   • procedural skills
   • traceable decisions

2. Example: If I ask “Where is Granny’s USB key?”, the system does not let the LLM invent a location. It checks its stored beliefs, can run a locate_object plan, combine memory and vision, then answer from its actual state. Next step is connecting the physical body through a robot_daemon for servos, CAN, IMU, sensors, and later locomotion. Basically: Body control should stay fast, local, and safety-critical. The cognitive layer handles memory, dialogue, initiative, planning, and high-level orchestration. But locomotion will come later. First, I need to be able to build reliable actuators.

[Granny]

Live voice first test

Voice Test

[Granny]

First Neck moves

Fisrt neck moves

[Granny]

face tracking is now ready

[Granny]

First look at the operational dashboard

[LazyYuuki_Menlo]

This is awesome stuff @Granny !!!

[Granny]

Small update on my Asimov internal system:

Before making the robot move, I wanted it to know what body it actually has.

So I added a first “body awareness” system to Asimov.

The robot now has a structured model of its own body: 25 articulations, each with a live state like available, unreliable, offline, or not wired yet.

The dashboard displays the full body map with one marker per joint, and the cognitive layer can summarize its own body state in natural language.

For example, Asimov can say that it has a complete body with 25 articulations, but that it cannot access all of them yet.

Right now, only the neck yaw is marked as available. Neck pitch is physically present but unreliable, probably because the head is not balanced enough yet. Most other joints are known by the body model, but not wired.

This is not motor control yet. It’s more like a first proprioception / body model layer.

The goal is that before any future action command, the robot can know:

• which joints exist
• which ones are wired
• which ones are reliable
• which ones are unsafe or unavailable
• which control interface should be used

So instead of a cognitive agent blindly asking to move an arm or the head, it first has access to the robot’s actual body state.

Next step: start testing homemade stepper-based actuators.