Giving a home robot agent a map it can browse

Oct 31, 2026 Last reply: 1 day ago 3 Replies
Jump to replies (3)

The detail that jumped out at me most: after Nav2 kept failing to dock near the fridge, the robot wrote its own parking rule into places/rest-spot/SKILL.md, then parked there successfully using it. An agent patching its own navigation behavior is the kind of thing you hope this kind of stack can do, and here it actually happened.



The build itself: the agent is Claude Code running on an XLeRobot body, a black wheeled multi-tiered cart carrying a forward-facing camera plus planar LiDAR. Compute is an Orange Pi 5 Ultra with an optional LAN GPU box, and the SLAM layer, EvergreenSLAM, needs nothing more than a 2D lidar plus an RK3588 board. The lidar came in at roughly 30 USD, which keeps the sensing side cheap. Navigation runs on ROS 2 Humble with Nav2 using MPPI.



What stands out is how the agent talks to the map: a command-line tool called egs, where every location is a folder anchored to the lidar map, holding a notes.md file and often a photo. That structure pays off in the demo. Asked about scissors, the agent checked four known places and needed a hint on Day 1, but on a later run found them at the side table and sent a photo 44 seconds after reading the question. It also drove to the balcony on the strength of an inferred line in its own notes and photographed a spot it had never captured before. Booting on Day 2 at the parked spot with no relocalize command is a strong result too.



Not everything is closed off. The Day 1 scissors hunt took about ten minutes and four questions, the GPU box stopped responding mid-take, the agent sat on a message for around 90 seconds waiting for a listen call to end, and the demo included neither teleoperation nor arms. Six places saved in about six minutes of autonomous exploration on Day 1 is a solid start.



Project by Thomas. Watch it on their YouTube channel:

formatting link


Treating each place as a plain folder with a notes.md inside seems like the smartest call in the whole design. An agent already knows how to read and write files, so the spatial memory needs no new interface to learn. The trade-off I see is that the map data and the notes live in separate layers, and keeping those anchored together is the part most likely to drift over time.

How does the Day 2 boot actually manage with no relocalize command? Does EvergreenSLAM match the live lidar scan against the saved map on startup by itself, or does the agent get a say in when localization is trusted? Also curious whether all six saved places came through a full power cycle with their notes and photos intact.

One variation I would suggest: after saving a new place, have the agent schedule a short return visit to photograph it. The balcony run showed a location can exist in the notes as text only until someone happens to ask about it, so a quick verification photo pass would catch bad entries early. It would also give that self-written parking rule more chances to prove itself around the rest of the house.

Join the Discussion

Have something to add? Share your thoughts — no account required.

Didn't find your answer?

Ask the community — no account required