Built myself a browser sandbox for testing robot code, wish I'd done it years ago

I hit the same wall a couple years back, except my bottleneck was an aging laptop running the sim and a Raspberry Pi running the controller. Every OS update on either side broke something and I'd lose an evening reinstalling packages instead of testing. What finally stuck for me was rebuilding the whole environment from one script, so a broken setup became a ten minute fix instead of a weekend project. Not browser based like yours, but the fresh environment per run idea is the part that actually made the difference for me too. On the lag, don't sit on it like I did. I ignored latency in my sensor feed for months and it made tuning miserable, because I kept second guessing wheather a bad run was my code or the feed itself. Turned out the wifi hop between laptop and controller was most of it. Once I ran a cable between them the results became repeatable and I could finally trust a pass or a fail.

For the replay feature, log timestamps on both sides before you build the viewer. I skipped that and had to guess at timing when reviewing crashes, which became its own headache. Even raw logs with a crude playback got me most of what the fancy tools promised.

I went through a similar phase last year with a differential drive robot I was tuning path planning on. Ended up keeping the sim local on an old ThinkPad instead of moving anything server side, mostly because my wifi adds 40 to 80 ms of jitter and that made every run feel diferent. Your lag on sensor data might be the same thing, so it's worth checking whether it's the network or the container spin-up before you chase it. The replay feature you're planning is the part I'd prioritize. I bolted one onto my setup after a week of scratching my head over why the robot cut a corner too tight. Turns out the encoder readings were fine, my waypoint tolerance was just set tighter than the sim's noise floor. Watching the sensor trace frame by frame found it in ten minutes after two days of guessing.

Also, simplified physics is fine for planning work. Just don't be surprised when something that glides through your sandbox slides off a table edge on the real hardware.

How are you handling code versions between runs? I kept losing track of which tweak belonged to which test until I made the sandbox auto save the code snapshot with each run's results. Now I can look back and see exactly what produced that wierd overshoot instead of guessing. Also curious if the sim runs at a fixed tick rate, that made my results way more repeatable once I stopped letting it float.

One thing to watch with the browser side is that tabs running in the background get throttled hard, so if you switch to another window mid run your connection can stutter even when the container itself is fine. I lost a whole afternoon to that once and blamed my control loop before I figured out what was happening. Keeping the tab focused during a test run saved me alot of false failures. Does your setup pause the run when the tab loses focus, or does the container just keep going and log whatever happened while you were looking away?

The couch testing bit rang a bell. I kept losing my session halfway through a run because the laptop would doze off or I'd close the lid without thinking. Putting a tmux session on the server changed everything, I just reconnect from wherever and the run is still sitting there exactly where I left it. Sounds minor but it's the reason my weird half formed ideas actually got tested instead of dying at the setup stage. Might slot in nicely before you build that replay viewer.