Robot learning foundations (optional)

Why it matters

Robot learning applies the ideas of this branch under physical constraints: data is expensive, evaluation needs real trials, and simulation is cheap but systematically wrong. If you are not targeting robotics or embodied AI, awareness is enough; if you are, interviewers expect you to discuss imitation-learning failure modes and how you would evaluate a policy honestly.

Core ideas

Behavior cloning fits a policy to expert state-action pairs with supervised learning. Its weakness is covariate shift: the learner's small mistakes take it to states the expert never visited, where its errors are larger, so errors compound with the horizon. DAgger fixes the data rather than the model: run the learned policy, ask the expert to label the states it actually reaches, aggregate and retrain, which turns imitation into a no-regret online learning problem.

Demonstrations are often multimodal, for example passing an obstacle on the left or on the right. A unimodal Gaussian policy averages the modes into an invalid action. Recent methods predict short sequences of actions or sample actions with a diffusion model, which represents several modes and reduces compounding between steps.

Sim-to-real. Simulators differ from reality in appearance, contact dynamics, latency and sensing. Visual domain randomization trains on widely varied textures, lighting and cameras so the real world looks like one more variation; dynamics randomization varies masses, friction and delays. System identification instead measures real parameters to make the simulator closer, and adaptation methods infer them online.

Evaluation. Real-robot results usually rest on tens of trials per condition, so success rates carry wide confidence intervals. Fix the number of trials, the initial-state distribution and the reset procedure in advance, report intervals and failure examples, and include simple baselines.

Common misconceptions

• "Behavior cloning fails because the network is too small." The main failure is distribution shift from compounding errors, which more capacity does not fix.
• "More randomization is always better." Very wide ranges can make the task harder to learn and hurt performance; ranges need justification.
• "A high success rate in a demo shows reliability." Without trial counts, a reset protocol and failure cases, the number is not interpretable.

How to approach it

Use the CS 185/285 imitation-learning lectures and homework to implement behavior cloning and DAgger in simulation, read the DAgger and randomization papers, then skim Diffusion Policy for modern practice. Treat evaluation design as part of the answer to any robot-learning question.