Physical Simulation-Based Correction of LLM-Generated Tool-Using Primitives

초록

Large language models can translate natural language into robot code but often produce physically infeasible behaviours, particularly in tool-use tasks requiring precise contact reasoning. We propose a closed-loop framework that integrates code generation, physical simulation, and iterative correction to ensure physically valid tool-use primitives. Failures such as collisions and misalignments are detected via simulation feedback and corrected by targeted parameter updates while preserving task structure. Across five tool-use tasks in simulation and real-world tests, our method achieves a 92% average success rate, representing a 119% improvement over prompt-based generation and a 16% improvement over vision-language model-only correction. These results highlight the importance of simulation-guided reasoning for robust language-driven manipulation.

키워드