Prior to joining the CoMMA Lab, I completed my B.S. in Computer Engineering at the University of Pittsburgh and my M.S. in Electrical and Computer Engineering at Carnegie Mellon University. During my time at CMU, I worked in the Search-based Planning Lab from The Robotics Institute, advised by Professor Maxim Likhachev.
My research interests are in developing real-time robotic algorithms leveraging high-performance computing techniques. I am also interested in the decision and planning problems in robotics that require reasoning about physics and dynamics in the environment. I believe that providing solutions to these challenges is essential for robots to perform dexterous tasks in unstructured settings, such as domestic environments.
2025
arXiv
Parallel Heuristic Search as Inference for Actor-Critic Reinforcement Learning Models
Actor-Critic models are a class of model-free deep reinforcement learning (RL) algorithms that have demonstrated effectiveness across various robot learning tasks. While considerable research has focused on improving training stability and data sampling efficiency, most deployment strategies have remained relatively simplistic, typically relying on direct actor policy rollouts. In contrast, we propose PACHS (Parallel Actor-Critic Heuristic Search), an efficient parallel best-first search algorithm for inference that leverages both components of the actor-critic architecture: the actor network generates actions, while the critic network provides cost-to-go estimates to guide the search. Two levels of parallelism are employed within the search – actions and cost-to-go estimates are generated in batches by the actor and critic networks respectively, and graph expansion is distributed across multiple threads. We demonstrate the effectiveness of our approach in robotic manipulation tasks, including collision-free motion planning and contact-rich interactions such as non-prehensile pushing.
@misc{yang2025pachs,title={Parallel Heuristic Search as Inference for Actor-Critic Reinforcement Learning Models},author={Yang, Hanlan and Mishani, Itamar and Pivetti, Luca and Kingston, Zachary and Likhachev, Maxim},eprint={2509.25402},archiveprefix={arXiv},primaryclass={cs.RO},year={2025},note={Under Review},}
