An Extensible Software Architecture for Composing Motion and Task Planners Zakary Littlefield, Athanasios Krontiris, Andrew Kimmel, Andrew Dobson, Rahul Shome, Kostas E. Bekris Computer Science, Rutgers University, NJ, USA October 21, SIMPAR 2014 Bergamo, Italy
High- level Mo,va,on Complex robot behaviors can be achieved by bringing together the contributions of different sub-fields in robotics Need for software tools/abstractions for better integration of: low-level controllers, motion planners and high-level task planning symbolic, logic-based, combinatorial, discrete, etc. reasoning Related software tools: OMPL, MoveIt!, Reflexxes, Gazebo, etc. Focus: So)ware abstrac0ons for composing mo0on and task planners
Domains of Application Manipula0on and Locomo0on (Humanoids) State X Time Planning Mul,- modal Mo,on Planning Object Rearrangement [Hauser et al. ISRR 07] [Kron,ris et al. Humanoids 14] Sensor- based Task Planning, Explora,on, Coverage, etc. Planning Among Dynamic Obstacles [Gashler et al. ICRA- work. 13] [van den Berg et al. ICRA 06] Mul,- goal challenges, e.g., robo,c TSPs [Plaku et al. MIG 14] Task Sequencing Mo,on Coordina,on [Bekris et al. MONE 09] Mul0- Robot Challenges
Why an Interface for Task Planners? For each challenge: Write one new planner from scratch specifically for this problem or compose exis,ng planning solu,ons for subproblems: Reusability: do not rewrite the same code again, reuse contribu,ons of others Modularity: easily switch components and evaluate their effects Challenges: Task planning is by default abstract and relates to many applica,ons Many compe,ng methodologies exist (e.g., LTL, PDDL, etc.) Dominant theme we adopt: hierarchical composi0on of planners As we work in new challenges we come across these abstrac,ons: We make the code available We want to communicate our ideas to receive your feedback!
Overall So`ware Architecture
High-level Components Separate processes for different components that interact with (ROS) messages Task and Mo,on Planning Focus on this interac,on and modules Control, Sensing & Simula,on Loop Visualiza,on Common U,li,es Package : Message- passing communica,on : Depends upon There can be many different processes for different planning nodes and different control loops: There is one ground truth simula,on node
Interac,on of Planning with Control Loop Mo,on and Task Planning Control, Simula,on & Sensing Loop Planning Applica,on Mo,on Planners Planning Modules e.g., local planner Hierarchy of Task Planners World Model Predic,ve Simulator plans C O M M U N I C A T I O N ground truth state Collision Checking Applica,on Ground Truth Simulator Hierarchy of Controllers Physical Plants Sensing Set of Sensors and Data : Message- passing : Makes use of module
Control Loop Abstractions Every moving body is modeled as a dynamical system: x = f(x,u) System Controller Plant Simulator : Inherits from Control u and control space U Physical plant state: x state space: X Capability to update state given a control Input control u in Control-space U in Controller state: x state- space: X Output control u out Control-space U out sensing data s(x) from sensor
x = < x 1, x 2, x 3, x 4, x 5, x 6, x 7, x 8, x 9, x 10 > A Hierarchy of Controllers Opera,ons: a. Set State b. Set Control and Propagate u < x 1, x 2, x 3, x 4, x 5, x 6, x 7, x 8, x 9, x 10 > Simulator with c. Get State state x u 1 u 2 Controller with many output spaces u 3 u 4 u 5 u 6 Controller with many output spaces u 7 u 8 u 9 u 10 x 9 x 9 x x 10 u 11 u12 u 13 u 14 u 15 u 16 u 17 u 18 x x 2 x x 3 x x x x x 2 6 1 x x 4 3 x x 6 5 x x 8 77
Controllers can also access sensing informa,on Ø Updated by sensors stored by the sensing model Integrating Sensing The simulator ini,ates a sensing update Ø upon demand or given a certain frequency
Interac,on of Planning with Control Loop Mo,on and Task Planning Control, Simula,on & Sensing Loop Planning Applica,on Mo,on Planners Planning Modules e.g., local planner Hierarchy of Task Planners World Model Predic,ve Simulator C O M M U N I C A T I O N Collision Checking Applica,on Ground Truth Simulator Hierarchy of Controllers Physical Plants Sensing Set of Sensors and Data : Message- passing : Makes use of module
Planning Hierarchies
Integrating Motion & Task Planners Abstract Planner Task Planner Mo,on Planner : Inherits from Mo,on and task planners: extensions of abstract planners access planning modules Task planners: can be composed hierarchically Mo,on planners: exist at the lowest level have more concrete interface Planners and planning modules: can access the world model
World Model Interface The world model allows planners to access an internal simulator: to model the evolu,on of the scene and the control systems inside it Exposes to planners: Ø a state space and Ø a control space. Simulator Different planners working with the same world model may need to switch planning context. Controller x x 2 x 4 1 x 3 Controller x x x 5 6 x 8 7
World Model Interface A planning context defines 3 subspaces: a planning space: Ø the space planning operates over an object space: Ø DOFs of objects used for validity checking and an inac0ve space: Ø systems ignored by the planner Simulator These spaces may be: direct subsets of the full space or correspond to embeddings of subsets of the state space. Controller x x 2 x 4 1 x 3 Controller x x x 5 6 x 8 7
Planners Interface Planners communicate via: specifica0ons and queries Specifica0ons define the problem type the lower- level planner must solve Queries: When linked, provide the ini,al and final condi,ons of the problem When resolved, they contain the answer to the problem Ø Before resolving a query, the lower level planner may call other planners to resolve mul,ple sub- queries The type of specifica,on/query generated by the high- level planner must match the type expected by the lower- level
Use Cases
Challenge: Push the objects further back in the space Observa,ons: Needs a single- object manipula,on planner Manipula,on planner needs mo,on planners Ø Transit planner: no cup in hand Ø Transfer planner: arm + cup in hand [Kron,ris et al. Humanoids 14] Rearrangement Using Baxter We use the same PRM* code for both the transit and transfer planner: different planning context The rearrangement planner generates on the fly problem specifica,ons and queries for the single- object manipula,on planner
Planning Among Dynamic Obstacles Challenge: Compute a plan for a car that avoids moving obstacles Behavior of moving obstacles defined by controllers in control loop The states of the moving objects correspond to object space, for which the planner does not reason over: Used only for state validity checking
Decentralized Mul,- Robot Coordina,on Challenge: Mul,ple agents replan on the fly and use VOs for collision avoidance Simula0on side: a consumer controller for each system receives plans from a planning node Planning side: build plan that minimizes conflicts with observed poses of neighbors using internal simulator [Kimmel et al. DARS 14]
Discussion
Contributed Features The community needs software that will glue together existing contributions towards solving more complex problems Next steps: We would like to see it adopted by others: Ø Considering independent release of manipula,on code for Baxter Ø Providing crowd simula,on for the PA s Bus Terminal in Manhatan Provide standard modules for abstract task planners (LTL, PDDL, etc.) Beter compa,bility with exis,ng so`ware tools Now the programmer fully describes the planning tools/tasks: Can we use this infrastructure when a task hierarchy needs to be automa,cally learned from demonstra,on and experience?
PRACSYS Team Zakary Littlefield Athanasios Krontiris Andrew Kimmel Andrew Dobson Thank you! Rahul Shome If you are interested in the framework, contact us: http://www.pracsyslab.org or check the Sourceforge page: http://sourceforge.net/projects/pracsys/ The work of Zakary Littlefield has been supported by a NASA graduate fellowship The work of Andrew Dobson has been supported by a DHS graduate fellowship Earlier versions of the software were supported by an NSF CPS grant