SC-BPE: Skeleton-Conditioned Base Placement Evaluation

Overview

Mobile manipulators usually pick a base pose for the next skill and reposition for the one after. Often a single pose could support the whole task, but finding it is hard: grasps and arm motions are not yet decided, and early choices change what later skills can reach.

SC-BPE is a learned evaluator that scores a base pose together with a plan skeleton, the ordered Pick and Place actions with their objects and regions. A task and motion planner then refines the highest-scoring pairs first, keeping the base fixed for the whole sequence.

Swapping two objects between occupied trays: where should the robot stand to complete the whole task, and how base suitability depends on the temporary buffer — Swapping two objects between occupied trays needs a temporary location. Which base poses work depends on that choice (the two skeletons at right differ only in the buffer). Click to enlarge.

TL;DR: Predicts whether a whole plan can work from one base pose, so planning tries promising bases first and saves time.

Method

The evaluator scores a base pose together with the whole plan skeleton, so the same symbolic plan can rank differently from different bases.

Overview

Given a task goal and an RGB-D observation, a symbolic planner proposes plan skeletons. SC-BPE scores each skeleton together with a sampled base pose, and the TAMP planner refines the most promising pairs first, with the base fixed for the whole sequence.

Click the figure to enlarge it.

Results

With one fixed base pose, SC-BPE plans faster than random ordering of the same candidates, its scores follow how the plan changes the best base, and it is competitive with methods that move the base between skills.

Tasks

SC-BPE task 1: shelf placement — Task 1: a TidyBot transfers objects from a table to a designated shelf level without overlap (2 to 5 objects).

Compared methods

SC-BPE (ours): Ranks complete skeleton–base pairs by surrogate score; the base stays fixed.
Random Ranking: The same pairs in random order, with the same candidate evaluation.
Per-Skill Rebase: Geometric base selection, and the base may change between skills.
Atomic-BPE Rebase: Learned scores for the current skill only, with rebasing between skills.

Each setting uses 29 held-out scenes; ON means N objects.

1. Does ranking make planning faster?

Yes. SC-BPE has higher success and lower planning time than random ordering of the same candidates in all eight settings. On Task 2–O3 it solves all 29 scenes (random: 28.3%), and its first attempt already succeeds in 86.2% of scenes (random: 2.1%).

Task 2–O3: cumulative success versus (a) candidate attempts and (b) planning time, on 29 test scenes (60 s per candidate, 180 s per scene).

Cumulative scene success versus planning attempts and planning time on Task 2-O3

2. Do the scores reflect the plan?

Yes. The three skeletons below differ only in the temporary buffer, yet get different score maps. Successful base positions sit toward the left for B1, the right for B3, and a broader central region for B2, and the high-score regions follow.

Surrogate score maps over sampled base positions for three skeletons that differ only in the temporary buffer — Task 2–O3. Maps show the maximum surrogate score over five headings. Circles and crosses mark base positions with at least one and no successful candidate evaluation; stars mark predicted maxima. On a phone, swipe sideways to see the whole figure.

3. How does a fixed base compare with rebasing?

It is competitive. SC-BPE has higher success and lower planning time than both rebasing methods on Task 1. On Task 2–O4 it ties Per-Skill Rebase at 89.7% success while cutting mean planning time from 124.4 to 75.0 s. Rebasing can win when moving helps: on Task 3–O2, Per-Skill Rebase reaches 96.6% success against 86.2%.

Setting	SC-BPE (ours)			Random Ranking			Per-Skill Rebase			Atomic-BPE Rebase
Setting	SR ↑	Plan ↓	Exec ↓	SR ↑	Plan ↓	Exec ↓	SR ↑	Plan ↓	Exec ↓	SR ↑	Plan ↓	Exec ↓
Task 1 · O2	100.0	8.9	27.5	86.4	21.1	27.5	93.1	20.3	31.5	93.1	19.9	29.4
Task 1 · O3	100.0	27.3	39.2	83.8	43.4	39.0	86.2	41.5	45.9	89.7	42.0	42.3
Task 1 · O4	100.0	46.0	54.9	80.6	91.0	52.5	72.4	93.0	59.7	79.3	88.0	54.8
Task 1 · O5	82.8	121.2	65.9	69.4	162.9	64.7	55.2	163.2	72.4	69.0	146.3	67.3
Task 2 · O3	100.0	26.4	29.3	28.3	147.1	30.3	86.2	72.9	31.6	89.7	77.6	31.9
Task 2 · O4	89.7	75.0	47.9	49.0	211.8	49.2	89.7	124.4	54.0	82.8	128.6	51.2
Task 3 · O2	86.2	52.4	34.9	22.1	108.1	34.8	96.6	46.9	38.5	58.6	75.8	36.5
Task 3 · O3	51.7	127.9	54.0	8.7	173.3	52.1	44.8	145.9	61.4	31.0	157.7	56.1

Swipe sideways to see all columns.

Hover (or tap) a method name for its definition and setting. SR: success rate (%). Plan: mean planning time over all scenes, including failures (s). Exec: mean execution time over successful scenes (s). Bold marks the best result and underline the second best when the best is unique. In each setting, all methods share the same scenes, goal checks, and time budget.

Real-to-sim-to-real demo

A physical mobile manipulator transfers two bananas from a coffee table onto a notebook on the sofa.

The evaluator is trained only on labels from a simulated reconstruction of the scene. Visual servoing corrects errors during execution.

How the demo works

(a) real-world navigation, (b) simulation navigation, (c) simulation trajectory, (d) real-world execution — Orange: placement trajectory. Cyan: post-placement retreat.

The room is reconstructed in simulation from real captures. Candidate outcomes collected there are the labels for training the evaluator.
SC-BPE ranks skeleton–base pairs, and the selected manipulation base becomes the navigation goal.
On the robot, point-cloud registration (small_gicp) localizes it as it approaches the base (a). The same approach is replayed in simulation (b), and the refined plan gives the manipulation trajectory (c).
The robot follows the simulated action sequence. Once the target is detected, visual servoing corrects accumulated and sim-to-real errors during execution (d).

Contact: [email protected], [email protected]