Semi-Empirical Simulation of Learned Force Response Models

for Heterogeneous Elastic Objects

¹ Y. Zhu and K. Hauser are with the Departments of Computer Science, University of Illinois at Urbana-Champaign, IL, USA. {yifan16, kkhauser}@illinois.edu}
² K. Lu is with the Department of Automation, Tsinghua University, Beijing, China. {lu-k16@mails.tsinghua.edu.cn}

paper link

This paper presents a semi-empirical method for simulating contact with elastically deformable objects whose force response is learned using entirely data-driven models. A point-based surface representation and inhomogeneous, nonlinear force response model are learned from a robotic arm acquiring force-displacement curves from a small number of poking interactions. The simulator then estimates displacement and force response when the deformable object is in contact with an arbitrary rigid object. It does so by summing the expected forces at individual points through querying the learned point stiffness models as a function of their expected displacement. Experiments on a variety of challenging objects show that our approach learns force response with sufficient accuracy to generate plausible contact response for novel rigid objects.

We evaluate this approach using an experimental testbed onsisting of a robot arm equipped with a RGB- D camera, a tapered probe attached at the end-effector, and a force-torque (F/T) sensor. We include 5 different testing objects in our experiments, Sloth, Vest, Lamb, Shoe, and Bird. Each object has varying stiffness across the object surface.

The flow of our overall system is illustrated in the aboved image. The first stage of our system learns a visuo-tactile model of the deformable object using data. We use a wrist mounted RGB-D camera to build a 3D point cloud, and poke a set of locations with a point probe to capture ground truth force response data. Then we use AutoML package from scikit-learn to perform the regression tasks. The second phase of our pipeline estimates reaction forces between the deformable object and a contacting rigid object. The rigid object is represented as a 3D surface mesh M defined in the global frame. The contact detector determines whether it is in contact with the deformable object’s equilibrium surface model S, and if so, which points must be displaced to resolve the contact. The contact force solver then computes the 3D reaction force and torque and the deformed shape of the object.

Step 1: Point Deformation Model Learning

Force predictions on the Lamb object, learned with different numbers of poking locations. Poking locations are indicated as circles, colored by the true force. d is set to be 0.5 mm. Because of camera bias, there is already significant amount of force at this depth.

Breakdown of learning curves for learning force response models. Regressors are trained over 10 sampled subsets of the ground truth dataset, and plots show RMSE mean and variance across samples.

Step 2: Semi-Empirical Simulator

Nominal Displacement Calculation: To calculate nominal displacements of the deformable object’s surface points we use collision detection and estimate penetration depths.

Force Solving: The force solver uses the nominal dis- placements calculated above combined with an indentation basis function to compute the actual displacements.

Results show that our approach can learn a point deformation model that predicts force response 0.49N of root mean squared error (RMSE) over 5 test objects with 10 interactions each. When making contact with novel rigid probes, which are a line-shaped paddle and a cylinder, our simulator’s force prediction has 1.5 N RMSE. This accuracy is comparable to learned models obtained with the same number of interactions.

Simulation Visualization: We illustrate our simulator by visualizing the simulation of a line probe poking straight at the Lamb based on Klampt and Open3D.

Application to a Packing Problem: We consider a simple packing problem as an illustration of our approach. Given a rigid box with a deformable object (the Lamb) already packed, we would like to know if the robot can pack another rigid item into the box.

On average in this experiment, a contact reaction force query takes 0.086s. Each query consists of 0.0738s of collision detection, 0.010 s of querying the point model, and 0.0017 s of LCP solving by our simulator.

---

Contact

Have any questions, please feel free to contact Yifan Zhu, Kai Lu

---

January, 2020
Copyright © Kai Lu