To Örebro University

oru.seÖrebro University Publications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Learn to Predict Posterior Probability in Particle Filtering for Tracking Deformable Linear Objects
Örebro University, School of Science and Technology. (AASS)ORCID iD: 0000-0003-1528-4301
Örebro University, School of Science and Technology. (AASS)ORCID iD: 0000-0003-3958-6179
Örebro University, School of Science and Technology. Department of Computing and Software, McMaster University, Canada. (AASS)ORCID iD: 0000-0002-6013-4874
2022 (English)In: 3rd Workshop on Robotic Manipulation of Deformable Objects: Challenges in Perception, Planning and Control for Soft Interaction (ROMADO-SI), IROS 2022, Kyoto, Japan, 2022Conference paper, Published paper (Refereed)
Abstract [en]

Tracking deformable linear objects (DLOs) is a key element for applications where robots manipulate DLOs. However, the lack of distinctive features or appearance on the DLO and the object’s high-dimensional state space make tracking challenging and still an open question in robotics. In this paper, we propose a method for tracking the state of a DLO by applying a particle filter approach, where the posterior probability of each sample is estimated by a learned predictor. Our method can achieve accurate tracking even with no prerequisite segmentation which many related works require. Due to the differentiability of the posterior probability predictor, our method can leverage the gradients of posterior probabilities with respect to the latent states to improve the motion model in the particle filter. The preliminary experiments suggest that the proposed method can provide robust tracking results and the estimated DLO state converges quickly to the true state if the initial state is unknown.

Place, publisher, year, edition, pages
2022.
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
URN: urn:nbn:se:oru:diva-102743OAI: oai:DiVA.org:oru-102743DiVA, id: diva2:1731600
Conference
35th IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2022), Kyoto, Japan, October 24-26, 2022
Funder
Vinnova, 2019-05175Wallenberg AI, Autonomous Systems and Software Program (WASP)Available from: 2023-01-27 Created: 2023-01-27 Last updated: 2023-09-18Bibliographically approved
In thesis
1. Advancing Modeling and Tracking of Deformable Linear Objects for Real-World Applications
Open this publication in new window or tab >>Advancing Modeling and Tracking of Deformable Linear Objects for Real-World Applications
2023 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Deformable linear objects (DLOs), such as cables, wires, ropes, and sutures, are important components in various applications in robotics. Although automating DLO manipulation tasks through robot deployment can offer benefits in terms of cost reduction and increased efficiency, it presents difficult challenges. Unlike rigid objects, DLOs can deform and possess high-dimensionalstate space, significantly amplifying the complexity of their dynamics. These inherent characteristics, combined with the absence of distinctive features and the occurrence of occlusion, contribute to the difficulties involved in DLO manipulation tasks.

This dissertation focuses on developing novel approaches for two aspects: modeling and tracking DLOs. Both aspects are important in DLO manipulation, yet they remain open research questions. Current analytical physics-based methods for modeling DLO dynamics are either time-consuming or inaccurate and often undifferentiable, which hampers their applications in robot planning and control. Although deep learning methods have shown promise in modeling object dynamics, there is still a gap in learning DLO dynamics in a 3D environment. As for the tracking, many current methods rely on assumptions such as knowing the DLO initial state and segmented DLO point sets, which are rarely fulfilled in real-world scenarios, significantly limiting their practical applicability.

This dissertation aims to answer three research questions: How can data-driven models be used for learning DLO dynamics? How can the data-driven models be efficiently trained for real-world DLO manipulation tasks? How can images be used to track the state of DLOs during manipulation in uncontrolled real-world settings?

The first contribution of this dissertation is a data-driven model that effectively simulates DLO state transitions. To bridge the current gap in learning full 3D DLO dynamics, a new DLO representation and a recurrent network module are introduced to facilitate better effect propagation between different segments along the DLO. Meanwhile, the model is differentiable, enabling efficient model predictive control for real-world DLO shape control tasks. However, data-driven approaches demand a large amount of training data, which can be time-consuming and laborious to collect in practice. Thus, the second and third contributions propose two frameworks for minimizing the burden incurred by the data collection process. Specifically, a framework is proposed for learning the data-driven model on synthetic data from simulation. Parameters of the simulation model are identified by solving an optimization problem using the differential evolution algorithm with only a few trajectories of a real DLO required. This dissertation also proposes a trial-and-error interaction approach inspired by model-based reinforcement learning, which significantly reduces the need for training data and automates the data collection process.

The above contributions rely on artificial markers for tracking the DLO state during data collection and closed-loop control, which is acknowledged as a limitation. To address this, the fourth contribution proposes a novel approach that utilizes a particle filter within a low-dimensional state embedding learned by an autoencoder. This approach achieves robust tracking under occlusion and eliminates the need for high-fidelity physics simulations or manually designed constraints. Furthermore, the particle-filter-based method is employed and extended to track the state of a branched deformable linear object (BDLO), which is more challenging because of its complex branched structure. The proposed approach learns a likelihood prediction function directly from depth images in simulation, without requiring segmented point sets of the BDLO.

In conclusion, with the proposed methods for modeling and tracking DLOs, this dissertation contributes to advancing a broad range of applications, including DLO simulation, tracking, and manipulation. The development of these approaches lays the foundations for various directions of future research, which are further discussed in the dissertation.

Place, publisher, year, edition, pages
Örebro: Örebro University, 2023. p. 65
Series
Örebro Studies in Technology, ISSN 1650-8580 ; 99
Keywords
Deformable Linear Object, Model Learning, Model-based Control, Tracking, Robustness
National Category
Computer Sciences
Identifiers
urn:nbn:se:oru:diva-107846 (URN)9789175295220 (ISBN)
Public defence
2023-10-13, Örebro universitet, Långhuset, Hörsal L2, Fakultetsgatan 1, Örebro, 13:15 (English)
Opponent
Supervisors
Available from: 2023-08-25 Created: 2023-08-25 Last updated: 2023-09-20Bibliographically approved

Open Access in DiVA

Learn to Predict Posterior Probability in Particle Filtering for Tracking Deformable Linear Objects(852 kB)189 downloads
File information
File name FULLTEXT01.pdfFile size 852 kBChecksum SHA-512
ee811e7886c13e9b06eb1f81119ace54b0cbf498058d85aba40dcee78671d8e6535d62bd2ccb4d9ab6c6046d401b6fdfc126135460414e11940ba1e7921af3cd
Type fulltextMimetype application/pdf

Other links

3rd Workshop on Robotic Manipulation of Deformable Objects. Contributed papers

Authority records

Yang, YuxuanStork, Johannes AndreasStoyanov, Todor

Search in DiVA

By author/editor
Yang, YuxuanStork, Johannes AndreasStoyanov, Todor
By organisation
School of Science and Technology
Computer Vision and Robotics (Autonomous Systems)

Search outside of DiVA

GoogleGoogle Scholar
Total: 189 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 401 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf