Related Researcher

Author's Photo

Kim, Kwang In
Machine Learning and Vision Group
Research Interests
  • Neural networks, semi-supervised learning; unsupervised learning; learning on Riemannian manifolds and graph-structured data; human body pose estimation; human hand pose estimation; image and video enhancement.


S2Contact: Graph-based network for 3D hand-object contact estimation with semi-supervised learning

DC Field Value Language Tse, Tze Ho Elden ko Zhang, Zhongqun ko Kim, Kwang In ko Leonardis, Aleš ko Zheng, Feng ko Chang, Hyung Jin ko 2022-10-05T02:35:26Z - 2022-10-03 ko 2022-10-26 ko
dc.identifier.citation European Conference on Computer Vision ko
dc.identifier.uri -
dc.description.abstract Despite the recent efforts in accurate 3D annotations in hand and object datasets, there still exist gaps in 3D hand and object reconstructions. Existing works leverage contact maps to refine inaccurate hand-object pose estimations and generate grasps given object models. However, they require explicit 3D supervision which is seldom available and therefore, are limited to constrained settings, e.g., where thermal cameras observe residual heat left on manipulated objects. In this paper, we propose a novel semi-supervised framework that allows us to learn contact from monocular images. Specifically, we leverage visual and geometric consistency constraints in large-scale datasets for generating pseudo-labels in semi-supervised learning and propose an efficient graph-based network to infer contact. Our semi-supervised learning framework achieves a favourable improvement over the existing supervised learning methods trained on data with ‘limited’ annotations. Notably, our proposed model is able to achieve superior results with less than half the network parameters and memory access cost when compared with the commonly-used PointNet-based approach. We show benefits from using a contact map that rules hand-object interactions to produce more accurate reconstructions. We further demonstrate that training with pseudo-labels can extend contact map estimations to out-of-domain objects and generalise better across multiple datasets. Project page is available. ko
dc.language 영어 ko
dc.publisher ECCV 2022 ko
dc.title S2Contact: Graph-based network for 3D hand-object contact estimation with semi-supervised learning ko
dc.type CONFERENCE ko
dc.identifier.scopusid 2-s2.0-85142760771 ko
dc.type.rims CONF ko
dc.identifier.url ko
Appears in Collections:
AI_Conference Papers

find_unist can give you direct access to the published full text of this article. (UNISTARs only)

Show simple item record


  • mendeley


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.