Hands are essential to human interaction, and understanding contact between
hands and the world can promote comprehensive understanding of their function.
Recently, there have been growing number of hand interaction datasets that
cover interaction with object, other hand, scene, and body. Despite the
significance of the task and increasing high-quality data, how to effectively
learn dense hand contact estimation remains largely underexplored. There are
two major challenges for learning dense hand contact estimation. First, there
exists class imbalance issue from hand contact datasets where majority of
samples are not in contact. Second, hand contact datasets contain spatial
imbalance issue with most of hand contact exhibited in finger tips, resulting
in challenges for generalization towards contacts in other hand regions. To
tackle these issues, we present a framework that learns dense HAnd COntact
estimation (HACO) from imbalanced data. To resolve the class imbalance issue,
we introduce balanced contact sampling, which builds and samples from multiple
sampling groups that fairly represent diverse contact statistics for both
contact and non-contact samples. Moreover, to address the spatial imbalance
issue, we propose vertex-level class-balanced (VCB) loss, which incorporates
spatially varying contact distribution by separately reweighting loss
contribution of each vertex based on its contact frequency across dataset. As a
result, we effectively learn to predict dense hand contact estimation with
large-scale hand contact data without suffering from class and spatial
imbalance issue. The codes will be released.
