CORSMAL challenge: A/V object classification for human-robot collaboration

(The Zoom link will be provided upon completion of the Eventbrite registration)

Title: The CORSMAL challenge: Audio-visual object classification for human-robot collaboration

Description

CORSMAL organise a 2-hour virtual workshop on the results of the challenge Audio-visual Object Classification for Human-Robot Collaboration. The workshop features live presentations by the authors of the accepted papers at the 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (IEEE ICASSP2022), an open discussion with Q&A, and the award of the challenge winner.

The CORSMAL challenge is about understanding from audio-visual data different physical properties of household containers for food and drinks. These containers are manipulated by a person prior to a human-robot collaboration and could be relevant for assistive scenarios at home or in a workplace. Containers vary in their shape, size, weight, or appearance, or they can be occluded by the human hand during manipulation. The challenge provides a benchmarking framework that supports the design and evaluation of novel audio-visual solutions for the estimation of the physical properties of containers in these challenging scenarios. The solutions should be accurate, robust, safe, and adaptive to changing conditions in the context of human-to-robot handovers.

The CORSMAL challenge focuses on the estimation of the capacity, dimensions, and mass of containers, the type, mass, and filling (percentage of the container with content), and the overall mass of the container and filling. The specific containers and fillings are unknown to the robot: the only prior is a set of object categories (drinking glasses, cups, food boxes) and a set of filling types (water, pasta, rice). The challenge also provides a real-to-simulation framework that visualizes the safeness and assesses the physical properties estimated by the participants to the challenge into a simulated human-to-robot handover scenario.

Programme

• 11:45: Virtual doors opening
• 12:00: Welcome and introduction: Collaborative Object Recognition, Shared Manipulation and Learning
• 12:15: Audio-Visual Object Classification For Human-Robot Collaboration
• 12:30: Container localisation and mass estimation with an RGB-D camera
• 12:45: Shared Transformer Encoder with Mask-based Volume Estimation for Filling Mass Estimation
• 13:00: Improving generalization of deep networks for estimating physical properties of containers and fillings
• 13:15: Q&A with the speakers and discussion
• 13:45: Challenge leaderboards and award
• 13:55: Closing

Host: Alessio Xompero, Centre for Intelligent Sensing, Queen Mary University of London

Know what’s Happening Next — before everyone else does.

Get App

Host or Publisher Centre for Intelligent Sensing