ECE Seminar Series on Modern AI — José Álvarez

Name: ECE Seminar Series on Modern AI — José Álvarez
Start: 2024-10-08T11:30:00-04:00
End: 2024-10-08T12:30:00-04:00
Location: 370 Jay Street, Room 1201

Schedule

Tue Oct 08 2024 at 11:30 am to 12:30 pm

UTC-04:00

Location

370 Jay Street, Room 1201 | New York, NY

This event is part of the NYU Tandon ECE Seminar Series on Modern AI, featuring José Álvarez, director at NVIDIA
https://engineering.nyu.edu/events/2024/10/08/towards-robust-and-reliable-autonomous-vehicles-foundation-models
Zoom Link: https://nyu.zoom.us/meeting/register/tJItcemsqzkoE9TL9Nlymzg0M5kd_NbzxIuB#/registration
Abstract:
In this talk, I will present our latest progress in vision-centric and data-driven methods for autonomous vehicles. First, I will introduce Hydra, our end-to-end architecture using a multi-target distillation paradigm, and then I will explain how we leverage MLLMs within the autonomous driving ecosystem. In the car, I will discuss Omnidrive, an LLM-Agent for driving with 3D perception, reasoning, and planning. I will introduce SSE in the cloud, a multimodal semantic data selection framework to scale up datasets for effectively training AV models. Finally, I will share some insights on the remaining challenges to achieving robust autonomous driving systems.
Bio:
José Álvarez is a director at NVIDIA, leading the perception research for the Autonomous Driving team. The focus of the team and his interests are the scalability of deep learning and its applicability to autonomous driving. Prior to NVIDIA, he was a research scientist at TRI and NICTA, and worked as a postdoc researcher at NYU under Yann LeCun. He did his PhD in computer vision prior to the Deep Learning revolution, working on road scene understanding for autonomous driving when datasets were scarce and small.
Overall, his work has been centered around dynamic scene understanding and focused on the applicability of AI in real world problems such as bionic vision, medicine, agriculture, and autonomous vehicles.