Visual Recognition beyond Appearances, and its Robotic Applications

1 111
Published on 17 Sep 2021, 0:30
The goal of Computer Vision, as coined by Marr, is to develop algorithms to answer What are Where at When from visual appearance. The speaker, among others, recognizes the importance of studying underlying entities and relations beyond visual appearance, following an Active Perception paradigm. This talk will present the speaker's efforts over the last decade, ranging from 1) reasoning beyond appearance for visual question answering, image/video captioning tasks, and their evaluation, through 2) temporal and self-supervised knowledge distillation with incremental knowledge transfer, till 3) their roles in a Robotic visual learning framework via a Robotic Indoor Object Search task. The talk will also feature the Active Perception Group (APG)’s ongoing projects (NSF RI, NRI and CPS, DARPA KAIROS, and Arizona IAM) addressing emerging challenges of the nation in autonomous driving and AI security domains, at the ASU School of Computing, Informatics, and Decision Systems Engineering (CIDSE).

Speaker: Yezhou Yang, School of Computing, Informatics, and Decision Systems Engineering, Arizona State University

List of major papers covered in the talk:

V&L model robustness:
ECCV 2020: VQA-LOL: Visual Question Answering under the Lens of Logic
ACL 2021: SMURF: SeMantic and linguistic UndeRstanding Fusion for Caption Evaluation via Typicality Analysis
EMNLP 2020: MUTANT: A Training Paradigm for Out-of-Distribution Generalization in Visual Question Answering Video2Commonsense: Generating Commonsense Descriptions to Enrich Video Captioning

Robotic object search:
CVPR 2021: Hierarchical and Partially Observable Goal-driven Policy Learning with Goals Relational Graph
ICRA 2021/RA-L: Efficient Robotic Object Search via HIEM: Hierarchical Policy Learning with Intrinsic-Extrinsic Modeling

Other teasers:
AI security/GAN attribution
ICLR 2021: Decentralized Attribution of Generative Models
AAAI 2021: Attribute-Guided Adversarial Training for Robustness to Natural Perturbations

Microsoft Research Deep Learning team: