Social Interactions as Recursive MDPs

Ravi Tejwani*, Yen-Ling Kuo*, Tianmin Shu, Boris Katz, and Andrei Barbu.

5th Conference on Robot Learning (CoRL), 2021.

Paper   Project

A Unified Psychological Space for Human Perception of Physical and Social Events

Tianmin Shu, Yujia Peng, Song-Chun Zhu, and Hongjing Lu.

Cognitive Psychology, 128: 101398, 2021.

Paper   Code (Stimuli Generation)   Code (Model)   Data

AGENT: A Benchmark for Core Psychological Reasoning

Tianmin Shu, Abhishek Bhandwaldar, Chuang Gan, Kevin A. Smith, Shari Liu, Dan Gutfreund, Elizabeth Spelke, Joshua B. Tenenbaum, and Tomer D. Ullman.

38th International Conference on Machine Learning (ICML), 2021.

Paper   Supp   Project Code (Data Generation)

Watch-And-Help: A Challenge for Social Perception and Human-AI Collaboration

Xavier Puig, Tianmin Shu, Shuang Li, Zilin Wang, Joshua B. Tenenbaum, Sanja Fidler, and Antonio Torralba.

9th International Conference on Learning Representations (ICLR), 2021. (Spotlight presentation)

(Best Paper Award at NeurIPS Cooperative AI Workshop, 2020)

Paper   Project   Code

PHASE: PHysically-grounded Abstract Social Events for Machine Social Perception

Aviv Netanyahu*, Tianmin Shu*, Boris Katz, Andrei Barbu, and Joshua B. Tenenbaum.

35th AAAI Conference on Artificial Intelligence (AAAI), 2021.

(Best Paper Award at NeurIPS Shared Visual Representations in Human and Machine Intelligence Workshop, 2020)

Paper   Supp   Project

Exploring Biological Motion Perception in Two-stream Convolutional Neural Networks

Yujia Peng, Hannah Lee, Tianmin Shu, and Hongjing Lu

Vision Research, 178: 28-40, 2021.

Paper

Adventures in Flatland: Perceiving Social Interactions Under Physical Dynamics

Tianmin Shu, Marta Kryven, Tomer D. Ullman, and Joshua B. Tenenbaum.

42nd Annual Meeting of the Cognitive Science Society (CogSci), 2020.

Paper   Bibtex

Joint Mind Modeling for Explanation Generation in Complex Human-Robot Collaborative Tasks

Xiaogeng Gao*, Ran Gong*, Yizhou Zhao, Shu Wang, Tianmin Shu, and Song-Chun Zhu.

International Conference on Robot & Human Interactive Communication (RO-MAN), 2020.

Paper   Project   Code

Active Visual Information Gathering for Vision-Language Navigation

Hanqing Wang, Wenguan Wang, Tianmin Shu, Wei Liang, and Jianbing Shen.

European Conference on Computer Vision (ECCV), 2020.

Paper   Code

Learning to Infer Human Attention in Daily Activities

Zhixiong Nan, Tianmin Shu, Ran Gong, Shu Wang, Ping Wei, Song-Chun Zhu, and Nanning Zheng.

Pattern Recognition, 103: 107314, 2020.

Paper

Partitioning the Perception of Physical and Social Events Within a Unified Psychological Space

Tianmin Shu, Yujia Peng, Hongjing Lu, and Song-Chun Zhu.

41st Annual Meeting of the Cognitive Science Society (CogSci), 2019.

(Oral presentation)

Paper   Project   Bibtex

VRKitchen: an Interactive 3D Environment for Learning Real Life Cooking Tasks

Xiaofeng Gao, Ran Gong, Tianmin Shu, Xu Xie, Shu Wang, and Song-Chun Zhu.

ICML workshop on Reinforcement Learning for Real Life, 2019.

Paper   Project   Code   Media   Bibtex

M^3RL: Mind-aware Multi-agent Management Reinforcement Learning

Tianmin Shu and Yuandong Tian.

7th International Conference on Learning Representations (ICLR), 2019.

Paper   Code   Bibtex

Interactive agent modeling by learning to probe

Tianmin Shu, Caiming Xiong, Ying Nian Wu, and Song-Chun Zhu.

NeurIPS Deep Reinforcement Learning Workshop, 2018.

Paper

Where and Why Are They Looking? Jointly Inferring Human Attention and Intentions in Complex Tasks

Ping Wei, Yang Liu, Tianmin Shu, Nanning Zheng, and Song-Chun Zhu.

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.

Paper   Bibtex

Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learning

Tianmin Shu, Caiming Xiong, and Richard Socher.

6th International Conference on Learning Representations (ICLR), 2018.

Paper   Media   Bibtex

Learning and Inferring "Dark Matter"
and Predicting Human Intents and Trajectories in Videos

Dan Xie, Tianmin Shu, Sinisa Todorovic, and Song-Chun Zhu.

IEEE Trans. on Pattern Analysis and Machine Intelligence (TPAMI), 40(7): 1639-1652, 2018.

Paper   Project   Bibtex

Perception of Human Interaction Based on Motion Trajectories: from Aerial Videos to Decontextualized Animations

Tianmin Shu*, Yujia Peng*, Lifeng Fan, Hongjing Lu, and Song-Chun Zhu.

Topics in Cognitive Science, 10(1): 225 - 241, 2018.

Paper   Bibtex

Inferring Human Interaction from Motion Trajectories in Aerial Videos

Tianmin Shu*, Yujia Peng*, Lifeng Fan, Hongjing Lu, and Song-Chun Zhu.

39th Annual Meeting of the Cognitive Science Society (CogSci), 2017.

(Computational Modeling Prize)

Paper   Project   Bibtex

CERN: Confidence-Energy Recurrent Network for Group Activity Recognition

Tianmin Shu, Sinisa Todorovic, and Song-Chun Zhu.

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.

Paper   Supp   Bibtex

Learning Social Affordance Grammar from Videos:
Transferring Human Interactions to Human-Robot Interactions

Tianmin Shu, Xiaofeng Gao, Michael S. Ryoo, and Song-Chun Zhu.

IEEE International Conference on Robotics and Automation (ICRA), 2017.

Paper   Project   Slides   Media   Bibtex

Critical Features of Joint Actions that Signal Human Interaction

Tianmin Shu*, Steven Thurman*, Dawn Chen, Song-Chun Zhu, and Hongjing Lu.

38th Annual Meeting of the Cognitive Science Society (CogSci), 2016.

Paper   Bibtex

Learning Social Affordance for Human-Robot Interaction

Tianmin Shu, M. S. Ryoo, and Song-Chun Zhu.

25th International Joint Conference on Artificial Intelligence (IJCAI), 2016.

Paper   Project   Slides   Bibtex

Joint Inference of Groups, Events and Human Roles in Aerial Videos

Tianmin Shu, Dan Xie, Brandon Rothrock, Sinisa Todorovic, and Song-Chun Zhu.

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.

(Oral presentation, acceptance rate: 71/2123 = 3.3%)

Paper   Project   Supp  Slides   Bibtex