No other species possesses a social intelligence quite like that of humans. Our ability to understand one another’s minds and actions, and to interact with one another in rich and complex ways, is the basis for much of our success, from governments to symphonies to the scientific enterprise. This course will discuss the principles of human social cognition, how we can use machine learning and AI models to computationally capture these principles, how these principles can help us build human-level machine social intelligence, and how social intelligence can enable the engineering of AI systems that can understand and interact with humans safely and productively in real-world settings. In this seminar course, we will read and discuss literature that cover diverse topics on social intelligence in humans and machines. These include (but are not limited to) Theory of Mind, coordination, assistnace, communication, social learning, cultural transimission, and moral judgment.

Relation to Cognitive AI (EN.601.473/673): This course will specifically focus on advanced topics in social intelligence, whereas Cognitive AI is an introductory course on cognitive modeling for human-like AI. Students do not have to take Cognitive AI prior to this course.

Prerequisites: Linear Algebra, Probability and Statistics, and Calculus. ML/AI courses such as 601.475 (Machine Learning), EN.601.464. (Artificial Intelligence), or EN.601.473/673 (Cognitive AI). Students must be comfortable reading recent research papers and discussing key concepts and ideas.

Instructors

Tianmin Shu (tianmin.shu@jhu.edu)
Chuanyang Jin (Head TA) & Tianai Yue
(Teaching Assistants)
Acknowledgements Website template from Prof. Anjalie Field, Prof. Daniel Khashabi, and Prof. Ziang Xiao.

Schedule

The schedule and the readings are subject to change.

< <
Date Topic Readings Work Due
Jan 20 Introduction No Required Reading
Jan 22 Background: decision making No Required Reading
Jan 27 Background: decision making No Required Reading
Jan 29 Background: inverse decision making No Required Reading
Feb 3 Emergent social intelligence via MARL Main:
  1. Reward is Enough
  2. Human-level performance in 3D multiplayer games with population-based reinforcement learning
Suggested:
  1. Emergent Tool Use From Multi-Agent Autocurricula
  2. “Other-Play” for Zero-Shot Coordination
Reading Responses by 12 pm
Feb 5 Emergent social intelligence via LLMs Main:
  1. Generative Agents: Interactive Simulacra of Human Behavior
  2. SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents
Suggested:
  1. SOTOPIA-π: Interactive Learning of Socially Intelligent Language Agents
  2. Is this the real life? Is this just fantasy? The Misleading Success of Simulating Social Interactions With LLMs
 Reading Responses by 12 pm
Feb 10 The need for a human model Main:
  1. On the Utility of Learning about Humans for Human-AI Coordination
  2. Human-level play in the game of Diplomacy by combining language models with strategic reasoning
Suggested:
  1. Learning to Influence Human Behavior with Offline Reinforcement Learning
  2. Learning to Cooperate with Humans using Generative Agents
 Reading Responses by 12 pm
Feb 12 How can social cognition help? Main:
  1. Socially intelligent machines that learn from humans and help humans learn
  2. Building Machines that Learn and Think with People
  3. Socially intelligent robots: dimensions of human–robot interaction (Section 1-3)
 Reading Responses by 12 pm
Feb 17 Evaluating Theory of Mind in humans and machines Main:
  1. Baby Intuitions Benchmark (BIB): Discerning the goals, preferences, and actions of others
  2. Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models
Suggested:
  1. Understanding Social Reasoning in Language Models with Language Models
  2. Teleological reasoning in infancy: the naı̈ve theory of rational action
  3. AGENT: A Benchmark for Core Psychological Reasoning
 Reading Responses by 12 pm
Feb 19 Cognitive modeling for Theory of Mind Main:
  1. Theory of mind and inverse decision-making (Chapter 14 of Bayesian Models of Cognition: Reverse Engineering the Mind)
  2. Planning with Theory of Mind
Suggested:
  1. Action understanding as inverse planning
  2. Rational quantitative attribution of beliefs, desires and percepts in human mentalizing
  3. Computational Models of Emotion Inference in Theory of Mind: A Review and Roadmap
  4. Emotion prediction as computation over a generative theory of mind
  5. Intervening on Emotions by Planning Over a Theory of Mind
  6. Human-like Affective Cognition in Foundation Models
 Reading Responses by 12pm
Feb 24Pragmatic reasoning Main:
  1. Pragmatics in Language Grounding: Phenomena, Tasks, and Modeling Approaches
  2. Pragmatic Language Interpretation as Probabilistic Inference
Suggested:
  1. Learning to refer informatively by amortizing pragmatic reasoning
  2. Reasoning about Pragmatics with Neural Listeners and Speakers
  3. A fine-grained comparison of pragmatic language understanding in humans and language models
 Reading Responses by 12 pm
Feb 26Instruction following Main:
  1. HandMeThat: Human-Robot Communication in Physical and Social Environments
  2. Pragmatic Instruction Following and Goal Assistance via Cooperative Language-Guided Inverse Planning
Suggested:
  1. Situated Instruction Following
  2. Learning to communicate about shared procedural abstractions
 Reading Responses by 12 pm
Mar 3 Multi-agent planning and Theory of Minds Main:
  1. Too many cooks: Bayesian inference for coordinating multi-agent collaboration
  2. Theory of Minds: Understanding Behavior in Groups Through Inverse Planning
Suggested:
  1. Coordinate to cooperate or compete: Abstract goals and joint intentions in social interaction
  2. Goal Inference Improves Objective and Perceived Performance in Human-Robot Collaboration
 Reading Responses by 12pm
Mar 5 Proactive assistance Main:
  1. AvE: Assistance via Empowerment
  2. COOPERA: Continual Open-Ended Human-Robot Assistance
Suggested:
  1. The Lumiere Project: Bayesian User Modeling for Inferring the Goals and Needs of Software Users
  2. NOPA: Neurally-guided Online Probabilistic Assistance for Building Socially Intelligent Home Assistants
  3. Proactive Robot Assistance via Spatio-Temporal Object Modeling
 Reading Responses by 12pm; Project Proposal by Mar 8th, 11:59 pm
Mar 10 Understanding suboptimal behavior Main:
  1. Online Bayesian Goal Inference for Boundedly-Rational Planning Agents
  2. Where Do You Think You're Going?: Inferring Beliefs about Dynamics from Behavior
Suggested:
  1. Modeling the Mistakes of Boundedly Rational Agents Within a Bayesian Theory of Mind
  2. Explainable Procedural Mistake Detection
 Reading Responses by 12pm
Mar 12 Cogntivie models meet foundation models Main:
  1. Discovering Symbolic Cognitive Models from Human and Animal Behavior
  2. AutoToM: Scaling Model-based Mental Inference via Automated Agent Modeling
Suggested:
  1. Hypothesis-Driven Theory-of-Mind Reasoning for Large Language Models
  2. Hypothetical Minds: Scaffolding Theory of Mind for Multi-Agent Tasks with Large Language Models
  3. Towards Automation of Cognitive Modeling using Large Language Models
  4. Human-like Few-Shot Learning via Bayesian Reasoning over Natural Language
  5. From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought (Section 3.3)
 Reading Responses by 12pm
Mar 17 Spring Break
Mar 19 Spring Break
Mar 24 Nonverbal communication Main:
  1. Planning for Autonomous Cars that Leverage Effects on Human Actions
  2. Gesture-Informed Robot Assistance via Foundation Models
Suggested:
  1. The eyes have it: the neuroethology, function and evolution of social gaze
  2. Legibility and predictability of robot motion
  3. Emergence of Grounded Compositional Language in Multi-Agent Populations
  4. Social Eye Gaze in Human-Robot Interaction: A Review
 Reading Responses by 12pm
Mar 26 Cooperative verbal communication Main:
  1. RoCo: Dialectic Multi-Robot Collaboration with Large Language Models
  2. Cooperative Explanation as Rational Communication
Suggested:
  1. GOMA: Proactive Embodied Cooperative Communication via Goal-Oriented Mental Alignment
  2. Building cooperative embodied agents modularly with large language models
  3. Towards Collaborative Plan Acquisition through Theory of Mind Modeling in Situated Dialogue
  4. Mutual Theory of Mind
 Reading Responses by 12pm; midway progress report by Mar 30th, 11:59 pm
Mar 31 Reinforcement learning from human feedback Main:
  1. Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
  2. Causal Confusion and Reward Misidentification in Preference-Based Reward Learning
Suggested:
  1. The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Models
  2. Human-centric dialog training via offline reinforcement learning
 Reading Responses by 12pm
Apr 2 Human social learning Main:
  1. Inferential social learning: cognitive foundations of human social learning and teaching
  2. Natural pedagogy
Suggested:
  1. Adaptive Social Learning using Theory of Mind
 Reading Responses by 12pm
Apr 7 Machine social learning Main:
  1. Cooperative Inverse Reinforcement Learning
  2. Language and Experience: A Computational Model of Social Learning in Complex Tasks
Suggested:
  1. Yell At Your Robot: Improving On-the-Fly from Language Corrections
  2. How to talk so AI will learn: Instructions, descriptions, and autonomy
  3. Pragmatic Feature Preferences: Learning Reward-Relevant Preferences from Human Input
  4. Vocal Sandbox: Continual Learning and Adaptation for Situated Human-Robot Collaboration
  5. Understanding Teacher Gaze Patterns for Robot Learning
  6. Pragmatic-Pedagogic Value Alignment
  7. Learning Robot Objectives from Physical Human Interaction
  8. On Using Social Signals to Enable Flexible Error-Aware HRI
 Reading Responses by 12pm
Apr 9 Collaborative multi-agent problem solving Main:
  1. CooperBench: Why Coding Agents Cannot be Your Teammates Yet
  2. Language Model Teams as Distributed Systems
Suggested:
  1. Improving Factuality and Reasoning in Language Models through Multiagent Debate
  2. Talk Isn't Always Cheap: Understanding Failure Modes in Multi-Agent Debate
  3. τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environment
  4. The Virtual Lab of AI agents designs new SARS-CoV-2 nanobodies
 Reading Responses by 12pm
Apr 14 Cultural learning/transmission Main:
  1. Cultural Learning (Section 1-2)
  2. Learning few-shot imitation as cultural transmission
 Reading Responses by 12pm
Apr 16 Moral decison making Main:
  1. Computational ethics
  2. When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment
Suggested:
  1. Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties
  2. The logic of universalization guides moral judgment
 Reading Responses by 12pm
Apr 21 Project presentation
Apr 23 Project presentation Final report due by May 3rd, 11:59 pm

Policies

Attendance policy This is a graduate-level course revolving around in-person discussion. Students are expected to attend class and may notify instructors if there are extenuating circumstances.

Course Conduct This is a discussion class focused on cutting-edge research. All students are expected to respect everyone's perspective and input and to contribute towards creating a welcoming and inclusive climate. We the instructors will strive to make this classroom an inclusive space for all students, and we welcome feedback on ways to improve.

Academic Integrity This course will have a zero-tolerance philosophy regarding plagiarism or other forms of cheating, and incidents of academic dishonesty will be reported. A student who has doubts about how the Honor Code applies to this course should obtain specific guidance from the course instructor before submitting the respective assignment.

AI Use Policy All written reponses and presentations must be prepared by the students without the help of AI. It is okay to use AI in the projects (for coding, model development and evaluation, and report editing). However, the students cannot use the AI to directly produce the project proposal, presentations, and the final reports.

Discrimination and Harrasment The Johns Hopkins University is committed to equal opportunity for its faculty, staff, and students. To that end, the university does not discriminate on the basis of sex, gender, marital status, pregnancy, race, color, ethnicity, national origin, age, disability, religion, sexual orientation, gender identity or expression, veteran status, military status, immigration status or other legally protected characteristic. The University's Discrimination and Harassment Policy and Procedures provides information on how to report or file a complaint of discrimination or harassment based on any of the protected statuses listed in the earlier sentence, and the University’s prompt and equitable response to such complaints.

Personal Well-being Take care of yourself! Being a student can be challenging and your physical and mental health is important. If you need support, please seek it out. Here are several of the many helpful resources on campus: