The lab is run by Yonatan Bisk and is housed primarily in the Language Technologies Institute, but includes members from the Machine Learning Department, Robotics Institute, and Computer Science Dept.
PhD Students
@SoYeonTiffMin she/her
(co- Rus Salakhutdinov)
Embodied-RAG: General non-parametric Embodied Memory for Retrieval and Generation
Situated Instruction Following
Tools Fail: Detecting Silent Errors in Faulty Tools
AgentKit: Flow Engineering with Graphs, not Coding
SPRING: GPT-4 Out-performs RL Algorithms by Studying Papers and Reasoning
Plan, Eliminate, and Track -- Language Models are Good Teachers for Embodied Agents
EXCALIBUR: Encouraging and Evaluating Embodied Exploration
Self-Supervised Object Goal Navigation with In-Situ Finetuning
Tackling AlfWorld with Action Attention and Common Sense from Language Models
Don't Copy the Teacher: Data and Model Challenges in Embodied Dialogue
FILM: Following Instructions in Language with Modular Methods
Embodied-RAG: General non-parametric Embodied Memory for Retrieval and Generation
Situated Instruction Following
Tools Fail: Detecting Silent Errors in Faulty Tools
AgentKit: Flow Engineering with Graphs, not Coding
SPRING: GPT-4 Out-performs RL Algorithms by Studying Papers and Reasoning
Plan, Eliminate, and Track -- Language Models are Good Teachers for Embodied Agents
EXCALIBUR: Encouraging and Evaluating Embodied Exploration
Self-Supervised Object Goal Navigation with In-Situ Finetuning
Tackling AlfWorld with Action Attention and Common Sense from Language Models
Don't Copy the Teacher: Data and Model Challenges in Embodied Dialogue
FILM: Following Instructions in Language with Modular Methods
@viddivj she/her
ANAVI: Audio Noise Awareness by Visual Interaction
Vid2Robot: End-to-end Video-conditioned Policy Learning with Cross-Attention Transformers
Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis
Spatial-Language Attention Policies for Efficient Robot Learning
HomeRobot: Open-Vocabulary Mobile Manipulation
Towards Open-World Mobile Manipulation in Homes: Lessons from the Neurips 2023 HomeRobot Open Vocabulary Mobile Manipulation Challenge
MAEA: Multimodal Attribution for Embodied AI
Transformers are Adaptable Task Planners
ANAVI: Audio Noise Awareness by Visual Interaction
Vid2Robot: End-to-end Video-conditioned Policy Learning with Cross-Attention Transformers
Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis
Spatial-Language Attention Policies for Efficient Robot Learning
HomeRobot: Open-Vocabulary Mobile Manipulation
Towards Open-World Mobile Manipulation in Homes: Lessons from the Neurips 2023 HomeRobot Open Vocabulary Mobile Manipulation Challenge
MAEA: Multimodal Attribution for Embodied AI
Transformers are Adaptable Task Planners
@_Yingshan she/her
Tools Fail: Detecting Silent Errors in Faulty Tools
Language Models Need Inductive Biases to Count Inductively
DiffusionPID: Interpreting Diffusion via Partial Information Decomposition
VISREAS: Complex Visual Reasoning with Unanswerable Questions
Skews in the Phenomenon Space Hinder Generalization in Text-to-Image Generation
Training Vision-Language Transformers from Captions
WebQA: Multihop and Multimodal QA
Tools Fail: Detecting Silent Errors in Faulty Tools
Language Models Need Inductive Biases to Count Inductively
DiffusionPID: Interpreting Diffusion via Partial Information Decomposition
VISREAS: Complex Visual Reasoning with Unanswerable Questions
Skews in the Phenomenon Space Hinder Generalization in Text-to-Image Generation
Training Vision-Language Transformers from Captions
WebQA: Multihop and Multimodal QA
Masters, Independent Studies, and Visitors
Cats
Star 🌟
she/her
Comet 💫
he/him
Alumni, Undergraduates, and Visitors
| 2024 | Hao Zhu | PhD | → Postdoc @ Stanford |  |
| 2024 | Minyoung Hwang | Intern | → PhD @ MIT | |
| 2022 | Xiang Fan | Undergraduate | → PhD @ UWashington | |
| 2022 | Andy Liu | Intern | → PhD @ CMU | |
| 2022 | Deogratias Amani | Intern | → | |
| 2022 | Josh Zhanson | MLT | → Data Scientist @ MSFT | |
| 2022 | Xiaoyu (Erin) Zhang | 5th year MS | → PhD @ UIUC | |
| 2021 | Feiyu (Gavin) Zhu | Undergrad | → PhD @ CMU | |
| 2021 | Blessing Bassey | Intern | → | |
| 2021 | Melanie Sclar | Intern | → PhD @ UWashington | |