1 Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA. [email protected].
2 McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, USA. [email protected].
3 Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA.
4 McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, USA.
5 Computer Science & Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA.
6 MIT-IBM Watson AI Lab, Cambridge, MA, USA.
7 Quest for Intelligence, Massachusetts Institute of Technology, Cambridge, MA, USA.
8 Neuro-X Institute, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland.
9 Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN, USA.
10 Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA. [email protected].
11 McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, USA. [email protected].
12 Program in Speech and Hearing Bioscience and Technology, Harvard University, Cambridge, MA, USA. [email protected].
Item in Clipboard
Greta Tuckute et al.
Nat Hum Behav.
.
Display options
Format
doi: 10.1038/s41562-023-01783-7.
Online ahead of print.
Affiliations
1 Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA. [email protected].
2 McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, USA. [email protected].
3 Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA.
4 McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, USA.
5 Computer Science & Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, USA.
6 MIT-IBM Watson AI Lab, Cambridge, MA, USA.
7 Quest for Intelligence, Massachusetts Institute of Technology, Cambridge, MA, USA.
8 Neuro-X Institute, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland.
9 Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN, USA.
10 Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA. [email protected].
11 McGovern Institute for Brain Research, Massachusetts Institute of Technology, Cambridge, MA, USA. [email protected].
12 Program in Speech and Hearing Bioscience and Technology, Harvard University, Cambridge, MA, USA. [email protected].
Item in Clipboard
Display options
Format
Abstract
Transformer models such as GPT generate human-like language and are predictive of human brain responses to language. Here, using functional-MRI-measured brain responses to 1,000 diverse sentences, we first show that a GPT-based encoding model can predict the magnitude of the brain response associated with each sentence. We then use the model to identify new sentences that are predicted to drive or suppress responses in the human language network. We show that these model-selected novel sentences indeed strongly drive and suppress the activity of human language areas in new individuals. A systematic analysis of the model-selected sentences reveals that surprisal and well-formedness of linguistic input are key determinants of response strength in the language network. These results establish the ability of neural network models to not only mimic human language but also non-invasively control neural activity in higher-level cortical areas, such as the language network.
Fedorenko, E., Hsieh, P.-J., Nieto-Castañón, A., Whitfield-Gabrieli, S. & Kanwisher, N. New method for fMRI investigations of language: defining ROIs functionally in individual subjects. J. Neurophysiol. 104, 1177–1194 (2010).
MacSweeney, M. et al. Neural systems underlying British Sign Language and audio-visual English processing in native users. Brain J. Neurol. 125, 1583–1593 (2002).
