Deep learning algorithms, natural language processing, and the brain, with Jean-Rémi King

In this episode, I talk with Masud Husain, Professor of Neurology and Cognitive Neuroscience at the University of Oxford, about his recent editorial ‘A mountain of small things’.Husain websiteHusain M. A mountain of small things. Brain 2024; 147: 739. [doi]

In this episode, I talk with Maaike Vandermosten, Associate Professor in the Department of Neurosciences at KU Leuven, about the neural basis of developmental dyslexia, and neuroplasticity in recovery from aphasia.Vandermosten websiteVanderauwera J, Wouters J, Vandermosten M, Ghesquière P. Early dynamics of white matter deficits in children developing dyslexia. Dev Cogn Neurosci 2017; 27: 69-77. [doi]Beelen C, Vanderauwera J, Wouters J, Vandermosten M, Ghesquière P. Atypical gray matter in children with dyslexia before the onset of reading instruction. Cortex 2019; 121: 399–413. [doi]Phan TV, Sima D, Smeets D, Ghesquière P, Wouters J, Vandermosten M. Structural brain dynamics across reading development: A longitudinal MRI study from kindergarten to grade 5. Hum Brain Mapp 2021; 42: 4497-509. [doi]Clercq PD, Gonsalves AR, Gerrits R, Vandermosten M. Individualized functional localization of the language and multiple demand network in chronic post-stroke aphasia. bioRxiv 2024; 2024.01.12.575350. [doi]

In this episode, I talk with Dorothy Bishop, Emeritus Professor of Developmental Neuropsychology at the University of Oxford, about her work on developmental langauge disorder and its neural basis.Bishop websiteBishop DVM. Comprehension in developmental language disorders. Dev Med Child Neurol. 1979;21:225-38. [doi]Bishop DVM, Snowling MJ, Thompson PA, Greenhalgh T, CATALISE consortium. CATALISE: A multinational and multidisciplinary Delphi consensus study: Identifying language impairments in children. PLoS One. 2016;11:e0158753. [doi]Bishop DVM, Snowling MJ, Thompson PA, Greenhalgh T, CATALISE‐2 consortium. Phase 2 of CATALISE: a multinational and multidisciplinary Delphi consensus study of problems with language development: Terminology. J Child Psychol Psychiatry. 2017;58:1068-80. [doi]Wilson AC, Bishop DVM. Resounding failure to replicate links between developmental language disorder and cerebral lateralisation. PeerJ. 2018;6:e4217. [doi]Bishop D. Rein in the four horsemen of irreproducibility. Nature. 2019;568:435. [doi]

In this episode, I talk with Rob Cavanaugh, Research Data Analyst at the Observational Health Data Sciences and Informatics Center at Northeastern University, about his dissertation ‘Determinants of multilevel discourse outcomes in anomia treatment for aphasia’.Cavanaugh websiteCavanaugh, R. Determinants of multilevel discourse outcomes in anomia treatment for aphasia. Doctoral dissertation, University of Pittsburgh. [dissertation]

In the episode, I talk with Jean-Rémi King, Research scientist and team leader at Meta AI, and Associate Researcher at CNRS, École Normale Supérieure, about three recent papers from his lab on deep learning algorithms, natural language processing, and the brain.King websiteMillet J, Caucheteux C, Orhan P, Boubenec Y, Gramfort A, Dunbar E, Pallier C, King J-R. Toward a realistic model of speech processing in the brain with self-supervised learning. In Advances in Neural Information Processing Systems (NeurIPS) 2022. [doi]Caucheteux C, King JR. Brains and algorithms partially converge in natural language processing. Commun Biol. 2022;5:134. [doi]Caucheteux C, Gramfort A, King JR. Evidence of a predictive coding hierarchy in the human brain listening to speech. Nat Hum Behav. 2023;7:430-41. [doi]

In this episode, I talk with Laura Gwilliams, soon-to-be Assistant Professor of Psychology, Neuroscience, and Data Science at Stanford University, about her recent paper ‘Neural dynamics of phoneme sequences reveal position-invariant code for content and order’.Gwilliams lab websiteGwilliams L, King JR, Marantz A, Poeppel D. Neural dynamics of phoneme sequences reveal position-invariant code for content and order. Nat Commun 2022; 13: 6606. [doi]

In this episode, I talk with Alexander Huth, Assistant Professor of Neuroscience and Computer Science at the University of Texas, Austin, about his work using functional imaging and advanced computational methods to model how the brain processes language and represents meaning.Huth lab websiteHuth AG, Nishimoto S, Vu AT, Gallant JL. A continuous semantic space describes the representation of thousands of object and action categories across the human brain. Neuron 2012; 76: 1210-24. [doi]Huth AG, de Heer WA, Griffiths TL, Theunissen FE, Gallant JL. Natural speech reveals the semantic maps that tile human cerebral cortex. Nature 2016; 532: 453-8. [doi]Jain S, Huth AG. Incorporating context into language encoding models for fMRI. Proceedings of the 32nd International Conference on Neural Information Processing Systems 2018, pp. 6629-38. [doi]Tang J, LeBel A, Jain S, Huth AG. Semantic reconstruction of continuous language from non-invasive brain recordings. Nat Neurosci in press. [doi]

In this episode, I talk with Liina Pylkkänen, Professor of Linguistics and Psychology at NYU, about her research program, and in particular her recent paper ‘Disentangling semantic composition and semantic association in the left temporal lobe’.Pylkkänen lab websiteLi J, Pylkkänen L. Disentangling semantic composition and semantic association in the left temporal lobe. J Neurosci 2021; 41: 6526-38. [doi]

In this episode, I talk with Eddie Chang, Professor of Neurological Surgery at the University of California, San Francisco, about his recent paper ‘Speech computations of the human superior temporal gyrus’.Chang lab websiteBhaya-Grossman I, Chang EF. Speech computations of the human superior temporal gyrus. Annu Rev Psychol 2022; 73: 79-102. [doi | pdf]Chang EF, Rieger JW, Johnson K, Berger MS, Barbaro NM, Knight RT. Categorical speech representation in human superior temporal gyrus. Nat Neurosci 2010; 13: 1428-33. [doi]Sjerps MJ, Fox NP, Johnson K, Chang EF. Speaker-normalized sound representations in the human auditory cortex. Nat Commun 2019; 10: 2465. [doi]Leonard MK, Baud MO, Sjerps MJ, Chang EF. Perceptual restoration of masked speech in human cortex. Nat Commun 2016; 7: 13619. [doi]Hamilton LS, Edwards E, Chang EF. A spatial map of onset and sustained responses to speech in the human superior temporal gyrus. Curr Biol 2018; 28: 1860-71. [doi]Oganian Y, Chang EF. A speech envelope landmark for syllable encoding in human superior temporal gyrus. Sci Adv 2019; 5: eaay6279. [doi]

In this episode, I talk with Olivia Leow, who experienced an awake craniotomy for resection of a brain tumor surrounded by language areas in her left posterior temporal lobe.Vanderbilt Brain Cancer Patient Assistance Fund, established by Olivia LeowDiachek E, Morgan VL, Wilson SM. Adaptive language mapping paradigms for presurgical language mapping. AJNR Am J Neuroradiol 2022; in press. [pdf]Wilson SM, Yen M, Eriksson DK. An adaptive semantic matching paradigm for reliable and valid language mapping in individuals with aphasia. Hum Brain Mapp 2018; 39: 3285-307. [doi]Yen M, DeMarco AT, Wilson SM. Adaptive paradigms for mapping phonological regions in individual participants. NeuroImage 2019; 189: 368-79. [doi]Wilson SM, Lam D, Babiak M, Perry D, Shih T, Hess CP, Berger MS, Chang EF. Transient aphasias after left hemisphere resective surgery. J Neurosurg 2015; 123: 581-93. [doi]McCarron A, Chavez A, Babiak MC, Berger MS, Chang EF, Wilson SM. Connected speech in transient aphasias after left hemisphere resective surgery. Aphasiology 2017; 31: 1266-81. [doi]Penfield W, Roberts L. Speech and Brain-Mechanisms. Princeton University Press; 1959. [book]Long Monday by John PrineClay Pigeons by John Prine