Responsable: Jérôme Sackur
E-mail : jerome.sackur@gmail.com
COGNITION ET NEUROSCIENCE
Perception visuelle anorthoscopique d’objets
Lorsqu’un objet (ou une scène visuelle) est vu au travers d’une fente (« slit viewing »), comme cela peut être le cas dans la vie réelle lorsque l’on perçoit une scène au travers d’un store ou d’une barrière par exemple , le système visuel est capable d’identifier et reconnaitre cet objet, bien que la même région de la rétine soit stimulée d’instant en instant. Quels sont les mécanismes qui permettent la reconstruction de cet objet et de le reconnaitre ? Quelle est la part de la dynamique du mouvement et des informations de formes dans ce processus ?
- Transfert inter hémisphérique de traitements visuels
La perception du champ visuel droit et gauche est assurée par les hémisphères gauche et droit, respectivement. Lorsqu’une stimulation visuelle franchit le méridien vertical, quelles sont informations, prédictions et traitements transférés d’un hémisphère à l’autre (vraisemblablement par l’intermédiaire du corps calleux qui réunit les deux hémisphères) ? Des observations informelles suggèrent que les traitements corticaux réalisés dans un hémisphère ne sont pas transférés à l’autre, et que les traitements visuels –en particulier ceux nécessaire à la perception du mouvement- doivent être recalculés. Le projet vise à quantifier cette observation et à identifier les constantes de temps mise en jeux.
- Oculomotricité et « Eye-Gaming »
l’étude de l’oculomotricité et de son répertoire (fixation, saccade, poursuite, vergence, activité pupillaire) est généralement réalisée en laboratoire avec des paradigmes stricts, mais peu attirants (faire 100 saccade vers une cible, par exemple). Le projet vise à collecter des données oculomotrices dans le cadre de « jeux sérieux » impliquant les mouvements oculaires (« Eye-Gaming ») et de déterminer si les mouvements oculaires mis en œuvre dans ces situations sont susceptibles d’apporter des informations pertinentes pour l’étude du système oculomoteur (et de « remplacer » ou compléter les paradigmes expérimentaux de laboratoire). A plus long terme, le projet vise à développer des bornes interactives utilisant les mouvements oculaires pour le pilotage d’interfaces, de collecter etd’ étudier ces mouvements oculaires sur une grande population (Cohorte).
- Sonification des mouvements oculaires
Bien que les yeux bougent sans arrêt (3 saccades par seconde), ces mouvements sont largement non-conscient, et souvent perturbés dans nombres de pathologies. Le projet vise à déterminer dans quelle mesure la sonification des mouvements oculaires (couplages mouvement oculaires/sons) est utile pour faciliter et améliorer le contrôle oculomoteur. A long terme ce projet vise à développer des applications cliniques pour le diagnostic et la remédiation de troubles oculomoteurs.
- La réponse pupillaire comme mesure de l'attention visuelle
La taille de la pupille varie en fonction de l'illumination de la scène, mais reflète aussi des fonctions cognitives telles que l'orientation de l'attention. En effet, lorsqu'on prête attention à un objet lumineux placé en périphérie, la pupille se rétracte par rapport à la situation où l'objet auquel on prête attention est sombre. L'oscillation de la luminance (sombre-clair) à une certaine fréquence induit des oscillations de la pupille (jusqu'à 3Hz environ). Il est possible de présenter différents objets dont la luminance oscille à différentes fréquences, et de déterminer celui auquel le sujet prête attention et regardant l'amplitude des fréquences dans une décomposition de Fourier du signal pupillaire - une technique appelée "frequency tagging". L'objectif de ce stage est de déterminer les contraintes spatiales de ce frequency tagging de manière à exploiter la réponse pupillaire comme mesure attentionnelle.
Ce stage comprend toutes ou une partie des étapes suivantes: la mise en place du protocole expérimental, programmation de l'expérience (matlab), formation à l'utilisation d'un oculomètre et aux méthodes psychophysiques pertinentes pour récolter les données, analyse des données.
- Développement d'un jeu pour évaluer les rythmes attentionnels des enfants
La capacité à rester concentré sur une même tâche varie considérablement dans la population générale, sans parler des pathologies de l'attention comme le Trouble Déficit de l'Attention / Hyperactivité (TDAH). Une manière traditionnelle d'étudier cela en sciences cognitives consiste à questionner à intervalles irréguliers des participants pendant qu'ils et elles effectuent une tâche (la lecture d'un texte rébarbatif par exemple) afin de déterminer si et quand leur pensée vagabonde. Le projet serait la première étape d'un autre approche, consistant à déterminer pour chaque sujet un "biais d'alternance" entre deux tâches. Pour ce faire, il faudrait concevoir une tâche bi-partite qui pourrait prendre ensuite la forme d'un jeu vidéo, dans laquelle les participants devraient répartir leur attention sur deux sous-tâches (deux parties de l'écran). Il faudrait concevoir une contrainte telle que, en fonction des performances de chacun-e, nous puissions déterminer le taux d'alternance optimal, qui ne serait pas accessible au sujet. Nous pourrons alors mesurer le biais de chaque sujet par rapport à ce taux optimal.
Les étapes du stage sont : 1/ définir les contraintes du jeu ; 2/ effectuer des simulations pour tester leur robustesse ; 3/ concevoir et analyser une expérience de laboratoire pour valider la mesure du biais ; 4/ réaliser la version « jeu vidéo ».
Pré-requis: programmation.
Les projets suivants sont en lien avec l'équipe d'Emmanuel Dupoux « The Synthetic Language Learner ». Ils demandent de bonnes connaissances en algèbre linéaire ou statistique ainsi qu'en programmation.
- Deep language learning from scratch
Deep Neural Networks (DNNs) have recently broken ground on state-of-the-art in several areas (image recognition, speech recognition, etc.) . However, these algorithms depend on large human annotated datasets. Yet, infants spontaneously achieve similar performance without direct supervision; the internship explores various ideas to 'de-supervise' deep learning using side information, loss functions or architectures inspired by research in human infants .
- Learning the laws of physics with a deep recurrent network
Recurrent networks can be used to learn regularities in video or audio sequences . This internship will use a game engine to learn the underlying physical regularities of interactions between macroscopic objects and compare it to results of infant's perception of possible versus impossible events . It will be conducted in collaboration with Facebook AI Research.
- Time invariance in speech perception.
Speech perception is invariant with respect to large variations in speech rate . How is this achieved ? The internship will explore time normalization using various computational architectures for speech recognition (convolutional coding, networks of oscillators, etc.) and compare the results to human data .
- The role of prosody in language bootstrapping.
Speech prosody is the 'melody' and 'rhythm' of language, and infants are very sensitive to it. We think that it provides bootstrapping into linguistic structures at many levels (lexical, grammatical). The internship will explore this using a variety of speech technology techniques (signal processing, spoken term discovery, word segmentation, etc.) .
- Rules and meaning
The human language faculty is unique in its ability to combine a finite number of categories to express infinitely varied meanings . The internship addresses how the basic constituents of langage (categories and rules) could be learned during infancy focusing on two ideas: extracting proto categories and rules from the sensory inputs using clustering or sparse coding techniques , and using mutual constraints linking the different levels of linguistic structures .
- Multimodal language learning
At four months of age, infants recognize a few very common words (their names, mommy, daddy, etc) , even though they are unable to produce them. This internship tests whether multimodal DNNs can simultaneously learn words and their approximate meaning on a parallel dataset of audio and video tracks This internship will be conducted in collaboration with Microsoft Research at Redmond, USA.
- Massive baby home data collection
Big baby data is essential to uncover the mysteries of early language acquisition . Here, we develop dense data recording in baby's homes using arrays of audio/3D video sensors , as well as toy-based evaluation of preverbal infant language acquisition, and we analyze the data in relation to computational models with unsupervised algorithms.
- Cracking the neural code for speech
How does the brain encode speech sounds? Progress in neuroimaging (ECoG, intracerebral electrical recording, etc) have resulted in a flow of data, both in human and animals. The internship will apply neural decoding methods and apply to neural data and data generated from deep neural architectures to explore hypotheses about the neural code for speech.
Langue du cours : Français
Credits ECTS : 20
Professor in charge: Jérôme Sackur
E-mail : jerome.sackur@gmail.com
COGNITION AND NEUROSCIENCE
Visual anorthoscopic perception of objects
When an object (or a visual scene) is seen through a narrow slot ("slit viewing"), as it can be in real life when a scene is through a blind or a barrier for example, the visual system is able to identify and recognize this object, although the same area of the retina is moment to moment stimulated. What are the mechanisms allowing the reconstruction of this objets and to recognize it? What is the role of the movement dynamic and shape information in this process?
- Inter-hemispheric transfer of visual processing
The perception of the left and right visual field is covered by, respectively, the left and right hemispheres. When a visual stimulation crosses the vertical meridian, what are information, predictions and transferred treatment from an hemisphere to another (likely by means of the corpus callosum that joins the two hemispheres)? Informal observations suggest that cortical treatments realized in one hemisphere are not transferred to the other, and that visual treatments - particularly those required for motion perception - must be recalculated. The project aims to quantify this observation and to identify the time constants involved.
- Oculomotricity and "Eye-Gaming"
The study of oculomotricity and its (fixation, saccade, pursuit, vergence, pupilary activity) is generally realized in laboratory with strict paradigms, but unattractive (e.g. make 100 saccades towards a target). The project aims to collect oculomotor data in the context of "serious games" involving ey movements ("Eye-Gaming") and to determine is the implemented eye movement might orovide relevant information for the study of the oculomotor system (and to "substitute" or complete laboratory experimental paradigms). In the longer term, the projects aims to develop interactive terminals using eye movements for the interface control, to collect and study these eye movements on a large population (cohort).
- Sonification of eye movement
Even though eyes are constently moving (3 saccades per second), these movements are largely unaware, and often disturbed in many pathologies. The project aims to determine how sonification of eye movements (couplings of eye/sound movement) is useful to ease and improve the oculomotor control. In the long term this project aims to develop clinical applications for diagnostic and remediation of oculomotor disorders.
- Pupilary response as a measure of visual attention
The size of the pupil varies depending on scene lighting, but reflect also cognitive functions such as orientation of attention. When we pay attention to a luminous object located on the periphery, the pupil retracts compared to the situation where the object that we are paying attention to is dark. The oscillation of luminance (dark-bright) to a certain frequency inducts pupil oscillations (to approximatively 3Hz). It is possiblie to present different objects with the luminance oscillating at different frequences, and to determine which one the subject is paying attention to and looking at the amplitude of frequency in a Fourier decomposition of the pupil signal - a technique named "frequency tagging". The aim of the internship is to determine the spatial constraint of this frequency tagging in order to exploit the pupilary response as attentional measure.
This internship includes all or part of the following steps: experimental protocol implementation, experience programming (matlab), training for eye tracker use and to relevant psychological methods to collect data, data analysis.
- Game development to assess the children's attentional ryhtms
The ability to stay focused on the same task varies significantly in the general population, not to mention attention pathologies as the attention-deficit/hyperactivity disorder (ADHD). A traditional way to study it in cognitive science is to question participants while they (e.g. reading an off-putting text) in order to determine if and when their though wanders. The project would be the first step in another approach, which consists of determining for each subject a "alternance bias" between two tasks. To do this, a bi-partite task should be created and could later take the form of a video game, in which participants should divide their attention on two sub-tasks (two parts of the screen). A constraint should be set to, depending of everybody's achievements, we could determine the rates of optimal alternence, that would not be available to the subject. We could then measure the bias of each subject in comparison with this optimal rates.
The internship stages are: 1/ define the game constraints; 2/ run simulations to test their robustness; 3/ design and analyse an laboratory experience to validate the bias measure; 4/ make the "video game" version.
Prerequisite: programming.
The following projects are related withe the Emmanuel Dupoux's team "The Synthetic Language Learner". They ask a good knowledge of linear or statistical algebra and of programming.
- Deep language learning from scratch
Deep Neural Networks (DNNs) have recently broken ground on state-of-the-art in several areas (image recognition, speech recognition, etc.) . However, these algorithms depend on large human annotated datasets. Yet, infants spontaneously achieve similar performance without direct supervision; the internship explores various ideas to 'de-supervise' deep learning using side information, loss functions or architectures inspired by research in human infants .
- Learning the laws of physics with a deep recurrent network
Recurrent networks can be used to learn regularities in video or audio sequences . This internship will use a game engine to learn the underlying physical regularities of interactions between macroscopic objects and compare it to results of infant's perception of possible versus impossible events . It will be conducted in collaboration with Facebook AI Research.
- Time invariance in speech perception.
Speech perception is invariant with respect to large variations in speech rate . How is this achieved ? The internship will explore time normalization using various computational architectures for speech recognition (convolutional coding, networks of oscillators, etc.) and compare the results to human data .
- The role of prosody in language bootstrapping.
Speech prosody is the 'melody' and 'rhythm' of language, and infants are very sensitive to it. We think that it provides bootstrapping into linguistic structures at many levels (lexical, grammatical). The internship will explore this using a variety of speech technology techniques (signal processing, spoken term discovery, word segmentation, etc.) .
- Rules and meaning
The human language faculty is unique in its ability to combine a finite number of categories to express infinitely varied meanings . The internship addresses how the basic constituents of langage (categories and rules) could be learned during infancy focusing on two ideas: extracting proto categories and rules from the sensory inputs using clustering or sparse coding techniques , and using mutual constraints linking the different levels of linguistic structures .
- Multimodal language learning
At four months of age, infants recognize a few very common words (their names, mommy, daddy, etc) , even though they are unable to produce them. This internship tests whether multimodal DNNs can simultaneously learn words and their approximate meaning on a parallel dataset of audio and video tracks This internship will be conducted in collaboration with Microsoft Research at Redmond, USA.
- Massive baby home data collection
Big baby data is essential to uncover the mysteries of early language acquisition . Here, we develop dense data recording in baby's homes using arrays of audio/3D video sensors , as well as toy-based evaluation of preverbal infant language acquisition, and we analyze the data in relation to computational models with unsupervised algorithms.
- Cracking the neural code for speech
How does the brain encode speech sounds? Progress in neuroimaging (ECoG, intracerebral electrical recording, etc) have resulted in a flow of data, both in human and animals. The internship will apply neural decoding methods and apply to neural data and data generated from deep neural architectures to explore hypotheses about the neural code for speech.
Course language: French
- Profesor: Sackur Jérôme