Professor in charge: Jérôme Sackur
E-mail : jerome.sackur@gmail.com
COGNITION AND NEUROSCIENCE
Visual anorthoscopic perception of objects
When an object (or a visual scene) is seen through a narrow slot ("slit viewing"), as it can be in real life when a scene is through a blind or a barrier for example, the visual system is able to identify and recognize this object, although the same area of the retina is moment to moment stimulated. What are the mechanisms allowing the reconstruction of this objets and to recognize it? What is the role of the movement dynamic and shape information in this process?
- Inter-hemispheric transfer of visual processing
The perception of the left and right visual field is covered by, respectively, the left and right hemispheres. When a visual stimulation crosses the vertical meridian, what are information, predictions and transferred treatment from an hemisphere to another (likely by means of the corpus callosum that joins the two hemispheres)? Informal observations suggest that cortical treatments realized in one hemisphere are not transferred to the other, and that visual treatments - particularly those required for motion perception - must be recalculated. The project aims to quantify this observation and to identify the time constants involved.
- Oculomotricity and "Eye-Gaming"
The study of oculomotricity and its (fixation, saccade, pursuit, vergence, pupilary activity) is generally realized in laboratory with strict paradigms, but unattractive (e.g. make 100 saccades towards a target). The project aims to collect oculomotor data in the context of "serious games" involving ey movements ("Eye-Gaming") and to determine is the implemented eye movement might orovide relevant information for the study of the oculomotor system (and to "substitute" or complete laboratory experimental paradigms). In the longer term, the projects aims to develop interactive terminals using eye movements for the interface control, to collect and study these eye movements on a large population (cohort).
- Sonification of eye movement
Even though eyes are constently moving (3 saccades per second), these movements are largely unaware, and often disturbed in many pathologies. The project aims to determine how sonification of eye movements (couplings of eye/sound movement) is useful to ease and improve the oculomotor control. In the long term this project aims to develop clinical applications for diagnostic and remediation of oculomotor disorders.
- Pupilary response as a measure of visual attention
The size of the pupil varies depending on scene lighting, but reflect also cognitive functions such as orientation of attention. When we pay attention to a luminous object located on the periphery, the pupil retracts compared to the situation where the object that we are paying attention to is dark. The oscillation of luminance (dark-bright) to a certain frequency inducts pupil oscillations (to approximatively 3Hz). It is possiblie to present different objects with the luminance oscillating at different frequences, and to determine which one the subject is paying attention to and looking at the amplitude of frequency in a Fourier decomposition of the pupil signal - a technique named "frequency tagging". The aim of the internship is to determine the spatial constraint of this frequency tagging in order to exploit the pupilary response as attentional measure.
This internship includes all or part of the following steps: experimental protocol implementation, experience programming (matlab), training for eye tracker use and to relevant psychological methods to collect data, data analysis.
- Game development to assess the children's attentional ryhtms
The ability to stay focused on the same task varies significantly in the general population, not to mention attention pathologies as the attention-deficit/hyperactivity disorder (ADHD). A traditional way to study it in cognitive science is to question participants while they (e.g. reading an off-putting text) in order to determine if and when their though wanders. The project would be the first step in another approach, which consists of determining for each subject a "alternance bias" between two tasks. To do this, a bi-partite task should be created and could later take the form of a video game, in which participants should divide their attention on two sub-tasks (two parts of the screen). A constraint should be set to, depending of everybody's achievements, we could determine the rates of optimal alternence, that would not be available to the subject. We could then measure the bias of each subject in comparison with this optimal rates.
The internship stages are: 1/ define the game constraints; 2/ run simulations to test their robustness; 3/ design and analyse an laboratory experience to validate the bias measure; 4/ make the "video game" version.
Prerequisite: programming.
The following projects are related withe the Emmanuel Dupoux's team "The Synthetic Language Learner". They ask a good knowledge of linear or statistical algebra and of programming.
- Deep language learning from scratch
Deep Neural Networks (DNNs) have recently broken ground on state-of-the-art in several areas (image recognition, speech recognition, etc.) . However, these algorithms depend on large human annotated datasets. Yet, infants spontaneously achieve similar performance without direct supervision; the internship explores various ideas to 'de-supervise' deep learning using side information, loss functions or architectures inspired by research in human infants .
- Learning the laws of physics with a deep recurrent network
Recurrent networks can be used to learn regularities in video or audio sequences . This internship will use a game engine to learn the underlying physical regularities of interactions between macroscopic objects and compare it to results of infant's perception of possible versus impossible events . It will be conducted in collaboration with Facebook AI Research.
- Time invariance in speech perception.
Speech perception is invariant with respect to large variations in speech rate . How is this achieved ? The internship will explore time normalization using various computational architectures for speech recognition (convolutional coding, networks of oscillators, etc.) and compare the results to human data .
- The role of prosody in language bootstrapping.
Speech prosody is the 'melody' and 'rhythm' of language, and infants are very sensitive to it. We think that it provides bootstrapping into linguistic structures at many levels (lexical, grammatical). The internship will explore this using a variety of speech technology techniques (signal processing, spoken term discovery, word segmentation, etc.) .
- Rules and meaning
The human language faculty is unique in its ability to combine a finite number of categories to express infinitely varied meanings . The internship addresses how the basic constituents of langage (categories and rules) could be learned during infancy focusing on two ideas: extracting proto categories and rules from the sensory inputs using clustering or sparse coding techniques , and using mutual constraints linking the different levels of linguistic structures .
- Multimodal language learning
At four months of age, infants recognize a few very common words (their names, mommy, daddy, etc) , even though they are unable to produce them. This internship tests whether multimodal DNNs can simultaneously learn words and their approximate meaning on a parallel dataset of audio and video tracks This internship will be conducted in collaboration with Microsoft Research at Redmond, USA.
- Massive baby home data collection
Big baby data is essential to uncover the mysteries of early language acquisition . Here, we develop dense data recording in baby's homes using arrays of audio/3D video sensors , as well as toy-based evaluation of preverbal infant language acquisition, and we analyze the data in relation to computational models with unsupervised algorithms.
- Cracking the neural code for speech
How does the brain encode speech sounds? Progress in neuroimaging (ECoG, intracerebral electrical recording, etc) have resulted in a flow of data, both in human and animals. The internship will apply neural decoding methods and apply to neural data and data generated from deep neural architectures to explore hypotheses about the neural code for speech.
Course language: French
- Teaching coordinator: Sackur Jérôme