Topic outline

  • General

    Collapse all Expand all

      Schedule 2023-2024

    • Lectures (9:30-12:30) + labs (13:340-16:30) : 4 weeks on Mondays 

            Sept 18, 25 + Oct 9 and 16 ---- break on October 2
          includes Bibliography talks (eval 1) on Oct 16
      

    • Project defenses (eval 2) on Monday November 6th.

  • Presentation of the course

    This course is a joint course between the MScT AI-ViC of Ecole Polytechnique and the Computer Science master IGD (Interaction, Graphics, Design) of IP Paris. It takes place during 4 consecutive weeks, from September 19 to October 10, 2022, with 3 hours of class in the morning (9h30-12h30, Room Yvan Sutherland and 3 hours of lab in the afternoon (13h30-16h30)  (see "Lab Sessions" below for lab-related instructions). Attending both is mandatory!

    The goal of this course is to explore recent advances in Computer Graphics with strong connections with Artificial Intelligence. We will cover two main topics:

    • Part 1: Creative AI (Weeks 1 and 2): Expressive 3D modeling, using smart geometry controlled by gestures. Applications span from the design of individual shapes to the creation of  full, animated virtual worlds.
    • Part 2: Character animation (Weeks 3 and 4):  Animating and training, possibly autonomous, virtual creatures, based on 3D graphics, knowledge and learning.

    Background:  A master 1 level in Computer Graphics, with some knowledge in 3D modeling, rendering and computer animation, is mandatory to follow this course. Each week, the necessary background on each subtopic will be briefly reviewed before studying recent advances linked to AI. In case you missed some of this background, catching up using any available online material is strongly recommended.

    Schedule 2023-2024 :
      4 weeks on Mondays: Sept 18, 25 + Oct 9 and 16 ---- break on October 2
       includes Bibliography talks (eval 1) on Oct 16
      + Project defenses (eval 2) on Monday November 6th.

    • Lectures: Marie-Paule Cani (weeks 1, 2, 4) and Julien Pettré (week 3)
    • Labs: Paul Boursin and ‪Ramana Sundararaman‬ (Teaching Assistants)
    In case you need to miss either a lecture or a lab session, please tell the teachers by email, preferably in advance.

    Evaluation is composed of two parts, both performed by groups of 3 students (see details in the "Evaluation" section below): 

    1. A bibliography study consisting in reading and comparing 2 research papers, followed by a talk in front of the other students during the week 3 course session. 
    2. A programming project which is the continuation of the lab sessions. 
    We hope you will enjoy both! Don't hesitate to contact your teachers in case of problems or questions.

  • Week 1: From 3D modeling to Creative AI

    Part 1, week 1 (Marie-Paule Cani)

    The concept of "User-centered creative AI", also called "Expressive 3D modeling", is presented in the context of the creation of individual, static shapes, after a brief review of shape representations in Computer Graphics.

    Chapters:

    • Shape representations for interactive modeling (slides + 2 videos)
    • Expressive 3D modeling (slides + 2 videos)

    If you missed the lecture, I suggest you to annotate the slides while watching the two videos. Please ask your questions directly to the prof just before the next course or during the coffee break.

  • Week 2: Designing animated virtual worlds

    Part 1, week 2 (Marie-Paule Cani)

    Let us now study how user-centered creative methods, ie. smart geometry controlled by gestures, can be extended to the design of full, animated virtual worlds. In a first chapter, we review standard approaches for creating and animating virtual worlds, before exploring how creative AI can help, thanks to the combination of knowledge, learning and user's control.

    Chapters:

    • Creating and animating virtual worlds (slides + 2 videos)
    • Extension of expressive design to virtual worlds (slides + 2 videos)

    If you missed the lecture, I suggest you to annotate the slides while watching the two videos. Please ask your questions directly to the prof just before the next course or during the coffee break.

  • Week 3: Motion planning for characters and crowds

    Part 2, chapter 2 (Julien Pettré)

    This last part of the course covers motion planning for both individual characters and for crowds.


  • Week 4: Animation of individual characters

    Part 2, chapter 1 (Marie-Paule Cani )

    We will study the animation of individual characters, focusing on the design and control of their skeletal motion. While the lecture will review standard methods in Computer Graphics, students talks will be exploring recent advances towards either animation authoring or the creation of autonomous creatures able to learn to use their body to move. 

    I will only provide a 1 hour lecture, introducing the topic and giving a brief overview of the history of skeletal motion control in Computer Graphics. Side videos are provided for students who would like to know about the other layers of a character model (the animation of skin, clothes and hair, from the skeletal animation).

    Students talks, starting at 10:30, will then enable us to discuss character motion control more in depth, including:

    • The interactive authoring of  kinematic skeletal  animations
    • Pioneer work on controllers generation for physically-based creatures
    • Recent uses of machine learning techniques to improve motion control 

    (see the "Evaluation Part 1" below for the choice of your own contribution)

  • Lab sessions

    Programming modules in Unity (Paul Boursin, Ramana Sundararaman)

    Important Links:

    The sessions will be held in person, in the teaching space of the Alan Turing building (RdC). 

    Preparations before the first session:

    • Join the Slack dedicated to this course. It will be used for general communication with the coordinators outside the lab hours. You will be able to ask questions, to ask for help on specific problems, etc...
    • Have a look to the Github repository for the lab session and to the README.md.
    • The computers in the class will be equipped with Unity. However, if you wish to bring your own laptops and go though the lab with them, you are welcome! In that case: Follow the instructions in the README.md described above to install Unity. Make sure that you can launch it before the session by following the instructions in the Github Page and that there is no errors (i.e., the project contained in the repository opens successfully in Unity and it does not have any graphics/version issues).
    • We recommend to fork the original repository, so you can push it to your own Github account and keep your progress saved (for example, if you make changes in the class computer, and then you want to keep working from your own laptop at home). You can decide which workflow suits you the best.
    • During the session we will perform the task in groups (2-3 people), so each group will need at least one working computer.

    Other aspects of working with Unity:

    • Create a Unity account. It will be required when you launch Unity. You can select the free option for personal use.
    • Make sure that you have a text editor installed and configured for C#. Visual Studio Code is a popular option, but any other code editor can be used.

    For the time during/after the sessions:

    • The lab sessions are described in the Github Page for the repository.
    • All sessions are visible from the beginning.

    Attending the lab sessions is mandatory. The exercises will be the basis of the evaluated mini-project (see below).

  • Evaluation Part 1: Bibliographic study on character motion control

    The first part of the evaluation will be based on paper reading & comparison, and take place on October 16th, 2023, during the course session.

    Each group of 3 students will present and compare two research papers either trying to solve the same challenge with two different strategies, or being the continuation of one another.  All of you should read the two papers carefully and meet online to discuss your understanding of the methods, and the pros and cons of each approach. You will then prepare a 10 minutes presentation (10 slides max) which should include:

    • A brief presentation of main contribution of each paper : input, output of the method, and key insight to solve the problem: Try to extract the main novel idea or novel technical method, something future work could inspire from.
    • A joint discussion of the two papers: similarities and differences, advantages/drawback of each method, application cases where each approach would be best suited.

    The slides describing the suggested choices for pairs of papers are attached.
    Once you have formed a group and made a choice among the topics still available in the list below, please email the professor, to indicate your choice: 
    1 Expressive design of kinematic animations. Alexandra Pilipyuk, Pedro Andrade Ferreira Sobrinho, Andrei Ostanin, Zhaoyang Chen2
    3
    4
    5 How far should we use anatomy? Muscle-based motion control. Aude Bouillé, Julien Jampsin, Benjamin Moreau. 
    6 Deep learning for motion control. Mathieu Gierski, Maximilien Bohm et Hanna Mergui
    7 Learning how to fly. Haiyang JIANG , Jackson SUNNY, Pierre OLLEON 
    8 Should we learn human gaits by imitation or discover new motion? Jingnan CAO , Ranyu FAN, and Wenqing QU.
    9
    10 Handling interactions with the scene. Ghislain Kengne Gumete, Yi-Hsuan Lee, Barthélemy Paléologue.
    11 Learning style and combining motions? Combining Deep RL with Adversarial networks. Célia Nouri, Ribal Teeny, Victor Barberteguy.

  • Evaluation Part 2

    Projects defense, Monday November 6th, in the morning

    Groups of 3 students will present their mini-project with a video of the environment that they have developed during the four lab sessions and finished from home. Defenses will be 20 minutes each (10 minutes presentation + 5 to 10 minutes questions). It is also recommended to have the Unity project ready to work on real-time in case it is checked during the presentation.

    Presentations are open to the other students from the class.  

    We recommend making a video where you show the tools you have created, how they work, and finally the final result of your environment. The criteria for a project with the highest grade is:

    - TP1: At least 1 complex edition tool (Perlin noise, erosion, etc.) or a brush more complex than traditional image editing tools.
    - TP2: Multiple placement brushes with different parameters (shape, height, steepness, etc.) or a brush learning from example distribution (difficult).
    - TP3: Quadrupeds moving procedurally being able to adapt to complex terrains (e.g., steep slopes, irregular grounds...). Other ideas: introduce new procedural functions to define leg's movement and overall gait of the animal (walk, trot, pace...); multi-layer, controllable characters using IK to adapt the feet to the terrain, or adapt their body to external conditions (e.g., tilting upper-body forward when going up-hill); new skins or more complex skeletons for your characters.
    - TP4: Experiments on simulation changes (new/improved receptors & actuators, brain, genetic algorithm) and being able to show/explain their effect during the presentation. Other ideas: explanation about how the ecosystem evolves and reaches an equilibrium; development of more complex relations between the agents (reproduction, prey-predator system).

    - Overall: Nice looking terrain; nice video (e.g., as it was a wildlife documentary, focusing on the important details: how the animals behave, move on complex terrains or relate with other agents). The best two videos will be selected and might be published online to serve as inspiration for future students!
    - Bonus: Interactions between animals and terrain (e.g., avoid water, steep slopes, trees); Impact of agents in the environment (e.g., animals leaving trails in those places where they walk the most).

    Defenses will take place in room Yvan Sutherland of the Turing building, from 9:30 to 12:30. Please register here:
    • Time slot --- student names
    • 9:30 - 9:50 --- 
    • 9:50 - 10:10 --- 
    • 10:10 - 10:30 --- 
    • 10:30 - 10:50 --- 
    • Coffee break!
    • 11:10 - 11:30 --- 
    • 11:30 - 11:50 --- 
    • 11:50 - 12:10 ---