Objectives

The objective of this course is to introduce linear and non-linear regression (logistic regression and generalized linear models). Regression plays a key role in many problems and it is absolutely essential for a datascientist to understand the theory and the practice of regression analysis. It is also an important vehicle to address the statistical challenges in statistical learning : model selection, penalisation, resampling (bootstrap, cross-validation) robustness, detection of outliers, and also methods to detect deviations from an assumed model. The course will also serve as a motivation to sharpen the understanding of statistical techniques, covering both estimation and tests.

Syllabus

  1. Least Square Multivariate Regression
    • Least Square and Projection
    • First Order and Second Order Statistical Model
    • Asymptotic Analysis
  2. Gaussian Linear Regression
    • Gaussian case
    • Link with Maximum Likelihood
    • Confidence Region
  3. Tests in the Gaussian Model
    • Tests
    • Significant coefficients and ANOVA
    • Asymptotic Analysis
  4. Projection Based Linear Regression
    • Linear Regression and Models
    • Choice Criterion: Prediction Error or Prob. Distance
    • Prediction Error Analysis
  5. Kernel based linear regression
    • Kernel estimation
    • Model Selection and Empirical Error
    • Cross Validation
  6. Bias correction
    • Prediction Error and Bias Correction
    • Bias correction and multiple test
  7. AIC/BIC, General Penalization Scheme and Practical Model Selection
    • Probabilistic distance and AIC / BIC heuristics
    • Linear Model(s) and General Penalization
    • Practical Model Selection
      • Smooth Optimization
      • Exploration
      • Convex Optimization
  8. Binary Outcome and Logistic Regression
    • Binary outcome, Bernoulli Models and Parameterization
    • Logistic Regression
    • Logistic Regression, Confidence Zones and Tests
    • Logistic Regression and Models
    • Generalized Linear Model
  • Modalités d'évaluation : Examen final

    Langue du cours : Français avec transparents en anglais