{"id":389,"date":"2011-02-14T21:00:59","date_gmt":"2011-02-14T20:00:59","guid":{"rendered":"http:\/\/mathfinance.sns.it\/new_site\/index.php\/statistics-arlot\/"},"modified":"2015-07-17T09:54:17","modified_gmt":"2015-07-17T08:54:17","slug":"statistics-arlot","status":"publish","type":"post","link":"http:\/\/mathfinance.sns.it\/index.php\/statistics-arlot\/","title":{"rendered":"Sylvain Arlot, &#8220;Advanced Course on Statistics&#8221;"},"content":{"rendered":"<p style=\"margin: 1em 0px; padding: 0px; text-align: center;\"><span lang=\"IT\" style=\"font-size: 10pt;\">Monday February 14 2011,<\/span><span lang=\"IT\" style=\"font-size: 10pt;\">\u00a014.00 &#8211; 16.00 Aula Bianchi<\/span><\/p>\n<p style=\"margin: 1em 0px; padding: 0px; text-align: center;\"><span lang=\"IT\" style=\"font-size: 10pt;\">Tuesday February 15 2011,<\/span><span style=\"font-size: 10pt;\">\u00a09.00 &#8211; 11.00 Aula Fermi<br \/>\n<\/span><\/p>\n<p style=\"margin: 1em 0px; padding: 0px; text-align: center;\"><span lang=\"IT\" style=\"font-size: 10pt;\">Thursday February 17 2011,<\/span><span style=\"font-size: 10pt;\">\u00a014.00 &#8211; 16.00 Aula Fermi<br \/>\n<\/span><\/p>\n<p style=\"margin: 1em 0px; padding: 0px; text-align: center;\"><span lang=\"IT\" style=\"font-size: 10pt;\">Tuesday February 22 2011,<\/span><span style=\"font-size: 10pt;\">\u00a014.00 &#8211; 16.00 Aula Dini<br \/>\n<\/span><\/p>\n<p style=\"margin: 1em 0px; padding: 0px; text-align: center;\"><span lang=\"IT\" style=\"font-size: 10pt;\">Wednesday February 23 2011,<\/span><span style=\"font-size: 10pt;\">\u00a09.00 &#8211; 11.00 Aula Bianchi<\/span><\/p>\n<p style=\"text-align: center;\">Scuola Normale Superiore<\/p>\n<p style=\"text-align: center;\"><b><strong><span style=\"font-size: 12pt;\"><span lang=\"EN-GB\">SYLVAIN ARLOT<\/span><\/span><\/strong><br \/>\n<\/b><span style=\"font-size: 10pt;\">CNRS-INRIA and ENS, Paris<\/span><\/p>\n<p style=\"text-align: center;\"><strong><span style=\"font-size: 12pt;\">Advanced Course on Statistics<\/span><\/strong><\/p>\n<p style=\"text-align: center;\"><strong><span style=\"font-size: 10pt;\">Lecture 1. (Monday February 14) Statistical learning <\/span><\/strong><\/p>\n<ul style=\"text-align: center;\">\n<li><span style=\"font-size: 10pt;\">the statistical learning learning problem<\/span><\/li>\n<li><span style=\"font-size: 10pt;\">examples: prediction, regression, classification, density estimation<\/span><\/li>\n<li><span style=\"font-size: 10pt;\">estimators: definition, consistency, examples<\/span><\/li>\n<li><span style=\"font-size: 10pt;\">universal learning rates and No Free Lunch Theorems [1]<\/span><\/li>\n<li><span style=\"font-size: 10pt;\">the estimator selection paradigm, bias-variance decomposition of the risk<\/span><\/li>\n<li><span style=\"font-size: 10pt;\">data-driven selection procedures and the unbiased risk estimation principle<\/span><\/li>\n<\/ul>\n<p class=\"contentpane\" style=\"text-align: center;\"><strong><span style=\"font-size: 10pt;\">Lecture 2. (Tuesday February 15) Model selection for least-squares regression <\/span><\/strong><\/p>\n<ul style=\"text-align: center;\">\n<li><span style=\"font-size: 10pt;\">ideal penalty, Mallows&#8217; Cp<\/span><\/li>\n<li><span style=\"font-size: 10pt;\">oracle inequality for Cp (i.e., non-asymptotic optimality of the corresponding model selection procedure), corresponding learning rates [2]<\/span><\/li>\n<li><span style=\"font-size: 10pt;\">the variance estimation problem<\/span><\/li>\n<li><span style=\"font-size: 10pt;\">minimal penalties and data-driven calibration of penalties: the slope heuristics [3,4]<\/span><\/li>\n<li><span style=\"font-size: 10pt;\">algorithmic and other practical issues [5]<\/span><\/li>\n<\/ul>\n<p class=\"contentpane\" style=\"text-align: center;\"><strong><span style=\"font-size: 10pt;\">Lecture 3. (Thursday February 17) Linear estimator selection for least-squares regression [6] <\/span><\/strong><\/p>\n<ul style=\"text-align: center;\">\n<li><span style=\"font-size: 10pt;\">linear estimators: (kernel) ridge regression, smoothing splines, k-nearest neighbours, Nadaraya-Watson estimators<\/span><\/li>\n<li><span style=\"font-size: 10pt;\">bias-variance decomposition of the risk <\/span><\/li>\n<li><span style=\"font-size: 10pt;\">the linear estimator selection problem: CL penalty <\/span><\/li>\n<li><span style=\"font-size: 10pt;\">oracle inequality for CL <\/span><\/li>\n<li><span style=\"font-size: 10pt;\">data-driven calibration of penalties: a new light on the slope heuristics<\/span><\/li>\n<\/ul>\n<p class=\"contentpane\" style=\"text-align: center;\"><strong><span style=\"font-size: 10pt;\">Lecture 4. (Tuesday February 22) Resampling and model selection <\/span><\/strong><\/p>\n<ul style=\"text-align: center;\">\n<li><span style=\"font-size: 10pt;\">regressograms in heteroscedastic regression: the penalty cannot be a function of the dimensionality of the models [7]<\/span><span style=\"font-size: 10pt;\">\u00a0<\/span><\/li>\n<li><span style=\"font-size: 10pt;\">resampling in statistics: general heuristics, the bootstrap, exchangeable weighted bootstrap [8] <\/span><span style=\"font-size: 10pt;\">\u00a0<\/span><\/li>\n<li><span style=\"font-size: 10pt;\">study of a case example: estimating the variance by resampling <\/span><span style=\"font-size: 10pt;\">\u00a0<\/span><\/li>\n<li><span style=\"font-size: 10pt;\">resampling penalties: why do they work for heteroscedastic regression? oracle-inequality. comparison of the resampling weights [9]<\/span><\/li>\n<\/ul>\n<p class=\"contentpane\" style=\"text-align: center;\"><strong><span style=\"font-size: 10pt;\">Lecture 5. (Wendsday February 23) Cross-validation and model\/estimator selection [10] <\/span><\/strong><span style=\"font-size: 10pt;\">\u00a0<\/span><\/p>\n<ul style=\"text-align: center;\">\n<li><span style=\"font-size: 10pt;\">cross-validation: principle, main examples<\/span><span style=\"font-size: 10pt;\">\u00a0<\/span><\/li>\n<li><span style=\"font-size: 10pt;\">cross-validation for estimating of the prediction risk: bias, variance <\/span><span style=\"font-size: 10pt;\">\u00a0<\/span><\/li>\n<li><span style=\"font-size: 10pt;\">cross-validation for selecting among a family of estimators: main properties, how should the splits be chosen? <\/span><span style=\"font-size: 10pt;\">\u00a0<\/span><\/li>\n<li><span style=\"font-size: 10pt;\">illustration of the robustness of cross-validation: detecting changes in the mean of a signal with unknown and non-constant variance [11] <\/span><span style=\"font-size: 10pt;\">\u00a0<\/span><\/li>\n<li><span style=\"font-size: 10pt;\"><span style=\"font-size: 10pt;\">correcting the bias of cross-validation: V-fold penalization. Oracle-inequality. [12]<\/span><\/span><!--more--><\/li>\n<\/ul>\n<p class=\"contentpane\" style=\"text-align: center;\"><strong><span style=\"font-size: 10pt;\">References<\/span><\/strong><\/p>\n<p class=\"contentpane\" style=\"padding-left: 30px; text-align: center;\"><span style=\"font-size: 10pt;\"><strong>[1]<\/strong> Luc Devroye, Laszlo Gyorfi, and Gabor Lugosi. A probabilistic theory of pattern recognition, volume 31 of <\/span><br \/>\n<span style=\"font-size: 10pt;\">Applications of Mathematics (New York). Springer-Verlag, New York, 1996.<\/span><\/p>\n<p class=\"contentpane\" style=\"padding-left: 30px; text-align: center;\"><span style=\"font-size: 10pt;\"><strong>[2]<\/strong> Pascal Massart. Concentration Inequalities and Model Selection, volume 1896 of Lecture Notes in Mathematics. <\/span><br \/>\n<span style=\"font-size: 10pt;\">Springer, Berlin, 2007. Lectures from the 33rd Summer School on Probability Theory held in Saint-Flour, <\/span><span style=\"font-size: 10pt;\">July 6-23, 2003.<\/span><\/p>\n<p class=\"contentpane\" style=\"padding-left: 30px; text-align: center;\"><span style=\"font-size: 10pt;\"><strong>[3]<\/strong> Lucien Birge and Pascal Massart. Minimal penalties for Gaussian model selection. Probab. Theory Related Fields, 138(1-2):33-73, 2007.<\/span><\/p>\n<p class=\"contentpane\" style=\"padding-left: 30px; text-align: center;\"><span style=\"font-size: 10pt;\"><strong>[4]<\/strong> Sylvain Arlot and Pascal Massart. Data-driven calibration of penalties for least-squares regression. J. Mach. Learn. Res., 10:245-279 (electronic), 2009. <a class=\"moz-txt-link-freetext\" href=\"http:\/\/jmlr.csail.mit.edu\/papers\/v10\/arlot09a.html\" target=\"_blank\">http:\/\/jmlr.csail.mit.edu\/papers\/v10\/arlot09a.html<\/a><\/span><\/p>\n<p class=\"contentpane\" style=\"padding-left: 30px; text-align: center;\"><span style=\"font-size: 10pt;\"><strong>[5]<\/strong> Jean-Patrick Baudry, Cathy Maugis, and Bertrand Michel. Slope Heuristics : Overview and Implementation. <\/span><br \/>\n<span style=\"font-size: 10pt;\">Technical Report 7223, INRIA, 2010. <a class=\"moz-txt-link-freetext\" href=\"http:\/\/hal.archives-ouvertes.fr\/hal-00461639\/en\/\" target=\"_blank\">http:\/\/hal.archives-ouvertes.fr\/hal-00461639\/en\/<\/a><\/span><\/p>\n<p class=\"contentpane\" style=\"padding-left: 30px; text-align: center;\"><span style=\"font-size: 10pt;\"><strong>[6]<\/strong> Sylvain Arlot and Francis Bach. Data-driven calibration of linear estimators with minimal penalties. Proceedings of NIPS 2009. <a class=\"moz-txt-link-freetext\" href=\"http:\/\/arxiv.org\/abs\/0909.1884\" target=\"_blank\">http:\/\/arxiv.org\/abs\/0909.1884<\/a><\/span><\/p>\n<p class=\"contentpane\" style=\"padding-left: 30px; text-align: center;\"><span style=\"font-size: 10pt;\"><strong>[7]<\/strong> Sylvain Arlot. Choosing a penalty for model selection in heteroscedastic regression. Preprint. 2010. <a class=\"moz-txt-link-freetext\" href=\"http:\/\/arxiv.org\/abs\/0812.3141\" target=\"_blank\">http:\/\/arxiv.org\/abs\/0812.3141<\/a><\/span><\/p>\n<p class=\"contentpane\" style=\"padding-left: 30px; text-align: center;\"><span style=\"font-size: 10pt;\"><strong>[8]<\/strong> Bradley Efron and Robert J. Tibshirani. An Introduction to the Bootstrap, volume 57 of Monographs on <\/span><br \/>\n<span style=\"font-size: 10pt;\">Statistics and Applied Probability. Chapman and Hall, New York, 1993.<\/span><\/p>\n<p class=\"contentpane\" style=\"padding-left: 30px; text-align: center;\"><span style=\"font-size: 10pt;\"><strong>[9]<\/strong> Sylvain Arlot. Model selection by resampling penalization. Electronic Journal of Statistics, 3, (2009), 557-624 (electronic). <a class=\"moz-txt-link-freetext\" href=\"http:\/\/projecteuclid.org\/DPubS?service=UI&amp;version=1.0&amp;verb=Display&amp;handle=euclid.ejs\/1245415825\" target=\"_blank\">http:\/\/dx.doi.org\/10.1214\/08-EJS196<\/a><\/span><\/p>\n<p class=\"contentpane\" style=\"padding-left: 30px; text-align: center;\"><span style=\"font-size: 10pt;\"><strong>[10]<\/strong> Sylvain Arlot and Alain Celisse. A survey of cross-validation procedures for model selection. Statist. Surv., 4:40-79, 2010. <a class=\"moz-txt-link-freetext\" href=\"http:\/\/www.i-journals.org\/ss\/viewarticle.php?id=54&amp;layout=abstract\" target=\"_blank\">http:\/\/dx.doi.org\/10.1214\/09-SS054<\/a><\/span><\/p>\n<p class=\"contentpane\" style=\"padding-left: 30px; text-align: center;\"><span style=\"font-size: 10pt;\"><strong>[11]<\/strong> Sylvain Arlot and Alain Celisse. Segmentation of the mean of heteroscedastic data via cross-validation. Statistics and Computing, 2010. <a class=\"moz-txt-link-freetext\" href=\"http:\/\/arxiv.org\/abs\/0902.3977\" target=\"_blank\">http:\/\/arxiv.org\/abs\/0902.3977<\/a><\/span><\/p>\n<p class=\"contentpane\" style=\"padding-left: 30px; text-align: center;\"><span style=\"font-size: 10pt;\"><strong>[12]<\/strong> Sylvain Arlot. V-fold cross-validation improved: V-fold penalization. Preprint. 2008. <a class=\"moz-txt-link-freetext\" href=\"http:\/\/fr.arxiv.org\/abs\/0802.0566\" target=\"_blank\">http:\/\/fr.arxiv.org\/abs\/0802.0566<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Monday February 14 2011,\u00a014.00 &#8211; 16.00 Aula Bianchi Tuesday February 15 2011,\u00a09.00 &#8211; 11.00 Aula Fermi Thursday February 17 2011,\u00a014.00 &#8211; 16.00 Aula Fermi Tuesday February 22 2011,\u00a014.00 &#8211; 16.00 Aula Dini Wednesday February 23 2011,\u00a09.00 &#8211; 11.00 Aula Bianchi Scuola Normale Superiore SYLVAIN ARLOT CNRS-INRIA and ENS, Paris Advanced Course on Statistics Lecture 1. [&hellip;]<\/p>\n","protected":false},"author":7,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[43],"tags":[],"_links":{"self":[{"href":"http:\/\/mathfinance.sns.it\/index.php\/wp-json\/wp\/v2\/posts\/389"}],"collection":[{"href":"http:\/\/mathfinance.sns.it\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/mathfinance.sns.it\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/mathfinance.sns.it\/index.php\/wp-json\/wp\/v2\/users\/7"}],"replies":[{"embeddable":true,"href":"http:\/\/mathfinance.sns.it\/index.php\/wp-json\/wp\/v2\/comments?post=389"}],"version-history":[{"count":3,"href":"http:\/\/mathfinance.sns.it\/index.php\/wp-json\/wp\/v2\/posts\/389\/revisions"}],"predecessor-version":[{"id":574,"href":"http:\/\/mathfinance.sns.it\/index.php\/wp-json\/wp\/v2\/posts\/389\/revisions\/574"}],"wp:attachment":[{"href":"http:\/\/mathfinance.sns.it\/index.php\/wp-json\/wp\/v2\/media?parent=389"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/mathfinance.sns.it\/index.php\/wp-json\/wp\/v2\/categories?post=389"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/mathfinance.sns.it\/index.php\/wp-json\/wp\/v2\/tags?post=389"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}