sutton001

The transfer between the Excel spreadsheet and Sipina was improved on the databases of moderate size (on large databases, several hundreds of thousands of rows, it is better to perform a direct importation of data file in text format TXT). The management of the decimal point has been improved. Now, the automatic processing is much faster than before. The precision of the numerical cut points displayed in a decision tree becomes customizable. The users dispose of a new item into the menu "Tree Management". Sipina website : Sipina Download : Setup file

In these slides, we study the analogy between the linear discriminant analysis and the regression on an indicator variable when we deal with a binary classification problem. The tests for the global significance of the model and the individual significance of the coefficients are equivalent. The coefficients are proportional, including the intercepts when we treat the balanced case. In the other case (unbalanced classes), an additional correction of the regression intercept is needed to obtain the linear discriminant analysis intercept. For the multiclass classification, the equivalence between the regression and the linear discriminant analysis is no longer valid. Keywords : supervised learning, linear discriminant analysis, multiple linear regression, R2, wilks lambda Slides : Classification via regression References : R.O. Duda, P.E. Hart, D. Stork, « Pattern Classification », 2nd Edition, Wiley, 2000. C.J. Huberty, S. Olejnik, « Applied MANOVA and Discriminant Analysi...

Here are the slides I use for my course about “Naive Bayes Classifier”. The main originality of this presentation is that I show it is possible to extract an explicit model based on the calculations of the conditional distributions. This makes highly easier the deployment of the model in real case studies. This aspect is commonly unknown. The feature selection problem in the context of Naive Bayes Classifier learning is also highlighted. Keywords : machine learning, supervised methods, naive bayes, independence assumption, independent feature model, feature selection, cfs Slides : Naive Bayes classifier References : T. Mitchell, " Generative and discriminative classifiers: naive bayes and logistic regression ", in "Machine Learning", McGraw Hill, 2010; Draft of January 2010. Wikipedia, " Naive Bayes classifier ".

Here are the slides I use for my course about “Linear Discriminant Analysis” (LDA). The two main assumptions which enable to obtain a linear classifier are highlighted. The LDA is very interesting because we can interpret the classifier in different ways: it is a parametric method based on the MAP (maximum a posteriori) decision rule; it is a classifier based on a distance to the conditional centroids; it is a linear separator which defines various regions in the representation space. Statistical tools for the overall model evaluation and the checking of the relevance of the predictive variables are presented. Keywords : machine learning, supervised methods, discriminant analysis, predictive discriminant analysis, linear discriminant analysis, linear classification functions, wilks lambda, stepdisc, feature selection Slides : linear discriminant analysis References : J. Gareth, D. Witten, T. Hastie, R. Tibshirani, "An introduction to statistical learning with applications in R...

Here are the slides I use for my course about “Regression Trees”. Because this course comes after the one about “Decision Trees”, only the special features for the handling of a continuous target attribute are highlighted. The described algorithms correspond roughly to the AID and the CART approaches. Keywords : machine learning, supervised methods, regression tree, aid, cart, continuous class attribute Slides : regression trees References : L. Breiman, J. Friedman, R. Olshen and C. Stone, “Classification and Regression Trees”, Wadsworth Int. Group, 1984. J. Morgan, J.A. Sonquist, "Problems in the Analysis of Survey Data and a Proposal", JASA, 58:415-435, 1963.

Here are the slides I use for my course about the existing decision tree learning algorithms. Only the most popular ones are described: C4.5, CART and CHAID (a variant). The differences between these approaches are highlighted according: the splitting measure; the merging strategy during the splitting process; the approach for determining the right sized tree. Keywords : machine learning, supervised methods, decision tree learning, classification tree, chaid, cart, c4.5 Slides : C4.5, CART and CHAID References : L. Breiman, J. Friedman, R. Olshen and C. Stone, “Classification and Regression Trees”, Wadsworth Int. Group, 1984. G. Kass, “An exploratory technique for Investigating Large Quantities of Categorical Data”, Applied Statistics, 29(2), 1980, pp. 119-127. R. Quinlan, “C4.5: Programs for machine learning”, Morgan Kaufman, 1993.

Here are the lecture notes I use for my course “Introduction to Decision Trees”. The basic concepts of the decision tree algorithm are described. The underlying method is rather similar to the CHAID approach. Keywords : machine learning, supervised methods, decision tree learning, classification tree Slides : Introduction to Decision Trees References : T. Mitchell, " Decision Tree Learning ", in "Machine Learning", McGraw Hill, 1997; Chapter 3, pp. 52-80. L. Rokach, O. Maimon, " Decision Trees ", in  "The Data Mining and Knowledge Discovery Handbook", Springer, 2005; Chapter 9, pp. 165-192.