Weka Online

It seems that there is now the possibility to use Data Mining online using Weka. A company, CEO delegates, has built a web site where some arff files can be loaded as well as the corresponding data mining algorithms chosen.

It was actually partly one of the things I planed to do. But there are still lots of supplementary things which can be done. So let's see what happens.

Combining Data Mining Models

Here is a little summary of the possible way of combining multiple models for Data Mining (I use as first resource the "Data Mining" Book from Ian H. Witten und Eibe Frank):

• Bagging
• Bagging with costs
• Randomization
• Boosting
• Additive regression
• Additive logistic regression
• Option trees
• Logistic model trees
• Stacking
• Error correcting output codes

Bagging

The principle of Bagging is to let create a number of models for a training set, and use the class returned the most frequently for a specific instance for each of these models. In other word, it is important that the different model return the same set of possible class output.

Bagging with costs

This extension of the Bagging approach uses a cost model. It is particularly useful when the predictions made by the models used in the bagging provide probabilities telling how likely is to be exact.

Randomization

The idea here is to introduce some kind of randomization in the model creation in order to create different models. Depending on the stability of the process, a certain class can then be chosen as prediction.

Boosting

Similarly to the bagging approach, boosting tries to create models as a kind of cascade, each model is built with the purpose of classifying better the instances which have not been suitably classified by previous models. This is a type of forward stagewise additive modelling

Additive regression

The additive regression is alse a kind of forward stagewise additive modelling which is suitable for numeric prediction with regression. Here again the principe is to use a serie of regressions which try to classify better the elements which were incorrectly classified.

Additive logistic regression

This type of regression is an adapation of the previous combination approach but for logistic regression.

Option trees

I still have to describe this but the concept is quite simple

Logistic model trees

I still have to describe this but the concept is quite simple

Stacking

The purpose of stacking is to combine different types of models which might not have the same labels. In order to achieve this, a meta learner is introduced. A number of models are built for the data. A meta learner, i.e a model which decides from the learning output of other learners, created in order to classify and adapt to all the models from the first phase.

Error correcting output codes

I still have to describe this but the concept is quite simple

Of course, all these mechanisms have been implemented in Weka.

Data Mining

This is the beginning of a series of post on the topics of Data Mining and Knowledge Discovery or KDD. There exists a large amount of resources on these topics. I plan to draw a picture of the possible open source solutions for Data Mining. I also want to recall which are the main applications of Data Mining. Open Source Systems The main open source systems for Data Mining are:

• Pentaho (used to be called WEKA)
• Rapid Miner (used to be called YALE, some parts of the systems were using Weka)
• Orange

for Text Mining, it is certainly important to add: the GATE system. I will review each of these systems. Supervised and unsupervised tasks One of the main distinction for algorithms and problems in Data Mining is whether the task to be performed is supervised or unsupervised. An algorithm is supervised if the goal of the algorithm is to create a model which is appropriate to categorise data according to an existing model. On the contrary, an unsupervised task consists in building a model which is appropriate to describe the data. For example, suppose we have a list of topics and news items which should be categorised in this list of topic. This is a typical supervised task. If we had no topics and were supposed to group the news items together according to some similarity criteria. This would be a unsupervised task. N.B: This entry does not really please me yet and is subject to change.