AdaBoost vs. BayesianBoosting

cherokeecherokee MemberPosts:82Guru
edited June 2019 inHelp
Hi!

I'm goint o try some boosting for my bachelor's thesis. I haven't yet decided whether using AdaBoost or BayesianBoosting. Actually I don't understand all the differences. What I do understand is that BayesianBoosing can use different fractions of the example set for model fitting and performance estimation. I understand that it is able to reweight examples to ensure equally distributed labels. But what exactly means [tt]allow_marginal_skews[/tt]?

Martin Scholz (the author of the operator) cites in the help text Scholz/2005b. Can anyone give me publication details on (t)his work. I think I would understand the differencen in detail if i read it.

Best regards,
chero
Tagged:

Answers

  • 韦塞尔韦塞尔 MemberPosts:537Guru
    Hmm, let me first see if I understand the terminology correctly.

    Bayesian Boosting is like creating all possible models and weighting them according their accuracy?
    (Maybe you also need to weigh them according the model prior).
    Most of the time this is not feasible in practice.

    演算法,自适应增强, setting the iteration parameter to n, creates n models.
    Models are weighted by their accuracy.
    New training data is created by reweighing examples.
    If examples are correctly classified in the previous iteration their weight goes down,
    if examples are incorrectly classified their weight goes up.
    Freund and Shaffire prove that the error on the training set goes down exponentially fast, using adaboost.

    This paper is really good?
    vorlon.case.edu/~sray/eecs600_fall08/ensembles_survey.ps
    Ensemble Methods in Machine Learning Dietterich filetype:pdf
  • IngoRMIngoRM Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, Community Manager, RMResearcher, Member, University ProfessorPosts:1,751RM Founder
    Hi,

    you can find Martin's publication here:

    http://www-ai.cs.uni-dortmund.de/auto?self=$Publication_e9zx9gcx

    Here are his other publications on this and related topics:

    http://www-ai.cs.uni-dortmund.de/PERSONAL/scholz.html

    Cheers,
    Ingo
  • 韦塞尔韦塞尔 MemberPosts:537Guru
    This paper is about subgroup detection.

    Not about ensembles or boosting.
  • IngoRMIngoRM Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, Community Manager, RMResearcher, Member, University ProfessorPosts:1,751RM Founder
    Yes, but the basic algorithm - and hence also the operator - can be used for both problem types. It's the paper cited by Martin in his implementation comment so I assume it is the one he thinks it is most appropriate even if the title suggests something else. If you find another one of his papers more appropriate: just go ahead and cite it here. They all can be found on the web site stated above.

    Cheers,
    Ingo
  • Stefan_EStefan_E MemberPosts:53Guru
    Ingo,

    Thanks a lot for the links - was searching the same just over the weekend :-)

    would propose to add these references to the wiki. There is a general lack of documentation where the various algorithms come from... so the wiki would be the natural place growing such a knowledge library.

    Problem is that it appears that the wiki wants to be a duplication of the built-in help. If this is so, you'll end up in a maintenance problem as I expect there is only a one-way conversion path?

    Stefan
  • IngoRMIngoRM Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, Community Manager, RMResearcher, Member, University ProfessorPosts:1,751RM Founder
    Yes, you are right. There was actually a discussion about connecting the operator help and the Wiki some time ago:

    http://rapid-i.com/rapidforum/index.php/topic,2013.msg8148.html#msg8148

    But I must admit that I am not sure about the current state of this.

    Cheers,
    Ingo
Sign InorRegisterto comment.