Compare ROC Parameter

PlatyQPlatyQ MemberPosts:4Contributor I
edited September 2020 inHelp

Hello,

I have some questions but I am not anglophone and may be it will difficult to you to understand my questions and difficult to me to understand your answer.

Please if I am not understandable say it to me.

I try to make a learning machine to a stage in computer science and I want compare some models to this.

I have very few examples (20) but I have to try anyway.

My troubles are to adjust the parameters of "compare ROC", I think understand what is a ROC curve and how interpreted but may be I make a mistake cause I don't understand why they are two parameters :
  • number_of_folds.
  • split_ratio.

According to me if i put 4 in number_of_foldsfor each models Rapidminer compares 4 training machine with 15 examples to learn and evaluate them with the last 5 and after create for each model the ROC Curves. And it match with that I want (compare some models to my 20 examples).

But split ratio can not be disable so it can play a role in the roc curve but i don't understand which. May be if I put 1 in this parameter it work as I think?


Thank for your help
Tagged:

Answers

  • lionelderkrikorlionelderkrikor Moderator, RapidMiner Certified Analyst, MemberPosts:1,195Unicorn

    Hi@PlatyQ,

    Good question.

    I asked myself the same question when I used this operator for the first time, but I never .......dared to ask the question.

    Your post prompted me to look for an answer in the documentation :

    According to the documentation,Compare ROCbehaves like aSplit ValidationoraCross Validationoperator :

    - ifnumber of folds = -1, thenCompare ROCbehaves like aSplit Validationwith the definedsplit ratio.

    - ifnumber of folds is in Range ]1, infinite[, thenCompare ROCbehaves like aCross Validationwith the definednumber of folds. (and in this case, the definedsplit ratiois not taken into account).

    - ifnumber of folds = 0 or 1, then RapidMiner raise an error .... and it's logic.

    NB : That's what I understand from the documentation, but thanks to the experts to confirm or..... invalidate this explanation of the behaviour ofCompare ROCoperator.

    I hope it helps,

    Regards,

    Lionel

  • PlatyQPlatyQ MemberPosts:4Contributor I

    Hi@lionelderkrikorand thank for your response

    I also search in documentation before ask the question but I read and read again and no see that "if number of folds is in Range ]1, infinite[ [...] (and in this case, the defined split ratio is not taken into account)". I think your right cause it seems more logic than use the two options in the same time but I regret than it is not well-defined I the documentation.

    I am reassure by your response but I hope we can have a second opinion.

    (And may be somebody know how to ask modifications in the documentation? If two people have doubts may be we are not the only ones.)

    thanks

Sign InorRegisterto comment.