Can I optimize a custom performance metric?

anaRodrigues

Hi,
I want to generate a F beta score and change beta according to the weight I want to put on precision or recall. And then I would like to add it to a performance vector so I can optimize it. I know there's a 'Performance to data' operator, but what I would need is the reverse. Is there any way I can do this?

EDIT: I found a similar question, but no solution:https://community.www.turtlecreekpls.com/discussion/38190/generate-own-performance-vector/p1

MartinLiebig

Hi@anaRodrigues,

ehm, F1-Measure is already part of the operator Performance (Binominal)?

Anyway, you can use Performance to Data to get tp, fp, tn and fn and then calculate the F1 score by hand. Then you can use Extract Performance to get it as a performance vector again. Attached is an example.

Cheers,

Martin

<参数键= value =“process_duration_for_mail30"/>






























In the training phase, a model is built on the current training data set. (90 % of data by default, 10 times)
















<参数键= " recall" value="false"/>



























The model created in the Training step is applied to the current test set (10 %).<br/>The performance is evaluated and sent to the operator results.

<描述一致=“中心”颜色=“透明”有限公司lored="false" width="126">A cross-validation evaluating a decision tree model.

MartinLiebig

Hi,

The operator you search for is called Extract Performance.

Best,

Martin

anaRodrigues

Hi Martin,
Thank you for your reply, but I don't see how that operator can help me. It only allows me to calculate four statistics: min, max, average and count of a specific attribute. What I want is to calculate the F metric, which comes from the confusion matrix values.
Thank you

anaRodrigues

Hi Martin,

Yes the F1-measure is available, but I wanted to calculate F2, F3, F0.5 etc.. which was not possible before.

Thank you this solves the problem!