"How are L pos and L neg set in an SVM?"

eraskin

Noob here. I am working on a basic classifcation problem (responders vs non-responders), where the cost of missing a responder is much higher than the cost of including a non-responder. Specifically, the average sale is $115 and the average cost to mail somebody is $0.68. We want to bias in favor of maximizing sales over minimizing cost to mail.

I can't find any documentation anywhere that shows how the "L pos" and "L neg" settings in an SVM are actually set. Are they numbers between 0 and 1? Are they relative weights for mis-classifying a positive "L pos" and a negative "L neg"? In my case, I would interpret that as setting "L pos" much higher than "L neg", since I want to ensure that we avoid as many false negatives as possible.

Can somebody help me understand how this works?

Thanks in advance...

Eric

lel5287

我知道这是几个月的时间,但我的工作on a very similar problem, so if anyone has any insight on this, that would be greatly appreciated.

Additionally, I'm still exploring classification methods for what I'm working on. Is it possible to set similar loss criteria in other classification methods? (False positives aren't too big of a deal, but false negatives are very costly--also, the test set includes very few responders compared to the number of non-responders, so a lot of methods that just look to improve accuracy without weighing false positives against false negatives classify almost everyone as a non-responder.)

MartinLiebig

Hi,

i don't know what it is but i got eager to find it out.

So apprently it is used in /src/main/java/com/rapidminer/operator/learner/functions/kernel/jmysvm/svm/SVM.java

I figured out there that the balance cost option sets the same values.

double generalCpos = paramOperator.getParameterAsDouble(JMySVMLearner.PARAMETER_L_POS);
double generalCneg = paramOperator.getParameterAsDouble(JMySVMLearner.PARAMETER_L_NEG);
if (paramOperator.getParameterAsBoolean(JMySVMLearner.PARAMETER_BALANCE_COST)) {
generalCpos *= the_examples.count_examples()
/ (2.0d * (the_examples.count_examples() - the_examples.count_pos_examples()));
generalCneg *= ((the_examples.count_examples()) / (2.0d * the_examples.count_pos_examples()));
}
for (int i = 0; i < cPos.length; i++) {
cPos= generalCpos;
cNeg= generalCneg;

So i did a quick test (process XML below). I classified golf with and without balance cost. Further i added weights to ensure class balance this way.

Weights w/o balance cost and w/o class balancing

Temperature -0.32830007919309123
Outlook = sunny -0.2990901852444362
Humidity -0.2759359389650189
Wind = true -0.23094010767584963
Outlook = rain 0.0
Wind = false 0.17320508075688723
Outlook = overcast 0.35248116362113585

w balance cost but w/o class balancing

Outlook = sunny -0.33391948463996746
Humidity -0.2776748983378322
Wind = true -0.25278262899833226
Outlook = rain -0.12521980673998773
Temperature -0.060810818269701084
Wind = false 0.18958697174874925
Outlook = overcast 0.5411008440732543

and finallyw/ scaling but w/o balancing

Outlook = sunny -0.33391948463996746
Humidity -0.2776748983378322
Wind = true -0.25278262899833226
Outlook = rain -0.12521980673998773
Temperature -0.060810818269701084
Wind = false 0.18958697174874925
Outlook = overcast 0.5411008440732543

So apprently the balancing here does the same thing than balancing via weights. Thus I expect LPos and LNeg to be class_weights.

Cheers,
Martin

JEdward

For other classifiers (& SVM too) another way to tackle this with different costs is to use cost based modelling.
Here's an example on how it works. Please note, I chose to use Naive Bayes because it demonstrated the extemes well chosing to send to the entire dataset to 'maximise profit' (Have the least cost)

You can chose to either use the Meta Cost operator or use the Performance (Costs) operators separately.
Meta cost builds several models to favour the cost.
Performance (Costs) tells you the expected cost of the model. (I tend to combine this one with an Optimise operator and then build my model parameters with the ones that decrease the cost the most).

<操作符= " true " class = " naive_激活bayes" compatibility="7.0.000" expanded="true" height="82" name="Naive Bayes" width="90" x="179" y="136"/>
















<操作符= " true " class = " performance_cost激活s" compatibility="7.0.000" expanded="true" height="82" name="Performance (Cost Meta)" width="90" x="112" y="187">
























<操作符= " true " class = " naive_激活bayes" compatibility="7.0.000" expanded="true" height="82" name="Naive Bayes (2)" width="90" x="45" y="34"/>










<操作符= " true " class = " performance_cost激活s" compatibility="7.0.000" expanded="true" height="82" name="Performance (Cost Without Meta)" width="90" x="45" y="238">























<运营商激活= " true " class = " extract_macro“compatibility="7.0.000" expanded="true" height="68" name="Extract Macro" width="90" x="179" y="136">






<运营商激活= " true " class = " extract_macro“compatibility="7.0.000" expanded="true" height="68" name="Extract Macro (2)" width="90" x="179" y="238">

MartinLiebig

While i totally support you i would like to add, that in a best case scenario you can give each example (e.g. customer) a cost (or revenue). Than you can aggregate on this and optimize on the costs. This is very similar but even more detailed. I think this is a better way - if the data is available.

~Martin