Performance (Ranking)
Synopsis
This operator delivers a performance value representing costs for the confidence rank of the true label.
Description
The Performance (Ranking) operator should be used for tasks, where it is not only important that the real class is selected, but also that it receives a comparably high confidence.
This operator will sort the confidences for each label and depending on the rank position of the real label, costs are generated. You can define these costs by the parameter ranking_costs. The costs are entered for whole intervals, so you don't have to enter a cost value for each rank. These intervals are defined by their start rank and range either until the start of the next interval or infinite. Everything before the first mentioned rank will receive costs of 0. The counting of rank starts with 0, so the most confident label is rank 0.
The costs are entered on the right side of the table.
For example, if you want to assign costs of zero if the true label is predicted with the highest confidence, 1 for the second place, 2 for the third and 10 for each following, you have to enter:
１ １
2 2
3 10
Input
labeled data
This input port expects a labeled ExampleSet. The
Output
example set
ExampleSet that was given as input is passed without change to this output port. This is usually used to reuse the same ExampleSet in further operators or to view the ExampleSet in the Results Workspace.
performance
This port delivers a Performance Vector (we call itoutput-performance-vectorfor now). The Performance Vector is a list of performance criteria values. The Performance vector is calculated on the basis of thelabelattribute and thepredictionattribute of the input ExampleSet. Theoutput-performance-vectorcontains performance criteria calculated by this Performance operator (we call itcalculated-performance-vectorhere). If a Performance Vector was also fed at theperformanceinput port (we call itinput-performance-vectorhere), criteria of theinput-performance-vectorare also added in theoutput-performance-vector. If theinput-performance-vectorand thecalculated-performance-vectorboth have the same criteria but with different values, the values ofcalculated-performance-vectorare delivered through the output port. This concept can be easily understood by studying the attached Example Process.
Parameters
Ranking costs
Table defining the costs when the real label isn't the one with the highest confidence