Multi Label Performance

Synopsis

This operator evaluates the performance of a Multi Label Model.

Description

A Multi Label Model is a meta model able to predict multiple label attributes at once. For each label attribute, the Multi Label Model contains a Prediction Model. It can be trained by the Multi Label Modeling operator.

The Multi Label Performance operator can be used to evaluate the performance of such a Multi Label Model after it was applied on a test ExampleSet by using the Apply Model operator. The result of the application of the Multi Label Model on a test ExampleSet has to be provided to thelabelled setinput port.

The operator can automatically detect the label and prediction attributes for which the performances shall be evaluated. All special attributes with a roleprediction_<name of label attribute>are considered to be prediction attributes, while the name of the label attribute is extracted from the role. The label attributes can also be selected by anattribute filterif the parameterauto detect label and prediction attributesis deselected.

For each of the selected label attributes, the ExampleSet at thelabelled setinput port has to contain the label attribute and the corresponding prediction attribute. If the label attribute is nominal, the ExampleSet has to contain also the corresponding confidence attributes.

The inner subprocess of the operator is executed for each selected label attribute. The roles of the current label, current prediction and (if existing) the current confidence attributes are set to the corresponding roles and the input data is provided at the innerlabelled setport. It can be used to evaluate the performance for the corresponding label attribute. A Performance Vector has to be provided to theperformanceport of the inner subprocess. Additional objects can be passed in and out of the subprocess through theinputandoutputport extender.

All Performance Vectors are collected and provided as a collection at thecollection of performances输出端口。如果可能的话Perfor的平均值mance Vectors is calculated (only possible if the types and number of Performance Criteria in each Performance Vector are the same). The averaged Performance Vector for all label attributes is provided at theperformance输出端口。

To ensure a defined behavior the input ExampleSet is not allowed to have a normal 'Label' or 'Prediction' attribute. If the input data has such attributes, their roles have to be set to different roles with theSet Roleoperator. Note that the role of all other attributes are not changed. The name and type of the current label attribute in the subprocess can be added as macros if the parameteradd macrosis selected.

Input

labelled set

The input ExampleSet for which the multi label prediction performances are evaluated. The input ExampleSet needs to have the label, the prediction attribute and (if the label is nominal) confidence attributes for each selected label attribute.

input

This port is a port extender, which means if a port is connected a newinputport is created. Any IOObject can be connected to the port and is passed to the corresponding innerinputport for each iteration.

Output

collection of performances

A collection of Performance Vectors for each of the selected label and prediction attribute.

performance

An averaged Performance Vector of the performances of all selected label and prediction attributes. The calculation of the averaged Performance Vector is only possible if the types and number of Performance Criteria in each Performance Vector are the same.

example set

The ExampleSet that was given as input is passed through without changes.

output

This port is a port extender, which means if a port is connected a newoutputport is created. The port collects every result that is provided by the inner process and returns a collections of all iterations.

Parameters

Auto detect label and prediction attributes

If selected the label and prediction attributes are automatically detected. All special attributes with a roleprediction_<name of label attribute>are considered to be prediction attributes, while the name of the label attribute is extracted from the role.

Attribute filter type

This parameter allows you to select the filter for the label attributes selection; the method you want to select the attributes for which performances are evaluated. Note that the filter is applied on all attributes indepent of their special role (the parameterinclude special attributesis set totruefor this attribute selection and cannot be changed). The different filter types are:

• all: This option selects all attributes of the ExampleSet to be label attributes. This is the default option.
• single: This option allows the selection of a single label attribute. The required attribute is selected by theattributeparameter.
• subset: This option allows the selection of multiple label attributes through a list (see parameterattributes). If the meta data of the ExampleSet is known all attributes are present in the list and the required ones can easily be selected.
• regular_expression: This option allows you to specify a regular expression for the label attribute selection. The regular expression filter is configured by the parametersregular expression, use except expression and except expression.
• value_type: This option allows selection of all the attributes of a particular type to be label attributes. It should be noted that types are hierarchical. For example real and integer types both belong to the numeric type. The value type filter is configured by the parametersvalue type, use value type exception, except value type.
• block_type: This option allows the selection of all the attributes of a particular block type to be label attributes. It should be noted that block types may be hierarchical. For example value_series_start and value_series_end block types both belong to the value_series block type. The block type filter is configured by the parametersblock type, use block type exception, except block type.
• no_missing_values: This option selects all attributes of the ExampleSet as label attributes which do not contain a missing value in any example. Attributes that have even a single missing value are not selected.
• numeric_value_filter: All numeric attributes whose examples all match a given numeric condition are selected as label attributes. The condition is specified by thenumeric conditionparameter.

Attribute

The required attribute can be selected from this option. The attribute name can be selected from the drop down box of the parameter if the meta data is known.

Attributes

The required attributes can be selected from this option. This opens a new window with two lists. All attributes are present in the left list. They can be shifted to the right list, which is the list of selected label attributes.

Regular expression

Attributes whose names match this expression will be selected. The expression can be specified through theedit and preview regular expressionmenu. This menu gives a good idea of regular expressions and it also allows you to try different expressions and preview the results simultaneously.

Use except expression

If enabled, an exception to the first regular expression can be specified. This exception is specified by theexcept regular expressionparameter.

Except regular expression

This option allows you to specify a regular expression. Attributes matching this expression will not be selected even if they match the first expression (expression that was specified inregular expressionparameter).

Value type

This option allows to select a type of attribute.

Use value type exception

If enabled, an exception to the selected type can be specified. This exception is specified by theexcept value typeparameter.

Except value type

The attributes matching this type will not be selected even if they matched the before selected type, specified by thevalue typeparameter.

Block type

这个选项允许选择一块类型的素质bute.

Use block type exception

If enabled, an exception to the selected block type can be specified. This exception is specified by theexcept block typeparameter.

Except block type

The attributes matching this block type will not be selected even if they matched the before selected type by theblock typeparameter.

Numeric condition

The numeric condition used by the numeric condition filter type. A numeric attribute is selected if all examples match the specified condition for this attribute. For example the numeric condition '>6' will keep all numeric attributes having a value of greater than 6 in every example. A combination of conditions is possible: '>6 &&<11' or '<= 5 ||<0'. But && and || cannot be used together in one numeric condition. Conditions like '(>0 &&<2) || (>10 &&<12)' are not allowed because they use both && and ||.

Invert selection

If this parameter is set to true the selection is reversed. In that case all attributes not matching the specified condition are selected as label attributes.

Add macros

If selected macros containing the name and the value type of the current label attribute are added in each iteration of the subprocess.

Current label name macro

Ifadd macrosis true, this parameter defines the name of the macro holding the current label attribute name.

Current label type macro

Ifadd macrosis true, this parameter defines the name of the macro holding the current label attribute type.

Enable parallel execution

该参数使t的并行执行he inner processes. Please disable the parallel execution if you run into memory problems.