Skip to main content

Replace Missing Values (Series)

Synopsis

This operator replaces missing values in time series.

Description

的参数replace type numerical,replace type nominalandreplace type date timedefines the kind of replacement which is used, respectively for the type of the time series.. The parametersskip other missings,replace infinity,replace empty stringsandensure finite valueshandles how neighboring missing values, positive and negative infinity, empty strings and missing values at the start/end of the series are handled. Be aware that only whenensure finite valuesis set to true it can be ensured that no invalid values (missing, positive/negative infinity, emtpy strings) remain in the series after the replacement.

This operator works on all time series (numerical, nominal and time series with date time values).

Differentiation

Replace Missing Values

The standard Replace Missing Values operator from RapidMiner replaces every missing value with a constant value. This series based operator on the other hand, replaces missing values context based. This means, that a missing value is replaced based on a selected rule taking neighboring values into account.

Input

example set

The ExampleSet which contains the time series data as attributes.

Output

example set

The ExampleSet after applying the replacement. In case ofoverwrite attributesis true original time series attributes are overwritten, if not new attributes with the replaced values are added. For the name of the new attributes a postfix, specified by thenew attributes postfixparameter, is added to the name of the original attributes. Other attributes are not changed.

Parameters

Attribute filter type

This parameter allows you to select the filter for the time series attributes selection filter; the method you want to select the attributes which holds the time series values. The different filter types are:

  • all: This option selects all attributes of the ExampleSet to be time series attributes. This is the default option.
  • single: This option allows the selection of a single time series attribute. The required attribute is selected by theattributeparameter.
  • subset: This option allows the selection of multiple time series attributes through a list (see parameterattributes). If the meta data of the ExampleSet is known all attributes are present in the list and the required ones can easily be selected.
  • regular_expression:该选项允许您指定一个常规expression for the time series attribute selection. The regular expression filter is configured by the parametersregular expression, use except expression and except expression.
  • value_type: This option allows selection of all the attributes of a particular type to be time series attributes. It should be noted that types are hierarchical. For example real and integer types both belong to the numeric type. The value type filter is configured by the parametersvalue type, use value type exception, except value type.
  • block_type: This option allows the selection of all the attributes of a particular block type to be time series attributes. It should be noted that block types may be hierarchical. For example value_series_start and value_series_end block types both belong to the value_series block type. The block type filter is configured by the parametersblock type, use block type exception, except block type.
  • no_missing_values: This option selects all attributes of the ExampleSet as time series attributes which do not contain a missing value in any example. Attributes that have even a single missing value are not selected.
  • numeric_value_filter: All numeric attributes whose examples all match a given numeric condition are selected as time series attributes. The condition is specified by thenumeric conditionparameter.

Attribute

The required attribute can be selected from this option. The attribute name can be selected from the drop down box of the parameter if the meta data is known.

属性

The required attributes can be selected from this option. This opens a new window with two lists. All attributes are present in the left list. They can be shifted to the right list, which is the list of selected time series attributes.

Regular expression

属性whose names match this expression will be selected. The expression can be specified through theedit and preview regular expressionmenu. This menu gives a good idea of regular expressions and it also allows you to try different expressions and preview the results simultaneously.

Use except expression

If enabled, an exception to the first regular expression can be specified. This exception is specified by theexcept regular expressionparameter.

Except regular expression

This option allows you to specify a regular expression. Attributes matching this expression will be filtered out even if they match the first expression (expression that was specified inregular expressionparameter).

值type

This option allows to select a type of attribute.

Use value type exception

If enabled, an exception to the selected type can be specified. This exception is specified by the除了值类型parameter.

Except value type

The attributes matching this type will be removed from the final output even if they matched the before selected type, specified by thevalue typeparameter.

Block type

This option allows to select a block type of attribute.

Use block type exception

If enabled, an exception to the selected block type can be specified. This exception is specified by theexcept block typeparameter.

Except block type

The attributes matching this block type will be removed from the final output even if they matched the before selected type by theblock typeparameter.

Numeric condition

The numeric condition used by the numeric condition filter type. A numeric attribute is selected if all examples match the specified condition for this attribute. For example the numeric condition '>6' will keep all numeric attributes having a value of greater than 6 in every example. A combination of conditions is possible: '>6 &&<11' or '<= 5 ||<0'. But && and || cannot be used together in one numeric condition. Conditions like '(>0 &&<2) || (>10 &&<12)' are not allowed because they use both && and ||.

Invert selection

If this parameter is set to true the selection is reversed. In that case all attributes not matching the specified condition are selected as time series attributes. Special attributes are not selected independent of theinvert selectionparameter as along as theinclude special attributesparameter is not set to true. If so the condition is also applied to the special attributes and the selection is reversed if this parameter is checked.

Include special attributes

Special attributes are attributes with special roles. These are: id, label, prediction, cluster, weight and batch. Also custom roles can be assigned to attributes. By default special attributes are not selected as time series attributes irrespective of the filter conditions. If this parameter is set to true, special attributes are also tested against conditions specified and those attributes are selected that match the conditions.

Has indices

This parameter indicates if there is an index attribute associated with the time series. If this parameter is set to true, the index attribute has to be selected.

Indices attribute

If the parameterhas indicesis set to true, this parameter defines the associated index attribute. It can be either a date, date_time or numeric value type attribute. The attribute name can be selected from the drop down box of the parameter if the meta data is known.

Overwrite attributes

This parameter indicates if the original time series attributes are overwritten by the resulting time series. If this parameter is set to false, the resulting new time series are added as new attributes to the ExampleSet. The name of these new attributes will be the name of the original time series with a postfix added. The postfix is specified by the parameternew attributes postfix.

New attributes postfix

Ifoverwrite attributesis false, this parameter specifies the postfix which is added to the names of the original time series to create the new attribute names.

Replace type numerical

The kind of replacement which is used to replace the missing values of numeric time series.

  • previous value: The previous value in the series is used as a replacement. If the parameterskip other missingsis set to true, neighboring missing values are all replaced by the first previous valid value. Missing values at the start of a series are not replaced, unless the parameterensure finite valuesis set to true. Than the next valid value is used as a replacement.
  • next value: The next value in the series is used as a replacement. If the parameterskip other missingsis set to true, neighboring missing values are all replaced by the next valid value. Missing values at the end of a series are not replaced, unless the parameterensure finite valuesis set to true. Than the first previous valid value is used as a replacement.
  • average: The average of the neighboring values in the series is used as a replacement. If the parameterskip other missingsis set to true, neighboring missing values are all replaced by the average of the neighboring valid values. Missing values at the start and end of a series are not replaced, unless the parameterensure finite valuesis set to true. Than the next, respectively previous valid value is used as a replacement.
  • linear interpolation: A linear interpolation (using the index values from theindex attribute) between the two neighboring values in the series is used to calculate the replacement value. If the parameterskip other missingsis set to true, the next neighboring valid values are used to perform a linear interpolation and all missing values are replaced by the replacement values calculated by the linear interpolation (using the index values from theindex attribute). Missing values at the start and end of a series are not replaced, unless the parameterensure finite valuesis set to true. Than the next, respectively previous valid value is used as a replacement.
  • value: All missing values are replaced by a constant value, specified by thereplace value numericalparameter.

Replace type nominal

The kind of replacement which is used to replace the missing values of nominal time series.

  • previous value: The previous value in the series is used as a replacement. If the parameterskip other missingsis set to true, neighboring missing values are all replaced by the first previous valid value. Missing values at the start of a series are not replaced, unless the parameterensure finite valuesis set to true. Than the next valid value is used as a replacement.
  • next value: The next value in the series is used as a replacement. If the parameterskip other missingsis set to true, neighboring missing values are all replaced by the next valid value. Missing values at the end of a series are not replaced, unless the parameterensure finite valuesis set to true. Than the first previous valid value is used as a replacement.
  • value: All missing values are replaced by a constant value, specified by thereplace value nominalparameter.

Replace type date time

The kind of replacement which is used to replace the missing values of time series with date time values (this is not used for theindices attribute)

  • previous value: The previous value in the series is used as a replacement. If the parameterskip other missingsis set to true, neighboring missing values are all replaced by the first previous valid value. Missing values at the start of a series are not replaced, unless the parameterensure finite valuesis set to true. Than the next valid value is used as a replacement.
  • next value: The next value in the series is used as a replacement. If the parameterskip other missingsis set to true, neighboring missing values are all replaced by the next valid value. Missing values at the end of a series are not replaced, unless the parameterensure finite valuesis set to true. Than the first previous valid value is used as a replacement.
  • average: The average of the neighboring values in the series is used as a replacement. If the parameterskip other missingsis set to true, neighboring missing values are all replaced by the average of the neighboring valid values. Missing values at the start and end of a series are not replaced, unless the parameterensure finite valuesis set to true. Than the next, respectively previous valid value is used as a replacement.
  • linear interpolation: A linear interpolation (using the index values from theindex attribute) between the two neighboring values in the series is used to calculate the replacement value. If the parameterskip other missingsis set to true, the next neighboring valid values are used to perform a linear interpolation and all missing values are replaced by the replacement values calculated by the linear interpolation (using the index values from theindex attribute). Missing values at the start and end of a series are not replaced, unless the parameterensure finite valuesis set to true. Than the next, respectively previous valid value is used as a replacement.
  • value: All missing values are replaced by a constant value, specified by thereplace value date timeparameter.

Replace value numerical

Ifreplace type numericalis set tovalue该参数指定的替换值all missing values of numerical time series.

Replace value nominal

Ifreplace type nominalis set tovalue该参数指定的替换值all missing values of nominal time series.

Replace value date time

Ifreplace type date timeis set tovalue该参数指定的替换值all missing values of time series with date time values.

Skip other missings

If this parameter is set to true, other neighboring values which are also missing are not considered for the determination of the replacement value. If this parameter is set to false and a replacement value would be also a missing value (e.g.,replace type numericalisnext valueand the next value would be missing), the missing value is not replaced.

Replace infinity

If this parameter is set to true, also positive and negative infinity values are replaced in numerical time series. Otherwise they are handled as valid values and are not replaced and considered in the determination of the replacement value for a missing value (e.g.replace type numericalisaverageand one neighboring value is positive infinity, than the replacement value is also positive infinity).

Replace empty strings

If this parameter is set to true, also empty strings are replaced in nominal time series. Otherwise they are handled as valid values and are not replaced and considered in the determination of the replacement value for a missing value (e.g.replace type nominalisnext valueand the next value is an empty string, than the replacement value is also an empty string).

Ensure finite values

If this parameter is set to true, the operator ensures that no invalid values (missing, positive/negative infinity, empty strings) remain in the series after the replacement. The parametersskip other missings,replace infinityandreplace emtpy stringsare automatically set to true. It is also ensured that invalid values at the start/end of a series are replaced with valid ones. See the description of the differentreplace typesfor details.