How to store performance metrics from each 10 fold cross validation?

archu92

Hi experts,

I need help in storing performance metrics calculated for each 10 folds validation. I used log operator but it only shows three performance values with standard deviation.

Please help.

Thank you,

Archana

MartinLiebig

Hi Archana,

i think there are several ways to do this. A quick one is to store each fold in the repo with a different name. Attached is a processes doing it. It uses the macro %{a} which is always the number of executions of each operators. Be careful if you use this in a loop/optimize. You need to store the process in the repo in order to make it work.

~Martin

<连接from_port="training set" to_op="Decision Tree" to_port="training set"/>
<连接from_op="Decision Tree" from_port="model" to_port="model"/>














<连接from_port="model" to_op="Apply Model" to_port="model"/>
<连接from_port="test set" to_op="Apply Model" to_port="unlabelled data"/>
<连接from_op="Apply Model" from_port="labelled data" to_op="Performance" to_port="labelled data"/>
<连接from_op="Performance" from_port="performance" to_op="Store" to_port="input"/>
<连接from_op="Store" from_port="through" to_port="performance 1"/>








<连接from_op="Retrieve Golf" from_port="output" to_op="Cross Validation" to_port="example set"/>

archu92

Hi Martin,

Thank you for quick reply. I imported attached processes and saved in repo but it is not storing set of values for each fold in validation.

Attached the error message while executing process.

MartinLiebig

Hey,

it should create a new folder results with the performance vectors, did you check for this?

~Martin

MartinLiebig

我再一次,

alternativly you can use another way to do this missusing the App Objects. Just have a look.

~Martin

<连接from_port="training set" to_op="Decision Tree" to_port="training set"/>
<连接from_op="Decision Tree" from_port="model" to_port="model"/>





















<运营商激活= " true " class = " recall_from_app”compatibility="7.3.001" expanded="true" height="82" name="Recall from App" width="90" x="45" y="85">








<连接from_port="in 1" to_op="Collect" to_port="input 1"/>
<连接from_op="Recall from App" from_port="result" to_op="Collect" to_port="input 2"/>
<连接from_op="Collect" from_port="collection" to_op="Publish to App" to_port="store"/>
<连接from_op="Publish to App" from_port="stored" to_port="out 1"/>









<连接from_port="in 1" to_op="Publish to App (2)" to_port="store"/>
<连接from_op="Publish to App (2)" from_port="stored" to_port="out 1"/>






<连接from_port="model" to_op="Apply Model" to_port="model"/>
<连接from_port="test set" to_op="Apply Model" to_port="unlabelled data"/>
<连接from_op="Apply Model" from_port="labelled data" to_op="Performance" to_port="labelled data"/>
<连接from_op="Performance" from_port="performance" to_op="Multiply" to_port="input"/>
<连接from_op="Multiply" from_port="output 1" to_op="Handle Exception" to_port="in 1"/>
<连接from_op="Multiply" from_port="output 2" to_port="performance 1"/>








<运营商激活= " true " class = " recall_from_app”compatibility="7.3.001" expanded="true" height="103" name="Recall from App (2)" width="90" x="380" y="136">



<连接from_op="Retrieve Golf" from_port="output" to_op="Cross Validation" to_port="example set"/>
<连接from_op="Cross Validation" from_port="performance 1" to_op="Recall from App (2)" to_port="through 1"/>
<连接from_op="Recall from App (2)" from_port="result" to_port="result 1"/>

archu92

Hi,

Problem solved, now it is making result folder and storing all the values. If i need other performance metric such as precision or specificity?

Thank you