How to apply different neural network model inside loop attribute?

binsetyawan

hello everyone, im working with neural network for forecasting

in my project, i got IO Object Collection that contain example sets from loop attributes.

Example Sets on IO Object Collection

内循环的属性,我把验证操作s to forecast with neural network operator so each example set have their own prediction.sliding window validation

To get the best model for each example set, i used optimization (grid) operator and each example set got their own best model (different training cycle, learning rate and momentum).Result on Optimization Parameter Grid (on each example)

When i got their own best model, how do i apply the result of optimization parameter to each example sets in my process? because if i put it on neural net operator, it only apply an model.inside validation operator

binsetyawan

excuse me sir, when i put write model operator after optimize parameter (grid) operator, is the model that come out from mod port is the best model? if its yes, the model name parameter i set test_%{loop_attribute} and the result is i got .mod file for each loop, so i can apply it with read model and connect it to apply model for testing

Thomas_Ott

That is correct, the model that comes out of the Optimize Parameters operator is the best model.

Thomas_Ott

Couldn't you store each model right after the validation and save it with a macro? like model_%{loop_attribute} ? This way you can store each iteration of the model and then use it to score your individual data sets later.

binsetyawan

yeah, i've tried the macro to save the model, i put it adter validation operator, but the name of the model is always overwrite and i dont know what values should i fill on it. so its not possible to appy if they have different amount of nodes in hidden layer?

Thomas_Ott

To prevent it from overwriting, you need to use a valid macro. I choose %{loop_attribute} because it's already defined by the Loop Attribiutes operator. Another option is to use %{t} which is the timestamp when it gets executed.

binsetyawan

im sorry i still confusing with your explanation

Thomas_Ott

You can save the model after each iteration provided that you append the model name with a macro like the timestamp. You would need to save it as "mymodel_%{t}" this way it doesn't overwrite it.

binsetyawan

instead of using macro, is it possible if i made a list of value parameter of ANN model for each example sets and apply it in my process and made each example sets recognize their model? i've tried write parameter operator but the problem is the values overwrite with the latest loop_attribute and if the problem resolved how to apply the result of write parameter operator on my process?

Thomas_Ott

Have you thought of using the Recall and Remember operator? You might be able to load in the write parameters in a loop, Remember them for the one iteration and then load in the second one for hte second iteration.

binsetyawan

Sorry, I'm more confused
I'm going to edit my topic and add pictures to make it easy to understand and know where to add operator

binsetyawan

and i didn't find any solution yetim so confused

Thomas_Ott

An optimized model with the LR and MOM values are exported with each parameter optimization. I see that you have a lot of them in your output because of the loop and using them is going to be tricky but I don't think that hard.

I would consider scoring your data right after the parameter optimization. In this way you would loop your training and optimization and then simulatenously score your data inside the loop, then save each score set with a macro or use annotation to differeniate them.

binsetyawan

yeah i got lot of them because its the best parameter for each example set that i got from loop and join. but i need to know the trick to apply the model that i got because it is the core of the project.

Should i really use macro operator? because when i got best model from optimization parameter operator, i will remove optimization parameter operator and swtich with validation operator which was previously inside optimization parameter operator and apply the model that i got and apply the mod port to data testing.

binsetyawan

up

binsetyawan

so... its not possible?

Thomas_Ott

It's probably possible but very complex. I would post your process and data here and see if someone wants to take up the challenge. Actually, we're starting data challenges if you want to throw out a prize like here:http://community.www.turtlecreekpls.com/t5/General-Chit-Chat/RapidMiner-Data-Modeling-CHALLENGE-200-in-cash-and-prizes/m-p/39159

Thomas_Ott

I just realized you did post your process and data.

Something like this doesn't work?

<运营商激活= " true "类=“乘”兼容ibility="7.5.001" expanded="true" height="103" name="Multiply" width="90" x="447" y="34"/>


























<运营商激活= " true "类=“乘”兼容ibility="7.5.001" expanded="true" height="103" name="Multiply (2)" width="90" x="514" y="370"/>











































输入<连接from_port = " 2 " to_op = to_port =“加入”"right"/>

binsetyawan

ah im sorry it isn't

what im looking for is how to apply different parameter values that i got (TC, LR, MOM) for each example sets to neural net operator

binsetyawan

我想如果我有更多的做这样的事情money :smileysad:

Thomas_Ott

I think the answer is to use Set Macros for your MOM, LR, etc.

<运营商激活= " true "类=“乘”兼容ibility="7.5.001" expanded="true" height="103" name="Multiply" width="90" x="447" y="34"/>













<操作符= " true " class = " optimize_paramet激活ers_grid" compatibility="7.5.001" expanded="true" height="145" name="Optimize Parameters (Grid)" width="90" x="246" y="34">


























<运营商激活= " true "类=“乘”兼容ibility="7.5.001" expanded="true" height="103" name="Multiply (2)" width="90" x="581" y="238"/>





































输入<连接from_port = " 2 " to_op = to_port =“加入”"right"/>

binsetyawan

okay, i know how the grand design from your recommendation... but when you put 2 macros at there, i ithink its similiar with when the validation is inside the optimization parameter and optimize neural net parameter or my something wrong with my understanding?

so i have to run the model to get the best model (which it need much time) and store the model after that for testing i use retrieve operator ro apply the model? hmm somehow, the best model that i've found is useless and cant be applied :catsad: :smileymad:

binsetyawan

and for testing, my data only 12 row (2016, monthly), for windowing operator, parameter that i use is 12 for window size, 1 step size and 1 for horizon. i want to know how good is the performance is so i put performance operator after apply model, but it comes to error, it said that there is no label, so i edit on windowing operator to create label but it comes error too, it said that the examples is low 12+1=13 but the examples only 12.

how i can measure the performance for testing?@Thomas_Ott

Thomas_Ott

To test the performance you will need a label. Usually we do this via a Validation type operator. With respect to your data set, you don't have enough testing data.

binsetyawan

i see, so that is the problem...

may be i should add more data for testing 2015+2016= 24 row, window size 12, step size 1 and horizon 12 to forecast 2017