"time series prediction"

Vikas

Hi everyone

I am new user of RapidMiner and this is my first post, I have 11 months electric feeder load time series data so I want to forecast one day ahead feeder load with the help of this data.so can anyone guide me how can I do this with the help of RapidMiner ?
Data Format:-
date hour1 hour2 hour3 hour4 hour5 hour6 ......... hour24
10/01/2010 .2934 .1983 .1328 .2032 .1002 .1834 ......... .2903
10/02/2010 .2367 . 1298 .1289 .1901 .1192 .1920 ........ .1902
................. ................................................................................
280 days 24 hour

Thanks

Vikas Gupta

韦塞尔

First convert your data to:

day_time, load
1, .2934
2, .1983
....
n .1902


Then use the windowing operator with the appropriate embedding dimension.
Then use k-nn or linear regression as a learner.

If you upload like 50 rows of data I'll make you an example process.

IngoRM

Dear Vikas,

please post your questions only once in the most appropriate board and not in every board here. Thanks.

Cheers,
Ingo

韦塞尔

Actually I have no idea how to convert this data to 1 column, using rapidminer, so I'm gonna use VIM or python.

Like:
loaddata #<-- this is a comment
0.5144 #<-- first data point
0.5144
0.5144
0.6001
0.6001
0.6859
0.6859
0.7716
0.7716
1.286
1.286
1.286
1.2003
1.2003
1.2003
1.286
1.286
1.5432
1.8004
1.6289
1.5432
1.3717
1.1145
0.8573
0.9431 #<-- day 2
...
1.286 #<-- last data point, etc

edit:
okay here is the data I will be using:

load
0.5144
0.5144
0.5144
0.6001
0.6001
0.6859
0.6859
0.7716
0.7716
1.286
1.286
1.286
1.2003
1.2003
1.2003
1.286
1.286
1.5432
1.8004
1.6289
1.5432
1.3717
1.1145
0.8573
0.9431
0.6859
0.6859
0.6859
0.7716
0.7716
0
0
0
1.6289
1.3717
1.3717
1.3717
0
1.5432
1.4575
1.4575
1.6289
1.8004
1.7147
1.5432
1.3717
1.1145
0.8573
0.7716
0.6859
0.6859
0.7716
0.9431
1.0288
2.0288
1.2003
1.286
1.286
1.4575
1.3717
1.3717
0
1.286
1.4575
1.5432
1.5432
1.8004
1.8004
1.5432
1.3717
1.2003
0.9431
0.8573
0.7716
0.7716
0.8573
0.8573
0.9431
1.0288
1.2003
1.286
1.3717
1.4575
1.3717
1.286
1.2003
1.286
0
0
1.7147
1.8861
1.8861
1.6289
1.3717
1.0288
0.7716
0.6859
0.6001
0.6001
0.6859
0.7716
0.8573
0
0
1.8861
1.5432
1.5432
1.5432
1.5432
1.286
1.3717
1.3717
1.5432
1.8004
1.8004
1.7147
1.5432
1.3717
1.1145
0.9431
0.8573
0.7716
0.6859
0.6859
0.7716
0.9431
1.0288
1.2003
1.2003
1.3717
1.4575
1.6289
1.5432
0
0
0
0
1.3717
1.3717
1.3717
1.2003
1.0288
0.8573
0.6859
0.6001
0.5144
0.5144
0.6001
0.6859
0.6859
0
0.8573
0.9431
0.9431
0.9431
0.7716
0.7716
0
0
0
0.8573
1.0288
1.1145
1.1145
1.0288
0.9431
0.7716
0.6001
0.6859
0.6001
0.6001
0.6001
0.6859
0.6859
0.7716
0.8573
1.0288
1.0288
1.5432
1.0288
0
0
1.286
1.1145
1.1145
1.286
1.3717
1.286
1.1145
1.0288
0.8573
0.6001
0.5144
0.5144
0.5144
0.5144
0.6001
0.6859
0
0.9431
1.1145
1.0288
1.1145
1.0288
0
0
1.286
1.2003
1.1145
1.3717
1.3717
1.286
1.1145
0.9431
0.8573
0.6001
0.5144
0.5144
0.5144
0.5144
0.6001
0.6859
0
0
1.3717
1.1145
1.1145
1.0288
0
0
1.2003
1.1145
1.1145
1.2003
1.286
1.1145
0.9431
0.9431
0.8573
0.5144
0.4287
0.4287
0.4287
0.4287
0.5144
0.6001
0.6859
0.7716
0.8573
0.8573
0.9431
0.9431
0.8573
0
0.9431
0.9431
0.9431
1.2003
1.286
1.2003
1.1145
0.9431
1.0288
0.7716
0.7716
0.6859
0.6859
0.6859
0.7716
0.9431
0
1.286
1.286
1.286
1.3717
1.4575
1.286
1.2003
1.286
1.286
1.5432
1.7147
1.8861
1.7147
1.6289
1.3717
1.2003
0.8573
0.7716
0.6859
0.6859
0.6859
0.7716
0.8573
0.9431
1.1145
1.286
1.3717
1.4575
1.3717
0
1.6289
1.3717
1.286
0.9431
1.1145
1.1145
1.0288
1.0288
0.9431
0.6001
0.5144
0.5144
0.5144
0.5144
0.5144
0.5144
0.5144
0.5144
0.6859
0.6859
0.7716
0.7716
0.6859
0.6859
0.6001
0.6001
0.6001
0.6001
0.8573
1.0288
0.9431
0.9431
0.7716
0.6001
0.5144
0.3429
0.3429
0.3429
0.3429
0.5144
0.6859
0.6859
0.8573
0.8573
0.8573
0.9431
0.9431
0.8573
0.8573
0.8573
0.7716
0.8573
0.9431
1.0288
1.0288
0.9431
0.7716
0.6859
0.5144
0.3429
0.3429
0.3429
0.3429
0.5144
0.5144
0.6859
0.7716
0.8573
0.8573
1.1145
1.2003
1.2003
1.2003
1.2003
1.2003
1.286
1.5432
1.6289
1.5432
1.286
1.1145
0.9431
0.6859
0.5144
0.5144
0.5144
0.6859
0.8573
0.8573
0.9431
1.2003
1.1145
1.2003
1.286
1.3717
0
0
1.3717
1.3717
1.4575
1.7147
1.8004
1.7147
1.3717
1.2003
0.9431
0.7716
0.6859
0.6001
0.6001
0.6001
0.6859
0.7716
1.0288
1.1145
1.2003
1.286
1.4575
1.4575
0
0
1.1145
1.3717
1.5432
1.7147
1.8861
1.6289
1.286
1.1145
0.9431
0.7716
0.6859
0.5144
0.5144
0.5144
0.6859
0.8573
0.9431
1.0288
1.1145
1.2003
1.3717
1.3717
1.286
1.2003
1.286
1.286
1.3717
1.6289
1.7147
1.6289
1.3717
1.1145
1.0288
0.6859
0.5144
0.5144
0.5144
0.5144
0.6859
0.8573
0.8573
1.1145
1.1145
1.2003
1.3717
1.3717
0
1.0288
1.3717
1.2003
1.286
1.5432
1.6289
1.4575
1.286
1.1145
1.0288
0.7716
0.6859
0.6859
0.6859
0.6859
0.6859
0.7716
0.8573
0.9431
1.1145
1.0288
1.1145
1.0288
0.9431
0.7716
0.8573
0.8573
0.8573
1.1145
1.286
1.286
1.1145
1.0288
0.8573
0.7716
0.6859
0.6001
0.6001
0.6859
0.7716
0.8573
1.0288
1.0288
1.0288
1.1145
1.0288
1.0288
0
1.286
1.2003
1.2003
1.2003
1.5432
1.5432
1.4575
1.286
1.1145
0.9431
0.6859
0.6859
0.6001
0.6001
0.6859
0.7716
0.8573
0.9431
1.1145
1.1145
1.2003
1.286
1.286
0
1.2003
1.2003
1.2003
1.3717
1.6289
1.6289
1.5432
1.286
1.1145
0.8573
0.7716
0.6001
0.6001
0.6001
0.6859
0.7716
0.8573
0.9431
1.2003
1.1145
1.1145
1.2003
1.3717
0
1.286
1.2003
1.286
1.3717
1.6289
1.8861
1.8004
1.4575
1.2003
0.8573
0.6859
0.6001
0.6001
0.6001
0.6859
0.8573
0.8573
0.9431
1.1145
1.1145
1.1145
0
0
0
0
0
0
1.7147
1.8004
1.8861
1.5432
1.3717
1.1145
0.7716
0.6859
0.6001
0.6001
0.6001
0.6859
0.6859
0.7716
1.0288
1.1145
1.1145
1.2003
1.2003
1.2003
1.2003
1.0288
1.0288
1.1145
1.3717
1.5432
1.6289
1.5432
1.5432
1.286
1.1145
0.8573
0.6001
0.5144
0.5144
0.5144
0.6859
0.7716
0.9431
1.1145
1.2003
1.2003
1.3717
1.3717
1.2003
1.1145
1.1145
1.2003
1.286
1.5432
1.6289
1.4575
1.286
1.1145
0.8573
0.6859
0.5144
0.5144
0.5144
0.5144
0.6859
0.8573
1.1145
1.3717
1.6289
1.6289
1.6289
1.4575
1.4575
1.4575
1.3717
1.3717
1.4575
1.8861
1.9719
1.9719
1.6289
1.6289
1.3717
1.2003
0.9431
0.8573
0.8573
0.8573
0.8573
1.2003
1.3717
1.4575
1.5432
1.8861
2.1434
2.1434
2.1434
1.9719
2.0576
1.2003
2.0576
2.572
2.572
2.3148
1.8861
1.7147
1.3717
1.3717
1.1145
1.1145
0
0.8573
0.9431
1.2003
1.286
1.5432
1.6289
1.8004
2.1434
2.1434
2.0576
1.8861
2.0576
2.0576
2.2291
2.4006
2.4863
2.3148
1.9719
1.7147
1.4575
1.286
0.9431
0.9431
0.8573
0.8573
0.9431
1.1145
1.3717
1.4575
1.6289
1.8004
2.1434
2.1434
2.1434
1.9719
1.9719
2.1434
2.1434
2.4006
2.4863
2.3148
2.0576
1.7147
1.5432
1.1145
1.0288
0.9431
0.9431
0.8573
1.0288
1.2003
1.3717
1.5432
1.6289
1.8861
2.1434
2.0576
2.0576
0
2.0576
2.0576
2.0576
2.4006
2.4006
2.2291
1.9719
1.7147
1.4575
1.2003
0.9431
0.8573
0.8573
0.8573
0.8573
1.2003
1.3717
1.5432
1.5432
1.8004
2.1434
2.0576
2.0576
1.9719
1.8861
2.0576
2.0576
2.3148
2.4006
2.2291
1.7147
1.8004
1.3717
1.1145
0.9431
0.8573
0.8573
0.8573
0.8573
1.1145
1.2003
1.4575
1.5432
1.7147
2.1434
2.0576
1.9719
1.8004
1.8861
1.8861
1.8861
2.2291
2.3148
2.2291
2.1434
1.6289
1.3717
1.0288
0.8573
0.8573
0.7716
0.7716
0.7716
0.9431
1.1145
1.286
1.3717
1.5432
1.6289
1.5432
1.3717
1.286
1.2003
1.2003
1.286
1.6289
1.8004
1.8004
1.6289
1.3717
1.2003
1.0288
0.8573
0.8573
0.8573
1.0288
1.2003
1.286
1.286
1.3717
1.5432
1.7147
1.8861
1.9719
1.8004
1.8004
1.7147
1.8004
1.8861
2.0576
2.2291
2.0576
1.7147
1.5432
1.2003
1.0288
0.8573
0.8573
0.8573
0.8573
1.0288
1.2003
1.286
1.4575
1.4575
1.6289
1.8004
1.7147
1.6289
1.6289
1.6289
1.7147
1.8004
1.9719
2.3148
2.2291
1.8861
1.5432
1.286
0.9431
0.8573
0.7716
0.7716
0.7716
0.8573
1.1145
1.286
1.4575
1.5432
1.7147
1.8004
1.8861
1.8004
1.8004
1.8861
1.8861
1.9719
2.3148
2.572
2.3148
1.8004
1.6289
1.3717
1.0288
0.8573
0.8573
0.8573
0.8573
1.0288
1.2003
1.2003
1.4575
1.6289
1.7147
1.8861
1.8861
1.8004
1.8861
1.8861
1.8004
0
2.4006
2.3148
2.0576
1.8004
1.6289
1.286
1.0288
0.9431
0.8573
0.6859
0.6859
0.8573
1.2003
1.286
1.4575
1.6289
1.8004
1.9719
2.1434
1.9719
1.8861
1.8861
1.9719
2.0576
2.3148
2.4863
2.1434
1.9719
1.8004
1.4575
1.1145
0.8573
0.7716
0.7716
0.7716
0.8573
1.0288
1.2003
1.4575
1.4575
1.8004
1.8861
1.8861
1.8004
1.7147
1.6289
1.7147
1.7147
2.0576
2.3148
2.1434
1.8861
1.5432
1.3717
1.1145
0.8573
0.7716
0.9431
0.9431
1.0288
1.1145
1.3717
1.7147
1.8004
2.0576
1.6289
1.4575
1.3717
1.286
1.2003
0
1.286
1.8004
1.8004
1.8861
1.6289
1.4575
1.4575
1.1145
0.8573
0.7716
0.7716
0.7716
0.7716
1.0288
1.286
1.3717
1.5432
1.8861
2.0576
2.1434
2.0576
1.8861
1.8861
1.8861
1.9719
2.3148
2.2291
2.1434
1.9719
1.6289
1.4575
1.0288
0.8573
0.8573
0.7716
0.7716
0.7716
1.0288
1.286
1.5432
1.7147
1.8861
2.0576
2.0576
1.8861
0
1.9719
1.9719
2.0576
2.2291
2.2291
2.0576
1.8861
1.6289
1.3717
1.0288
0.8573
0.7716
0.7716
0.7716
0.7716
0.9431
1.286
1.3717
1.4575
1.7147
1.8861
1.8861
1.8861
1.7147
1.7147
1.7147
1.8861
2.1434
2.3148
2.0576
1.8004
1.5432
1.286
0.9431
0.8573
0.7716
0.7716
0.7716
0.7716
1.0288
1.2003
1.3717
1.5432
1.8004
1.8004
1.8861
1.8004
1.7147
1.7147
1.7147
2.2291
2.2291
2.2291
2.0576
1.8004
1.4575
1.2003
0.9431
0.7716
0.7716
0.7716
0.9431
0.9431
1.2003
1.4575
1.8004
1.8004
2.1434
2.1434
2.2291
2.2291
2.3148
2.2291
2.1434
1.8004
2.2291
2.3148
1.9719
1.7147
1.4575
1.286
1.0288
0.7716
0.7716
0.6859
0.6859
0.7716
0.9431
1.2003
1.3717
1.6289
1.8004
2.4006
2.4006
2.3148
2.2291
2.2291
2.4006
2.4006
2.7435
2.7435
2.6578
2.3148
1.9719
1.6289
1.2003
1.0288
1.0288
0.9431
1.0288
1.1145
1.2003
1.3717
1.5432
1.8004
1.8861
1.8004
1.7147
1.5432
1.6289
1.5432
1.4575
1.5432
1.9719
2.3148
2.2291
1.9719
1.8861
1.6289
1.2003
1.0288
0.9431
0.9431
0.9431
1.1145
1.3717
1.5432
1.8004
1.8861
2.1434
2.3148
2.2291
2.2291
0
2.1434
2.1434
2.2291
2.572
2.7435
2.572
2.3148
2.0576
1.7147
1.286

Vikas

Thanks for help me Wessel

Please help me about windowing operator(horizon,window size) to forecast the feeder load one day ahead.

Thanks
Vikas

韦塞尔

Your problem does not seem to be really interesting.
So your better of with classical statistics. No need for windowing, embedding, and machine learning here.

http://devio.us/~wessel/load/load.jpeg
http://devio.us/~wessel/load/load2.jpeg

Vikas

实际上我想建立一个模型,我可以predict the load in advance(1 or 2 day ahead) with the help of previous load data which can improve load shedding management of electric feeder.

韦塞尔

Meh, if you insist creating a model using heavy number crunching machine learning....

here is the process:

< portSpacing端口= " sink_performance”间隔= " 0 " / >



















































<连接from_op = "lect by Weights" from_port="example set output" to_op="Validation2" to_port="training"/>
<连接from_op = "lect by Weights" from_port="original" to_port="result 3"/>

韦塞尔

Result:
correlation: 0.795 +/- 0.136 (mikro: 0.786)

Vikas

If we see the data there are many outlier(0 and equal load) or human intervention so for better result should I perform outlier analysis before the forecasting ?

韦塞尔

I don't know, it depends on your application.

A correlation of 0.8 is already really good.

Depends also on how much noise your sensor has.

韦塞尔

So, ehm, you got the "process" to run?

Vikas

Dear Wessel

I got the process but please give me some help abut it's Output

1:- Prediction trend accuracy and correlation both are same thing?
2:-Can you give me some explanation about its output of process?

Thanks
Vikas

韦塞尔

No they are not the same, but they are related.

Look at the scatter plot of predicted load vs actual load.
When a data point is predicted correctly it lies exactly on the diagonal.
You see that all data points that are not 0 are predicted with only a small error.

The error is bigger in data points that are 0, which is expected because they are anomalous values.

I could have used "mean absolute error" instead of "correlation".
But the nice thing about "correlation" is that its invariant to the dataset.
If I would multiply all data points by a factor 100, "mean absolute error" would go up by a factor 100.
Correlation stays the same, since its normalized between -1 and 1.

Vikas

Hi Wessel

Can you help me about this linear regression generated by process
0.299 * load-23 - 0.041 * load-19 + 0.006 * load-15 - 0.007 * load-8 - 0.014 * load-5 + 0.217 * load-1 + 0.407 * load-0 + 0.182
for forecasting of one day ahead load.

韦塞尔

What you want to know about this?

Vikas

this is regression equation so how can I forecast(calculate) of load at 12,11,10,7 hour can you show me one example?

韦塞尔

I'm not sure I understand what you are asking.

Vikas

Can you suggest me any other alternative for prediction of (one day load) with the help of previous model ?

韦塞尔

An alternative would be: Arima:
http://en.wikipedia.org/wiki/Autoregressive_integrated_moving_average

Vikas

Can I apply ARIMA using RapidMiner operators for forecasting the load?

韦塞尔

No.

Vikas

Hi Wessel

Can you give me some idea about one hour ahead load prediction using same data set ?

Thanks

韦塞尔

I gave you a full implementation in Rapid Miner, and a link to alternative approach,...

isn't that ideas enough?