"Problem with R-script results"
Contributor I
Hi everyone,
I used the R extension before and always managed to get the data out using "outdata <-as.data.frame(Results)"
I'm currently working on a script that download twitter followers using the twitter API, everything work fine and I managed to create the results data-frame but Rapidminer doesn't like it and spit out this error:
"In order to import an R Data Frame as example set the data frame must provide attribute names"
I'm confused because when I check my result variable with colnames(outdata), the column names are assigned as you can see here:
colnames(Results)
[1] "id_str" "name" "screen_name"
[4] "url" "profile_image_url" "description"
[7] "location" "followers_count" "friends_count"
[10] "statuses_count" "created_at"
Anyone else had this problem before?
Here's the process and the script:
I used the R extension before and always managed to get the data out using "outdata <-as.data.frame(Results)"
I'm currently working on a script that download twitter followers using the twitter API, everything work fine and I managed to create the results data-frame but Rapidminer doesn't like it and spit out this error:
"In order to import an R Data Frame as example set the data frame must provide attribute names"
I'm confused because when I check my result variable with colnames(outdata), the column names are assigned as you can see here:
colnames(Results)
[1] "id_str" "name" "screen_name"
[4] "url" "profile_image_url" "description"
[7] "location" "followers_count" "friends_count"
[10] "statuses_count" "created_at"
Anyone else had this problem before?
Here's the process and the script:
https://api.twitter.com/oauth2/token", ; config(httpheader = c( "Authorization" = paste("Basic", secret), "Content-Type" = "application/x-www-form-urlencoded;charset=UTF-8" )), body = "grant_type=client_credentials", encode = "multipart" ); #Extract the access token token <- paste("Bearer", content(req)$access_token) #Request Followers count url <- paste("https://api.twitter.com/1.1/users/lookup.json?screen_name=",target,sep=""); req& lt; - GET (url配置(httpheader = c(“Authorization" = token))); json <- content(req, as = "text"); tmp <- fromJSON(json); num_followers <- tmp[[1]]$followers_count; #If more than 5000 followers split ID request otherwise retrieve all in one request if (num_followers<5000) { #Request <5000 IDs url <- paste("https://api.twitter.com/1.1/followers/ids.json?screen_name=",target,sep=""); req& lt; - GET (url配置(httpheader = c(“Authorization" = token))); json <- content(req, as = "text"); IDs <- fromJSON(json); IDlist <- data.frame(IDs$ids); } else { #Request batches of IDs batches <- ceiling(num_followers/5000); batch_size <- ceiling(num_followers/batches); curs <- "-1"; IDlist <- data.frame(); for (i in 1:batches) { url <- paste("https://api.twitter.com/1.1/followers/ids.json?cursor=",curs,"&screen_name=",target,"&count=",batch_size,sep=""); req& lt; - GET (url配置(httpheader = c(“Authorization" = token))); json <- content(req, as = "text"); IDs <- fromJSON(json); tmp <- data.frame(IDs$ids); IDlist <- rbind(IDlist, tmp); curs <- IDs$next_cursor_str; Sys.sleep(60); }; }; #Loop through IDs in batches to retrieve full profiles batches <- ceiling(num_followers/100); batch_size <- ceiling(num_followers/batches); BatchIDcounter <- 0; FollowersList <- data.frame(); for (i in 1:batches) { start <- BatchIDcounter; end <- BatchIDcounter+batch_size; if (end>length(IDlist[,1])) { end <-length(IDlist[,1]); batch_size <- (end-start); }; BatchIDs <- paste(IDlist[start:end,1],collapse=","); url <- paste("https://api.twitter.com/1.1/users/lookup.json?include_entities=false&user_id=",BatchIDs,sep=""); req& lt; - GET (url配置(httpheader = c(“Authorization" = token))); json <- content(req, as = "text"); profiles <- fromJSON(json); for (j in 1:batch_size) { tmp <- data.table(id_str="",name="",screen_name="",url="",profile_image_url="",description="",location="",followers_count="",friends_count="",statuses_count="",created_at=""); if (length(profiles[]$id_str)==0) {tmp$id_str=""} else{tmp$id_str=head(profiles[]$id_str[1])}; if (length(profiles[]$name)==0) {tmp$name=""} else{tmp$name=head(profiles[]$name[1])}; if (length(profiles[]$screen_name)==0) {tmp$screen_name=""} else{tmp$screen_name=head(profiles[]$screen_name[1])}; if (length(profiles[]$url)==0) {tmp$url=""} else{tmp$url=head(profiles[]$url[1])}; if (length(profiles[]$profile_image_url)==0) {tmp$profile_image_url=""} else{tmp$profile_image_url=head(profiles[]$profile_image_url[1])}; if (length(profiles[]$description)==0) {tmp$description=""} else{tmp$description=head(profiles[]$description[1])}; if (length(profiles[]$location)==0) {tmp$location=""} else{tmp$location=head(profiles[]$location[1])}; if (length(profiles[]$url)==0) {tmp$url=""} else{tmp$url=head(profiles[]$url[1])}; if (length(profiles[]$followers_count)==0) {tmp$followers_count=""} else{tmp$followers_count=head(profiles[]$followers_count[1])}; if (length(profiles[]$friends_count)==0) {tmp$friends_count=""} else{tmp$friends_count=head(profiles[]$friends_count[1])}; if (length(profiles[]$statuses_count)==0) {tmp$statuses_count=""} else{tmp$statuses_count=head(profiles[]$statuses_count[1])}; if (length(profiles[]$created_at)==0) {tmp$created_at=""} else{tmp$created_at=head(profiles[]$created_at[1])}; FollowersList <- rbind(FollowersList, tmp); }; BatchIDcounter <- end; Sys.sleep(5); }; #Output outdata <- as.data.frame(FollowersList); "/>
Tagged:
0
Answers
Here the log details:
I see that the same problem appeared in another posthttp://rapid-i.com/rapidforum/index.php/topic,7128.msg24720.html#msg24720.
"tennenrishin" reported that this issue only happened when NA values were present however I tested that and in my case it happens even if all the values are present.
I even tried to explicitly declare every column of the DataFrame when creating it but no luck...
This was actually one of the main reasons why I stopped using RM and moved to do most of my work directly in R... But that's a real pity as there are many things that I like about RM.
Any possible solution?
I had the same problem. What I figured is when you put data from R to Rapidminer you must assure that all character data is a factor. Example
If you have a input variable name as "input" and you want to create a column "newcolumn" with the value "newvalue" may you type
input$newcolumn<-"newvalue"
But this expression will give you a error.
However, if you try
input$newcolumn<-as.factor("newvalue" )
Rapidminer will recognize the new attribute
https://marketplace.www.turtlecreekpls.com/UpdateServer/faces/product_details.xhtml?productId=rmx_r_scripting
Thanks