"Importing large local JSON file into rapid miner"

cjc2238cjc2238 MemberPosts:1Contributor I
edited June 2019 inHelp

Hey all,

I'm trying to import a ~30gb dataset into rapidminer but it keeps crashing. I suspect I am doing this incorrectly as I have never worked with JSON files before. From my research it seems I need to connect to an online database to import the data into rapid miner but I am trying to work from a local file.

If it helps at all, the file I am working wth is located here:https://www.reddit.com/r/datasets/comments/3bxlg7/i_have_every_publicly_available_reddit_comment/

Any help wpuld be greatly appreciated~

-C

Tagged:

Answers

  • Thomas_OttThomas_Ott RapidMiner Certified Analyst, RapidMiner Certified Expert, MemberPosts:1,761Unicorn

    Studio will trying to load that entire 30gb set into memory. Do you have enough memory to available to handle that large file?

Sign InorRegisterto comment.