Problem with text preprocessing

ghamgham MemberPosts:24
edited December 2018 inHelp

嗨

How do I use the stop word dictionary to abbreviate words and replace them with the original words and delete negative words like not?
Where do I download the stopword dictinary?
Thankful

Tagged:

Answers

  • ghamgham MemberPosts:24

    help me...:smileysad:

  • kypexinkypexin Moderator, RapidMiner Certified Analyst, MemberPosts:290Unicorn

    Hi@gham

    I suggest you to look at 'Text processing' extension for RapidMiner which contains lots of very useful operators for working with texts; among them, there's a group of operators named 'Filter stopwords' which are doing the exact thing you need, for different languages.

    As for downloading the dictionary, I guess there's no 'unified' dictionary as every text processing task would basically need different stoword list, so this part need to be completed manually. Maybe you can just google and download any ready dictionaries / lists someone has built before and shared online.

    --
    Vladimir
    http://whatthefraud.wtf
    yyhuang
  • Telcontar120Telcontar120 Moderator, RapidMiner Certified Analyst, RapidMiner Certified Expert, MemberPosts:1,635Unicorn

    If you are using one of the supported languages, there are operators with built-in dictionaries (e.g., English, German, etc.) , so you don't need to download them. If you are using another language, or if you want a custom stopword dictionary, you will need to create it yourself, or find one on the web and download it. That's not functionality which is handled inside RapidMiner, but a little bit of web searching will turn up many useful references.

    Brian T.
    Lindon Ventures
    Data Science Consulting from Certified RapidMiner Experts
    sgenzer
  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM ModeratorPosts:2,959Community Manager

    hello@ghamSome quick recommendations for you:
    •y后our XML process here in this thread (seehttps://youtu.be/KkgB5QXWXJ8and "Read Before Posting" on right when you reply)
    • Attach your dataset if possible (use a fictionalized version if there are privacy concerns)
    • Make sure you have all necessary extensions installed (seehttps://youtu.be/pjBqG3xtXx4)

    Scott

  • ghamgham MemberPosts:24

    Please help my friend help me out.

  • Thomas_OttThomas_Ott RapidMiner Certified Analyst, RapidMiner Certified Expert, MemberPosts:1,761Unicorn

    @ghamdid you post your process like what@sgenzersaid? You have to do some work here, help us troubleshoot were you're getting stuck. We're not going to buildyour process for you.

    sgenzer
  • ghamgham MemberPosts:24

    Hello
    How do I remove the negation?

    help me please

    12. jpg

    12. jpg 0B
  • sgenzersgenzer Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM ModeratorPosts:2,959Community Manager

    I'm sorry - I have no idea what "remove the negation" means.


    Scott

Sign InorRegisterto comment.