How to extract a piece of text occurring before a certain type of formatting in html?

MRNJEM001

Hi community

Beginner here, I've tried my best to figure it out but unfortunately haven't cracked the case.

I have a piece of software that outputs text in html with certain words in red. I need to get a document full of the immediately preceding word to all red words.

例如,我需要thiswordandalsothatword as output.

Here is an example extract of the software output:

<跨类=span-black>substantialclass=apple-converted-space> class=span-red>challengesclass=apple-converted-space> class=span-black>in

As you can see, it is quite messy (a random   in between each word).

In this extract,challengesis the flagged word and I need to output substantial. In a document there are a few hundred red words.

Is there any way I can accomplish this in RapidMiner? I've tried using Cut Document, Documents to Data. Also the Rosette Text Analytics and Information-Extraction extensions, but I'm quite lost.

Thanks!

JEdward

I took a very similar approach to Kayman for this. I first replaced the &nbps with a space.

Then I replaced all occurances of the redword class span with REDWORDWOO

Next I removed all other HTML tags from the document. After tokenizing and generating 2Grams I can then select only the tokens that end "_REDWORDWOO“ which are the words before a redword.

<参数键=“文本”值= " = sp & lt;跨度# 10类an-black><span lang=EN-ZA style='font-size:10.0pt;line-height:107%;
font-family:"FreeSans",serif;color:#4E4E4A'>a</span></span><span
class=apple-converted-space><span lang=EN-ZA style='font-size:10.0pt;
line-height:107%;font-family:"FreeSans",serif;color:#4E4E4A'>&nbsp;</span></span><span
class=span-black><span lang=EN-ZA style='font-size:10.0pt;line-height:107%;
font-family:"FreeSans",serif;color:#4E4E4A'>reasonably</span></span><span
class=apple-converted-space><span lang=EN-ZA style='font-size:10.0pt;
line-height:107%;font-family:"FreeSans",serif;color:#4E4E4A'>&nbsp;</span></span><span
class=span-red><span lang=EN-ZA style='font-size:10.0pt;line-height:107%;
font-family:"FreeSansBold",serif;color:#E74E31'>good</span></span>"/>







1. Replace nbsp<br/>2. Change all occurances of Red colour into REDWORDWOO<br/>3. Remove all other HTML content.



Generate 2-grams



Remove any tokens that don't end in _REDWORDWOO





Remove _REDWORDWOO

Telcontar120

Is the exact html formatting around the words of interest always the same? If so, you should be able to extract them pretty consistently using the Cut Document & Extract Information operators with an appropriately crafted regular expression or regular region selection. Of course, if there are other words that have the same formatting, then you might end up with some "false positive" matches as well, but at least you would have the terms of interest.

MRNJEM001

The trouble is there are thousands of words formatted in black. All the exact same html. I need to extract only the words just before the red formatted words. Would the method you describe be possible for that?

The example below reads "a reasonablygood" and I need to extract ie. "reasonably", but not "a" (and other words before that).

class=span-black>aclass=apple-converted-space> class=span-black>reasonablyclass=apple-converted-space> class=span-red>good

Telcontar120

Ah, I see, so my "simple" solution of taking everything between the "black" and "red" codes would create way too many false positives to be helpful.

It still seems like a properly crafted regex should be able to get this using a backwards lookup, but it's beyond my limited skills with all the html formatting in the way. Maybe one of our other regex experts could weigh in?@Edin_Klapic@BalazsBarany@kayman@JEdward@Thomas_Ott

kayman

Hmm, nice challenge...

Not sure if it is the best approach but a possible approach would be to make use of the fact that red words are by default encapsulated by a span containing the style. So a multistep approach might work.

Some assumptions I make based on your code snippet :

Blackwords are in a so called span-black tag, red words in the span-red tags, other spans can be removed because no real use.

First capture the red tags as follows

(?ms)(.*?)<\/span><\/span>

and replace with redword:$1
next capture the black tags as follows :

(?ms)(.*?)<\/span><\/span>

and replace for instance with blackword:$1

和finally remove all of the other span tags as follows

(?ms)<\/?span.*?>

And replace with nothing .

This would give you something like

blackword:a blackword:reasonably redword:good

So, now we still have this dreadfull non breaking spaces, which we just replace with a space so we get

blackword:a blackword:reasonably redword:good

And now we get the blackwords followed by a redword as follows :

(?ms)blackword:(\w+)\s*redword:\w+

and replace with $1, leaving you with

blackword:合理

So now the only thing left is to get rid of every word starting with blackword (since these were not followed by a redword) and you have your 'first black before a red one' word

(?ms)blackword:\w+\s*(\w+)

replace once again with $1 and what you get left with is reasonable.

Tried with multiple copies of your snippet and works pretty fine, but not guaranteed to work on actual data so you may need to tweak a bit

Telcontar120

I am always impressed by the creativity of the RapidMiner community!! Thanks@JEdwardand@kaymanfor these innovative solutions to a tricky problem.

Thomas_Ott

Thanks for the tag but I'm too late to this challenge.@Telcontar120said!

Thomas_Ott

Thanks for the tag but I'm too late to this challenge. What@Telcontar120said!

MRNJEM001

Thank you for the amazing solutions@JEdwardand@kayman, you are incredible. It's for a thesis, so if it ever gets published you should be in the acknowledgements

Thanks again!