NoClassDefFoundError Could not initialize class org.apache.hadoop.yarn.conf.YarnConfiguration

cazjck

Hi,

I try run Hadoop with single mode on Rapidminer. Howerver, I got issue . I using Rapidminer Studio 7.5

java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.yarn.conf.YarnConfiguration
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at com.rapidminer.pagerank.utilities.HadoopUtilities.runHadoopLocal(HadoopUtilities.java:173)
at com.rapidminer.pagerank.operator.PageRankOperator.doWork(PageRankOperator.java:78)
at com.rapidminer.operator.Operator.execute(Operator.java:1004)
at com.rapidminer.operator.execution.SimpleUnitExecutor.execute(SimpleUnitExecutor.java:77)
at com.rapidminer.operator.ExecutionUnit$3.run(ExecutionUnit.java:812)
at com.rapidminer.operator.ExecutionUnit$3.run(ExecutionUnit.java:1)
at java.security.AccessController.doPrivileged(Native Method)
at com.rapidminer.operator.ExecutionUnit.execute(ExecutionUnit.java:807)
at com.rapidminer.operator.OperatorChain.doWork(OperatorChain.java:428)
at com.rapidminer.operator.Operator.execute(Operator.java:1004)
at com.rapidminer.Process.run(Process.java:1315)
at com.rapidminer.Process.run(Process.java:1179)
at com.rapidminer.Process.run(Process.java:1132)
at com.rapidminer.Process.run(Process.java:1127)
at com.rapidminer.Process.run(Process.java:1117)
at com.rapidminer.gui.ProcessThread.run(ProcessThread.java:65)

In build.gradle

dependencies {
//compile 'com.google.guava:guava:18.0'
编译”org.apache.hadoop: hadoop -hdfs:2.7.3"
编译”org.apache.hadoop: hadoop -client:2.7.3"
编译”org.apache.hadoop: hadoop -mapreduce-client-core:2.7.3"
编译”org.apache.hadoop: hadoop -yarn-common:2.7.3"
编译”org.apache.hadoop: hadoop -mapreduce-client-jobclient:2.7.3"
编译”org.apache.hadoop: hadoop -yarn-client:2.7.3"
编译”org.apache.hadoop: hadoop -yarn:2.7.3"
编译”org.apache.hadoop: hadoop -common:2.7.3"
}

Can everyone help me?

cazjck

Help me.

Thomas_Ott

You posted this twice. Doing so will not get your question answered quicker. I will delete the second topic.

Pavithra_Rao

Hi,

I guess if you provide the full path for the yarn.application.classpath property, the NoClassDefFoundError issue gets resolved.

YARN configuration XML could be found here:/etc/hadoop/conf/yarn-site.xml

Hope this helps, let us know.

Cheers,

cazjck

Thanks you so much.

However, When I run in Eclipse with Hadoop single mode && cluster mode , it's ok .

After I install my extension , I run extension in Rapidminer, occur exception.

I don't know, How to add classpath in extension of Rapidminer ? Can you provide for me one example ?

Sorry, I speak english not good.

Pavithra_Rao

Hi,

请检查纱线类路径是defined in the following XML file:

YARN configuration XML could be found here:/etc/hadoop/conf/yarn-site.xml

[email protected]$ sudo vi /etc/hadoop/conf/yarn-site.xml [ ... ]  yarn.application.classpath  $HADOOP_CONF_DIR, $HADOOP_COMMON_HOME/*, $HADOOP_COMMON_HOME/lib/*, $HADOOP_HDFS_HOME/*, $HADOOP_HDFS_HOME/lib/*, $HADOOP_MAPRED_HOME/*, $HADOOP_MAPRED_HOME/lib/*, $YARN_HOME/*, $YARN_HOME/lib/*, /etc/hadoop/conf, /usr/lib/hadoop/*, /usr/lib/hadoop/lib, /usr/lib/hadoop/lib/*, /usr/lib/hadoop-hdfs, /usr/lib/hadoop-hdfs/*, /usr/lib/hadoop-hdfs/lib/*, /usr/lib/hadoop-yarn/*, /usr/lib/hadoop-yarn/lib/*, /usr/lib/hadoop-mapreduce/*, /usr/lib/hadoop-mapreduce/lib/*  

Cheers,

cazjck

Thanks you.

But in code initialize Configuration throw exception. You can see some picture below:

Thomas_Ott

Why don't you just use the Radoop Extension?

cazjck

I want to create a extension using MapReduce, I don't know use Hive and Spark .