maven - hadoop mapreduce java program exception: java.lang.NoSuchMethodError -

-

this question has answer here:

this first experience hadoop, , need solve problem stuck in (as shown in title).

i found project looking for: https://github.com/tzulitai/distributed-svm

before starting run mapreduce job, executed 3 commands on terminal, build info said:

$ git clone https://github.com/tzulitai/distributed-svm.git $ cd distributed-svm $ mvn clean install

a jar file cascade-svm-mr-0.0.1-snapshot.jar generated, , used on the command below run mapreduce job:

$ '/usr/local/hadoop/bin/hadoop' jar \ >'/home/hduser/distributed-svm/cascade-svm-mr/target/cascade-svm-mr-0.0.1-snapshot.jar'\ > ncku.hpds.tzulitai.mapreduce.svm.cascade.cascadesvm input output

here part of exception met:

job[] prepartitionjobs = new job[prepartitionjobcount];      prepartitionjobs[0] = new job(prepartitionconfs[0], "cascade svm: partitioning training data, phase 1");     prepartitionjobs[0].setjarbyclass(cascadesvm.class);     prepartitionjobs[0].setnumreducetasks(0);   // map-only job     prepartitionjobs[0].setmapperclass(prestatcountermapper.class);     prepartitionjobs[0].setoutputkeyclass(nullwritable.class);     prepartitionjobs[0].setoutputvalueclass(text.class);     fileinputformat.addinputpath(prepartitionjobs[0], new path(otherargs[0]));     fileoutputformat.setoutputpath(prepartitionjobs[0], new path(otherargs[1]+"/tmp"));     prepartitionjobs[0].waitforcompletion(true);

(the whole code cascadesvm.java can found in link above).

the error met in last line in code above:

exception in thread "main" java.lang.nosuchmethoderror: org.apache.hadoop.ipc.rpc.getproxy(ljava/lang/class;jljava/net/inetsocketaddress;lorg/apache/hadoop/security/usergroupinformation;lorg/apache/hadoop/conf/configuration;ljavax/net/socketfactory;ilorg/apache/hadoop/io/retry/retrypolicy;z)lorg/apache/hadoop/ipc/versionedprotocol; @ org.apache.hadoop.mapred.jobclient.createrpcproxy(jobclient.java:505) @ org.apache.hadoop.mapred.jobclient.init(jobclient.java:496) @ org.apache.hadoop.mapred.jobclient.<init>(jobclient.java:479) @ org.apache.hadoop.mapreduce.job$1.run(job.java:563) @ java.security.accesscontroller.doprivileged(native method) @ javax.security.auth.subject.doas(subject.java:422) @ org.apache.hadoop.security.usergroupinformation.doas(usergroupinformation.java:1657) @ org.apache.hadoop.mapreduce.job.connect(job.java:561) @ org.apache.hadoop.mapreduce.job.submit(job.java:549) @ org.apache.hadoop.mapreduce.job.waitforcompletion(job.java:580) @ ncku.hpds.tzulitai.mapreduce.svm.cascade.cascadesvm.main(cascadesvm.java:485) @ sun.reflect.nativemethodaccessorimpl.invoke0(native method) @ sun.reflect.nativemethodaccessorimpl.invoke(nativemethodaccessorimpl.java:62) @ sun.reflect.delegatingmethodaccessorimpl.invoke(delegatingmethodaccessorimpl.java:43) @ java.lang.reflect.method.invoke(method.java:498) @ org.apache.hadoop.util.runjar.run(runjar.java:221) @ org.apache.hadoop.util.runjar.main(runjar.java:136)

according exception, noticed method waitforcompletion not recognized or not exist in class job, when imported package job mapreduce.

i thought maybe project using old version of hadoop 2.4.1 , using hadoop 2.7.2, modified hadoop version in pom.xml file, stil doesn't solve problem.

this pom.xml file:

<project xmlns="http://maven.apache.org/pom/4.0.0" xmlns:xsi="http://www.w3.org/2001/xmlschema-instance" xsi:schemalocation="http://maven.apache.org/pom/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">   <modelversion>4.0.0</modelversion>   <groupid>ncku.hpds.tzulitai</groupid>   <artifactid>distributed-svm</artifactid>   <version>0.0.1-snapshot</version>   <packaging>pom</packaging>   <name>distributed-svm</name>   <description>distributed svm approaches implemented mapreduce</description>   <modules>     <module>cascade-svm-mr</module>     <module>bagging-svm-mr</module>   </modules>   <dependencies>     <dependency>         <groupid>org.apache.hadoop</groupid>         <artifactid>hadoop-common</artifactid>         <version>2.7.2</version>     </dependency>     <dependency>         <groupid>org.apache.hadoop</groupid>         <artifactid>hadoop-mapreduce-client-core</artifactid>         <version>2.7.2</version>     </dependency>     <dependency>         <groupid>tw.edu.ntu.csie</groupid>         <artifactid>libsvm</artifactid>         <version>3.17</version>     </dependency>   </dependencies>   <build>   <plugins>     <plugin>         <artifactid>maven-compiler-plugin</artifactid>         <version>2.3.2</version>         <configuration>             <source>1.8</source>             <target>1.8</target>         </configuration>     </plugin>    </plugins>   </build> </project>

sorry if question long, need solve error, appreciated, regards.


Comments

Post a Comment

Popular posts from this blog

ios - RestKit 0.20 — CoreData: error: Failed to call designated initializer on NSManagedObject class (again) -

-
this error has shown on many times, seems. prominent answer issue seems restkit 0.20 — coredata: error: failed call designated initializer on nsmanagedobject class however, solution posted doesn't work me. let's start beginning: i want upload file server. want parse response make sure uploaded. if not, try again later. model has boolean variable called nsnumber *senttoserver purpose (in coredata, bool saved nsnumber ) . because don't know whether sending going successful, save entity coredata model first before sending via restkit. when post request successful, server returns json string this: {"id":14,"created":true} notice that, of course, id not set on coredata model yet, both id , created new informations model. here's model: @interface recording : nsmanagedobject @property (nonatomic, retain) nsstring * audiofilepath; @property (nonatomic, retain) nsdate * createdat; @property (nonatomic, retain) nsstring * deviceid; @
Read more

laravel - PDOException in Connector.php line 55: SQLSTATE[HY000] [1045] Access denied for user 'root'@'localhost' (using password: YES) -

-
my .env: app_env=local app_debug=true app_key=myappkey app_url=http://localhost db_connection=mysql db_host=localhost db_port=3306 db_database=gestiontaller db_username=root db_password='root' cache_driver=file session_driver=file queue_driver=sync redis_host=127.0.0.1 redis_password=null redis_port=6379 mail_driver=smtp mail_host=mailtrap.io mail_port=2525 mail_username=null mail_password=null mail_encryption=null i've tried password ", ' , without, restarting server every time. i've tried changing localhost 172.0.0.1. yes, password correct, running mysql -u root -p introducing password root in prompt can go mysql. users table: +------------------+-----------+ | user | host | +------------------+-----------+ | debian-sys-maint | localhost | | mysql.sys | localhost | | root | localhost | | root1 | localhost | +------------------+-----------+ i've added homestead.yaml this: database_port: 3306
Read more

java - Digest auth with Spring Security using javaconfig -

-
so i'm trying create digest authentication spring following documentation trying translate xml "requirements" in java requirements. let's have xml in docs: <bean id="digestfilter" class= "org.springframework.security.web.authentication.www.digestauthenticationfilter"> <property name="userdetailsservice" ref="jdbcdaoimpl"/> <property name="authenticationentrypoint" ref="digestentrypoint"/> <property name="usercache" ref="usercache"/> </bean> <bean id="digestentrypoint" class= "org.springframework.security.web.authentication.www.digestauthenticationentrypoint"> <property name="realmname" value="contacts realm via digest authentication"/> <property name="key" value="acegi"/> <property name="noncevalidityseconds" value="10"/> </b
Read more