FW: ClassNotFound error trying to run rdf_example.pig
by Rob Vesse
Forwarding my reply with the solution to the problem to the list
The key was that I was testing with Pig's local execution mode so needed to use the PIG_CLASSPATH environment variable instead
Rob
From: Rob Vesse <rvesse(a)yarcdata.com<mailto:rvesse@yarcdata.com>>
Date: Monday, 20 January 2014 16:04
To: "Datta, Kushal" <kushal.datta(a)intel.com<mailto:kushal.datta@intel.com>>
Subject: Re: [GraphBuilder] ClassNotFound error trying to run rdf_example.pig
As I already said in my email I had done this. I've tried putting the GraphBuilder JAR in both the HADOOP_CLASSPATH and the CLASSPATH and neither one makes any difference.
The key thing here appears to be that I am using Pig's local execution mode which means neither of the above are honoured, changing to use PIG_CLASSPATH resolves this problem
Rob
From: <Datta>, Kushal <kushal.datta(a)intel.com<mailto:kushal.datta@intel.com>>
Date: Monday, 20 January 2014 13:53
To: GraphBuilder Users List <graphbuilder(a)lists.01.org<mailto:graphbuilder@lists.01.org>>
Subject: Re: [GraphBuilder] ClassNotFound error trying to run rdf_example.pig
Hi Rob,
Please extend HADOOP_CLASSPATH with the path to the graphbuilder jar before you use the library.
$ export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/path_to_your_graphbuilder_jar
Thanks,
-Kushal.
From: GraphBuilder [mailto:graphbuilder-bounces@lists.01.org] On Behalf Of Rob Vesse
Sent: Monday, January 20, 2014 1:52 PM
To: GraphBuilder Users List
Subject: [GraphBuilder] ClassNotFound error trying to run rdf_example.pig
I'm trying to run the examples with the 2.0 alpha build and I'm getting the following error:
2014-01-20 13:42:13,101 [Thread-6] WARN org.apache.hadoop.mapred.LocalJobRunner - job_local_0001
java.io.IOException: Could not find class com.intel.hadoop.graphbuilder.graphelements.SerializedGraphElementStringTypeVids, while attempting to de-serialize it
at org.apache.pig.data.BinInterSedes.readWritable(BinInterSedes.java:293)
at org.apache.pig.data.BinInterSedes.readDatum(BinInterSedes.java:422)
at org.apache.pig.data.BinInterSedes.readDatum(BinInterSedes.java:318)
at org.apache.pig.data.utils.SedesHelper.readGenericTuple(SedesHelper.java:144)
at org.apache.pig.data.BinInterSedes.readTuple(BinInterSedes.java:149)
at org.apache.pig.data.BinInterSedes.readTuple(BinInterSedes.java:1374)
at org.apache.pig.impl.io.NullableTuple.readFields(NullableTuple.java:55)
at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:67)
at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:40)
at org.apache.hadoop.mapreduce.ReduceContext.nextKeyValue(ReduceContext.java:116)
at org.apache.hadoop.mapreduce.ReduceContext.nextKey(ReduceContext.java:92)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:175)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:649)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:417)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:260)
Caused by: java.lang.ClassNotFoundException: com.intel.hadoop.graphbuilder.graphelements.SerializedGraphElementStringTypeVids
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:190)
at org.apache.pig.data.BinInterSedes.readWritable(BinInterSedes.java:291)
This is when running in local mode using the following command:
> pig –x local –f examples/rdf_example.pig
Note that I have double checked that the graph builder 2.0 with dependencies JAR is in the class path (I've tried both with CLASSPATH and HADOOP_CLASSPATH)
Anyone else run into this error or have any thoughts on the cause?
Rob
Rob Vesse
Software Engineer
Mobile +44 (0) 7791 638 873 | rvesse(a)yarcdata.com<mailto:rvesse@yarcdata.com> | Skype rvesse
One Kingdom Street | London | W2 6BD | www.yarcdata.com<http://www.yarcdata.com/>
[Description: cid:image002.jpg@01CE3B8C.715B1D60]<http://www.yarcdata.com/>
8 years, 8 months
ClassNotFound error trying to run rdf_example.pig
by Rob Vesse
I'm trying to run the examples with the 2.0 alpha build and I'm getting the following error:
2014-01-20 13:42:13,101 [Thread-6] WARN org.apache.hadoop.mapred.LocalJobRunner - job_local_0001
java.io.IOException: Could not find class com.intel.hadoop.graphbuilder.graphelements.SerializedGraphElementStringTypeVids, while attempting to de-serialize it
at org.apache.pig.data.BinInterSedes.readWritable(BinInterSedes.java:293)
at org.apache.pig.data.BinInterSedes.readDatum(BinInterSedes.java:422)
at org.apache.pig.data.BinInterSedes.readDatum(BinInterSedes.java:318)
at org.apache.pig.data.utils.SedesHelper.readGenericTuple(SedesHelper.java:144)
at org.apache.pig.data.BinInterSedes.readTuple(BinInterSedes.java:149)
at org.apache.pig.data.BinInterSedes.readTuple(BinInterSedes.java:1374)
at org.apache.pig.impl.io.NullableTuple.readFields(NullableTuple.java:55)
at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:67)
at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:40)
at org.apache.hadoop.mapreduce.ReduceContext.nextKeyValue(ReduceContext.java:116)
at org.apache.hadoop.mapreduce.ReduceContext.nextKey(ReduceContext.java:92)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:175)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:649)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:417)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:260)
Caused by: java.lang.ClassNotFoundException: com.intel.hadoop.graphbuilder.graphelements.SerializedGraphElementStringTypeVids
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:190)
at org.apache.pig.data.BinInterSedes.readWritable(BinInterSedes.java:291)
This is when running in local mode using the following command:
> pig –x local –f examples/rdf_example.pig
Note that I have double checked that the graph builder 2.0 with dependencies JAR is in the class path (I've tried both with CLASSPATH and HADOOP_CLASSPATH)
Anyone else run into this error or have any thoughts on the cause?
Rob
Rob Vesse
Software Engineer
Mobile +44 (0) 7791 638 873 | rvesse(a)yarcdata.com | Skype rvesse
One Kingdom Street | London | W2 6BD | www.yarcdata.com<http://www.yarcdata.com/>
[Description: cid:image002.jpg@01CE3B8C.715B1D60]<http://www.yarcdata.com/>
8 years, 8 months
Are there any Titan+Cassandra graph builder example similar to titan_bulk_load_example.pig out there?
by Tsui, Michael
Hi,
I saw a recent thread that was talking about graph builder with titan+cassandra. The recent thread says you can use the load_titan macro for titan+cassandra. The macro itself in the comment says it is looking for htable input name. I was wondering how that would work for Cassandra?
I am looking for an example similar to titan_bulk_load_example.pig. I’m an Avro Pig/GraphBuilder newbie so I’m just trying to figure out a way to re-use the example with the Titan-Cassandra Flavor.
Thanks for the help in advance,
Mike
8 years, 8 months
