Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.contrib.serde2.RegexSerDe

java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:431)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:371)
at org.apache.hadoop.mapred.Child$4.run(Child.java:259)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
at org.apache.hadoop.mapred.Child.main(Child.java:253)

Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
… 9 more

Caused by: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
… 14 more

Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
… 17 more

Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
… 22 more

Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassNotFoundException: org.apache.hadoop.hive.contrib.serde2.RegexSerDe
at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:406)
at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:90)
… 22 more

Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.contrib.serde2.RegexSerDe
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:819)
at org.apache.hadoop.hive.ql.exec.MapOperator.initObjectInspector(MapOperator.java:233)
at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:366)
… 23 more

 

1> Check if hive-contrib jar is in the hive-shell’s classpath –> http://www.techbuzz.in/how-to-check-the-list-of-jars-included-in-hive-classpath.php

2> If it’s not in the class path then add the jar to the classpath –> add jar <relative-classpath or absolute-classpath>/hive-contrib-0.8.1.jar

This will solve your problem.

How to Check the List of Jars Included in Hive ClassPath

At times you’ll need to be aware of all the jars that are available in Hive Classpath. The Query to get a list of all jars that are included in the ClassPath of Hive, run the following Query:

hive> list jars;

This will list all the jars, the ones that are added explicitly by you, and the ones that are available by default.

The output will look like:

/usr/local/…..*.jar

/usr/local/…..*.jar

file:/usr/local/…..*.jar

/usr/local/…..*.jar

The jar location with file: prefix is the one which was available in the ClassPath by default. The others were added explicitly by users using the “add jar <absolute / relative class path>

“Return Code 1 from org.apache.hadoop.hive.ql.exec.DDLTask” Error

While trying to execute a Hive query you can end up with the following error:

return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

Try re-running the same query, and if that doesn’t work, thenĀ Start Hadoop if it’s not running or restart Hadoop if it’s already up and if Hadoop’s running in cluster mode, check if DataNode is running.

This should solve the problem.

Exception in thread “main” java.lang.NoClassDefFoundError: start-all/sh Error

When trying to start hadoop, this error message is quite common :

Exception in thread “main” java.lang.NoClassDefFoundError: start-all/sh

Caused by: java.lang.ClassNotFoundException: start-all.sh
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
Could not find the main class: start-all.sh. Program will exit.

Here’s a quick check list to resolve this issue:

  1. Ensure you are calling the ‘start-all.sh‘ from within ‘bin‘. If you are inside ‘bin’ then the call should be ‘./start-all.sh‘ and if you are a level above bin, then try running: ‘bin/start-all.sh‘.
  2. One other common mistake I’ve seen is, people try running : ‘bin/hadoop start-all.sh‘. Please do not prefix ‘hadoop‘ to the start call.