Announcement Announcement Module
Collapse
No announcement yet.
Apache Hadoop 1.0.0.RC1 (SHDP) : Getting class cast error while executing Batch Jobs: Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Apache Hadoop 1.0.0.RC1 (SHDP) : Getting class cast error while executing Batch Jobs:

    Hi,

    I am new to SHDP and have been trying to build the environment for batch execution of Jobs.
    However, with the use of the eclipse Plugin (Map/reduce) to run the job using SHDP, I am getting the below error:
    I see the context initialization happening properly, however, I get the class casting exception from map-reduce job - HDP to spring job.
    Please refer the error log below: -

    Dec 11, 2012 9:23:41 AM org.springframework.context.support.AbstractApplic ationContext prepareRefresh
    INFO: Refreshing org.springframework.context.support.ClassPathXmlAp plicationContext@9a9b65: startup date [Tue Dec 11 09:23:41 IST 2012]; root of context hierarchy
    Dec 11, 2012 9:23:41 AM org.springframework.beans.factory.xml.XmlBeanDefin itionReader loadBeanDefinitions
    INFO: Loading XML bean definitions from class path resource [resources/launch-context.xml]
    Dec 11, 2012 9:23:43 AM org.springframework.beans.factory.xml.XmlBeanDefin itionReader loadBeanDefinitions
    INFO: Loading XML bean definitions from class path resource [META-INF/spring/batch-common.xml]
    Dec 11, 2012 9:23:43 AM org.springframework.beans.factory.xml.XmlBeanDefin itionReader loadBeanDefinitions
    INFO: Loading XML bean definitions from class path resource [META-INF/spring/hadoop-context.xml]
    Dec 11, 2012 9:23:43 AM org.springframework.beans.factory.xml.XmlBeanDefin itionReader loadBeanDefinitions
    INFO: Loading XML bean definitions from class path resource [META-INF/spring/wordcount-context.xml]
    Dec 11, 2012 9:23:44 AM org.springframework.core.io.support.PropertiesLoad erSupport loadProperties
    INFO: Loading properties file from class path resource [batch.properties]
    Dec 11, 2012 9:23:44 AM org.springframework.core.io.support.PropertiesLoad erSupport loadProperties
    INFO: Loading properties file from class path resource [hadoop.properties]
    Dec 11, 2012 9:23:44 AM org.springframework.beans.factory.support.DefaultL istableBeanFactory preInstantiateSingletons
    INFO: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultL istableBeanFactory@c51ce4: defining beans [org.springframework.beans.factory.config.PropertyP laceholderConfigurer#0,org.springframework.context .annotation.internalConfigurationAnnotationProcess or,org.springframework.context.annotation.internal AutowiredAnnotationProcessor,org.springframework.c ontext.annotation.internalRequiredAnnotationProces sor,org.springframework.context.annotation.interna lCommonAnnotationProcessor,jobRepository,transacti onManager,jobLauncher,hadoopConfiguration,org.spri ngframework.data.hadoop.scripting.HdfsScriptRunner #0,script-tasklet,wordcount-job,org.springframework.context.annotation.Configu rationClassPostProcessor.importAwareProcessor]; root of factory hierarchy
    Dec 11, 2012 9:23:44 AM org.springframework.batch.core.launch.support.Simp leJobLauncher afterPropertiesSet
    INFO: No TaskExecutor has been set, defaulting to synchronous executor.
    Dec 11, 2012 9:23:44 AM org.apache.hadoop.conf.Configuration <clinit>
    WARNING: DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
    Checking jobName [org.apache.hadoop.mapreduce.Job@1ee89c0]
    Executing job wordcount-job
    Exception in thread "main" org.springframework.beans.factory.BeanInitializati onException: Cannot execute job wordcount-job; nested exception is java.lang.ClassCastException: org.apache.hadoop.mapreduce.Job cannot be cast to org.springframework.batch.core.Job
    at org.apache.hadoop.jobManager.JobManager.startJobMa nager(JobManager.java:46)
    at org.springframework.data.hadoop.samples.wordcount. Main.main(Main.java:42)
    Caused by: java.lang.ClassCastException: org.apache.hadoop.mapreduce.Job cannot be cast to org.springframework.batch.core.Job

    at org.apache.hadoop.jobManager.JobManager.startJobMa nager(JobManager.java:42)
    ... 1 more

  • #2
    Re:

    The issue was resolved as I was using the mapred job reference instead of the spring batch job reference.
    Cheers!
    Pushkar




    Originally posted by pusingh View Post
    Hi,

    I am new to SHDP and have been trying to build the environment for batch execution of Jobs.
    However, with the use of the eclipse Plugin (Map/reduce) to run the job using SHDP, I am getting the below error:
    I see the context initialization happening properly, however, I get the class casting exception from map-reduce job - HDP to spring job.
    Please refer the error log below: -

    Dec 11, 2012 9:23:41 AM org.springframework.context.support.AbstractApplic ationContext prepareRefresh
    INFO: Refreshing org.springframework.context.support.ClassPathXmlAp plicationContext@9a9b65: startup date [Tue Dec 11 09:23:41 IST 2012]; root of context hierarchy
    Dec 11, 2012 9:23:41 AM org.springframework.beans.factory.xml.XmlBeanDefin itionReader loadBeanDefinitions
    INFO: Loading XML bean definitions from class path resource [resources/launch-context.xml]
    Dec 11, 2012 9:23:43 AM org.springframework.beans.factory.xml.XmlBeanDefin itionReader loadBeanDefinitions
    INFO: Loading XML bean definitions from class path resource [META-INF/spring/batch-common.xml]
    Dec 11, 2012 9:23:43 AM org.springframework.beans.factory.xml.XmlBeanDefin itionReader loadBeanDefinitions
    INFO: Loading XML bean definitions from class path resource [META-INF/spring/hadoop-context.xml]
    Dec 11, 2012 9:23:43 AM org.springframework.beans.factory.xml.XmlBeanDefin itionReader loadBeanDefinitions
    INFO: Loading XML bean definitions from class path resource [META-INF/spring/wordcount-context.xml]
    Dec 11, 2012 9:23:44 AM org.springframework.core.io.support.PropertiesLoad erSupport loadProperties
    INFO: Loading properties file from class path resource [batch.properties]
    Dec 11, 2012 9:23:44 AM org.springframework.core.io.support.PropertiesLoad erSupport loadProperties
    INFO: Loading properties file from class path resource [hadoop.properties]
    Dec 11, 2012 9:23:44 AM org.springframework.beans.factory.support.DefaultL istableBeanFactory preInstantiateSingletons
    INFO: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultL istableBeanFactory@c51ce4: defining beans [org.springframework.beans.factory.config.PropertyP laceholderConfigurer#0,org.springframework.context .annotation.internalConfigurationAnnotationProcess or,org.springframework.context.annotation.internal AutowiredAnnotationProcessor,org.springframework.c ontext.annotation.internalRequiredAnnotationProces sor,org.springframework.context.annotation.interna lCommonAnnotationProcessor,jobRepository,transacti onManager,jobLauncher,hadoopConfiguration,org.spri ngframework.data.hadoop.scripting.HdfsScriptRunner #0,script-tasklet,wordcount-job,org.springframework.context.annotation.Configu rationClassPostProcessor.importAwareProcessor]; root of factory hierarchy
    Dec 11, 2012 9:23:44 AM org.springframework.batch.core.launch.support.Simp leJobLauncher afterPropertiesSet
    INFO: No TaskExecutor has been set, defaulting to synchronous executor.
    Dec 11, 2012 9:23:44 AM org.apache.hadoop.conf.Configuration <clinit>
    WARNING: DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
    Checking jobName [org.apache.hadoop.mapreduce.Job@1ee89c0]
    Executing job wordcount-job
    Exception in thread "main" org.springframework.beans.factory.BeanInitializati onException: Cannot execute job wordcount-job; nested exception is java.lang.ClassCastException: org.apache.hadoop.mapreduce.Job cannot be cast to org.springframework.batch.core.Job
    at org.apache.hadoop.jobManager.JobManager.startJobMa nager(JobManager.java:46)
    at org.springframework.data.hadoop.samples.wordcount. Main.main(Main.java:42)
    Caused by: java.lang.ClassCastException: org.apache.hadoop.mapreduce.Job cannot be cast to org.springframework.batch.core.Job

    at org.apache.hadoop.jobManager.JobManager.startJobMa nager(JobManager.java:42)
    ... 1 more

    Comment

    Working...
    X