Announcement Announcement Module
Collapse
No announcement yet.
Hadoop build problem Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Hadoop build problem

    Hi,
    when I try to build spring-hadoop (from trunk) running ./gradlew -Pdistro=hadoop22 -Phd.fs=hdfs://localhost.localdomain:8020 build - The build fails due to unit test failures. Most of the tests fails with this output:
    10:19:21,444 INFO Test worker support.AbstractContextLoader - Detected default resource location "classpath:/org/springframework/data/hadoop/fs/FsShellTest-context.xml" for test class [org.springframework.data.hadoop.fs.FsShellTest].
    10:19:21,446 INFO Test worker support.AbstractDelegatingSmartContextLoader - GenericXmlContextLoader detected default locations for context configuration [[email protected] declaringClass = 'org.springframework.data.hadoop.fs.FsShellTest', locations = '{classpath:/org/springframework/data/hadoop/fs/FsShellTest-context.xml}', classes = '{}', inheritLocations = true, initializers = '{}', inheritInitializers = true, name = [null], contextLoaderClass = 'org.springframework.test.context.ContextLoader'].
    10:19:21,447 INFO Test worker support.AnnotationConfigContextLoaderUtils - Could not detect default configuration classes for test class [org.springframework.data.hadoop.fs.FsShellTest]: FsShellTest does not declare any static, non-private, non-final, inner classes annotated with @Configuration.
    10:19:21,447 INFO Test worker support.AbstractContextLoader - Detected default resource location "classpath:/org/springframework/data/hadoop/fs/AbstractROFsShellTest-context.xml" for test class [org.springframework.data.hadoop.fs.AbstractROFsShe llTest].
    10:19:21,447 INFO Test worker support.AbstractDelegatingSmartContextLoader - GenericXmlContextLoader detected default locations for context configuration [[email protected] declaringClass = 'org.springframework.data.hadoop.fs.AbstractROFsSh ellTest', locations = '{classpath:/org/springframework/data/hadoop/fs/AbstractROFsShellTest-context.xml}', classes = '{}', inheritLocations = true, initializers = '{}', inheritInitializers = true, name = [null], contextLoaderClass = 'org.springframework.test.context.ContextLoader'].
    10:19:21,447 INFO Test worker support.AnnotationConfigContextLoaderUtils - Could not detect default configuration classes for test class [org.springframework.data.hadoop.fs.AbstractROFsShe llTest]: AbstractROFsShellTest does not declare any static, non-private, non-final, inner classes annotated with @Configuration.
    10:19:21,470 INFO Test worker xml.XmlBeanDefinitionReader - Loading XML bean definitions from class path resource [org/springframework/data/hadoop/fs/AbstractROFsShellTest-context.xml]
    10:19:21,497 INFO Test worker xml.XmlBeanDefinitionReader - Loading XML bean definitions from class path resource [org/springframework/data/hadoop/hadoop-ctx.xml]
    10:19:21,533 INFO Test worker xml.XmlBeanDefinitionReader - Loading XML bean definitions from class path resource [org/springframework/data/hadoop/fs/FsShellTest-context.xml]
    10:19:21,541 INFO Test worker support.DefaultListableBeanFactory - Overriding bean definition for bean 'hadoopFs': replacing [Generic bean: class [org.springframework.data.hadoop.fs.FileSystemFacto ryBean]; scope=; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null] with [Generic bean: class [org.springframework.data.hadoop.fs.FileSystemFacto ryBean]; scope=; abstract=false; lazyInit=true; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null; defined in class path resource [org/springframework/data/hadoop/fs/FsShellTest-context.xml]]
    10:19:21,542 INFO Test worker support.GenericApplicationContext - Refreshing org.springframework.context.support.GenericApplica [email protected]: startup date [Tue Dec 24 10:19:21 PST 2013]; root of context hierarchy
    10:19:21,549 INFO Test worker config.PropertyPlaceholderConfigurer - Loading properties file from class path resource [test.properties]
    10:19:21,551 INFO Test worker annotation.AutowiredAnnotationBeanPostProcessor - JSR-330 'javax.inject.Inject' annotation found and supported for autowiring
    10:19:21,552 INFO Test worker support.DefaultListableBeanFactory - Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultL [email protected]: defining beans [ppc,hadoopFs,hadoopResourceLoader,hadoopConfigurat ion,cfg-init,fs-init,rl-init,org.springframework.data.hadoop.scripting.Hdf sScriptRunner#0,fsShell,org.springframework.contex t.annotation.internalConfigurationAnnotationProces sor,org.springframework.context.annotation.interna lAutowiredAnnotationProcessor,org.springframework. context.annotation.internalRequiredAnnotationProce ssor,org.springframework.context.annotation.intern alCommonAnnotationProcessor,org.springframework.co ntext.annotation.ConfigurationClassPostProcessor.i mportAwareProcessor]; root of factory hierarchy
    10:19:21,593 INFO Test worker Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
    10:19:21,608 INFO Test worker fs.TrashPolicyDefault - Namenode trash configuration: Deletion interval = 0 minutes, Emptier interval = 0 minutes.
    10:19:21,623 INFO Test worker support.DefaultListableBeanFactory - Destroying singletons in org.springframework.beans.factory.support.DefaultL [email protected]: defining beans [ppc,hadoopFs,hadoopResourceLoader,hadoopConfigurat ion,cfg-init,fs-init,rl-init,org.springframework.data.hadoop.scripting.Hdf sScriptRunner#0,fsShell,org.springframework.contex t.annotation.internalConfigurationAnnotationProces sor,org.springframework.context.annotation.interna lAutowiredAnnotationProcessor,org.springframework. context.annotation.internalRequiredAnnotationProce ssor,org.springframework.context.annotation.intern alCommonAnnotationProcessor,org.springframework.co ntext.annotation.ConfigurationClassPostProcessor.i mportAwareProcessor]; root of factory hierarchy
    10:19:21,623 ERROR Test worker context.TestContextManager - Caught exception while allowing TestExecutionListener [org.springframework.test.context.support.Dependenc [email protected]] to prepare test instance [[email protected] 0bd0d]
    java.lang.IllegalStateException: Failed to load ApplicationContext
    at org.springframework.test.context.CacheAwareContext LoaderDelegate.loadContext(CacheAwareContextLoader Delegate.java:99)
    at org.springframework.test.context.TestContext.getAp plicationContext(TestContext.java:122)
    at org.springframework.test.context.support.Dependenc yInjectionTestExecutionListener.injectDependencies (DependencyInjectionTestExecutionListener.java:109 )
    at org.springframework.test.context.support.Dependenc yInjectionTestExecutionListener.prepareTestInstanc e(DependencyInjectionTestExecutionListener.java:75 )
    at org.springframework.test.context.TestContextManage r.prepareTestInstance(TestContextManager.java:312)
    at org.springframework.test.context.junit4.SpringJUni t4ClassRunner.createTest(SpringJUnit4ClassRunner.j ava:211)
    at org.springframework.test.context.junit4.SpringJUni t4ClassRunner$1.runReflectiveCall(SpringJUnit4Clas sRunner.java:288)
    at org.junit.internal.runners.model.ReflectiveCallabl e.run(ReflectiveCallable.java:12)
    at org.springframework.test.context.junit4.SpringJUni t4ClassRunner.methodBlock(SpringJUnit4ClassRunner. java:284)
    at org.springframework.test.context.junit4.SpringJUni t4ClassRunner.runChild(SpringJUnit4ClassRunner.jav a:231)
    at org.springframework.test.context.junit4.SpringJUni t4ClassRunner.runChild(SpringJUnit4ClassRunner.jav a:88)
    at org.junit.runners.ParentRunner$3.run(ParentRunner. java:238)
    at org.junit.runners.ParentRunner$1.schedule(ParentRu nner.java:63)
    at org.junit.runners.ParentRunner.runChildren(ParentR unner.java:236)
    at org.junit.runners.ParentRunner.access$000(ParentRu nner.java:53)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRu nner.java:229)
    at org.springframework.test.context.junit4.statements .RunBeforeTestClassCallbacks.evaluate(RunBeforeTes tClassCallbacks.java:61)
    at org.springframework.test.context.junit4.statements .RunAfterTestClassCallbacks.evaluate(RunAfterTestC lassCallbacks.java:71)
    at org.junit.runners.ParentRunner.run(ParentRunner.ja va:309)
    at org.springframework.test.context.junit4.SpringJUni t4ClassRunner.run(SpringJUnit4ClassRunner.java:174 )
    at org.gradle.api.internal.tasks.testing.junit.JUnitT estClassExecuter.runTestClass(JUnitTestClassExecut er.java:80)
    at org.gradle.api.internal.tasks.testing.junit.JUnitT estClassExecuter.execute(JUnitTestClassExecuter.ja va:47)
    at org.gradle.api.internal.tasks.testing.junit.JUnitT estClassProcessor.processTestClass(JUnitTestClassP rocessor.java:69)
    at org.gradle.api.internal.tasks.testing.SuiteTestCla ssProcessor.processTestClass(SuiteTestClassProcess or.java:49)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Nativ e Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Native MethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(De legatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.gradle.messaging.dispatch.ReflectionDispatch.d ispatch(ReflectionDispatch.java:35)
    at org.gradle.messaging.dispatch.ReflectionDispatch.d ispatch(ReflectionDispatch.java:24)
    at org.gradle.messaging.dispatch.ContextClassLoaderDi spatch.dispatch(ContextClassLoaderDispatch.java:32 )


    Caused by: org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name 'org.springframework.data.hadoop.scripting.HdfsScr iptRunner#0': Invocation of init method failed; nested exception is org.springframework.scripting.ScriptCompilationExc eption: Could not compile script [static script []]: Execution failure; nested exception is javax.script.ScriptException: sun.org.mozilla.javascript.internal.WrappedExcepti on: Wrapped org.springframework.data.hadoop.HadoopException: Cannot test resource .staging;Failed on local exception: com.google.protobuf.InvalidProtocolBufferException : Protocol message contained an invalid tag (zero).; Host Details : local host is: "localhost.localdomain/127.0.0.1"; destination host is: "localhost.localdomain":8020; (<Unknown source>#8) in <Unknown source> at line number 8
    at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.initializeBean(Abstract AutowireCapableBeanFactory.java:1512)
    at org.springframework.beans.factory.support.Abstract AutowireCapableBeanFactory.doCreateBean(AbstractAu towireCapableBeanFactory.java:521)

    Caused by: javax.script.ScriptException: sun.org.mozilla.javascript.internal.WrappedExcepti on: Wrapped org.springframework.data.hadoop.HadoopException: Cannot test resource .staging;Failed on local exception: com.google.protobuf.InvalidProtocolBufferException : Protocol message contained an invalid tag (zero).; Host Details : local host is: "localhost.localdomain/127.0.0.1"; destination host is: "localhost.localdomain":8020; (<Unknown source>#8) in <Unknown source> at line number 8
    at com.sun.script.javascript.RhinoScriptEngine.eval(R hinoScriptEngine.java:224)
    at com.sun.script.javascript.RhinoScriptEngine.eval(R hinoScriptEngine.java:240)
    at javax.script.AbstractScriptEngine.eval(AbstractScr iptEngine.java:233)
    at org.springframework.data.hadoop.scripting.Jsr223Sc riptEvaluator.evaluate(Jsr223ScriptEvaluator.java: 79)

    Caused by: sun.org.mozilla.javascript.internal.WrappedExcepti on: Wrapped org.springframework.data.hadoop.HadoopException: Cannot test resource .staging;Failed on local exception: com.google.protobuf.InvalidProtocolBufferException : Protocol message contained an invalid tag (zero).; Host Details : local host is: "localhost.localdomain/127.0.0.1"; destination host is: "localhost.localdomain":8020; (<Unknown source>#8)
    at sun.org.mozilla.javascript.internal.Context.throwA sScriptRuntimeEx(Context.java:1798)
    at sun.org.mozilla.javascript.internal.MemberBox.invo ke(MemberBox.java:196)
    at sun.org.mozilla.javascript.internal.NativeJavaMeth od.call(NativeJavaMethod.java:245)
    My namenode information:
    NameNode 'localhost.localdomain:8020' (active)
    Started: Mon Dec 23 16:59:28 PST 2013
    Version: 2.0.5-alpha, dee8c65d6efb8244d16a3692a558c46744c87c92
    Compiled: 2013-06-09T06:06Z by jenkins from (no branch)
    Cluster ID: CID-50e1dd8a-f0c7-4508-83fd-e98013a4870f
    Block Pool ID: BP-548923788-127.0.0.1-1373495482411

    I'm running CentOS-6.4 64 bits in Virtual Box,
    java: oracle 7u45
    hadoop 2.0.5, single-node in a pseudo-distributed mode

    Any suggestions?
    Why spring-hadoop picking localhost as local host is: "localhost.localdomain/127.0.0.1" ?

    Victor

    I've tried to upload an attachment (logs), but always has an error ( ImageMagick exec error found:...)

  • #2
    You can't build and test with -Pdistro=hadoop22 against Hadoop 2.0.5. We no longer support building against Apache Hadoop 2.0 alpha versions, you need to upgrade your Hadoop install to Hadoop 2.2 stable.

    Comment

    Working...
    X