Announcement Announcement Module
Collapse
No announcement yet.
Using spring context in mapper, how to autowire Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Using spring context in mapper, how to autowire

    What is recommended to use Spring container in mapper?

    In hadoop mapper, i do not have object creation under control. Hadoop will create mapper object and then call its initialization method. I would like to use @Inject in mapper, it means to create application context by hand, this is fine with me, but how to get application context to autowire resources into already created object?

  • #2
    We don't support this scenario in master but there's a branch you might want to try that adds this ability.
    In a nutshell, you need to deploy Spring on all nodes and use a Utility mapper (potentially as a base class) that bootstraps Spring from a well-known location and then applies self-injection or autowiring.
    As for the autowiring process, see AutowireCapableBeanFactory & co [1]

    http://j.mp/MoWmI3

    Comment


    • #3
      Thanks for reply, branch is here: https://github.com/SpringSource/spri...doop/mapreduce

      Last commit to this branch is 8 months old. This POJO design pattern was abandoned or there are plans to merge it with current version or place that code in separate module?

      Comment


      • #4
        I am using in Mapper this
        springcontext.getAutowireCapableBeanFactory().auto wireBean(this);

        but no autowiring is done. Fields annotated with @Inject are still null and no error message.

        Comment


        • #5
          Do you have javax.inject in the classpath (that is on each cluster node)? You could try @Autowired as a replacement.
          If the annotation library is not found, its declaration is simply ignored.

          Comment


          • #6
            javax.inject is part of hadoop since 0.23 (hadoop-2).

            ls $YARN_HOME/share/hadoop/mapreduce/lib/javax*
            /usr/local/hadoop/hadoop-0.23.3-SNAPSHOT/share/hadoop/mapreduce/lib/javax.inject-1.jar

            its listed as guice dependency
            [INFO] +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:0.23.3-SNAPSHOTrovided
            [INFO] | +- org.apache.hadoop:hadoop-yarn-common:jar:0.23.3-SNAPSHOTrovided
            [INFO] | | +- org.apache.hadoop:hadoop-yarn-api:jar:0.23.3-SNAPSHOTrovided
            [INFO] | | +- com.google.inject:guice:jar:3.0rovided
            [INFO] | | | \- javax.inject:javax.inject:jar:1rovided

            No difference with @Autowired but maybe i should call something different then autowireBean(this); there is lot of similar functions in autowirecapablebeanfactory

            Comment


            • #7
              springcontext.getAutowireCapableBeanFactory().auto wireBeanProperties(this, AutowireCapableBeanFactory.AUTOWIRE_BY_TYPE, true); didnt worked either

              Comment


              • #8
                i didnt had context annotation processing turned on in XML file. now it works in combination with autowirebeanproperties

                Comment

                Working...
                X