Announcement Announcement Module
No announcement yet.
Using spring context in mapper, how to autowire Page Title Module
Move Remove Collapse
Conversation Detail Module
  • Filter
  • Time
  • Show
Clear All
new posts

  • Using spring context in mapper, how to autowire

    What is recommended to use Spring container in mapper?

    In hadoop mapper, i do not have object creation under control. Hadoop will create mapper object and then call its initialization method. I would like to use @Inject in mapper, it means to create application context by hand, this is fine with me, but how to get application context to autowire resources into already created object?

  • #2
    We don't support this scenario in master but there's a branch you might want to try that adds this ability.
    In a nutshell, you need to deploy Spring on all nodes and use a Utility mapper (potentially as a base class) that bootstraps Spring from a well-known location and then applies self-injection or autowiring.
    As for the autowiring process, see AutowireCapableBeanFactory & co [1]


    • #3
      Thanks for reply, branch is here:

      Last commit to this branch is 8 months old. This POJO design pattern was abandoned or there are plans to merge it with current version or place that code in separate module?


      • #4
        I am using in Mapper this
        springcontext.getAutowireCapableBeanFactory().auto wireBean(this);

        but no autowiring is done. Fields annotated with @Inject are still null and no error message.


        • #5
          Do you have javax.inject in the classpath (that is on each cluster node)? You could try @Autowired as a replacement.
          If the annotation library is not found, its declaration is simply ignored.


          • #6
            javax.inject is part of hadoop since 0.23 (hadoop-2).

            ls $YARN_HOME/share/hadoop/mapreduce/lib/javax*

            its listed as guice dependency
            [INFO] +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:0.23.3-SNAPSHOTrovided
            [INFO] | +- org.apache.hadoop:hadoop-yarn-common:jar:0.23.3-SNAPSHOTrovided
            [INFO] | | +- org.apache.hadoop:hadoop-yarn-api:jar:0.23.3-SNAPSHOTrovided
            [INFO] | | +-
            [INFO] | | | \- javax.inject:javax.inject:jar:1rovided

            No difference with @Autowired but maybe i should call something different then autowireBean(this); there is lot of similar functions in autowirecapablebeanfactory


            • #7
              springcontext.getAutowireCapableBeanFactory().auto wireBeanProperties(this, AutowireCapableBeanFactory.AUTOWIRE_BY_TYPE, true); didnt worked either


              • #8
                i didnt had context annotation processing turned on in XML file. now it works in combination with autowirebeanproperties