Announcement Announcement Module
Collapse
No announcement yet.
Programmatic JobConfiguration Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Programmatic JobConfiguration

    I want to use spring data to run hadoop jobs that need programmatic configuration.

    For example, when using HCatalog as job output, I need to call code such as

    Code:
    HCatSchema s = HCatOutputFormat.getTableSchema(job.getConfiguration());
    HCatOutputFormat.setSchema(job, s);
    in the job driver.

    To achieve this, I provided a custom implementation of "FactoryBean<Job>". Is there a more elegant way to do this?

  • #2
    Using the hdp:tool-runnner instead of hdp:job solved my problem.

    Comment


    • #3
      Thanks for posting the resolution, might help someone else.

      -Thomas

      Comment

      Working...
      X