Announcement Announcement Module
Collapse
No announcement yet.
Pig, HBase and Spring Data Hadoop Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Pig, HBase and Spring Data Hadoop

    Has anyone got experience of using Spring Data Hadoop to run a Pig script that connects to HBase using Elephant Bird's HBaseLoader?

    I'm new to all of the above, but need to take some existing Pig scripts that were executed via a shell script and instead wrap them up in a self-contained Java application. Currently the scripts are run from a specific server that has Hadoop, HBase and Pig installed, and config for all of the above in /etc/. Pig has the HBase config on its classpath, so I'm guessing this is how it know how to connect to HBase

    I want to have all configuration in Spring. Is this possible if I need Pig to connect to HBase? How do I configure HBase such that the Pig script and the Elephant Bird library will know how to connect to it?

  • #2
    Haven't tried this myself but I suspect Elephant Bird library would be registered in Pig through a script (which SHDP supports). Not sure what the HBase connectivity implies - I suspect another Pig function/library that gets loaded which you could parameterize so the hbase location/port are not hardcoded.

    Comment


    • #3
      Hi Costin,

      Thanks for your reply. As you may have guessed, my understanding of the Hadoop ecosystem is a little fuzzy!

      I have recently discovered what I think to be an unrelated bug in 1.0.0.M1: https://jira.springsource.org/browse/SHDP-38

      Comment

      Working...
      X