Announcement Announcement Module
No announcement yet.
Pig, HBase and Spring Data Hadoop Page Title Module
Move Remove Collapse
Conversation Detail Module
  • Filter
  • Time
  • Show
Clear All
new posts

  • Pig, HBase and Spring Data Hadoop

    Has anyone got experience of using Spring Data Hadoop to run a Pig script that connects to HBase using Elephant Bird's HBaseLoader?

    I'm new to all of the above, but need to take some existing Pig scripts that were executed via a shell script and instead wrap them up in a self-contained Java application. Currently the scripts are run from a specific server that has Hadoop, HBase and Pig installed, and config for all of the above in /etc/. Pig has the HBase config on its classpath, so I'm guessing this is how it know how to connect to HBase

    I want to have all configuration in Spring. Is this possible if I need Pig to connect to HBase? How do I configure HBase such that the Pig script and the Elephant Bird library will know how to connect to it?

  • #2
    Haven't tried this myself but I suspect Elephant Bird library would be registered in Pig through a script (which SHDP supports). Not sure what the HBase connectivity implies - I suspect another Pig function/library that gets loaded which you could parameterize so the hbase location/port are not hardcoded.


    • #3
      Hi Costin,

      Thanks for your reply. As you may have guessed, my understanding of the Hadoop ecosystem is a little fuzzy!

      I have recently discovered what I think to be an unrelated bug in 1.0.0.M1: