Announcement Announcement Module
Collapse
No announcement yet.
Using GiraphRunner Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Using GiraphRunner

    I am trying to use SHDP to run a Giraph job by using GiraphRunner and its associated switches like so:

    Code:
    hadoop jar giraphjob.jar 
                        org.apache.giraph.GiraphRunner 
                        com.myapp.MyVertex 
                        -w  3
                        -if  com.myapp.VertexInputFormat 
                        -of  com.myapp.VertexOutputFormat 
                        -ip  inputPath
                        -op  outputPath
    It is not obvious to me how to do this with <hdp:tool-runner>. Any help is appreciated.

    Thanks.

  • #2
    (I'm not familiar with english)

    I don't know Giraph.
    Hmm... GiraphRunner is implemented org.apache.hadoop.util.Tool.
    I will explain my setting/codes. (Perhaps, it is not good ways but run correctly)
    In xml
    <hdp:tool-runner id="giraphRunnerBean"
    tool-class="org.apache.giraph.GiraphRunner"
    jar="file://${giraphjob.jar.location}"
    run-at-startup="false">
    </hdp:tool-runner>
    'giraphRunnerBean' is general spring bean and extends org.springframework.data.hadoop.mapreduce.ToolRunn er.

    You have to write some another bean that is injected giraphRunnerBean.
    Inside the another bean (my case spring-batch Tasklet),
    giraphRunnerBean.setArguments(
    "com.myapp.MyVertex",
    "-w", "3",
    "-if", "com.myapp.VertexInputFormat",
    "-of", "com.myapp.VertexOutputFormat ", ...)
    giraphRunnerBean.call();
    I hope it helps.
    Last edited by shekhina; Oct 31st, 2012, 04:51 AM.

    Comment


    • #3
      It seems a little cumbersome to write two declarations for this one task, but I will try this approach. Thanks for your help.

      Comment


      • #4
        neilac333/ You are right.
        There is a simpler way.

        <hdp:tool-runner id="giraphRunnerBean"
        tool-class="org.apache.giraph.GiraphRunner"
        jar="file://${giraphjob.jar.location}"
        run-at-startup="false">

        <hdp:arg value="-a"/>
        <hdp:arg value="someVal"/>
        <hdp:arg value="-b"/>
        <hdp:arg value="otherVal"/>
        </hdp:tool-runner>

        Comment


        • #5
          Indeed - check out the documentation which has an entry exactly for this: http://static.springsource.org/sprin...p:tool-scripts

          Comment

          Working...
          X