Announcement Announcement Module
Collapse
No announcement yet.
hadoop 0.20.2 on windows Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • hadoop 0.20.2 on windows

    Hi, I need to make a fully distributed hadoop (version 0.20.2) with 1 master and 1 slave on windows. I have followed these steps but something goes wrong

    (Italic words are executed on each pc of the lan)

    on each pc of the lan modify c:\windows\system32\drivers\etc\hosts with the ip of the master and slaves
    master NNN.NNN.NNN.NNN
    slave NNN.NNN.NNN.NNN

    Unlock port 22 on the firewall

    Install cygwin with ssh

    On cygwin
    ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
    cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

    on the master: scp ~/.ssh/id_dsa.pub <slaveusername>@slave:~/.ssh/master-key.pub
    on the slave: cat ~/.ssh/master-key.pub >> ~/.ssh/authorized_keys
    on the slave: scp ~/.ssh/id_dsa.pub <masterusername>@master:~/.ssh/slave-key.pub
    on the master: cat ~/.ssh/slave-key.pub >> ~/.ssh/authorized_keys


    extract hadoop in c:\cygwin\usr\local

    in hadoop/conf modify:

    hadoop-env
    export JAVA_HOME=c:\\Program\ Files\\Java\\jdk1.7
    export HADOOP_IDENT_STRING=MYHADOOP

    core-site
    <property>
    <name>fs.default.name</name>
    <value>hdfs://master:54310</value>
    </property>

    mapread-site
    <property>
    <name>mapred.job.tracker</name>
    <value>master:54311</value>
    </property>

    hdfs-site
    <property>
    <name>dfs.replication</name>
    <value>2</value>
    </property>

    slaves
    master
    slave

    on the master start cygwin
    cd /usr/local/hadoop
    bin/start-dfs.sh
    bin/start-mapred.sh

    on the browser (of the master)
    http://localhost:50070/

    on the browser i can only see one live node, the master. Why? Thanks in advance

  • #2
    This is a Hadoop specific question which is best addressed on the Hadoop mailing list/docs.

    Comment

    Working...
    X