Announcement Announcement Module
Collapse
No announcement yet.
Pseudo distributed : Need to change number of mapper nodes Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Pseudo distributed : Need to change number of mapper nodes

    I am using a Intel(R) Core(TM)2 Duo processor. I have installed hadoop in pseudo distributed mode. I have written a program which needs 50 mappers nodes. Is it possible to have 50 mapper nodes in the pseudo distributed mode or I will be limited to 4 nodes(2 * number of cores) . I have tried setting "mapred.tasktracker.map.tasks.maximum" to 50, but there is no change in concurrency.

  • #2
    Not really. There are various materials on the internet about this (I believe there's also a question + answer on this forum from awhile back). Basically it boils down to the fact that the settings act as hints but in the end, the Hadoop algorithms decide what to do and inside a pseudo-configuration, the number of mapper/reducers isn't that flexible.

    Comment

    Working...
    X