Announcement Announcement Module
No announcement yet.
FileSystem closed error in hadoop 2.0.3-alpha Page Title Module
Move Remove Collapse
Conversation Detail Module
  • Filter
  • Time
  • Show
Clear All
new posts

  • FileSystem closed error in hadoop 2.0.3-alpha

    In hadoop 2, FileSystem for each type is singleton. Calling FileSystem.close() closes it for all future calls.

    After updating from RC1 to GA, i can not run tasks working with @Inject filesystem bean because fs is closed.

    I was able to discover only changes to HdfsResouceLoader which now closes Filesystem. I changed code to call it with filesystem instance to avoid closing fs, but it is still not enough to get it running.

    my code is using: ConfigurationFactoryBean, HdfsResourceLoader, DistributedCacheFactoryBean.

  • #2
    Problem is in DistributedCacheFactoryBean. It creates HDFSResourceLoader from configuration, instead of filesystem. Closing its internal resourceloader closes filesystem as well, causing hadoop 2.0 drivers (Tool) to stop working.