Announcement Announcement Module
No announcement yet.
XML partitioner Page Title Module
Move Remove Collapse
Conversation Detail Module
  • Filter
  • Time
  • Show
Clear All
new posts

  • XML partitioner

    If there a way to use a (custom) partitioner to be able to read a large xml file and pass smaller chunks for subsequent processing using the partitioning technique?

  • #2
    There's no efficient way to do it because an XML document is a single entity. You might get away with a partitioner that streams through the file counting items. I know that works for flat files, and occasionally improves performance, but that depends on the downstream processing. There's a pull request on github that works for flat files (maybe XML as well, I'm not sure) if you want to check it out.