Announcement Announcement Module
Collapse
No announcement yet.
Is Pageable implementation for mongo considering the mongo doc? Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Is Pageable implementation for mongo considering the mongo doc?

    I've been wondering if the Pageable implementation done via MongoRepository considers this stated on the mongodb docs?

    "Paging Costs
    Unfortunately skip can be (very) costly and requires the server to walk from the beginning of the collection, or index, to get to the offset/skip position before it can start returning the page of data (limit). As the page number increases skip will become slower and more cpu intensive, and possibly IO bound, with larger collections.
    Range based paging provides better use of indexes but does not allow you to easily jump to a specific page."

    http://www.mongodb.org/display/DOCS/...ip%28%29%7D%7D

    Any insights about this?

    Best

  • #2
    Range based paging provides better use of indexes but does not allow you to easily jump to a specific page.
    This pretty much sums up that part of the documentation as "if you need to do pagination, it can be slow potentially" but not actually documenting a better way to do it. So all we can do is use the existing API.

    Comment


    • #3
      Thanks Oliver, all this make sense. Let's hope for a better API in the near future on the meantime I'll use a range query to go over the data set

      Comment


      • #4
        Um, what you quoted and what you summarized seem to be different things. Range based paging is not slow, nor potentially slow. It is documented as a better way to do paging than using skip/limit and many people have provided example and samples of how to do it. Here are a few references:

        http://stackoverflow.com/questions/6...paging-mongodb
        (not the best question, but includes more links in the answer)
        http://stackoverflow.com/questions/8...ies-in-mongodb

        Comment


        • #5
          Thanks for the additional links, Scott. What I was a bit underwhelmed with about the reference documentation is that it essentially says: "well paging with skip/limit can be costly so you might wanna use range queries" but then doesn't give a link to what that actually means. Might be a good idea to add the ones you just listed here to the docs in general.

          Generally speaking the suggested range queries might offer a more performant access to paginated data but require deep knowledge about the data to be paginated plus a "previous" page which are unmeetable requirements if you want to build a general purpose "give my page x by a page size of y sorted by z" kind of API on top of MongoDB. Not to mention that you might not return correct data in case an element was inserted into the original page while accessing it.

          Long story short: you have a point that my statement should have been "if you want random access pagination" and it's cool to have the range query option listed here so that people can choose that style (which is usable just using the query API and a PageRequest object) in case they experience performance problems.

          Comment

          Working...
          X