Announcement Announcement Module
Collapse
No announcement yet.
Spring Data MongoDB aggregation framework implementation - exceptions Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Spring Data MongoDB aggregation framework implementation - exceptions

    Hello all,
    I'm trying to use the new implementation of the Mongo Aggregation framework by Spring Data team I'm using the spring-data version 1.3.0.RC1 ! and for reference I'm looking at https://jira.springsource.org/browse/DATAMONGO-586
    Here is my problem:

    Let's say we have collections with article items:
    Code:
    {
    "_id": {
    "$oid": "5214b5d529ee12460939e2ba"
    },
    "title": "this is my title",
    "author": "bob",
    "pageViews": 5,
    "tags": [
    "fun",
    "sport"
    ],
    "comments": [
    {
    "author": "alex",
    "text": "this is cool",
    "createdAt": 1
    },
    {
    "author": "sam",
    "text": "this is bad",
    "createdAt": 2
    },
    {
    "author": "jenny",
    "text": "this is bad",
    "createdAt": 3
    }
    ],
    "other": {
    "foo": 5
    }
    }
    .....
    How I can make an aggregation query like this:
    db.articles.aggregate( {$project:{author:1, comments:1}} , {$unwind:"$comments"} , {$sort:{"comments.createdAt":-1}} , {$group: {_id:null,comments:{$push:"$comments"}}} );

    I try with the following code snippet but every time exceptions are thrown:

    Code:
            Aggregation agg = newAggregation( //
                    project().andInclude("author").andInclude("comments"), //this should make the project part : {$project:{author:1, comments:1}} - actual result: { "comments" : "$comments" , "author" : "$author"}
                    unwind("comments"),//second part :  {$unwind:"$comments"} - this is correct
                    group("_id","comments").push("comments").as("comments") - throws invalid reference "_id"...
                    //sort(Direction.ASC,"comments.createdAt") - this didn't work with nested array properties
            );
    I want to transform articles collection to be displayed only with comments ordered by "createdAt" field.
    I don't know if my approach is wrong or the aggregation framework is not yet fully implemented.
    Thanks in advance

  • #2
    Originally posted by nickolayrusev View Post
    How I can make an aggregation query like this:
    db.articles.aggregate( {$project:{author:1, comments:1}} , {$unwind:"$comments"} , {$sort:{"comments.createdAt":-1}} , {$group: {_id:null,comments:{$push:"$comments"}}} );

    I try with the following code snippet but every time exceptions are thrown:

    Code:
            Aggregation agg = newAggregation( //
                    project().andInclude("author").andInclude("comments"), //this should make the project part : {$project:{author:1, comments:1}} - actual result: { "comments" : "$comments" , "author" : "$author"}
                    unwind("comments"),//second part :  {$unwind:"$comments"} - this is correct
                    group("_id","comments").push("comments").as("comments") - throws invalid reference "_id"...
                    //sort(Direction.ASC,"comments.createdAt") - this didn't work with nested array properties
            );
    Please have a look at these examples:

    https://github.com/SpringSource/spri...tionTests.java

    To just project some fields, the project() method takes a simple list of the fields names. Something like this:
    Code:
    newAggregation( project( "author", "comments"), ... )
    The missing field reference for _id may be be resolved if you include _id in the initial projection:
    Code:
    newAggregation( project( "_id", "author", "comments"), ... )
    HTH
    Tobias

    Comment


    • #3
      Hello,
      thanks for the response. Yeah I'm looking into the test cases. I tried to add the "_id" field into initial projection but the same result...
      Code:
      @Test
      	public void testMongoOperations(){
      		Aggregation agg = newAggregation( //
                      project("_id","author","comments"), //this should make the project part : {$project:{author:1, comments:1}} - actual result: { "comments" : "$comments" , "author" : "$author"}
                      unwind("comments"),//second part :  {$unwind:"$comments"} - this is correct
                      group("_id","comments").push("comments").as("comments") //- throws invalid reference "_id"...
                      //sort(Direction.ASC,"comments.createdAt") - this didn't work with nested array properties
              );
      		AggregationResults<Comment> aggregate = mongoOperations.aggregate(agg, "articles", Comment.class);
      		List<Comment> mappedResults = aggregate.getMappedResults();
      		
      	}
      Code:
      	java.lang.IllegalArgumentException: Invalid reference '_id'!
      	at org.springframework.data.mongodb.core.aggregation.ExposedFieldsAggregationOperationContext.getReference(ExposedFieldsAggregationOperationContext.java:63)
      	at org.springframework.data.mongodb.core.aggregation.ExposedFieldsAggregationOperationContext.getReference(ExposedFieldsAggregationOperationContext.java:47)
      	at org.springframework.data.mongodb.core.aggregation.GroupOperation.toDBObject(GroupOperation.java:292)
      	at org.springframework.data.mongodb.core.aggregation.Aggregation.toDbObject(Aggregation.java:228)
      	at org.springframework.data.mongodb.core.MongoTemplate.aggregate(MongoTemplate.java:1256)
      	at org.springframework.data.mongodb.core.MongoTemplate.aggregate(MongoTemplate.java:1245)
      	at org.mongo.domain.MongoOperationsTest.testMongoOperations(MongoOperationsTest.java:63)
      I didn't get some of the things:

      1. Why project("author") produces: {$group:{"author":"$author"}} but not: {$group:{"author":1}}?
      2.why i can't resolve the field "_id" even it's exposed and how I can pass the "_id" field value "null" like this "$group: {_id:null"} ?
      Thanks in advance

      Comment


      • #4
        1. There a few slight inconsistencies in the aggregation framework which makes it hard to easily build a reference checked pipeline. Thus we decided to always explicitly define the output names of an aggregation step. The expression you see is logically equivalent to what you expect.

        2. It's important to understand that you define aggregations in terms of type properties, not document field names. Beyond that _id has special semantics in the aggregation case as it is used to separate non-syntheic fields from synthetic ones.

        I am a bit puzzled as you seem to want two things: first, you were asking to include the _id field into the group clause. This can be achieved by consistently referring to the id *property name* through all of the aggregation steps. In your last post you're asking to drop the id (I guess you're again referring to the id property). You can do this by simply not listing the property. If you want to refer to the _id used to refer to the previous aggregation step, use Aggregation.previousOperation(). This will express that you're not referring to an individual field.

        Is there a chance you provide a small sample project, that we can play with?

        Comment


        • #5
          Hello thanks for reply,
          Yeah,
          later in the day I will provide sample project.
          At the moment the only problem is that i can't create this part of the query :
          {$group: {_id:null}} because i cannot pass null to the "_id" ... and in the mongodb is allowed

          Code:
          java.lang.IllegalArgumentException: AggregationField name must not be null or empty!

          Comment


          • #6
            Here is the project:
            https://github.com/nickolayrusev/MongoDBAggregation

            Look into ExampleServiceTests.java , located in /MongoDBAggregation/src/test/java/org/main/app/ExampleServiceTests.java
            There is a description what should be the desired result. Thanks in advance !

            Comment


            • #7
              Hello,
              Did you have time to look at the sample project?
              thanks a lot

              Comment

              Working...
              X