Announcement Announcement Module
Collapse
No announcement yet.
LTWWorld.dumpState_cantFindTypeExceptions filling up, causes OutOfMemoryException Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • LTWWorld.dumpState_cantFindTypeExceptions filling up, causes OutOfMemoryException

    We are experiencing OutOfMemoryExceptions that appear to be caused by runaway growth of the dumpState_cantFindTypeExceptions collection in LTWWorld.java.

    At the time of the most recent heap exhaustion, the list had grown to consume 40% of the heap, containing 179,347 entries, each with the message "Can't find type com.backstopsolutions.common.aspects.IdempotentCac he$ajcMightHaveAspect".

    We're not sure how to proceed - I see there's an accept() method that can be used to truncate the list, is it really up to us to manage that?

    We're using:

    JVM:
    java version "1.5.0_13"
    Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_13-b05)
    Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_13-b05, mixed mode)

    aspectjrt.jar 1.5.4
    aspectjweaver.jar 1.5.4

    jboss 4.2.2.GA

    I'm attaching the ajcore dump file - please let me know if I can provide more information.

  • #2
    Well, I attempted to upload the ajcore file, it's 1.5M & disallowed. I would be happy to email.

    Comment


    • #3
      As a first step, I will certainly recommend switching over to the latest version of AspectJ. Andy Clement and the AspectJ team has put a lot of energy improving in this area and continues to do so.

      -Ramnivas

      Comment


      • #4
        We will certainly do so going forward, but our burn-in cycle isn't going to allow for that in our immediate situation. We recently upgraded to 1.5.4, we may have to roll back to what we were using before in the short term.

        The issue seems to surround a pointcut we defined with annotations - it appears to be weaving itself onto many objects in the system, not just the subset we're trying to limit it to. I'm wondering if that might be the source of all of the ajcMightHaveAspect notices.

        Is this an appropriate forum to request help with our pointcut definition?

        Comment


        • #5
          Show a couple of your pointcuts. I have intuitive feeling of what may be wrong. We will see if that turns out to be the case :-)

          -Ramnivas

          Comment


          • #6
            Thanks for taking a moment to check it out.

            Here is the code for the class at the center of things. Our intent is that this pointcut only be applied to classes with the '@Idempotent' annotation (I included the annotation at the bottom, just in case it's involved).

            I have inherited this code, and I confess to not fully understanding it. I have been spinning up on AOP & the pointcut _seems_ to be what we want, but it's hard to nail down.

            Code:
            @Aspect("pertarget( cached( java.lang.Object ) )")
            public class IdempotentCache{
                Log logger = LogFactory.getLog(IdempotentCache.class);
                private ConcurrentHashMap<String,Object> cache = new ConcurrentHashMap<String, Object>();
            
                @Pointcut("execution(public * *(..)) && " +
                        "!execution(public void *(..)) && " +
                        "!execution(public java.util.Map+ *(..)) && " +
                        "!execution(public java.util.Collection+ *(..)) && " +
                        "target(calculator) && " +
                        "within(@Idempotent *)")
                public void cached( Object calculator ){}
            
            	@Around("cached(calculator)")
                public Object aroundCached(ProceedingJoinPoint pjp, Object calculator) {
                    logger.info(calculator.getClass().getName());
                    StringBuffer key = new StringBuffer(""+pjp.getSignature().hashCode());
            		for(int i = 0; i < pjp.getArgs().length; i++) {
                           Object o = pjp.getArgs()[i];
               			key.append("_");
               			if(o instanceof EntityTypeIdentifiable) {
               				EntityTypeIdentifiable ie = (EntityTypeIdentifiable) o;
               			    key.append(ie.getId()).append(ie.getEntityTypeIdentifier());
               			} else {
                            key.append("").append(o.hashCode());
            			}
                    }
                    Object retValue = cache.get(key.toString());
                	if (retValue == null){
                        try {
                            retValue = pjp.proceed();
                            if (retValue == null){
                                retValue = NULL_MARKER;
                            }
                            Object oldReturnValue = cache.putIfAbsent(key.toString(), retValue);
                            if(oldReturnValue != null){
                                retValue = oldReturnValue;
                            }
                        } catch (Throwable throwable) {
                            LogFactory.getLog(this.getClass()).error(throwable.getMessage(), throwable);
                        }
            		}
                    if (retValue == NULL_MARKER){
                        retValue = null;
                    }
                    return retValue;
            	}
            
                public static final Object NULL_MARKER = new Object();
            }
            The @Idemptent annotation:

            @Retention(value = RetentionPolicy.RUNTIME)
            public @interface Idempotent {
            }

            Comment


            • #7
              Well, your pointcut looks innocent. I was suspecting the use of @annotation() leading to runtime checks, but that is not the case.

              I am not so sure, but you may want to try without pertarget. Instead use @DeclareParents to introduce the per-object cache (I am assuming that is what you want/need).

              -Ramnivas

              Comment


              • #8
                Thank you very much Ramnivas - we're going to give that a try & some other things. If & when we resolve it, I'll post back in the hope that it can be useful to someone else.

                Comment


                • #9
                  Hi,

                  I'm surprised you are getting a message about a missing *generated* type:

                  com.backstopsolutions.common.aspects.IdempotentCac he$ajcMightHaveAspect

                  as that will have been created during weaving so must be around (somewhere...). Are they really all for that same type? As Ramnivas says, the pointcut looks ok - I might use this() rather than target() as you are using execution() - but they should, of course, mean the same thing here.

                  I notice you don't qualify the annotation within the pointcut, is it in the same package as the aspect?

                  I've just put in a change to limit the growth of the dumpState_cantFindTypeExceptions list - if the limit is hit (100) there is clearly something really wrong and so no need to collect even more.

                  If you don't get anywhere, do let me know and we can dig a bit deeper.

                  regards,
                  Andy
                  ---
                  Andy Clement
                  AspectJ Development

                  Comment


                  • #10
                    Hi Andy - thanks for taking a look.

                    As far as I can tell, all are for the same type. I have an ajcore dump file that precedes the heap exhaustion - it seems to only have a small subset of what I'm seeing in the .hprof file. The ajcore file might have more information that you could make sense of, I'm not quite sure what to make of it. I wanted to attach it but it's 1.5M.

                    I removed the qualification from the annotation when I posted it, it was the beginning of a halfhearted and ultimately abandoned attempt at unnecessary obfuscation of posting code to the internet. Sorry for the confusion.

                    We're actually going to test our code with & without this aspect - upon review we don't think it's actually doing what we want. If the difference is not significant, I think our solution will just be to remove it.

                    I appreciate the time you guys have spent taking a look.

                    Tim

                    Comment

                    Working...
                    X