Announcement Announcement Module
Collapse

JavaConfig forum decommissioned in favor of Core Container

As described at

http://static.springsource.org/sprin...fig/README.TXT

key features of the Spring JavaConfig project have been migrated into the core Spring Framework as of version 3.0.

Please see the Spring 3.0 documentation on @Configuration and @Bean support:

http://static.springsource.org/sprin...tml#beans-java

For any questions related to @Configuration classes and @Bean methods in Spring 3.0, please post in the dedicated 'Core Container' forum at

http://forum.springsource.org/forumdisplay.php?f=26
See more
See less
Any know leaks with JC/cgilib? Page Title Module
Move Remove Collapse
X
Conversation Detail Module
Collapse
  • Filter
  • Time
  • Show
Clear All
new posts

  • Any know leaks with JC/cgilib?

    I'm now using JC in somewhat large system. The functional test suite has just under 1000 tests. In the setup for the abstract-test I have:

    Code:
    buyingPowerFactory = BeanFactoryInstance.getBean("com.its.trading.domain.buyingpower.config.BuyingPowerConfiguration");
    Where BuyingPowerConfiguration is a JC config loaded in into an ApplicationContext via bean post processing (along with lots of xml configs).

    Code:
    @Configuration
    @AnnotationDrivenConfig
    @WithBeanNamePrefix("buyingpower.")
    public class BuyingPowerConfiguration extends ConfigurationSupport implements BuyingPowerFactory {
    
        @Autowired
        private InstrumentDataHome instrumentDataHome;
    
        @Autowired(required=false)
        private Set<BuyingPowerListener> buyingPowerListeners = Collections.emptySet();
    
        @Bean
        public BuyingPowerHome home() {
            return new BuyingPowerHomeImpl();
        }
    
        @Bean(scope=BeanDefinition.SCOPE_PROTOTYPE)
        public BuyingPowerImpl buyingPower() {
            BuyingPowerImpl buyingPower = new BuyingPowerImpl();
            addBuyingPowerListeners(buyingPower);
            return buyingPower;
        }
     
        ...
    }
    Just the calling of that bean lookup method drops the suite to it knees. About half way thru the suite, memory usage starts to sharply rise and test methods take longer and longer, eventually terminating with OutOfMemoryError. Simply commenting out the getBean() call the test runs normally. Note config is not even used in the test suite.

    I should note that as the application context is being recreated for every test method! (Not my doing, its what I inherited. I'm trying to correct that, but its gonna take some time.)

    I've tried running the system with YourKit, but both with the latest version and their 8.0 EAP the JVM does a hard crash when snapshotting memory.

    I've also created a single parameterized test suite that simply creates the same application context as the real test suite and calls getBean() for the JC a thousand times. But this did not reproduce the problem. (I'll keep working on this to try to use more memory to more closely simulate the real suite.)

    So, I'm asking (probably way to late in the post to have kept your attention) ... Are there any know issues with memory leaks with JC or its use of gclib that might help me to pinpoint what is going on?
    Last edited by memelet; Dec 6th, 2008, 08:10 AM.

  • #2
    Solution

    Not a JC or cglib problem at all. It seems that junit does not actually free up the test objects for the duration of the suite. So the test reference to JC config instance was keeping the entire application context in memory.

    The existing suite nulled out the the top-level application context, but the introduction of the reference to the JC config instance made that non effective since the JC configure was contains a reference to its containing application context.

    As an experiment I added the following as part of the teardown:

    Code:
        private void nullAllInstanceFields() {
            ReflectionUtils.doWithFields(BaseTestCase.this.getClass(), new FieldCallback() {
                @Override
                public void doWith(Field field) throws IllegalArgumentException, IllegalAccessException {
                    field.setAccessible(true);
                    field.set(BaseTestCase.this, null);
                }
            }, new FieldFilter() {
                @Override
                public boolean matches(Field field) {
                    try {
                        if (Modifier.isStatic(field.getModifiers())) {
                            return false;
                        }
                        if (Modifier.isFinal(field.getModifiers())) {
                            return false;
                        }
                        if (field.isEnumConstant()) {
                            return false;
                        }
                        Class<?> testClazz = field.getDeclaringClass();
                        if (!BaseTestCase.class.isAssignableFrom(testClazz)) {
                            return false;
                        }
                        ReflectionUtils.makeAccessible(field);
                        Object object = ReflectionUtils.getField(field, BaseTestCase.this);
                        if (object == null) {
                            return false;
                        }
                        if (ClassUtils.isPrimitiveOrWrapper(object.getClass())) {
                            return false;
                        }
                        return true;
                    } catch (Exception e) {
                        e.printStackTrace();
                        return false;
                    }
                }
            });
        }
    While this may turn out to be overkill, a suite that sucked up 786M now runs in 64M !!

    Here is where I got tipped to this isssue: http://blogs.atlassian.com/developer...ory_usage.html

    Comment


    • #3
      Interesting.. glad to hear you've got this worked out (or at least worked around).

      Comment


      • #4
        Simpler solution

        The root of this problem is that JUnit38ClassRunner is retained by the suite and hence JUnit38ClassRunner.fTest retains a reference to the test instance. Simply clearing this field (via reflection) replaces all the crud in the code above.

        For our case, we have a custom suite builder so we are able to replace JUnit38ClassRunner with the following:

        Code:
            public static class MemoryReleasingJUnit38ClassRunner extends JUnit38ClassRunner {
        
                private Description description;
        
                public MemoryReleasingJUnit38ClassRunner(Class<?> klass) {
                    super(klass);
        
                }
                public MemoryReleasingJUnit38ClassRunner(Test test) {
                    super(test);
                }
        
                @Override
                public void run(RunNotifier notifier) {
                    super.run(notifier);
                    description = getDescription();
                    Field testField = ReflectionUtils.findField(this.getClass(), "fTest");
                    ReflectionUtils.makeAccessible(testField);
                    ReflectionUtils.setField(testField, this, null);
                }
        
                @Override
                public Description getDescription() {
                    if (description == null) {
                        return super.getDescription();
                    } else {
                        return description;
                    }
                }
            }
        And then inject into junit via:

        Code:
        public class ClasspathSuite extends Suite {
        
            public ClasspathSuite(Class<?> suiteClass) throws InitializationError {
                super(getRunnerBuilder(), suiteClass, asArray(sortTestClasses(findTestClasses())));
            }
        
            private static class MemoryReleasingJUnit3Builder extends org.junit.internal.builders.JUnit3Builder {
                @Override
                public Runner runnerForClass(Class<?> testClass) throws Throwable {
                    if (isPre4Test_(testClass)) {
                        return new MemoryReleasingJUnit38ClassRunner(testClass);
                    }
                    return null;
                }
        
                private boolean isPre4Test_(Class<?> testClass) {
                    return junit.framework.TestCase.class.isAssignableFrom(testClass);
                }
            }
        
            private static class AllDefaultPossibilitiesBuilder extends org.junit.internal.builders.AllDefaultPossibilitiesBuilder {
                public AllDefaultPossibilitiesBuilder(boolean canUseSuiteMethod) {
                    super(canUseSuiteMethod);
                }
                @Override
                protected MemoryReleasingJUnit3Builder junit3Builder() {
                    return new MemoryReleasingJUnit3Builder();
                }
            }
        
            private static RunnerBuilder getRunnerBuilder() {
                return new AllDefaultPossibilitiesBuilder(true);
            }
        
            ...
        FYI, this bit of complexity is only required if you have very large test suites. Our unit-test suite contains 10k+ tests and our functional-test suite contains 2k+ tests, hence this little hack is critical for us.

        Comment

        Working...
        X