Run long batch processing jobs in Grails without memory leaks
In one of our Grails applications we had to run a number of batch jobs. Nothing unusual and Grails supports it quite well with the excellent Quartz plugin.
But when we deployed application in production, we noticed that after running for some time, it consumed a lot of memory and JVM was spending all the time running garbage collection. The reason for it was that our jobs were quite long-running, taking several hours to complete, and Grails wasn’t really designed for such kind of use case.
First problem is actually well-known and documented — using single Hibernate session for a long time gets a lot of objects being added to cache. It is described for example in this blog post.
This basically goes into flushing and clearing Hibernate session like following:
But while less memory was being leaked after that, the problem still wasn’t resolved. However thankfully to this nice post about Grails memory leak I learned about another more obscure problem and got it fixed.
Main idea is that
save() method stores validation errors info in thread-local storage when
HttpServletRequest instance is not available. So this info needs to be cleared from time to time, which can be achieved by something like this:
Resolving these two issues solved our problem with memory usage. Hopefully this post prevents you from spending much time resolving the same issues.