I added a call to EntityManager.clear() after each transaction, but these object arrays still seem to be piling up. I am doing another test run with tracking allocations turned on, so hopefully I can pin down better where these objects are being allocated.
Does EntityManager.clear() clear the whole cache? I don't see a clear() method on EntityManagerFactory, which I understand wraps the ServerSession - is there something I can try re-setting the whole thing?
Thanks.
- DAP On Oct 20, 2009, at 11:57 AM, David Parker wrote: Thanks, everybody, for all the responses. I appreciate your help. Are you using JTA, or RESOURCE_LOCAL?
Local Are you using the same EntityManager, or do you create a new one for each transaction?
This thing runs as a big loop sucking records out of one database via a simple JDBC Connection, and for each record creating the JPA objects for that record, and persisting them. As part of creating the JPA objects there is some querying against the target database via SQL queries.
There is one EntityManager, and a single transaction per record. What cache setting are you using?
SoftWeak, but I am confused about what the best caching option is. I've read the doc, and I ~think I understand the technical description, but I'm still not sure when to use what. I would recommend you create a new EntityManager per transaction, otherwise the previous objects will still be part of the persistence context. You could also call clear().
I've started another run with the clear() after each commit.
You can try disabling the cache (shared=false), or using a weak cache.
Do you run out of memory, or just notice a memory increase? You may wish to try a memory profiler such as JProfiler.
Memory increases slowly but inexorably. I've had it blow the heap at -Xmx6G -d64 -server. I am hopeful that just being more aggressive about the cache clearing will fix the problem. The YourKit profiler doesn't flag much it thinks is a leal - just a few k. Also ensure that your application is not holding onto any or the objects. DavidParker wrote:
Yeah, we are using multiple transactions, i.e.
get an EntityTransaction
do work
EntityTransation.commit()
set the local tx to null
On Oct 19, 2009, at 5:59 PM, Dave Brosius wrote:
Is the import all in one transaction? I'd be curious if you broke up
the import (even for a test) into multiple transactions whether the
issue would go away.
-----Original Message-----
From: "David Parker" <dap@xxxxxxxxxxxxxxxxxxxx>
Sent: Monday, October 19, 2009 5:51pm
To: "EclipseLink User Discussions" <eclipselink-users@xxxxxxxxxxx>
Subject: [eclipselink-users] memory usage
I am running a long data import that uses eclipselink 1.1.2 for
persistence. I see memory increasing slowly but surely as it goes
along. The YourKit profiler does not identify anything as a link, but
I do seem to have an increasing "retained size" for
org.eclipse.persistence.internal.sessions.RepeatableWriteUnitOfWork
[Stack Local]
I see there is a clear method on TransactionWrapperImpl, but that does
not seem to be in the JPA EntityTransaction API.
Is there a best practice for long running data imports? Should I be
clearing something?
Thanks.
=============================================
David Parker
dap@xxxxxxxxxxxxxxxxxxxx
----- http://wiki.eclipse.org/User:James.sutherland.oracle.com James Sutherland http://www.eclipse.org/eclipselink/ EclipseLink , http://www.oracle.com/technology/products/ias/toplink/TopLink Wiki: http://wiki.eclipse.org/EclipseLink EclipseLink , http://wiki.oracle.com/page/TopLink TopLink Forums: http://forums.oracle.com/forums/forum.jspa?forumID=48 TopLink , http://www.nabble.com/EclipseLink-f26430.html EclipseLink Book: http://en.wikibooks.org/wiki/Java_Persistence Java Persistence -- View this message in context: http://www.nabble.com/memory-usage-tp25966341p25974597.htmlSent from the EclipseLink - Users mailing list archive at Nabble.com. _______________________________________________ eclipselink-users mailing list eclipselink-users@xxxxxxxxxxxhttps://dev.eclipse.org/mailman/listinfo/eclipselink-users
============================================= David Parker
_______________________________________________ eclipselink-users mailing list eclipselink-users@xxxxxxxxxxx https://dev.eclipse.org/mailman/listinfo/eclipselink-users
============================================= David Parker
|