Re: [eclipselink-users] memory usage

[David Parker <dap@xxxxxxxxxxxxxxxxxxxx>]

Thanks, everybody, for all the responses. I appreciate your help.

Are you using JTA, or RESOURCE_LOCAL?

Local

Are you using the same EntityManager, or do you create a new one for each
transaction?

This thing runs as a big loop sucking records out of one database via a simple JDBC Connection, and for each record creating the JPA objects for that record, and persisting them. As part of creating the JPA objects there is some querying against the target database via SQL queries.

There is one EntityManager, and a single transaction per record.

What cache setting are you using?

SoftWeak, but I am confused about what the best caching option is. I've read the doc, and I ~think I understand the technical description, but I'm still not sure when to use what.

I would recommend you create a new EntityManager per transaction, otherwise
the previous objects will still be part of the persistence context.  You
could also call clear().

I've started another run with the clear() after each commit.

You can try disabling the cache (shared=false), or using a weak cache.

Do you run out of memory, or just notice a memory increase?  You may wish to
try a memory profiler such as JProfiler.

Memory increases slowly but inexorably. I've had it blow the heap at -Xmx6G -d64 -server. I am hopeful that just being more aggressive about the cache clearing

will fix the problem. The YourKit profiler doesn't flag much it thinks is  a leal - just a few k.

Also ensure that your application is not holding onto any or the objects.


DavidParker wrote:

Yeah, we are using multiple transactions, i.e.

get an EntityTransaction

do work

EntityTransation.commit()

set the local tx to null

On Oct 19, 2009, at 5:59 PM, Dave Brosius wrote:

Is the import all in one transaction? I'd be curious if you broke up  

the import (even for a test) into multiple transactions whether the  

issue would go away.

-----Original Message-----

From: "David Parker" <dap@xxxxxxxxxxxxxxxxxxxx>

Sent: Monday, October 19, 2009 5:51pm

To: "EclipseLink User Discussions" <eclipselink-users@xxxxxxxxxxx>

Subject: [eclipselink-users] memory usage

I am running a long data import that uses eclipselink 1.1.2 for

persistence. I see memory increasing slowly but surely as it goes

along. The YourKit profiler does not identify anything as a link, but

I do seem to have an increasing "retained size" for

org.eclipse.persistence.internal.sessions.RepeatableWriteUnitOfWork

[Stack Local]

I see there is a clear method on TransactionWrapperImpl, but that does

not seem to be in the JPA EntityTransaction API.

Is there a best practice for long running data imports? Should I be

clearing something?

Thanks.

=============================================

David Parker

dap@xxxxxxxxxxxxxxxxxxxx

-----
http://wiki.eclipse.org/User:James.sutherland.oracle.com James Sutherland
http://www.eclipse.org/eclipselink/
EclipseLink ,  http://www.oracle.com/technology/products/ias/toplink/
TopLink
Wiki:  http://wiki.eclipse.org/EclipseLink EclipseLink ,
http://wiki.oracle.com/page/TopLink TopLink
Forums:  http://forums.oracle.com/forums/forum.jspa?forumID=48 TopLink ,
http://www.nabble.com/EclipseLink-f26430.html EclipseLink
Book:  http://en.wikibooks.org/wiki/Java_Persistence Java Persistence
--
View this message in context: http://www.nabble.com/memory-usage-tp25966341p25974597.html
Sent from the EclipseLink - Users mailing list archive at Nabble.com.

_______________________________________________
eclipselink-users mailing list
eclipselink-users@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/eclipselink-users

=============================================

David Parker

dap@xxxxxxxxxxxxxxxxxxxx