Re: [eclipselink-users] Threads waiting indefinitely in ConcurrencyManag

[Andrei Ilitchev <andrei.ilitchev@xxxxxxxxxx>]

Hi Shashi,

My understanding is that you don't see a deadlock but rather a 
performance problem caused by multiple thread concurrently requesting 
sequence numbers from the same sequence.

Eclipselink preallocates (caches) sequence values. When all the cached 
values are used Eclipselink executes a query to obtain more: the first 
thread to get to an empty sequence values' cache obtains a lock, runs 
the query, copies the obtained values into the cache, releases the lock.
After that all threads pick up the preallocated values without executing 
more queries.
Therefore the higher sequence prealloocation size the less sequence 
queries performed and less wait occurs.
The drawback is potentially not using up all the preallocated values.

I would suggest increasing sequence preallocation size in Eclipselink.
Note that the preallocation size should equal to "increment by" 
parameter of the sequence in the data base.

Newer (2.0.2 and later) versions of Eclipselink use a different pattern 
when preallocation size is 1: in this case only one new sequence value 
is allocated by the query, therefore there is nothing to share with 
other threads and the lock in never obtained.

Thanks,
Andrei

On 4/28/2011 7:35 PM, Shashikant Kale wrote:
> Analyzing further, I feel the issue is due to too many sequence values
> getting generated and each time the next sequence is asked for, the
> SequencingManager would obtain a lock using the ConcurrencyManager, I
> see in the thread dump that there are several (at a stretch 15) threads
> waiting on ConcurrencyManager since the same sequence is being used to
> get the next value. I am not sure if creating my own sequencing manager
> would help here.
>
> Please suggest.
>
> Regards,
> Shashi
>
> *From:* eclipselink-users-bounces@xxxxxxxxxxx
> [mailto:eclipselink-users-bounces@xxxxxxxxxxx] *On Behalf Of *Shashikant
> Kale
> *Sent:* Friday, April 29, 2011 4:23 AM
> *To:* EclipseLink User Discussions
> *Subject:* Re: [eclipselink-users] Threads waiting indefinitely in
> ConcurrencyManager.
>
> Few more updates,
>
> We have disabled the L2 cache in the application.
>
> I have observed that there is different object id on which the
> ConcurrencyManager waits for the same thread in different thread dumps.
> That means when I am looking at the thread dumps taken at different
> times, have the same web request thread stuck at the same place but
> actually is serving a different request. So does it mean that it is not
> a problem with the eclipselink however there are too many threads
> waiting for writing to the database and write operation is slow?
>
> Please throw some light.
>
> Thanks,
> Shashi
>
> *From:* eclipselink-users-bounces@xxxxxxxxxxx
> [mailto:eclipselink-users-bounces@xxxxxxxxxxx] *On Behalf Of *Shashikant
> Kale
> *Sent:* Friday, April 29, 2011 3:46 AM
> *To:* EclipseLink User Discussions
> *Subject:* [eclipselink-users] Threads waiting indefinitely in
> ConcurrencyManager.
>
> Hello,
>
> We have been using eclipselink 1.0 (Old application) for application
> development using JPA. For few days when we are executing performance
> tests for the application, we are observing threads waiting on object
> monitor at ConcurrencyManager. The thread stack shows
>
> "ajp-145.245.142.25-8309-79" daemon prio=6 tid=0x0000000008961800
> nid=0x1818 in Object.wait() [0x000000001afbd000]
>
> java.lang.Thread.State: WAITING (on object monitor)
>
> at java.lang.Object.wait(Native Method)
>
> - waiting on <0x00000000a3e60fe8> (a
> org.eclipse.persistence.internal.helper.ConcurrencyManager)
>
> at java.lang.Object.wait(Object.java:485)
>
> at
> org.eclipse.persistence.internal.helper.ConcurrencyManager.acquire(ConcurrencyManager.java:89)
>
> - locked <0x00000000a3e60fe8> (a
> org.eclipse.persistence.internal.helper.ConcurrencyManager)
>
> at
> org.eclipse.persistence.internal.helper.ConcurrencyManager.acquire(ConcurrencyManager.java:75)
>
> at
> org.eclipse.persistence.internal.sequencing.SequencingManager.acquireLock(SequencingManager.java:279)
>
> at
> org.eclipse.persistence.internal.sequencing.SequencingManager$Preallocation_NoTransaction_State.getNextValue(SequencingManager.java:590)
>
> at
> org.eclipse.persistence.internal.sequencing.SequencingManager.getNextValue(SequencingManager.java:884)
>
> at
> org.eclipse.persistence.internal.sequencing.ClientSessionSequencing.getNextValue(ClientSessionSequencing.java:86)
>
> at
> org.eclipse.persistence.internal.descriptors.ObjectBuilder.assignSequenceNumber(ObjectBuilder.java:258)
>
> at
> org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.assignSequenceNumber(UnitOfWorkImpl.java:405)
>
> at
> org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.registerNotRegisteredNewObjectForPersist(UnitOfWorkImpl.java:3879)
>
> at
> org.eclipse.persistence.internal.sessions.RepeatableWriteUnitOfWork.registerNotRegisteredNewObjectForPersist(RepeatableWriteUnitOfWork.java:369)
>
> at
> org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.registerNewObjectForPersist(UnitOfWorkImpl.java:3827)
>
> - locked <0x00000000fde013a8> (a
> org.eclipse.persistence.internal.sessions.RepeatableWriteUnitOfWork)
>
> at
> org.eclipse.persistence.internal.jpa.EntityManagerImpl.persist(EntityManagerImpl.java:219)
>
> at
> com.arisglobal.framework.services.audittrail.AuditTrailServiceImpl.saveAuditTrail(AuditTrailServiceImpl.java:40)
>
> I saw a similar bug
> <https://bugs.eclipse.org/bugs/show_bug.cgi?id=312110> fixed in
> eclipselink 1.2.1 release. However since we are using quite older
> version (not feasible to upgrade to the latest version due to
> application production release date) I was not sure how we can apply the
> patch to the eclipselink.
>
> Would appreciate any help regarding this.
>
> Thanks,
> Shashi
>
> ------------------------------------------------------------------------
>
>
> Disclaimer: This transmission, including attachments, is confidential,
> proprietary, and may be privileged. It is intended solely for the
> intended recipient. If you are not the intended recipient, you have
> received this transmission in error and you are hereby advised that any
> review, disclosure, copying, distribution, or use of this transmission,
> or any of the information included therein, is unauthorized and strictly
> prohibited. If you have received this transmission in error, please
> immediately notify the sender by reply and permanently delete all copies
> of this transmission and its attachments.
>
> ------------------------------------------------------------------------
>
>
> Disclaimer: This transmission, including attachments, is confidential,
> proprietary, and may be privileged. It is intended solely for the
> intended recipient. If you are not the intended recipient, you have
> received this transmission in error and you are hereby advised that any
> review, disclosure, copying, distribution, or use of this transmission,
> or any of the information included therein, is unauthorized and strictly
> prohibited. If you have received this transmission in error, please
> immediately notify the sender by reply and permanently delete all copies
> of this transmission and its attachments.
>
>
> ------------------------------------------------------------------------
>
> Disclaimer: This transmission, including attachments, is confidential,
> proprietary, and may be privileged. It is intended solely for the
> intended recipient. If you are not the intended recipient, you have
> received this transmission in error and you are hereby advised that any
> review, disclosure, copying, distribution, or use of this transmission,
> or any of the information included therein, is unauthorized and strictly
> prohibited. If you have received this transmission in error, please
> immediately notify the sender by reply and permanently delete all copies
> of this transmission and its attachments.
>
>
>
> _______________________________________________
> eclipselink-users mailing list
> eclipselink-users@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/eclipselink-users