Home » Modeling » EMF » Improving containment list performance in CDO/EMF
| Improving containment list performance in CDO/EMF [message #1846992] |
Mon, 11 October 2021 14:19  |
 Alain Picard
Messages: 266
Registered: July 2009 |
Senior Member |
|
|
Hi,
We are seeing some scalability issues with regards to ConcurrentAccessException when dealing with multi-valued containment references.
The scenario is essentially that the same container essentially contains most of the key new elements that are being added at a point in time. For example, a container for weekly orders containing all orders for a week. During that week, adding new orders will always add to the same container and create a "choke" point with that reference list.
The problem that we are observing, is that EMF translates a list add, which is an append operation, into an indexed add at the location of the size at the time of the add. This can quickly lead to ConcurrentAccessException in an environment like CDO where the index was computed as x and after another element was added, it will fail since the index should now be x+1.
What we would like to implement (and share back to EMF/CDO after), is the ability to do add that are real append, i.e. just append the element to the end of the list, whatever location that is. From what I can see, those adds would probably have to return the final insertion index back so that EMF can add those to notifications. On the CDO side there is still some transaction concurrency to address for sure.
What are your ideas? concerns ? etc.
Cheers,
Alain
On
|
|
| | | |
| Re: Improving containment list performance in CDO/EMF [message #1847011 is a reply to message #1847006] |
Tue, 12 October 2021 07:43  |
Alain Picard
Messages: 266
Registered: July 2009 |
Senior Member |
|
|
Ed,
I wouldn't consider it a bug from a transaction perspective. If your add got translated to add element at position x and then another transaction happened in the meantime, you get notified of the issue and can retry the transaction. There is no way to know if that position x meant just append to the end of the list, whatever it's at, or do what is said and put it at position x, in which case you should be notified that your referenced view of the element is out of sync. Imagine if you're sorting the list in some fashion, position x could have been the end of the list, but after the other transaction has happened, this can force you to re-evaluate. This is why the semantic of add and add at index are different.
OTOH, if you would submit a real add/append to CDO without any any index, then it can be clear that your intent is to simply append to such list, without any consideration for the position. Then the conflict can be managed in an optimized way.
Alain
|
|
| |
| Re: Improving containment list performance in CDO/EMF [message #1847016 is a reply to message #1847011] |
Tue, 12 October 2021 09:44 |
|
You're right, AbstractEList.add(E) delegates to addUnique(E), which calls doAddUnique(size(), E) in DelegatingNotifyingListImpl. In CDO that creates a CDOAddFeatureDelta with the insertion index that's computed in the committing client transaction. When that arrives at the server the underlying list/object can already be modified by another client and the insertion index (like any other feature delta in that revision delta) can have become meaningless. Treating an "append" differently is not easily possible because even when one would succeed to create a new kind of CDOAddFeatureDelta this feature delta would have to be interpreted differently in thousand places. More importantly, we can't generally establish that the doAddUnique(size(), E) behavior is NOT the intended behavior; applications could rely on EMF's known behavior.
But CDO offers automatic conflict resolvers on both the client and the server side. On the client side you can add one, for example, as follows:
transaction.options().addConflictResolver(new CDOMergingConflictResolver(new DefaultCDOMerger.PerFeature.ManyValued());
A CDOConflictResolver is comparingly cheap, but conflicts can still happen on the server if timing permits. In addition you can one on the server side, for example, as follows:
repository.setCommitConflictResolver(new ICommitConflictResolver.Merging());
An ICommitConflictResolver attempts to resolve possible conflicts during TransactionCommitContext.computeDirtyObjects().
Both resolvers use a DefaultCDOMerger.PerFeature.ManyValued, which is able to adjust the insertion index of an add() operation, if needed.
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
| |
| Re: Improving containment list performance in CDO/EMF [message #1847018 is a reply to message #1847016] |
Tue, 12 October 2021 09:57 |
Alain Picard
Messages: 266
Registered: July 2009 |
Senior Member |
|
|
Eike,
Thanks for the great info. As you know (from other threads here) we do have a CDOMergingConflictResolver, but not tailored to this and I get the hint to move to current version as 4,7 doesn't have the server side conflict resolution.
That leads me to one more question. How does one know that it is a valid index re-adjustment case. I was thinking that if the feature is not ordered, that is probably a good candidate that doesn't care about the index, but I do have, what is probably the main case, where ordered is true but insertion order is irrelevant. The ordering is set to true, since elements can be manually re-ordered by the user after the fact. Semantically would you see the pattern of add at Integer.MAX_VALUE to be an approach or what other mechanism would you suggest to provide the correct semantic?
Cheers,
Alain
|
|
|
| Re: Improving containment list performance in CDO/EMF [message #1847020 is a reply to message #1847018] |
Tue, 12 October 2021 10:16 |
|
Alain Picard wrote on Tue, 12 October 2021 11:57Semantically would you see the pattern of add at Integer.MAX_VALUE to be an approach or what other mechanism would you suggest to provide the correct semantic?
Semantically a new CDOFeatureDelta type for "append" would seem to be the right approach. But, as I mentioned before, many technical implications are not clear at this point. And getting it all right would be an extremely expensive undertaking.
It would be much simpler and safer to base the "insert vs. append" decision not on the operation type, but rather on some static feature information. Note that you can extend the DefaultCDOMerger and make it ignore certain features.
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
| |
Goto Forum:
Current Time: Thu May 02 09:41:38 GMT 2024
Powered by FUDForum. Page generated in 0.06694 seconds |
] 
Search
Help
Register
Login
Home
