Home » Modeling » EMF "Technology" (Ecore Tools, EMFatic, etc) » CDO performance problems
|
| Re: CDO performance problems [message #22018 is a reply to message #21567] |
Fri, 10 February 2006 11:18   |
Eclipse User |
|
|
|
Originally posted by: stepper.sympedia.de
Mark,
Sorry, that I did not respond immediately. I just returned from an abroad journey.
One major task for my next week is the actual migration of CDO to EMFT, and that will
establish a kind of third version of CDO. Nevertheless I will try to investigate your
issues during this weekend. There are some infos that you could provide to ease my start:
- Can you give me a timestamp of your CVS checkout?
- Is it possible that you send me a complete workspace, so that I can reproduce an identical environment?
If yes, just zip it and send it via email (if that doesn't work due to limitations of my pop account,
I could manage to create an FTP acoount for you)
- Do you have the feeling that you just uncovered an issue that happened to persist for a longer period?
Or does it seem to be associated with some kind of environmental change?
- Some time ago you applied some changes to Channel/Multiplexer threading model.
Do you have changes in this area, that are not committed to the code base?
- Is there an opportunity that you take this occasion as a trigger to reintegrate with
a newer version of CDO?
- (If "No" for the last question) Do you see a chance that we will at least be able to
test your environment with the newest version of CDO, so that we receive an indication
where to focus our efforts with the highest degree of synergy?
Another question that does not relate to your actual issue:
What is your opinion with respect to the usage of the Spring framework in CDO?
Currently I'm carrying ideas to replace the dynamic configuration of the Spring containers
by generated DI containers. Most recently I've published a cute generator framework (see http://genfw.berlios.de )
and a sophisticated container generator (named "density"), which has many advanced features over Spring.
I use it for my own products. The reason why I ask at all, is that the dynamic config files would disappear
then. Of course there will be no degradation of flexibility, I assume that I will be able to shift these kinds
of features towards Eclipse extension points (in the end they are XML configs, too).
Cheers
/Eike
geib schrieb:
> Eike,
>
> Sorry to bring up problems with older version of CDO, but we are having
> significant performance problems in our application using the orginal CDO.
>
> During startup, which now takes almost two minutes, one of the views
> loads all the objects of the first level of a tree. There are now more
> than two thousand of these objects.
>
> I have now installed a profiler and find on both the client side and the
> server side that nearly 40% of the time is spent in blockingDequeue() in
> MultiplexerImpl.java.
>
> Also, the profiler tells me that it is taking nearly 10 seconds to
> authenticate the client with the server, which seems like a very long time.
>
> Do you have any ideas on where I could look to help performance in
> general, or specifically address these hotspots.??
>
> Thanks,
> Mark.
>
|
|
|
| Re: CDO performance problems [message #23161 is a reply to message #22018] |
Thu, 16 February 2006 16:53 |
 Mark Geib
Messages: 432
Registered: July 2009 |
Senior Member |
|
|
Eike,
Please see replies below.
Mark.
Eike Stepper wrote:
> Mark,
> Sorry, that I did not respond immediately. I just returned from an abroad
journey.
> One major task for my next week is the actual migration of CDO to EMFT, and
that will
> establish a kind of third version of CDO. Nevertheless I will try to
investigate your
> issues during this weekend. There are some infos that you could provide to
ease my start:
> - Can you give me a timestamp of your CVS checkout?
I am sorry I don't have the date. But I am running code from the time that
I sent you the changes I had made, for example in ChannelImpl.java.
> - Is it possible that you send me a complete workspace, so that I can
reproduce an identical environment?
This is not very practical, at least at this time.
> If yes, just zip it and send it via email (if that doesn't work due to
limitations of my pop account,
> I could manage to create an FTP acoount for you)
> - Do you have the feeling that you just uncovered an issue that happened to
persist for a longer period?
I beleive this issue has become important as the size of our database and
the number of obejcts has increased. Everything seems to work as before,
but because of the number of objects involved the performance has crossed
a threshold that seems to bother the users. It takes to long to start.
I am continuing to work on more detailed profiling of the system. It seems
to simply be waiting alot for the other party. Both the client and server
side spend a lot of time waiting for a queue to not be empty, but I am not
sure if while one side is waiting the other is busy, or not.
> Or does it seem to be associated with some kind of environmental change?
> - Some time ago you applied some changes to Channel/Multiplexer threading
model.
> Do you have changes in this area, that are not committed to the code base?
I believe I sent you all the changes I have made.
> - Is there an opportunity that you take this occasion as a trigger to
reintegrate with
> a newer version of CDO?
Yes. I actually have a complete set of my projects/workspace without CDO
integrated at all. I have used this for standalone testing with XMI
persistence....
So I think I could this as a clean starting point for using the new
version of CDO.
> - (If "No" for the last question) Do you see a chance that we will at least
be able to
> test your environment with the newest version of CDO, so that we receive
an indication
> where to focus our efforts with the highest degree of synergy?
> Another question that does not relate to your actual issue:
> What is your opinion with respect to the usage of the Spring framework in
CDO?
> Currently I'm carrying ideas to replace the dynamic configuration of the
Spring containers
> by generated DI containers. Most recently I've published a cute generator
framework (see http://genfw.berlios.de )
> and a sophisticated container generator (named "density"), which has many
advanced features over Spring.
> I use it for my own products. The reason why I ask at all, is that the
dynamic config files would disappear
> then. Of course there will be no degradation of flexibility, I assume that I
will be able to shift these kinds
> of features towards Eclipse extension points (in the end they are XML
configs, too).
I don't think I have an opion here...I have not had any reason to deal
with the Spring framework at all.
> Cheers
> /Eike
> geib schrieb:
>> Eike,
>>
>> Sorry to bring up problems with older version of CDO, but we are having
>> significant performance problems in our application using the orginal CDO.
>>
>> During startup, which now takes almost two minutes, one of the views
>> loads all the objects of the first level of a tree. There are now more
>> than two thousand of these objects.
>>
>> I have now installed a profiler and find on both the client side and the
>> server side that nearly 40% of the time is spent in blockingDequeue() in
>> MultiplexerImpl.java.
>>
>> Also, the profiler tells me that it is taking nearly 10 seconds to
>> authenticate the client with the server, which seems like a very long time.
>>
>> Do you have any ideas on where I could look to help performance in
>> general, or specifically address these hotspots.??
>>
>> Thanks,
>> Mark.
>>
|
|
|
| Re: CDO performance problems [message #29170 is a reply to message #23161] |
Fri, 24 March 2006 09:03 |
| Eclipse User |
|
|
|
Originally posted by: stepper.sympedia.de
Mark,
Sorry that this has taken quite long now, but I've used the time
to completely reimplement the Net4j core. The main goal was to restore
the original characteristics of the I/O thread (Selector.Controller).
Now it is really non-blocking, buffered (but without the obvious
defects that you discovered and worked around formerly).
Before I go into details of my changes, the new code is approximately
37 times faster than the blocking version!!! By switching off
logging and deadlock detection you gain another 800%!
I've written a simple performance test case, that starts one server
container and one client container, same VM but connected through
SocketChannels. Client initiates a single TestProtocol channel and
bombs the server with 1000 sequential echo requests. Each time a
363 bytes, mixed datatype signal is transmitted and sent back.
Average times per request/indication/response/confirmation cycle:
740 millis (original code with your workarounds)
20 millis (new code -> 37 times faster!!!)
9 millis (new code without console)
2.53 millis (new code without logfile)
Summary of important changes:
1) Optimized buffer multiplexing for byte[], String und Serializable.
2) Simplified SelectorManager to only use a single Controller thread
(now possible because Sun fixed a Windows-related bug).
3) Simplified and optimized BufferPool
(formerly NOOPBufferPool was used -> no pooling).
4) Used JDK5 Concurrency API wherever appropriate, for example:
- Replaced the whole Dispatcher/Executor/ExecutorPool framework of
Net4j by the new ExecutorService API (JDK5)
- ConcurrentLinkedQueue or LinkedBlockingQueue instead of own sync
5) Most important: Removed all blocking calls from code that is executed
in the context of the main I/O thread. This made up more than 90% of
the performance gain, maybe even more in a server-only VM!
6) Reworked various synchronization relations on the way from initiating
transmitter over peer receiver, peer transmitter and local receiver.
Fixed several defects that formerly led to deadlocks during high
throughput situations.
At the moment the complete embedded transport does not work properly,
but socket transport has undergone stress tests and performs very well.
To accommodate missing embedded transport I have provided additional
environment plugins for distributed socket environments:
- org.eclipse.net4j.cdo.test.serverapp (headless socket server + mapper)
- org.eclipse.net4j.cdo.test.client (full IDE socket client + library example)
There are launch configs for both.
I hope that I can commit this new version to EMFT CVS today. Until then you
can use the BerliOS CVS at http://developer.berlios.de/cvs/?group_id=1840
Cheers
/Eike
P.S.: You complained about issues with Multiplexer.blockingDequeue(). This
method has disappeared completely during step 5) ;-)
|
|
|
| Re: CDO performance problems [message #29398 is a reply to message #29170] |
Sat, 01 April 2006 13:28 |
| Eclipse User |
|
|
|
Originally posted by: NOSPAMser.man.szczNOSPAMecin.pl
Fri, 24 Mar 2006 10:03:38 +0100, Eike Stepper wrote:
> I hope that I can commit this new version to EMFT CVS today. Until then
> you can use the BerliOS CVS at
> http://developer.berlios.de/cvs/?group_id=1840
Hi Eike,
I see many initial commits in dev.eclipse.org/org.eclipse.emft/(net4j|cdo).
Should I switch to this repository to get CDO?
org.eclipse.net4j.cdo have changed name to org.eclipse.emf.cdo. Do you
plan any other important for end user refactorings?
Eike, maybe you could write few words about near future of CDO. I'm great fan
of CDO and it would be nice to know what's going on.
Regards,
Seweryn Niemiec
|
|
|
| Re: CDO performance problems [message #29435 is a reply to message #29398] |
Sat, 01 April 2006 16:20 |
| Eclipse User |
|
|
|
Originally posted by: stepper.sympedia.de
This is a multi-part message in MIME format.
--------------030006050106030507020207
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit
Seweryn,
You're really fast ;-)
After working hard on the new version it got just committed to EMFT CVS.
Nick (and others) has helped me a lot and now we have to separate modules:
emft/net4j/
org.eclipse.net4j.*
emft/cdo/
org.eclipse.emf.cdo.*
org.eclipse.emf.dbgen.*
org.eclipse.emf.jdbc.*
This latest major release has 4 focusses:
1) Stability and performance of Net4j (described in the previous mail to
Mark)
2) New: Remote resource exploration protocol "cdores"
3) Usable socket based example
4) New: Example UI for cdo and cdores protocols (CDO Explorer and CDO
Editor)
I planned to let others do the first tests (of the public code, of
course I tested it before ;-) ),
but since you are asking and know the previous versions I will paste in
the hints that I sent to the EMF team:
<snip-snap>
I've just committed a working version to CVS. I'd appreciate if you
could test it (it's fun!).
In case I continue to work, I've labeled the working version with
"eike-initial001". Of course I always try to commit working versions ;-)
The example has a fancy UI now with a generic CDOEditor and a
CDOExplorer view
to browse all remote resources. The editor has a special
LoadResourceDialog.
Here is a short setup walkthrough:
1) Download mysql from http://dev.mysql.com/downloads/mysql/5.0.html
I use 4.11-nt, but 5.0 should also work. Will test that later...
2) Install (defaults, but with transactions, 4.11-nt) and start it.
3) Create a user "test" with password "test"
4) Install the test database (the script is in the
org.eclipse.emf.cdo.example.client plugin):
@echo off
set USER=test
set PASSWORD=test
set DATABASE=cdoTest
mysql -u %USER% -p%PASSWORD% -e "drop database if exists %DATABASE%;
create database %DATABASE%" mysql
5) Checkout all plugins from net4j/cdo modules in EMFT (net4j.tests are
not needed)
6) Download the Mysql JDBC Driver from
http://dev.mysql.com/downloads/connector/j/3.1.html
7) Copy mysql-connector-java-3.1.12-bin.jar to the lib folder of
org.eclipse.emf.jdbc.mysql.
8) Start the launch config "CDO Server". This is an Eclipse headless
launch! You should see the startup in the host console.
9) Start the launch config "CDO Client1". This is an Eclipse IDE launch!
You should see the startup in the host console.
10) In the client, open the view "CDO Explorer". It is empty for now.
Here you can create new CDO resources.
The explorer doesn't visually react on new resources for the moment.
To see them in the view you have to restart the client.
I will fix that soon...
11) To have 2 initially populated remote resources, you can also hit the
first of the ugly Eclipse buttons "Create Library CDO Resources".
Again you have to restart the client in order to see the new
resources in the view.
12) You can open the CDOEditor by double clicking into the CDOExplorer
viewer. Other resources can be loaded with "Load Resource..."
When you hit "Save", a transactional commit happens for the whole
ResourceSet. Only changes are committed and other clients are notified
over the net so that they can update their UI. I can't test this at the
moment (see below).
Step 11) demonstrates that resources can be committed to the db. But
unfortunately the editor does not permit any changes. I have no idea,
why it doesn't ;-(
I assume it's something trivial that I did not trace while EMF evolved.
Can you give me a hint?
<snip-snap>
Happy persisting
/Eike
P.S.: Severyn, thanks for your hint that the "distributed shared model"
aspect of CDO should be stressed. Ed agreed with that and I will respect
it in the new docs (when I come to write them ;-) )
Seweryn Niemiec schrieb:
> Fri, 24 Mar 2006 10:03:38 +0100, Eike Stepper wrote:
>
>
>> I hope that I can commit this new version to EMFT CVS today. Until then
>> you can use the BerliOS CVS at
>> http://developer.berlios.de/cvs/?group_id=1840
>>
>
> Hi Eike,
>
> I see many initial commits in dev.eclipse.org/org.eclipse.emft/(net4j|cdo).
> Should I switch to this repository to get CDO?
>
> org.eclipse.net4j.cdo have changed name to org.eclipse.emf.cdo. Do you
> plan any other important for end user refactorings?
>
> Eike, maybe you could write few words about near future of CDO. I'm great fan
> of CDO and it would be nice to know what's going on.
>
> Regards,
> Seweryn Niemiec
>
>
--------------030006050106030507020207
Content-Type: text/html; charset=ISO-8859-15
Content-Transfer-Encoding: 8bit
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Seweryn,<br>
<br>
You're really fast ;-)<br>
After working hard on the new version it got just committed to EMFT CVS.<br>
Nick (and others) has helped me a lot and now we have to separate
modules:<br>
<br>
emft/net4j/<br>
|
|
|
| Re: CDO performance problems [message #29472 is a reply to message #29398] |
Sat, 01 April 2006 17:50 |
| Eclipse User |
|
|
|
Originally posted by: stepper.sympedia.de
This is a multi-part message in MIME format.
--------------020002030001010906050300
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit
Seweryn,
You asked for further refactorings. I was just about to say that nothing
severe would happen in the near future.
But I just got some feedback from Ed on ugly names. Especially the plugins
org.eclipse.emf.dbgen.*
org.eclipse.emf.jdbc.*
will get a "cdo." inserted.
I will post here for you when I'm ready with refactoring ;-)
Cheers
/Eike
Seweryn Niemiec schrieb:
> Fri, 24 Mar 2006 10:03:38 +0100, Eike Stepper wrote:
>
>
>> I hope that I can commit this new version to EMFT CVS today. Until then
>> you can use the BerliOS CVS at
>> http://developer.berlios.de/cvs/?group_id=1840
>>
>
> Hi Eike,
>
> I see many initial commits in dev.eclipse.org/org.eclipse.emft/(net4j|cdo).
> Should I switch to this repository to get CDO?
>
> org.eclipse.net4j.cdo have changed name to org.eclipse.emf.cdo. Do you
> plan any other important for end user refactorings?
>
> Eike, maybe you could write few words about near future of CDO. I'm great fan
> of CDO and it would be nice to know what's going on.
>
> Regards,
> Seweryn Niemiec
>
>
--------------020002030001010906050300
Content-Type: text/html; charset=ISO-8859-15
Content-Transfer-Encoding: 8bit
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Seweryn,<br>
<br>
You asked for further refactorings. I was just about to say that
nothing severe would happen in the near future.<br>
But I just got some feedback from Ed on ugly names. Especially the
plugins<br>
<br>
|
|
|
| Re: CDO performance problems [message #566593 is a reply to message #21567] |
Fri, 10 February 2006 11:18 |
|
Mark,
Sorry, that I did not respond immediately. I just returned from an abroad journey.
One major task for my next week is the actual migration of CDO to EMFT, and that will
establish a kind of third version of CDO. Nevertheless I will try to investigate your
issues during this weekend. There are some infos that you could provide to ease my start:
- Can you give me a timestamp of your CVS checkout?
- Is it possible that you send me a complete workspace, so that I can reproduce an identical environment?
If yes, just zip it and send it via email (if that doesn't work due to limitations of my pop account,
I could manage to create an FTP acoount for you)
- Do you have the feeling that you just uncovered an issue that happened to persist for a longer period?
Or does it seem to be associated with some kind of environmental change?
- Some time ago you applied some changes to Channel/Multiplexer threading model.
Do you have changes in this area, that are not committed to the code base?
- Is there an opportunity that you take this occasion as a trigger to reintegrate with
a newer version of CDO?
- (If "No" for the last question) Do you see a chance that we will at least be able to
test your environment with the newest version of CDO, so that we receive an indication
where to focus our efforts with the highest degree of synergy?
Another question that does not relate to your actual issue:
What is your opinion with respect to the usage of the Spring framework in CDO?
Currently I'm carrying ideas to replace the dynamic configuration of the Spring containers
by generated DI containers. Most recently I've published a cute generator framework (see http://genfw.berlios.de )
and a sophisticated container generator (named "density"), which has many advanced features over Spring.
I use it for my own products. The reason why I ask at all, is that the dynamic config files would disappear
then. Of course there will be no degradation of flexibility, I assume that I will be able to shift these kinds
of features towards Eclipse extension points (in the end they are XML configs, too).
Cheers
/Eike
geib schrieb:
> Eike,
>
> Sorry to bring up problems with older version of CDO, but we are having
> significant performance problems in our application using the orginal CDO.
>
> During startup, which now takes almost two minutes, one of the views
> loads all the objects of the first level of a tree. There are now more
> than two thousand of these objects.
>
> I have now installed a profiler and find on both the client side and the
> server side that nearly 40% of the time is spent in blockingDequeue() in
> MultiplexerImpl.java.
>
> Also, the profiler tells me that it is taking nearly 10 seconds to
> authenticate the client with the server, which seems like a very long time.
>
> Do you have any ideas on where I could look to help performance in
> general, or specifically address these hotspots.??
>
> Thanks,
> Mark.
>
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
| Re: CDO performance problems [message #567526 is a reply to message #22018] |
Thu, 16 February 2006 16:53 |
Mark Geib
Messages: 432
Registered: July 2009 |
Senior Member |
|
|
Eike,
Please see replies below.
Mark.
Eike Stepper wrote:
> Mark,
> Sorry, that I did not respond immediately. I just returned from an abroad
journey.
> One major task for my next week is the actual migration of CDO to EMFT, and
that will
> establish a kind of third version of CDO. Nevertheless I will try to
investigate your
> issues during this weekend. There are some infos that you could provide to
ease my start:
> - Can you give me a timestamp of your CVS checkout?
I am sorry I don't have the date. But I am running code from the time that
I sent you the changes I had made, for example in ChannelImpl.java.
> - Is it possible that you send me a complete workspace, so that I can
reproduce an identical environment?
This is not very practical, at least at this time.
> If yes, just zip it and send it via email (if that doesn't work due to
limitations of my pop account,
> I could manage to create an FTP acoount for you)
> - Do you have the feeling that you just uncovered an issue that happened to
persist for a longer period?
I beleive this issue has become important as the size of our database and
the number of obejcts has increased. Everything seems to work as before,
but because of the number of objects involved the performance has crossed
a threshold that seems to bother the users. It takes to long to start.
I am continuing to work on more detailed profiling of the system. It seems
to simply be waiting alot for the other party. Both the client and server
side spend a lot of time waiting for a queue to not be empty, but I am not
sure if while one side is waiting the other is busy, or not.
> Or does it seem to be associated with some kind of environmental change?
> - Some time ago you applied some changes to Channel/Multiplexer threading
model.
> Do you have changes in this area, that are not committed to the code base?
I believe I sent you all the changes I have made.
> - Is there an opportunity that you take this occasion as a trigger to
reintegrate with
> a newer version of CDO?
Yes. I actually have a complete set of my projects/workspace without CDO
integrated at all. I have used this for standalone testing with XMI
persistence....
So I think I could this as a clean starting point for using the new
version of CDO.
> - (If "No" for the last question) Do you see a chance that we will at least
be able to
> test your environment with the newest version of CDO, so that we receive
an indication
> where to focus our efforts with the highest degree of synergy?
> Another question that does not relate to your actual issue:
> What is your opinion with respect to the usage of the Spring framework in
CDO?
> Currently I'm carrying ideas to replace the dynamic configuration of the
Spring containers
> by generated DI containers. Most recently I've published a cute generator
framework (see http://genfw.berlios.de )
> and a sophisticated container generator (named "density"), which has many
advanced features over Spring.
> I use it for my own products. The reason why I ask at all, is that the
dynamic config files would disappear
> then. Of course there will be no degradation of flexibility, I assume that I
will be able to shift these kinds
> of features towards Eclipse extension points (in the end they are XML
configs, too).
I don't think I have an opion here...I have not had any reason to deal
with the Spring framework at all.
> Cheers
> /Eike
> geib schrieb:
>> Eike,
>>
>> Sorry to bring up problems with older version of CDO, but we are having
>> significant performance problems in our application using the orginal CDO.
>>
>> During startup, which now takes almost two minutes, one of the views
>> loads all the objects of the first level of a tree. There are now more
>> than two thousand of these objects.
>>
>> I have now installed a profiler and find on both the client side and the
>> server side that nearly 40% of the time is spent in blockingDequeue() in
>> MultiplexerImpl.java.
>>
>> Also, the profiler tells me that it is taking nearly 10 seconds to
>> authenticate the client with the server, which seems like a very long time.
>>
>> Do you have any ideas on where I could look to help performance in
>> general, or specifically address these hotspots.??
>>
>> Thanks,
>> Mark.
>>
|
|
|
| Re: CDO performance problems [message #572703 is a reply to message #23161] |
Fri, 24 March 2006 09:03 |
|
Mark,
Sorry that this has taken quite long now, but I've used the time
to completely reimplement the Net4j core. The main goal was to restore
the original characteristics of the I/O thread (Selector.Controller).
Now it is really non-blocking, buffered (but without the obvious
defects that you discovered and worked around formerly).
Before I go into details of my changes, the new code is approximately
37 times faster than the blocking version!!! By switching off
logging and deadlock detection you gain another 800%!
I've written a simple performance test case, that starts one server
container and one client container, same VM but connected through
SocketChannels. Client initiates a single TestProtocol channel and
bombs the server with 1000 sequential echo requests. Each time a
363 bytes, mixed datatype signal is transmitted and sent back.
Average times per request/indication/response/confirmation cycle:
740 millis (original code with your workarounds)
20 millis (new code -> 37 times faster!!!)
9 millis (new code without console)
2.53 millis (new code without logfile)
Summary of important changes:
1) Optimized buffer multiplexing for byte[], String und Serializable.
2) Simplified SelectorManager to only use a single Controller thread
(now possible because Sun fixed a Windows-related bug).
3) Simplified and optimized BufferPool
(formerly NOOPBufferPool was used -> no pooling).
4) Used JDK5 Concurrency API wherever appropriate, for example:
- Replaced the whole Dispatcher/Executor/ExecutorPool framework of
Net4j by the new ExecutorService API (JDK5)
- ConcurrentLinkedQueue or LinkedBlockingQueue instead of own sync
5) Most important: Removed all blocking calls from code that is executed
in the context of the main I/O thread. This made up more than 90% of
the performance gain, maybe even more in a server-only VM!
6) Reworked various synchronization relations on the way from initiating
transmitter over peer receiver, peer transmitter and local receiver.
Fixed several defects that formerly led to deadlocks during high
throughput situations.
At the moment the complete embedded transport does not work properly,
but socket transport has undergone stress tests and performs very well.
To accommodate missing embedded transport I have provided additional
environment plugins for distributed socket environments:
- org.eclipse.net4j.cdo.test.serverapp (headless socket server + mapper)
- org.eclipse.net4j.cdo.test.client (full IDE socket client + library example)
There are launch configs for both.
I hope that I can commit this new version to EMFT CVS today. Until then you
can use the BerliOS CVS at http://developer.berlios.de/cvs/?group_id=1840
Cheers
/Eike
P.S.: You complained about issues with Multiplexer.blockingDequeue(). This
method has disappeared completely during step 5) ;-)
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
| |
| Re: CDO performance problems [message #572994 is a reply to message #29398] |
Sat, 01 April 2006 16:20 |
|
This is a multi-part message in MIME format.
--------------030006050106030507020207
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit
Seweryn,
You're really fast ;-)
After working hard on the new version it got just committed to EMFT CVS.
Nick (and others) has helped me a lot and now we have to separate modules:
emft/net4j/
org.eclipse.net4j.*
emft/cdo/
org.eclipse.emf.cdo.*
org.eclipse.emf.dbgen.*
org.eclipse.emf.jdbc.*
This latest major release has 4 focusses:
1) Stability and performance of Net4j (described in the previous mail to
Mark)
2) New: Remote resource exploration protocol "cdores"
3) Usable socket based example
4) New: Example UI for cdo and cdores protocols (CDO Explorer and CDO
Editor)
I planned to let others do the first tests (of the public code, of
course I tested it before ;-) ),
but since you are asking and know the previous versions I will paste in
the hints that I sent to the EMF team:
<snip-snap>
I've just committed a working version to CVS. I'd appreciate if you
could test it (it's fun!).
In case I continue to work, I've labeled the working version with
"eike-initial001". Of course I always try to commit working versions ;-)
The example has a fancy UI now with a generic CDOEditor and a
CDOExplorer view
to browse all remote resources. The editor has a special
LoadResourceDialog.
Here is a short setup walkthrough:
1) Download mysql from http://dev.mysql.com/downloads/mysql/5.0.html
I use 4.11-nt, but 5.0 should also work. Will test that later...
2) Install (defaults, but with transactions, 4.11-nt) and start it.
3) Create a user "test" with password "test"
4) Install the test database (the script is in the
org.eclipse.emf.cdo.example.client plugin):
@echo off
set USER=test
set PASSWORD=test
set DATABASE=cdoTest
mysql -u %USER% -p%PASSWORD% -e "drop database if exists %DATABASE%;
create database %DATABASE%" mysql
5) Checkout all plugins from net4j/cdo modules in EMFT (net4j.tests are
not needed)
6) Download the Mysql JDBC Driver from
http://dev.mysql.com/downloads/connector/j/3.1.html
7) Copy mysql-connector-java-3.1.12-bin.jar to the lib folder of
org.eclipse.emf.jdbc.mysql.
8) Start the launch config "CDO Server". This is an Eclipse headless
launch! You should see the startup in the host console.
9) Start the launch config "CDO Client1". This is an Eclipse IDE launch!
You should see the startup in the host console.
10) In the client, open the view "CDO Explorer". It is empty for now.
Here you can create new CDO resources.
The explorer doesn't visually react on new resources for the moment.
To see them in the view you have to restart the client.
I will fix that soon...
11) To have 2 initially populated remote resources, you can also hit the
first of the ugly Eclipse buttons "Create Library CDO Resources".
Again you have to restart the client in order to see the new
resources in the view.
12) You can open the CDOEditor by double clicking into the CDOExplorer
viewer. Other resources can be loaded with "Load Resource..."
When you hit "Save", a transactional commit happens for the whole
ResourceSet. Only changes are committed and other clients are notified
over the net so that they can update their UI. I can't test this at the
moment (see below).
Step 11) demonstrates that resources can be committed to the db. But
unfortunately the editor does not permit any changes. I have no idea,
why it doesn't ;-(
I assume it's something trivial that I did not trace while EMF evolved.
Can you give me a hint?
<snip-snap>
Happy persisting
/Eike
P.S.: Severyn, thanks for your hint that the "distributed shared model"
aspect of CDO should be stressed. Ed agreed with that and I will respect
it in the new docs (when I come to write them ;-) )
Seweryn Niemiec schrieb:
> Fri, 24 Mar 2006 10:03:38 +0100, Eike Stepper wrote:
>
>
>> I hope that I can commit this new version to EMFT CVS today. Until then
>> you can use the BerliOS CVS at
>> http://developer.berlios.de/cvs/?group_id=1840
>>
>
> Hi Eike,
>
> I see many initial commits in dev.eclipse.org/org.eclipse.emft/(net4j|cdo).
> Should I switch to this repository to get CDO?
>
> org.eclipse.net4j.cdo have changed name to org.eclipse.emf.cdo. Do you
> plan any other important for end user refactorings?
>
> Eike, maybe you could write few words about near future of CDO. I'm great fan
> of CDO and it would be nice to know what's going on.
>
> Regards,
> Seweryn Niemiec
>
>
--------------030006050106030507020207
Content-Type: text/html; charset=ISO-8859-15
Content-Transfer-Encoding: 8bit
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Seweryn,<br>
<br>
You're really fast ;-)<br>
After working hard on the new version it got just committed to EMFT CVS.<br>
Nick (and others) has helped me a lot and now we have to separate
modules:<br>
<br>
emft/net4j/<br>
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
| Re: CDO performance problems [message #573038 is a reply to message #29398] |
Sat, 01 April 2006 17:50 |
|
This is a multi-part message in MIME format.
--------------020002030001010906050300
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit
Seweryn,
You asked for further refactorings. I was just about to say that nothing
severe would happen in the near future.
But I just got some feedback from Ed on ugly names. Especially the plugins
org.eclipse.emf.dbgen.*
org.eclipse.emf.jdbc.*
will get a "cdo." inserted.
I will post here for you when I'm ready with refactoring ;-)
Cheers
/Eike
Seweryn Niemiec schrieb:
> Fri, 24 Mar 2006 10:03:38 +0100, Eike Stepper wrote:
>
>
>> I hope that I can commit this new version to EMFT CVS today. Until then
>> you can use the BerliOS CVS at
>> http://developer.berlios.de/cvs/?group_id=1840
>>
>
> Hi Eike,
>
> I see many initial commits in dev.eclipse.org/org.eclipse.emft/(net4j|cdo).
> Should I switch to this repository to get CDO?
>
> org.eclipse.net4j.cdo have changed name to org.eclipse.emf.cdo. Do you
> plan any other important for end user refactorings?
>
> Eike, maybe you could write few words about near future of CDO. I'm great fan
> of CDO and it would be nice to know what's going on.
>
> Regards,
> Seweryn Niemiec
>
>
--------------020002030001010906050300
Content-Type: text/html; charset=ISO-8859-15
Content-Transfer-Encoding: 8bit
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Seweryn,<br>
<br>
You asked for further refactorings. I was just about to say that
nothing severe would happen in the near future.<br>
But I just got some feedback from Ed on ugly names. Especially the
plugins<br>
<br>
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
Goto Forum:
Current Time: Sat Jul 27 12:38:33 GMT 2024
Powered by FUDForum. Page generated in 0.05216 seconds |
] 
Search
Help
Register
Login
Home
