Home » Modeling » EMF "Technology" (Ecore Tools, EMFatic, etc) » [Teneo/CDO] Managing huge models / unloading parts of models
| [Teneo/CDO] Managing huge models / unloading parts of models [message #107855] |
Wed, 23 January 2008 11:40  |
 Stefan Winkler
Messages: 307
Registered: July 2009
Location: Germany |
Senior Member |
|
|
Hi all,
I am working on a project which will have to deal with quite large (both
broad and deep) models. The models themselves are only read or created
(out of other models) but not modified themselves.
Regarding the creation task: this will be something like a custom model
transformation: an algorithm will read one model and produce another.
Problem is: I expect the models to be too big to fit into memory, so I
need a solution which allows only parts of models to be loaded and these
parts to be unloaded, when they are not being used anymore. In terms of
EMF, I want only parts of Resources to be loaded.
Ed Merks pointed out that CDO or Teneo could help here as they use lazy
loading strategies. So far I had a look into Teneo/Hibernate and I like
the interface so far. But: While I understand that Teneo supports lazy
loading of EMF Objects, does Teneo (or Hibernate behind this) have
support to unload objects that are no longer needed?
E.g.
A
|
+--- B
|
+---- C
|
+----- D
When I start at A (B,C,D are proxies, because policy says so), I access
B,C and D. Is it possible to unload C and D and make C Proxy again so
that C and D get garbage collected?
If Teneo does not support this, does CDO?
Thanks in advance!
Cheers,
Stefan
|
|
|
| Re: [Teneo/CDO] Managing huge models / unloading parts of models [message #107870 is a reply to message #107855] |
Wed, 23 January 2008 14:39  |
Eclipse User |
|
|
|
Originally posted by: stepper.sympedia.de
Hi Stefan,
I don't know if Teneo can do this, Martin will want to explain that.
CDO can do it. With CDO the modeled instances internally are not
interconnected with java reference handles. This way unneeded instances
can be completely garbage collected and transparently reloaded later on.
You don't need to write extra code for that to function.
If you plan to construct the output model programmatically everything
should work fine. If you plan to read your input models for example from
xml documents they should of course be small enough to fit into memory.
The resulting CDO repository resource(s) would have to remain within the
repository until you have enough memory on client side that you can load
them completely into an XML resource to save them to a file again.
If you plan to do your conversion within one VM (that is, not through a
networked remote repository) you can gain a lot of system resources
(memory, performance, ...) if you decide to use a Net4j IJVMConnector
instead of the usual ITCPConnector. And you can use an embedded Derby
Database to further optimize, but I guess that performance is not the
issue. If client memory gets low, it could even be better to use a
remote database server.
Regards,
Eike Stepper
----
http://wiki.eclipse.org/CDO
http://wiki.eclipse.org/Net4j
Stefan Winkler schrieb:
> Hi all,
>
> I am working on a project which will have to deal with quite large
> (both broad and deep) models. The models themselves are only read or
> created (out of other models) but not modified themselves.
>
> Regarding the creation task: this will be something like a custom
> model transformation: an algorithm will read one model and produce
> another.
>
> Problem is: I expect the models to be too big to fit into memory, so I
> need a solution which allows only parts of models to be loaded and
> these parts to be unloaded, when they are not being used anymore. In
> terms of EMF, I want only parts of Resources to be loaded.
>
> Ed Merks pointed out that CDO or Teneo could help here as they use
> lazy loading strategies. So far I had a look into Teneo/Hibernate and
> I like the interface so far. But: While I understand that Teneo
> supports lazy loading of EMF Objects, does Teneo (or Hibernate behind
> this) have support to unload objects that are no longer needed?
>
> E.g.
>
> A
> |
> +--- B
> |
> +---- C
> |
> +----- D
>
> When I start at A (B,C,D are proxies, because policy says so), I
> access B,C and D. Is it possible to unload C and D and make C Proxy
> again so that C and D get garbage collected?
>
>
> If Teneo does not support this, does CDO?
>
>
> Thanks in advance!
>
> Cheers,
>
> Stefan
>
|
|
|
| Re: [Teneo/CDO] Managing huge models / unloading parts of models [message #107884 is a reply to message #107870] |
Wed, 23 January 2008 15:26 |
Stefan Winkler
Messages: 307
Registered: July 2009
Location: Germany |
Senior Member |
|
|
Hi Eike,
thanks for your reply.
> I don't know if Teneo can do this, Martin will want to explain that.
Some research about Hibernate revealed that in Hibernate you are
supposed to use Sessions for scoping. So in order to unload things, you
should load these things before in a separate session and close and
dispose that session if no longer needed.
Martin, can you confirm that?
> CDO can do it. With CDO the modeled instances internally are not
> interconnected with java reference handles. This way unneeded instances
> can be completely garbage collected and transparently reloaded later on.
> You don't need to write extra code for that to function.
>
> If you plan to construct the output model programmatically everything
> should work fine. If you plan to read your input models for example from
> xml documents they should of course be small enough to fit into memory.
> The resulting CDO repository resource(s) would have to remain within the
> repository until you have enough memory on client side that you can load
> them completely into an XML resource to save them to a file again.
All models are stored in the database back-end and processed from there.
(Importing into the database is another issue and it is clear that
reading XMI for importing into the database requires enough memory to
hold the XMI resource data.)
> If you plan to do your conversion within one VM (that is, not through a
> networked remote repository) you can gain a lot of system resources
> (memory, performance, ...) if you decide to use a Net4j IJVMConnector
> instead of the usual ITCPConnector. And you can use an embedded Derby
> Database to further optimize, but I guess that performance is not the
> issue. If client memory gets low, it could even be better to use a
> remote database server.
What I am doing is more of a research prototype, so performance is not
an issue. I also don't need a client-server architecture or distributed
databases/repositories. So my architecture simply looks like this
[ DB to store repository ] <-{1}---------{1}-> [Eclipse-based client]
Is there any hello world tutorial with a simple scenario like this one
(using the IJVMConnector as you said)?
The tutorial nodes in the wiki are still quite empty ... ;-)
Cheers,
Stefan
|
|
|
| Re: [Teneo/CDO] Managing huge models / unloading parts of models [message #107897 is a reply to message #107870] |
Wed, 23 January 2008 15:27 |
| Eclipse User |
|
|
|
Originally posted by: stepper.sympedia.de
I've written a small example how to set up a standalone client and
server within one JVM:
package org.eclipse.emf.cdo.tests;
import org.eclipse.emf.cdo.CDOSession;
import org.eclipse.emf.cdo.CDOTransaction;
import org.eclipse.emf.cdo.server.CDOServerUtil;
import org.eclipse.emf.cdo.server.IRepository;
import org.eclipse.emf.cdo.server.IStore;
import org.eclipse.emf.cdo.server.StoreUtil;
import org.eclipse.emf.cdo.util.CDOUtil;
import org.eclipse.net4j.Net4jUtil;
import org.eclipse.net4j.connector.IConnector;
import org.eclipse.net4j.jvm.JVMUtil;
import org.eclipse.net4j.util.container.ContainerUtil;
import org.eclipse.net4j.util.container.IManagedContainer;
import org.eclipse.emf.ecore.EObject;
import org.eclipse.emf.ecore.resource.Resource;
/**
* @author Eike Stepper
*/
public class ImportXML
{
private static final String REPOSITORY_NAME = "repo1";
public static void main(String[] args)
{
// Prepare the standalone infra structure (not needed when
running inside Eclipse)
IManagedContainer container = ContainerUtil.createContainer();
// Create a wiring container
Net4jUtil.prepareContainer(container); // Prepare the Net4j kernel
JVMUtil.prepareContainer(container); // Prepare the JVM transport
CDOServerUtil.prepareContainer(container); // Prepare the CDO server
CDOUtil.prepareContainer(container, false); // Prepare the CDO
client
// Start the transport and create a repository
JVMUtil.getAcceptor(container, "default"); // Start the JVM
transport
CDOServerUtil.addRepository(container, createRepository()); //
Start a CDO respository
// Establish a communications connection and open a session with
the repository
IConnector connector = JVMUtil.getConnector(container,
"default"); // Open a JVM connection
CDOSession session = CDOUtil.openSession(connector,
REPOSITORY_NAME, true);// Open a CDO session
CDOTransaction transaction = session.openTransaction();// Open a
CDO transaction
Resource resource =
transaction.createResource("/my/big/resource");// Create a new EMF resource
// Work normally with the EMF resource
for (int i = 0; i < 100; i++)
{
EObject inputModel = getInputModel(i);
resource.getContents().add(inputModel);
transaction.commit();
}
session.close();
}
private static IRepository createRepository()
{
Map<String, String> props = new HashMap<String, String>();
props.put(IRepository.PROP_SUPPORTING_REVISION_DELTAS, "true");
props.put(IRepository.PROP_CURRENT_LRU_CAPACITY, "10000");
props.put(IRepository.PROP_REVISED_LRU_CAPACITY, "10000");
return CDOServerUtil.createRepository(REPOSITORY_NAME,
createStore(), props);
}
private static IStore createStore()
{
// You might want to create an IDBStore here instead if memory
is an issue!
return StoreUtil.createMEMStore();
}
private static EObject getInputModel(int i)
{
throw new UnsupportedOperationException("Load you model here");
}
}
It's also possible to create and wire the various components yourself,
but using a managed container with factories is easier. If running in
Eclipse even the container is created and configured automatically.
There's also a way to do the configuration via XML. Please tell me if
you need more infos.
Regards,
Eike Stepper
----
http://wiki.eclipse.org/CDO
http://wiki.eclipse.org/Net4j
Eike Stepper schrieb:
> Hi Stefan,
>
> I don't know if Teneo can do this, Martin will want to explain that.
>
> CDO can do it. With CDO the modeled instances internally are not
> interconnected with java reference handles. This way unneeded
> instances can be completely garbage collected and transparently
> reloaded later on. You don't need to write extra code for that to
> function.
>
> If you plan to construct the output model programmatically everything
> should work fine. If you plan to read your input models for example
> from xml documents they should of course be small enough to fit into
> memory. The resulting CDO repository resource(s) would have to remain
> within the repository until you have enough memory on client side that
> you can load them completely into an XML resource to save them to a
> file again.
>
> If you plan to do your conversion within one VM (that is, not through
> a networked remote repository) you can gain a lot of system resources
> (memory, performance, ...) if you decide to use a Net4j IJVMConnector
> instead of the usual ITCPConnector. And you can use an embedded Derby
> Database to further optimize, but I guess that performance is not the
> issue. If client memory gets low, it could even be better to use a
> remote database server.
>
> Regards,
> Eike Stepper
> ----
> http://wiki.eclipse.org/CDO
> http://wiki.eclipse.org/Net4j
>
>
>
>
>
> Stefan Winkler schrieb:
>> Hi all,
>>
>> I am working on a project which will have to deal with quite large
>> (both broad and deep) models. The models themselves are only read or
>> created (out of other models) but not modified themselves.
>>
>> Regarding the creation task: this will be something like a custom
>> model transformation: an algorithm will read one model and produce
>> another.
>>
>> Problem is: I expect the models to be too big to fit into memory, so
>> I need a solution which allows only parts of models to be loaded and
>> these parts to be unloaded, when they are not being used anymore. In
>> terms of EMF, I want only parts of Resources to be loaded.
>>
>> Ed Merks pointed out that CDO or Teneo could help here as they use
>> lazy loading strategies. So far I had a look into Teneo/Hibernate and
>> I like the interface so far. But: While I understand that Teneo
>> supports lazy loading of EMF Objects, does Teneo (or Hibernate behind
>> this) have support to unload objects that are no longer needed?
>>
>> E.g.
>>
>> A
>> |
>> +--- B
>> |
>> +---- C
>> |
>> +----- D
>>
>> When I start at A (B,C,D are proxies, because policy says so), I
>> access B,C and D. Is it possible to unload C and D and make C Proxy
>> again so that C and D get garbage collected?
>>
>>
>> If Teneo does not support this, does CDO?
>>
>>
>> Thanks in advance!
>>
>> Cheers,
>>
>> Stefan
>>
|
|
|
| Re: [Teneo/CDO] Managing huge models / unloading parts of models [message #107926 is a reply to message #107897] |
Wed, 23 January 2008 15:30 |
| Eclipse User |
|
|
|
Originally posted by: stepper.sympedia.de
Hmm, the format is ruined but say, 1 Minute to deliver this example? ;-)
Cheers
/Eike
Eike Stepper schrieb:
> I've written a small example how to set up a standalone client and
> server within one JVM:
>
> package org.eclipse.emf.cdo.tests;
> import org.eclipse.emf.cdo.CDOSession;
> import org.eclipse.emf.cdo.CDOTransaction;
> import org.eclipse.emf.cdo.server.CDOServerUtil;
> import org.eclipse.emf.cdo.server.IRepository;
> import org.eclipse.emf.cdo.server.IStore;
> import org.eclipse.emf.cdo.server.StoreUtil;
> import org.eclipse.emf.cdo.util.CDOUtil;
> import org.eclipse.net4j.Net4jUtil;
> import org.eclipse.net4j.connector.IConnector;
> import org.eclipse.net4j.jvm.JVMUtil;
> import org.eclipse.net4j.util.container.ContainerUtil;
> import org.eclipse.net4j.util.container.IManagedContainer;
> import org.eclipse.emf.ecore.EObject;
> import org.eclipse.emf.ecore.resource.Resource;
> /**
> * @author Eike Stepper
> */
> public class ImportXML
> {
> private static final String REPOSITORY_NAME = "repo1";
> public static void main(String[] args)
> {
> // Prepare the standalone infra structure (not needed when
> running inside Eclipse)
> IManagedContainer container = ContainerUtil.createContainer();
> // Create a wiring container
> Net4jUtil.prepareContainer(container); // Prepare the Net4j kernel
> JVMUtil.prepareContainer(container); // Prepare the JVM transport
> CDOServerUtil.prepareContainer(container); // Prepare the CDO
> server
> CDOUtil.prepareContainer(container, false); // Prepare the CDO
> client
> // Start the transport and create a repository
> JVMUtil.getAcceptor(container, "default"); // Start the JVM
> transport
> CDOServerUtil.addRepository(container, createRepository()); //
> Start a CDO respository
> // Establish a communications connection and open a session
> with the repository
> IConnector connector = JVMUtil.getConnector(container,
> "default"); // Open a JVM connection
> CDOSession session = CDOUtil.openSession(connector,
> REPOSITORY_NAME, true);// Open a CDO session
> CDOTransaction transaction = session.openTransaction();// Open
> a CDO transaction
> Resource resource =
> transaction.createResource("/my/big/resource");// Create a new EMF
> resource
> // Work normally with the EMF resource
> for (int i = 0; i < 100; i++)
> {
> EObject inputModel = getInputModel(i);
> resource.getContents().add(inputModel);
> transaction.commit();
> }
> session.close();
> }
> private static IRepository createRepository()
> {
> Map<String, String> props = new HashMap<String, String>();
> props.put(IRepository.PROP_SUPPORTING_REVISION_DELTAS, "true");
> props.put(IRepository.PROP_CURRENT_LRU_CAPACITY, "10000");
> props.put(IRepository.PROP_REVISED_LRU_CAPACITY, "10000");
> return CDOServerUtil.createRepository(REPOSITORY_NAME,
> createStore(), props);
> }
> private static IStore createStore()
> {
> // You might want to create an IDBStore here instead if memory
> is an issue!
> return StoreUtil.createMEMStore();
> }
> private static EObject getInputModel(int i)
> {
> throw new UnsupportedOperationException("Load you model here");
> }
> }
>
> It's also possible to create and wire the various components yourself,
> but using a managed container with factories is easier. If running in
> Eclipse even the container is created and configured automatically.
> There's also a way to do the configuration via XML. Please tell me if
> you need more infos.
>
> Regards,
> Eike Stepper
> ----
> http://wiki.eclipse.org/CDO
> http://wiki.eclipse.org/Net4j
>
>
> Eike Stepper schrieb:
>> Hi Stefan,
>>
>> I don't know if Teneo can do this, Martin will want to explain that.
>>
>> CDO can do it. With CDO the modeled instances internally are not
>> interconnected with java reference handles. This way unneeded
>> instances can be completely garbage collected and transparently
>> reloaded later on. You don't need to write extra code for that to
>> function.
>>
>> If you plan to construct the output model programmatically everything
>> should work fine. If you plan to read your input models for example
>> from xml documents they should of course be small enough to fit into
>> memory. The resulting CDO repository resource(s) would have to remain
>> within the repository until you have enough memory on client side
>> that you can load them completely into an XML resource to save them
>> to a file again.
>>
>> If you plan to do your conversion within one VM (that is, not through
>> a networked remote repository) you can gain a lot of system resources
>> (memory, performance, ...) if you decide to use a Net4j IJVMConnector
>> instead of the usual ITCPConnector. And you can use an embedded Derby
>> Database to further optimize, but I guess that performance is not the
>> issue. If client memory gets low, it could even be better to use a
>> remote database server.
>>
>> Regards,
>> Eike Stepper
>> ----
>> http://wiki.eclipse.org/CDO
>> http://wiki.eclipse.org/Net4j
>>
>>
>>
>>
>>
>> Stefan Winkler schrieb:
>>> Hi all,
>>>
>>> I am working on a project which will have to deal with quite large
>>> (both broad and deep) models. The models themselves are only read or
>>> created (out of other models) but not modified themselves.
>>>
>>> Regarding the creation task: this will be something like a custom
>>> model transformation: an algorithm will read one model and produce
>>> another.
>>>
>>> Problem is: I expect the models to be too big to fit into memory, so
>>> I need a solution which allows only parts of models to be loaded and
>>> these parts to be unloaded, when they are not being used anymore. In
>>> terms of EMF, I want only parts of Resources to be loaded.
>>>
>>> Ed Merks pointed out that CDO or Teneo could help here as they use
>>> lazy loading strategies. So far I had a look into Teneo/Hibernate
>>> and I like the interface so far. But: While I understand that Teneo
>>> supports lazy loading of EMF Objects, does Teneo (or Hibernate
>>> behind this) have support to unload objects that are no longer needed?
>>>
>>> E.g.
>>>
>>> A
>>> |
>>> +--- B
>>> |
>>> +---- C
>>> |
>>> +----- D
>>>
>>> When I start at A (B,C,D are proxies, because policy says so), I
>>> access B,C and D. Is it possible to unload C and D and make C Proxy
>>> again so that C and D get garbage collected?
>>>
>>>
>>> If Teneo does not support this, does CDO?
>>>
>>>
>>> Thanks in advance!
>>>
>>> Cheers,
>>>
>>> Stefan
>>>
|
|
|
| Re: [Teneo/CDO] Managing huge models / unloading parts of models [message #108036 is a reply to message #107884] |
Thu, 24 January 2008 10:18 |
Martin Taal
Messages: 5468
Registered: July 2009 |
Senior Member |
|
|
Hi Stefan,
You can also evict individual objects from the hibernate sesion by doing session.evict(object).
However, you would also need to re-proxy objects but I am not sure how to accomplish that.
Hibernate has proxies but you would need to dive into the hibernate internals to implement
reproxying of objects. I checked and there are some opportunities to look at (afaics implement your
own proxyfactory which returns re-proxyable proxies).
You can also try re-proxying through EMF but I am not sure how well this works with hibernate.
gr. Martin
Stefan Winkler wrote:
> Hi Eike,
>
> thanks for your reply.
>
>> I don't know if Teneo can do this, Martin will want to explain that.
> Some research about Hibernate revealed that in Hibernate you are
> supposed to use Sessions for scoping. So in order to unload things, you
> should load these things before in a separate session and close and
> dispose that session if no longer needed.
>
> Martin, can you confirm that?
>
>> CDO can do it. With CDO the modeled instances internally are not
>> interconnected with java reference handles. This way unneeded
>> instances can be completely garbage collected and transparently
>> reloaded later on. You don't need to write extra code for that to
>> function.
>>
>> If you plan to construct the output model programmatically everything
>> should work fine. If you plan to read your input models for example
>> from xml documents they should of course be small enough to fit into
>> memory. The resulting CDO repository resource(s) would have to remain
>> within the repository until you have enough memory on client side that
>> you can load them completely into an XML resource to save them to a
>> file again.
>
> All models are stored in the database back-end and processed from there.
> (Importing into the database is another issue and it is clear that
> reading XMI for importing into the database requires enough memory to
> hold the XMI resource data.)
>
>> If you plan to do your conversion within one VM (that is, not through
>> a networked remote repository) you can gain a lot of system resources
>> (memory, performance, ...) if you decide to use a Net4j IJVMConnector
>> instead of the usual ITCPConnector. And you can use an embedded Derby
>> Database to further optimize, but I guess that performance is not the
>> issue. If client memory gets low, it could even be better to use a
>> remote database server.
>
> What I am doing is more of a research prototype, so performance is not
> an issue. I also don't need a client-server architecture or distributed
> databases/repositories. So my architecture simply looks like this
>
> [ DB to store repository ] <-{1}---------{1}-> [Eclipse-based client]
>
> Is there any hello world tutorial with a simple scenario like this one
> (using the IJVMConnector as you said)?
> The tutorial nodes in the wiki are still quite empty ... ;-)
>
> Cheers,
> Stefan
--
With Regards, Martin Taal
Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal@springsite.com - mtaal@elver.org
Web: www.springsite.com - www.elver.org
|
|
|
| Re: [Teneo/CDO] Managing huge models / unloading parts of models [message #108068 is a reply to message #107897] |
Thu, 24 January 2008 11:05 |
| Eclipse User |
|
|
|
Originally posted by: stepper.sympedia.de
This is a multi-part message in MIME format.
--------------040801030404030100020306
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit
I found a nice plugin to format the java code to HTML:
|*package *org.eclipse.emf.cdo.tests;
*import *org.eclipse.emf.cdo.CDOSession;
*import *org.eclipse.emf.cdo.CDOTransaction;
*import *org.eclipse.emf.cdo.server.CDOServerUtil;
*import *org.eclipse.emf.cdo.server.IRepository;
*import *org.eclipse.emf.cdo.server.IStore;
*import *org.eclipse.emf.cdo.server.StoreUtil;
*import *org.eclipse.emf.cdo.util.CDOUtil;
*import *org.eclipse.net4j.Net4jUtil;
*import *org.eclipse.net4j.connector.IConnector;
*import *org.eclipse.net4j.jvm.JVMUtil;
*import *org.eclipse.net4j.util.container.ContainerUtil;
*import *org.eclipse.net4j.util.container.IManagedContainer;
*import *org.eclipse.emf.ecore.EObject;
*import *org.eclipse.emf.ecore.resource.Resource;
*import *java.util.HashMap;
*import *java.util.Map;
/**
* @author Eike Stepper
*/
*public class *ImportXML
{
*private static final *String REPOSITORY_NAME = "repo1";
*public static **void *main(String[] args)
{
// Prepare the standalone infra structure (not needed when running inside Eclipse)
IManagedContainer container = ContainerUtil.createContainer(); // Create a wiring container
Net4jUtil.prepareContainer(container); // Prepare the Net4j kernel
JVMUtil.prepareContainer(container); // Prepare the JVM transport
CDOServerUtil.prepareContainer(container); // Prepare the CDO server
CDOUtil.prepareContainer(container, *false*); // Prepare the CDO client
// Start the transport and create a repository
JVMUtil.getAcceptor(container, "default"); // Start the JVM transport
CDOServerUtil.addRepository(container, createRepository()); // Start a CDO respository
// Establish a communications connection and open a session with the repository
IConnector connector = JVMUtil.getConnector(container, "default"); // Open a JVM connection
CDOSession session = CDOUtil.openSession(connector, REPOSITORY_NAME, *true*);// Open a CDO session
CDOTransaction transaction = session.openTransaction();// Open a CDO transaction
Resource resource = transaction.createResource("/my/big/resource");// Create a new EMF resource
// Work normally with the EMF resource
*for *(*int *i = 0; i < 100; i++)
{
EObject inputModel = getInputModel(i);
resource.getContents().add(inputModel);
transaction.commit();
}
session.close();
}
*private static *IRepository createRepository()
{
Map<String, String> props = *new *HashMap<String, String>();
props.put(IRepository.PROP_SUPPORTING_REVISION_DELTAS, "true");
props.put(IRepository.PROP_CURRENT_LRU_CAPACITY, "10000");
props.put(IRepository.PROP_REVISED_LRU_CAPACITY, "10000");
*return *CDOServerUtil.createRepository(REPOSITORY_NAME, createStore(), props);
}
*private static *IStore createStore()
{
// You might want to create an IDBStore here instead if memory is an issue!
*return *StoreUtil.createMEMStore();
}
*private static *EObject getInputModel(*int *i)
{
*throw new *UnsupportedOperationException("Load you model here");
}
}|
Regards,
Eike Stepper
----
http://wiki.eclipse.org/CDO
http://wiki.eclipse.org/Net4j
Eike Stepper schrieb:
> I've written a small example how to set up a standalone client and
> server within one JVM:
>
> package org.eclipse.emf.cdo.tests;
> import org.eclipse.emf.cdo.CDOSession;
> import org.eclipse.emf.cdo.CDOTransaction;
> import org.eclipse.emf.cdo.server.CDOServerUtil;
> import org.eclipse.emf.cdo.server.IRepository;
> import org.eclipse.emf.cdo.server.IStore;
> import org.eclipse.emf.cdo.server.StoreUtil;
> import org.eclipse.emf.cdo.util.CDOUtil;
> import org.eclipse.net4j.Net4jUtil;
> import org.eclipse.net4j.connector.IConnector;
> import org.eclipse.net4j.jvm.JVMUtil;
> import org.eclipse.net4j.util.container.ContainerUtil;
> import org.eclipse.net4j.util.container.IManagedContainer;
> import org.eclipse.emf.ecore.EObject;
> import org.eclipse.emf.ecore.resource.Resource;
> /**
> * @author Eike Stepper
> */
> public class ImportXML
> {
> private static final String REPOSITORY_NAME = "repo1";
> public static void main(String[] args)
> {
> // Prepare the standalone infra structure (not needed when
> running inside Eclipse)
> IManagedContainer container = ContainerUtil.createContainer();
> // Create a wiring container
> Net4jUtil.prepareContainer(container); // Prepare the Net4j kernel
> JVMUtil.prepareContainer(container); // Prepare the JVM transport
> CDOServerUtil.prepareContainer(container); // Prepare the CDO
> server
> CDOUtil.prepareContainer(container, false); // Prepare the CDO
> client
> // Start the transport and create a repository
> JVMUtil.getAcceptor(container, "default"); // Start the JVM
> transport
> CDOServerUtil.addRepository(container, createRepository()); //
> Start a CDO respository
> // Establish a communications connection and open a session
> with the repository
> IConnector connector = JVMUtil.getConnector(container,
> "default"); // Open a JVM connection
> CDOSession session = CDOUtil.openSession(connector,
> REPOSITORY_NAME, true);// Open a CDO session
> CDOTransaction transaction = session.openTransaction();// Open
> a CDO transaction
> Resource resource =
> transaction.createResource("/my/big/resource");// Create a new EMF
> resource
> // Work normally with the EMF resource
> for (int i = 0; i < 100; i++)
> {
> EObject inputModel = getInputModel(i);
> resource.getContents().add(inputModel);
> transaction.commit();
> }
> session.close();
> }
> private static IRepository createRepository()
> {
> Map<String, String> props = new HashMap<String, String>();
> props.put(IRepository.PROP_SUPPORTING_REVISION_DELTAS, "true");
> props.put(IRepository.PROP_CURRENT_LRU_CAPACITY, "10000");
> props.put(IRepository.PROP_REVISED_LRU_CAPACITY, "10000");
> return CDOServerUtil.createRepository(REPOSITORY_NAME,
> createStore(), props);
> }
> private static IStore createStore()
> {
> // You might want to create an IDBStore here instead if memory
> is an issue!
> return StoreUtil.createMEMStore();
> }
> private static EObject getInputModel(int i)
> {
> throw new UnsupportedOperationException("Load you model here");
> }
> }
>
> It's also possible to create and wire the various components yourself,
> but using a managed container with factories is easier. If running in
> Eclipse even the container is created and configured automatically.
> There's also a way to do the configuration via XML. Please tell me if
> you need more infos.
>
> Regards,
> Eike Stepper
> ----
> http://wiki.eclipse.org/CDO
> http://wiki.eclipse.org/Net4j
>
>
> Eike Stepper schrieb:
>> Hi Stefan,
>>
>> I don't know if Teneo can do this, Martin will want to explain that.
>>
>> CDO can do it. With CDO the modeled instances internally are not
>> interconnected with java reference handles. This way unneeded
>> instances can be completely garbage collected and transparently
>> reloaded later on. You don't need to write extra code for that to
>> function.
>>
>> If you plan to construct the output model programmatically everything
>> should work fine. If you plan to read your input models for example
>> from xml documents they should of course be small enough to fit into
>> memory. The resulting CDO repository resource(s) would have to remain
>> within the repository until you have enough memory on client side
>> that you can load them completely into an XML resource to save them
>> to a file again.
>>
>> If you plan to do your conversion within one VM (that is, not through
>> a networked remote repository) you can gain a lot of system resources
>> (memory, performance, ...) if you decide to use a Net4j IJVMConnector
>> instead of the usual ITCPConnector. And you can use an embedded Derby
>> Database to further optimize, but I guess that performance is not the
>> issue. If client memory gets low, it could even be better to use a
>> remote database server.
>>
>> Regards,
>> Eike Stepper
>> ----
>> http://wiki.eclipse.org/CDO
>> http://wiki.eclipse.org/Net4j
>>
>>
>>
>>
>>
>> Stefan Winkler schrieb:
>>> Hi all,
>>>
>>> I am working on a project which will have to deal with quite large
>>> (both broad and deep) models. The models themselves are only read or
>>> created (out of other models) but not modified themselves.
>>>
>>> Regarding the creation task: this will be something like a custom
>>> model transformation: an algorithm will read one model and produce
>>> another.
>>>
>>> Problem is: I expect the models to be too big to fit into memory, so
>>> I need a solution which allows only parts of models to be loaded and
>>> these parts to be unloaded, when they are not being used anymore. In
>>> terms of EMF, I want only parts of Resources to be loaded.
>>>
>>> Ed Merks pointed out that CDO or Teneo could help here as they use
>>> lazy loading strategies. So far I had a look into Teneo/Hibernate
>>> and I like the interface so far. But: While I understand that Teneo
>>> supports lazy loading of EMF Objects, does Teneo (or Hibernate
>>> behind this) have support to unload objects that are no longer needed?
>>>
>>> E.g.
>>>
>>> A
>>> |
>>> +--- B
>>> |
>>> +---- C
>>> |
>>> +----- D
>>>
>>> When I start at A (B,C,D are proxies, because policy says so), I
>>> access B,C and D. Is it possible to unload C and D and make C Proxy
>>> again so that C and D get garbage collected?
>>>
>>>
>>> If Teneo does not support this, does CDO?
>>>
>>>
>>> Thanks in advance!
>>>
>>> Cheers,
>>>
>>> Stefan
>>>
--------------040801030404030100020306
Content-Type: text/html; charset=ISO-8859-15
Content-Transfer-Encoding: 8bit
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
I found a nice plugin to format the java code to HTML:<br>
<br>
<title></title>
<style type="text/css">
<!--code { font-family: Courier New, Courier; font-size: 10pt; margin: 0px; }-->
</style>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<!-- ======================================================== -->
<!-- = Java Sourcecode to HTML automatically converted code = --><!-- = Java2Html Converter 5.0 [2006-02-26] by Markus Gebhard markus@jave.de = -->
<!-- = Further information: http://www.java2html.de = -->
<div class="java" align="left">
<table bgcolor="#ffffff" border="0" cellpadding="3" cellspacing="0">
<tbody>
<tr>
<!-- start source code --> <td align="left" nowrap="nowrap"
valign="top"> <code><font color="#7f0055"><b>package
|
|
|
| Re: [Teneo/CDO] Managing huge models / unloading parts of models [message #108084 is a reply to message #108068] |
Thu, 24 January 2008 12:11 |
| Eclipse User |
|
|
|
Originally posted by: merks.ca.ibm.com
This is a multi-part message in MIME format.
--------------020000080300010401020708
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit
Eike,
Oooo. That's just so pretty! Where did you find it?
Eike Stepper wrote:
> I found a nice plugin to format the java code to HTML:
>
> |*package *org.eclipse.emf.cdo.tests;
>
> *import *org.eclipse.emf.cdo.CDOSession;
> *import *org.eclipse.emf.cdo.CDOTransaction;
> *import *org.eclipse.emf.cdo.server.CDOServerUtil;
> *import *org.eclipse.emf.cdo.server.IRepository;
> *import *org.eclipse.emf.cdo.server.IStore;
> *import *org.eclipse.emf.cdo.server.StoreUtil;
> *import *org.eclipse.emf.cdo.util.CDOUtil;
>
> *import *org.eclipse.net4j.Net4jUtil;
> *import *org.eclipse.net4j.connector.IConnector;
> *import *org.eclipse.net4j.jvm.JVMUtil;
> *import *org.eclipse.net4j.util.container.ContainerUtil;
> *import *org.eclipse.net4j.util.container.IManagedContainer;
>
> *import *org.eclipse.emf.ecore.EObject;
> *import *org.eclipse.emf.ecore.resource.Resource;
>
> *import *java.util.HashMap;
> *import *java.util.Map;
>
> /**
> * @author Eike Stepper
> */
> *public class *ImportXML
> {
> *private static final *String REPOSITORY_NAME = "repo1";
>
> *public static **void *main(String[] args)
> {
> // Prepare the standalone infra structure (not needed when running inside Eclipse)
> IManagedContainer container = ContainerUtil.createContainer(); // Create a wiring container
> Net4jUtil.prepareContainer(container); // Prepare the Net4j kernel
> JVMUtil.prepareContainer(container); // Prepare the JVM transport
> CDOServerUtil.prepareContainer(container); // Prepare the CDO server
> CDOUtil.prepareContainer(container, *false*); // Prepare the CDO client
>
> // Start the transport and create a repository
> JVMUtil.getAcceptor(container, "default"); // Start the JVM transport
> CDOServerUtil.addRepository(container, createRepository()); // Start a CDO respository
>
> // Establish a communications connection and open a session with the repository
> IConnector connector = JVMUtil.getConnector(container, "default"); // Open a JVM connection
> CDOSession session = CDOUtil.openSession(connector, REPOSITORY_NAME, *true*);// Open a CDO session
> CDOTransaction transaction = session.openTransaction();// Open a CDO transaction
> Resource resource = transaction.createResource("/my/big/resource");// Create a new EMF resource
>
> // Work normally with the EMF resource
> *for *(*int *i = 0; i < 100; i++)
> {
> EObject inputModel = getInputModel(i);
> resource.getContents().add(inputModel);
> transaction.commit();
> }
>
> session.close();
> }
>
> *private static *IRepository createRepository()
> {
> Map<String, String> props = *new *HashMap<String, String>();
> props.put(IRepository.PROP_SUPPORTING_REVISION_DELTAS, "true");
> props.put(IRepository.PROP_CURRENT_LRU_CAPACITY, "10000");
> props.put(IRepository.PROP_REVISED_LRU_CAPACITY, "10000");
> *return *CDOServerUtil.createRepository(REPOSITORY_NAME, createStore(), props);
> }
>
> *private static *IStore createStore()
> {
> // You might want to create an IDBStore here instead if memory is an issue!
> *return *StoreUtil.createMEMStore();
> }
>
> *private static *EObject getInputModel(*int *i)
> {
> *throw new *UnsupportedOperationException("Load you model here");
> }
> }|
>
>
> Regards,
> Eike Stepper
> ----
> http://wiki.eclipse.org/CDO
> http://wiki.eclipse.org/Net4j
>
>
>
> Eike Stepper schrieb:
>> I've written a small example how to set up a standalone client and
>> server within one JVM:
>>
>> package org.eclipse.emf.cdo.tests;
>> import org.eclipse.emf.cdo.CDOSession;
>> import org.eclipse.emf.cdo.CDOTransaction;
>> import org.eclipse.emf.cdo.server.CDOServerUtil;
>> import org.eclipse.emf.cdo.server.IRepository;
>> import org.eclipse.emf.cdo.server.IStore;
>> import org.eclipse.emf.cdo.server.StoreUtil;
>> import org.eclipse.emf.cdo.util.CDOUtil;
>> import org.eclipse.net4j.Net4jUtil;
>> import org.eclipse.net4j.connector.IConnector;
>> import org.eclipse.net4j.jvm.JVMUtil;
>> import org.eclipse.net4j.util.container.ContainerUtil;
>> import org.eclipse.net4j.util.container.IManagedContainer;
>> import org.eclipse.emf.ecore.EObject;
>> import org.eclipse.emf.ecore.resource.Resource;
>> /**
>> * @author Eike Stepper
>> */
>> public class ImportXML
>> {
>> private static final String REPOSITORY_NAME = "repo1";
>> public static void main(String[] args)
>> {
>> // Prepare the standalone infra structure (not needed when
>> running inside Eclipse)
>> IManagedContainer container = ContainerUtil.createContainer();
>> // Create a wiring container
>> Net4jUtil.prepareContainer(container); // Prepare the Net4j
>> kernel
>> JVMUtil.prepareContainer(container); // Prepare the JVM transport
>> CDOServerUtil.prepareContainer(container); // Prepare the CDO
>> server
>> CDOUtil.prepareContainer(container, false); // Prepare the CDO
>> client
>> // Start the transport and create a repository
>> JVMUtil.getAcceptor(container, "default"); // Start the JVM
>> transport
>> CDOServerUtil.addRepository(container, createRepository()); //
>> Start a CDO respository
>> // Establish a communications connection and open a session
>> with the repository
>> IConnector connector = JVMUtil.getConnector(container,
>> "default"); // Open a JVM connection
>> CDOSession session = CDOUtil.openSession(connector,
>> REPOSITORY_NAME, true);// Open a CDO session
>> CDOTransaction transaction = session.openTransaction();// Open
>> a CDO transaction
>> Resource resource =
>> transaction.createResource("/my/big/resource");// Create a new EMF
>> resource
>> // Work normally with the EMF resource
>> for (int i = 0; i < 100; i++)
>> {
>> EObject inputModel = getInputModel(i);
>> resource.getContents().add(inputModel);
>> transaction.commit();
>> }
>> session.close();
>> }
>> private static IRepository createRepository()
>> {
>> Map<String, String> props = new HashMap<String, String>();
>> props.put(IRepository.PROP_SUPPORTING_REVISION_DELTAS, "true");
>> props.put(IRepository.PROP_CURRENT_LRU_CAPACITY, "10000");
>> props.put(IRepository.PROP_REVISED_LRU_CAPACITY, "10000");
>> return CDOServerUtil.createRepository(REPOSITORY_NAME,
>> createStore(), props);
>> }
>> private static IStore createStore()
>> {
>> // You might want to create an IDBStore here instead if memory
>> is an issue!
>> return StoreUtil.createMEMStore();
>> }
>> private static EObject getInputModel(int i)
>> {
>> throw new UnsupportedOperationException("Load you model here");
>> }
>> }
>>
>> It's also possible to create and wire the various components
>> yourself, but using a managed container with factories is easier. If
>> running in Eclipse even the container is created and configured
>> automatically. There's also a way to do the configuration via XML.
>> Please tell me if you need more infos.
>>
>> Regards,
>> Eike Stepper
>> ----
>> http://wiki.eclipse.org/CDO
>> http://wiki.eclipse.org/Net4j
>>
>>
>> Eike Stepper schrieb:
>>> Hi Stefan,
>>>
>>> I don't know if Teneo can do this, Martin will want to explain that.
>>>
>>> CDO can do it. With CDO the modeled instances internally are not
>>> interconnected with java reference handles. This way unneeded
>>> instances can be completely garbage collected and transparently
>>> reloaded later on. You don't need to write extra code for that to
>>> function.
>>>
>>> If you plan to construct the output model programmatically
>>> everything should work fine. If you plan to read your input models
>>> for example from xml documents they should of course be small enough
>>> to fit into memory. The resulting CDO repository resource(s) would
>>> have to remain within the repository until you have enough memory on
>>> client side that you can load them completely into an XML resource
>>> to save them to a file again.
>>>
>>> If you plan to do your conversion within one VM (that is, not
>>> through a networked remote repository) you can gain a lot of system
>>> resources (memory, performance, ...) if you decide to use a Net4j
>>> IJVMConnector instead of the usual ITCPConnector. And you can use an
>>> embedded Derby Database to further optimize, but I guess that
>>> performance is not the issue. If client memory gets low, it could
>>> even be better to use a remote database server.
>>>
>>> Regards,
>>> Eike Stepper
>>> ----
>>> http://wiki.eclipse.org/CDO
>>> http://wiki.eclipse.org/Net4j
>>>
>>>
>>>
>>>
>>>
>>> Stefan Winkler schrieb:
>>>> Hi all,
>>>>
>>>> I am working on a project which will have to deal with quite large
>>>> (both broad and deep) models. The models themselves are only read
>>>> or created (out of other models) but not modified themselves.
>>>>
>>>> Regarding the creation task: this will be something like a custom
>>>> model transformation: an algorithm will read one model and produce
>>>> another.
>>>>
>>>> Problem is: I expect the models to be too big to fit into memory,
>>>> so I need a solution which allows only parts of models to be loaded
>>>> and these parts to be unloaded, when they are not being used
>>>> anymore. In terms of EMF, I want only parts of Resources to be loaded.
>>>>
>>>> Ed Merks pointed out that CDO or Teneo could help here as they use
>>>> lazy loading strategies. So far I had a look into Teneo/Hibernate
>>>> and I like the interface so far. But: While I understand that Teneo
>>>> supports lazy loading of EMF Objects, does Teneo (or Hibernate
>>>> behind this) have support to unload objects that are no longer needed?
>>>>
>>>> E.g.
>>>>
>>>> A
>>>> |
>>>> +--- B
>>>> |
>>>> +---- C
>>>> |
>>>> +----- D
>>>>
>>>> When I start at A (B,C,D are proxies, because policy says so), I
>>>> access B,C and D. Is it possible to unload C and D and make C Proxy
>>>> again so that C and D get garbage collected?
>>>>
>>>>
>>>> If Teneo does not support this, does CDO?
>>>>
>>>>
>>>> Thanks in advance!
>>>>
>>>> Cheers,
>>>>
>>>> Stefan
>>>>
--------------020000080300010401020708
Content-Type: text/html; charset=ISO-8859-15
Content-Transfer-Encoding: 8bit
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Eike,<br>
<br>
Oooo. That's just so pretty!
|
|
|
| Re: [Teneo/CDO] Managing huge models / unloading parts of models [message #108101 is a reply to message #108084] |
Thu, 24 January 2008 13:03 |
| Eclipse User |
|
|
|
Originally posted by: stepper.sympedia.de
This is a multi-part message in MIME format.
--------------060906030706060808070601
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit
Ed Merks schrieb:
> Eike,
>
> Oooo. That's just so pretty! Where did you find it?
Cute, isn't it?
Look at http://www.java2html.de/eclipse.html
Cheers
/Eike
>
>
> Eike Stepper wrote:
>> I found a nice plugin to format the java code to HTML:
>>
>> |*package *org.eclipse.emf.cdo.tests;
>>
>> *import *org.eclipse.emf.cdo.CDOSession;
>> *import *org.eclipse.emf.cdo.CDOTransaction;
>> *import *org.eclipse.emf.cdo.server.CDOServerUtil;
>> *import *org.eclipse.emf.cdo.server.IRepository;
>> *import *org.eclipse.emf.cdo.server.IStore;
>> *import *org.eclipse.emf.cdo.server.StoreUtil;
>> *import *org.eclipse.emf.cdo.util.CDOUtil;
>>
>> *import *org.eclipse.net4j.Net4jUtil;
>> *import *org.eclipse.net4j.connector.IConnector;
>> *import *org.eclipse.net4j.jvm.JVMUtil;
>> *import *org.eclipse.net4j.util.container.ContainerUtil;
>> *import *org.eclipse.net4j.util.container.IManagedContainer;
>>
>> *import *org.eclipse.emf.ecore.EObject;
>> *import *org.eclipse.emf.ecore.resource.Resource;
>>
>> *import *java.util.HashMap;
>> *import *java.util.Map;
>>
>> /**
>> * @author Eike Stepper
>> */
>> *public class *ImportXML
>> {
>> *private static final *String REPOSITORY_NAME = "repo1";
>>
>> *public static **void *main(String[] args)
>> {
>> // Prepare the standalone infra structure (not needed when running inside Eclipse)
>> IManagedContainer container = ContainerUtil.createContainer(); // Create a wiring container
>> Net4jUtil.prepareContainer(container); // Prepare the Net4j kernel
>> JVMUtil.prepareContainer(container); // Prepare the JVM transport
>> CDOServerUtil.prepareContainer(container); // Prepare the CDO server
>> CDOUtil.prepareContainer(container, *false*); // Prepare the CDO client
>>
>> // Start the transport and create a repository
>> JVMUtil.getAcceptor(container, "default"); // Start the JVM transport
>> CDOServerUtil.addRepository(container, createRepository()); // Start a CDO respository
>>
>> // Establish a communications connection and open a session with the repository
>> IConnector connector = JVMUtil.getConnector(container, "default"); // Open a JVM connection
>> CDOSession session = CDOUtil.openSession(connector, REPOSITORY_NAME, *true*);// Open a CDO session
>> CDOTransaction transaction = session.openTransaction();// Open a CDO transaction
>> Resource resource = transaction.createResource("/my/big/resource");// Create a new EMF resource
>>
>> // Work normally with the EMF resource
>> *for *(*int *i = 0; i < 100; i++)
>> {
>> EObject inputModel = getInputModel(i);
>> resource.getContents().add(inputModel);
>> transaction.commit();
>> }
>>
>> session.close();
>> }
>>
>> *private static *IRepository createRepository()
>> {
>> Map<String, String> props = *new *HashMap<String, String>();
>> props.put(IRepository.PROP_SUPPORTING_REVISION_DELTAS, "true");
>> props.put(IRepository.PROP_CURRENT_LRU_CAPACITY, "10000");
>> props.put(IRepository.PROP_REVISED_LRU_CAPACITY, "10000");
>> *return *CDOServerUtil.createRepository(REPOSITORY_NAME, createStore(), props);
>> }
>>
>> *private static *IStore createStore()
>> {
>> // You might want to create an IDBStore here instead if memory is an issue!
>> *return *StoreUtil.createMEMStore();
>> }
>>
>> *private static *EObject getInputModel(*int *i)
>> {
>> *throw new *UnsupportedOperationException("Load you model here");
>> }
>> }|
>>
>>
>> Regards,
>> Eike Stepper
>> ----
>> http://wiki.eclipse.org/CDO
>> http://wiki.eclipse.org/Net4j
>>
>>
>>
>> Eike Stepper schrieb:
>>> I've written a small example how to set up a standalone client and
>>> server within one JVM:
>>>
>>> package org.eclipse.emf.cdo.tests;
>>> import org.eclipse.emf.cdo.CDOSession;
>>> import org.eclipse.emf.cdo.CDOTransaction;
>>> import org.eclipse.emf.cdo.server.CDOServerUtil;
>>> import org.eclipse.emf.cdo.server.IRepository;
>>> import org.eclipse.emf.cdo.server.IStore;
>>> import org.eclipse.emf.cdo.server.StoreUtil;
>>> import org.eclipse.emf.cdo.util.CDOUtil;
>>> import org.eclipse.net4j.Net4jUtil;
>>> import org.eclipse.net4j.connector.IConnector;
>>> import org.eclipse.net4j.jvm.JVMUtil;
>>> import org.eclipse.net4j.util.container.ContainerUtil;
>>> import org.eclipse.net4j.util.container.IManagedContainer;
>>> import org.eclipse.emf.ecore.EObject;
>>> import org.eclipse.emf.ecore.resource.Resource;
>>> /**
>>> * @author Eike Stepper
>>> */
>>> public class ImportXML
>>> {
>>> private static final String REPOSITORY_NAME = "repo1";
>>> public static void main(String[] args)
>>> {
>>> // Prepare the standalone infra structure (not needed when
>>> running inside Eclipse)
>>> IManagedContainer container =
>>> ContainerUtil.createContainer(); // Create a wiring container
>>> Net4jUtil.prepareContainer(container); // Prepare the Net4j
>>> kernel
>>> JVMUtil.prepareContainer(container); // Prepare the JVM
>>> transport
>>> CDOServerUtil.prepareContainer(container); // Prepare the CDO
>>> server
>>> CDOUtil.prepareContainer(container, false); // Prepare the
>>> CDO client
>>> // Start the transport and create a repository
>>> JVMUtil.getAcceptor(container, "default"); // Start the JVM
>>> transport
>>> CDOServerUtil.addRepository(container, createRepository());
>>> // Start a CDO respository
>>> // Establish a communications connection and open a session
>>> with the repository
>>> IConnector connector = JVMUtil.getConnector(container,
>>> "default"); // Open a JVM connection
>>> CDOSession session = CDOUtil.openSession(connector,
>>> REPOSITORY_NAME, true);// Open a CDO session
>>> CDOTransaction transaction = session.openTransaction();//
>>> Open a CDO transaction
>>> Resource resource =
>>> transaction.createResource("/my/big/resource");// Create a new EMF
>>> resource
>>> // Work normally with the EMF resource
>>> for (int i = 0; i < 100; i++)
>>> {
>>> EObject inputModel = getInputModel(i);
>>> resource.getContents().add(inputModel);
>>> transaction.commit();
>>> }
>>> session.close();
>>> }
>>> private static IRepository createRepository()
>>> {
>>> Map<String, String> props = new HashMap<String, String>();
>>> props.put(IRepository.PROP_SUPPORTING_REVISION_DELTAS, "true");
>>> props.put(IRepository.PROP_CURRENT_LRU_CAPACITY, "10000");
>>> props.put(IRepository.PROP_REVISED_LRU_CAPACITY, "10000");
>>> return CDOServerUtil.createRepository(REPOSITORY_NAME,
>>> createStore(), props);
>>> }
>>> private static IStore createStore()
>>> {
>>> // You might want to create an IDBStore here instead if
>>> memory is an issue!
>>> return StoreUtil.createMEMStore();
>>> }
>>> private static EObject getInputModel(int i)
>>> {
>>> throw new UnsupportedOperationException("Load you model here");
>>> }
>>> }
>>>
>>> It's also possible to create and wire the various components
>>> yourself, but using a managed container with factories is easier. If
>>> running in Eclipse even the container is created and configured
>>> automatically. There's also a way to do the configuration via XML.
>>> Please tell me if you need more infos.
>>>
>>> Regards,
>>> Eike Stepper
>>> ----
>>> http://wiki.eclipse.org/CDO
>>> http://wiki.eclipse.org/Net4j
>>>
>>>
>>> Eike Stepper schrieb:
>>>> Hi Stefan,
>>>>
>>>> I don't know if Teneo can do this, Martin will want to explain that.
>>>>
>>>> CDO can do it. With CDO the modeled instances internally are not
>>>> interconnected with java reference handles. This way unneeded
>>>> instances can be completely garbage collected and transparently
>>>> reloaded later on. You don't need to write extra code for that to
>>>> function.
>>>>
>>>> If you plan to construct the output model programmatically
>>>> everything should work fine. If you plan to read your input models
>>>> for example from xml documents they should of course be small
>>>> enough to fit into memory. The resulting CDO repository resource(s)
>>>> would have to remain within the repository until you have enough
>>>> memory on client side that you can load them completely into an XML
>>>> resource to save them to a file again.
>>>>
>>>> If you plan to do your conversion within one VM (that is, not
>>>> through a networked remote repository) you can gain a lot of system
>>>> resources (memory, performance, ...) if you decide to use a Net4j
>>>> IJVMConnector instead of the usual ITCPConnector. And you can use
>>>> an embedded Derby Database to further optimize, but I guess that
>>>> performance is not the issue. If client memory gets low, it could
>>>> even be better to use a remote database server.
>>>>
>>>> Regards,
>>>> Eike Stepper
>>>> ----
>>>> http://wiki.eclipse.org/CDO
>>>> http://wiki.eclipse.org/Net4j
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Stefan Winkler schrieb:
>>>>> Hi all,
>>>>>
>>>>> I am working on a project which will have to deal with quite large
>>>>> (both broad and deep) models. The models themselves are only read
>>>>> or created (out of other models) but not modified themselves.
>>>>>
>>>>> Regarding the creation task: this will be something like a custom
>>>>> model transformation: an algorithm will read one model and produce
>>>>> another.
>>>>>
>>>>> Problem is: I expect the models to be too big to fit into memory,
>>>>> so I need a solution which allows only parts of models to be
>>>>> loaded and these parts to be unloaded, when they are not being
>>>>> used anymore. In terms of EMF, I want only parts of Resources to
>>>>> be loaded.
>>>>>
>>>>> Ed Merks pointed out that CDO or Teneo could help here as they use
>>>>> lazy loading strategies. So far I had a look into Teneo/Hibernate
>>>>> and I like the interface so far. But: While I understand that
>>>>> Teneo supports lazy loading of EMF Objects, does Teneo (or
>>>>> Hibernate behind this) have support to unload objects that are no
>>>>> longer needed?
>>>>>
>>>>> E.g.
>>>>>
>>>>> A
>>>>> |
>>>>> +--- B
>>>>> |
>>>>> +---- C
>>>>> |
>>>>> +----- D
>>>>>
>>>>> When I start at A (B,C,D are proxies, because policy says so), I
>>>>> access B,C and D. Is it possible to unload C and D and make C
>>>>> Proxy again so that C and D get garbage collected?
>>>>>
>>>>>
>>>>> If Teneo does not support this, does CDO?
>>>>>
>>>>>
>>>>> Thanks in advance!
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Stefan
>>>>>
>
--------------060906030706060808070601
Content-Type: text/html; charset=ISO-8859-15
Content-Transfer-Encoding: 8bit
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Ed Merks schrieb:
<blockquote cite="mid:fn9v9l$fsp$1@build.eclipse.org" type="cite">
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
Eike,<br>
<br>
Oooo. That's just so pretty!
|
|
|
| Re: [Teneo/CDO] Managing huge models / unloading parts of models [message #108165 is a reply to message #108068] |
Fri, 25 January 2008 10:57 |
Stefan Winkler
Messages: 307
Registered: July 2009
Location: Germany |
Senior Member |
|
|
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Eike,<br>
<br>
thanks for the code snippet.<br>
<br>
Could you again clarify a few things:<br>
<br>
When you write:<br>
<code><font color="#ffffff"><br>
</font><font color="#3f7f5f"> // Prepare the standalone infra structure (not needed when running inside Eclipse) </font><br>
<br>
</code>do you mean "not needed if the code is running in Eclipse" or
"not needed if a (separate) server is already running in Eclipse".<br>
Because: I checked out all.psf and as far as I can see the internat
server is only started, if it is run as an own product.<br>
Simply installing the plugins and features into my existing Eclipse IDE
does not start the server, right?<br>
<br>
<code><font color="#ffffff"></font><font color="#3f7f5f"> // You might want to create an IDBStore here instea d if memory is an issue! <br>
<br>
</font></code>How is this done (I'm going to use MySQL as a backend
database)?<br>
There is no StoreUtil.createIDBStore :-(<br>
<br>
Thanks for your support,<br>
<br>
Stefan<br>
<br>
<br>
<br>
Eike Stepper schrieb:
<blockquote cite="mid:fn9re7$q6m$1@build.eclipse.org" type="cite">
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
I found a nice plugin to format the java code to HTML:<br>
<br>
<title></title>
<style type="text/css">
<!--code { font-family: Courier New, Courier; font-size: 10pt; margin: 0px; }-->
</style>
<meta http-equiv="Content-Type" content="text/html; ">
<!-- ======================================================== -->
<!-- = Java Sourcecode to HTML automatically converted code = --><!-- = Java2Html Converter 5.0 [2006-02-26] by Markus Gebhard markus@jave.de = --><!-- = Further information: http://www.java2html.de = -->
<div class="java" align="left">
<table bgcolor="#ffffff" border="0" cellpadding="3" cellspacing="0">
<tbody>
<tr>
<!-- start source code --> <td align="left" nowrap="nowrap"
valign="top"> <code><font color="#7f0055"><b>package </b></font><font
color="#000000">org.eclipse.emf.cdo.tests;</font><br>
<br>
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.emf.cdo.CDOSession;</font><br >
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.emf.cdo.CDOTransaction;</font ><br>
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.emf.cdo.server.CDOServerUtil;</font ><br>
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.emf.cdo.server.IRepository;</font ><br>
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.emf.cdo.server.IStore;</font> <br>
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.emf.cdo.server.StoreUtil;</font ><br>
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.emf.cdo.util.CDOUtil;</font><br >
<br>
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.net4j.Net4jUtil;</font><br>
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.net4j.connector.IConnector;</font ><br>
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.net4j.jvm.JVMUtil;</font><br >
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.net4j.util.container.ContainerUtil; </font><br>
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.net4j.util.container.IManagedContainer; </font><br>
<br>
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.emf.ecore.EObject;</font><br >
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.emf.ecore.resource.Resource;</font ><br>
<br>
<font color="#7f0055"><b>import </b></font><font color="#000000">java.util.HashMap;</font><br>
<font color="#7f0055"><b>import </b></font><font color="#000000">java.util.Map;</font><br>
<br>
<font color="#3f5fbf">/**</font><br>
<font color="#ffffff"> </font><font color="#3f5fbf">* </font><font
color="#7f9fbf">@author </font><font color="#3f5fbf">Eike Stepper</font><br>
<font color="#ffffff"> </font><font color="#3f5fbf">*/</font><br>
<font color="#7f0055"><b>public class </b></font><font
color="#000000">ImportXML</font><br>
<font color="#000000">{</font><br>
<font color="#ffffff"> </font><font color="#7f0055"><b>private static final </b></font><font
color="#000000">String REPOSITORY_NAME = </font><font color="#2a00ff">"repo1"</font><font
color="#000000">;</font><br>
<br>
<font color="#ffffff"> </font><font color="#7f0055"><b>public static </b></font><font
color="#7f0055"><b>void </b></font><font color="#000000">main</font><font
color="#000000">(</font><font color="#000000">String</font><font
color="#000000">[] </font><font color="#000000">args</font><font
color="#000000">)</font><br>
<font color="#ffffff"> </font><font color="#000000">{</font><br>
<font color="#ffffff"> </font><font color="#3f7f5f"> // Prepare the standalone infra structure (not needed when running inside Eclipse) </font><br>
<font color="#ffffff"> </font><font color="#000000"> IManagedContainer container = ContainerUtil.createContain er </font><font
color="#000000">()</font><font color="#000000">; </font><font
color="#3f7f5f">// Create a wiring container</font><br >
<font color="#ffffff"> </font><font color="#000000">Net4jUtil.prepareContainer</font><font
color="#000000">(</font><font color="#000000">container</font><font
color="#000000">)</font><font color="#000000">; </font><font
color="#3f7f5f">// Prepare the Net4j kernel</font><br >
<font color="#ffffff"> </font><font color="#000000">JVMUtil.prepareContainer</font><font
color="#000000">(</font><font color="#000000">container</font><font
color="#000000">)</font><font color="#000000">; </font><font
color="#3f7f5f">// Prepare the JVM transport</font><br >
<font color="#ffffff"> </font><font color="#000000">CDOServerUtil.prepareContainer</font><font
color="#000000">(</font><font color="#000000">container</font><font
color="#000000">)</font><font color="#000000">; </font><font
color="#3f7f5f">// Prepare the CDO server</font><br>
<font color="#ffffff"> </font><font color="#000000">CDOUtil.prepareContainer</font><font
color="#000000">(</font><font color="#000000">container, </font><font
color="#7f0055"><b>false</b></font><font color="#000000">)</font><font
color="#000000">; </font><font color="#3f7f5f">// Prepare the CDO client</font><br>
<br>
<font color="#ffffff"> </font><font color="#3f7f5f">// Start the transport and create a repository </font><br>
<font color="#ffffff"> </font><font color="#000000">JVMUtil.getAcceptor</font><font
color="#000000">(</font><font color="#000000">container, </font><font
color="#2a00ff">"default"</font><font color="#000000">)</font><font
color="#000000">; </font><font color="#3f7f5f">// Start the JVM transport</font><br >
<font color="#ffffff"> </font><font color="#000000">CDOServerUtil.addRepository</font><font
color="#000000">(</font><font color="#000000">container, createRepository</font><font
color="#000000">())</font><font color="#000000">; </font><font
color="#3f7f5f">// Start a CDO respository</font><br >
<br>
<font color="#ffffff"> </font><font color="#3f7f5f"> // Establish a communications connection and open a session with the repository </font><br>
<font color="#ffffff"> </font><font color="#000000">IConnector connector = JVMUtil.getConnector </font><font
color="#000000">(</font><font color="#000000">container, </font><font
color="#2a00ff">"default"</font><font color="#000000">)</font><font
color="#000000">; </font><font color="#3f7f5f">// Open a JVM connection</font><br>
<font color="#ffffff"> </font><font color="#000000">CDOSession session = CDOUtil.openSession </font><font
color="#000000">(</font><font color="#000000">connector, REPOSITORY_NAME, </font><font
color="#7f0055"><b>true</b></font><font color="#000000">)</font><font
color="#000000">;</font><font color="#3f7f5f">// Open a CDO session</font><br>
<font color="#ffffff"> </font><font color="#000000">CDOTransaction transaction = session.openTransaction </font><font
color="#000000">()</font><font color="#000000">;</font><font
color="#3f7f5f">// Open a CDO transaction</font><br>
<font color="#ffffff"> </font><font color="#000000">Resource resource = transaction.createResource </font><font
color="#000000">(</font><font color="#2a00ff">"/my/big/resource"</font><font
color="#000000">)</font><font color="#000000">;</font><font
color="#3f7f5f">// Create a new EMF resource</font><br >
<br>
<font color="#ffffff"> </font><font color="#3f7f5f">// Work normally with the EMF resource </font><br>
<font color="#ffffff"> </font><font color="#7f0055"><b>for </b></font><font
color="#000000">(</font><font color="#7f0055"><b>int </b></font><font
color="#000000">i = </font><font color="#990000">0</font><font
color="#000000">; i < </font><font color="#990000">100</font><font
color="#000000">; i++</font><font color="#000000">)</font><br>
<font color="#ffffff"> </font><font color="#000000">{</font><br>
<font color="#ffffff"> </font><font color="#000000">EObject inputModel = getInputModel</font ><font
color="#000000">(</font><font color="#000000">i</font><font
color="#000000">)</font><font color="#000000">;</font><br>
<font color="#ffffff"> </font><font color="#000000">resource.getContents</font><font
color="#000000">()</font><font color="#000000">.add</font><font
color="#000000">(</font><font color="#000000">inputModel</font><font
color="#000000">)</font><font color="#000000">;</font><br>
<font color="#ffffff"> </font><font color="#000000">transaction.commit</font><font
color="#000000">()</font><font color="#000000">;</font><br>
<font color="#ffffff"> </font><font color="#000000">}</font><br>
<br>
<font color="#ffffff"> </font><font color="#000000">session.close</font><font
color="#000000">()</font><font color="#000000">;</font><br>
<font color="#ffffff"> </font><font color="#000000">}</font><br>
<br>
<font color="#ffffff"> </font><font color="#7f0055"><b>private static </b></font><font
color="#000000">IRepository createRepository</font><font
color="#000000">()</font><br>
<font color="#ffffff"> </font><font color="#000000">{</font><br>
<font color="#ffffff"> </font><font color="#000000">Map<String, String> props = </font><font
color="#7f0055"><b>new </b></font><font color="#000000">HashMap<String, String></font ><font
color="#000000">()</font><font color="#000000">;</font><br>
<font color="#ffffff"> </font><font color="#000000">props.put</font><font
color="#000000">(</font><font color="#000000">IRepository.PROP_SUPPORTING_REVISION_DELTAS, </font><font
color="#2a00ff">"true"</font><font color="#000000">)</font><font
color="#000000">;</font><br>
<font color="#ffffff"> </font><font color="#000000">props.put</font><font
color="#000000">(</font><font color="#000000">IRepository.PROP_CURRENT_LRU_CAPACITY, </font ><font
color="#2a00ff">"10000"</font><font color="#000000">)</font><font
color="#000000">;</font><br>
<font color="#ffffff"> </font><font color="#000000">props.put</font><font
color="#000000">(</font><font color="#000000">IRepository.PROP_REVISED_LRU_CAPACITY, </font ><font
color="#2a00ff">"10000"</font><font color="#000000">)</font><font
color="#000000">;</font><br>
<font color="#ffffff"> </font><font color="#7f0055"><b>return </b></font><font
color="#000000">CDOServerUtil.createRepository</font><font
color="#000000">(</font><font color="#000000">REPOSITORY_NAME, createStore</font><font
color="#000000">()</font><font color="#000000">, props</font><font
color="#000000">)</font><font color="#000000">;</font><br>
<font color="#ffffff"> </font><font color="#000000">}</font><br>
<br>
<font color="#ffffff"> </font><font color="#7f0055"><b>private static </b></font><font
color="#000000">IStore createStore</font><font color="#000000">()</font><br>
<font color="#ffffff"> </font><font color="#000000">{</font><br>
<font color="#ffffff"> </font><font color="#3f7f5f"> // You might want to create an IDBStore here instea d if memory is an issue! </font><br>
<font color="#ffffff"> </font><font color="#7f0055"><b>return </b></font><font
color="#000000">StoreUtil.createMEMStore</font><font color="#000000">()</font><font
color="#000000">;</font><br>
<font color="#ffffff"> </font><font color="#000000">}</font><br>
<br>
<font color="#ffffff"> </font><font color="#7f0055"><b>private static </b></font><font
color="#000000">EObject getInputModel</font><font color="#000000">(</font><font
color="#7f0055"><b>int </b></font><font color="#000000">i</font><font
color="#000000">)</font><br>
<font color="#ffffff"> </font><font color="#000000">{</font><br>
<font color="#ffffff"> </font><font color="#7f0055"><b>throw new </b></font><font
color="#000000">UnsupportedOperationException</font><font
color="#000000">(</font><font color="#2a00ff">"Load you model here"</font><font
color="#000000">)</font><font color="#000000">;</font><br>
<font color="#ffffff"> </font><font color="#000000">}</font><br>
<font color="#000000">}</font></code> </td>
<!-- end source code --> </tr>
</tbody>
</table>
</div>
<!-- = END of automatically generated HTML code = -->
<!-- ======================================================== --><br>
Regards,<br>
Eike Stepper<br>
----<br>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://wiki.eclipse.org/CDO">http://wiki.eclipse.org/CDO</a><br>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://wiki.eclipse.org/Net4j">http://wiki.eclipse.org/Net4j</a><br>
<br>
<br>
<br>
Eike Stepper schrieb:
<blockquote cite="mid:fn7mcv$nfg$2@build.eclipse.org" type="cite">I've
written a small example how to set up a standalone client and server
within one JVM: <br>
<br>
package org.eclipse.emf.cdo.tests; <br>
import org.eclipse.emf.cdo.CDOSession; <br>
import org.eclipse.emf.cdo.CDOTransaction; <br>
import org.eclipse.emf.cdo.server.CDOServerUtil; <br>
import org.eclipse.emf.cdo.server.IRepository; <br>
import org.eclipse.emf.cdo.server.IStore; <br>
import org.eclipse.emf.cdo.server.StoreUtil; <br>
import org.eclipse.emf.cdo.util.CDOUtil; <br>
import org.eclipse.net4j.Net4jUtil; <br>
import org.eclipse.net4j.connector.IConnector; <br>
import org.eclipse.net4j.jvm.JVMUtil; <br>
import org.eclipse.net4j.util.container.ContainerUtil; <br>
import org.eclipse.net4j.util.container.IManagedContainer; <br>
import org.eclipse.emf.ecore.EObject; <br>
import org.eclipse.emf.ecore.resource.Resource; <br>
/** <br>
* @author Eike Stepper <br>
*/ <br>
public class ImportXML <br>
{ <br>
private static final String REPOSITORY_NAME = "repo1"; <br>
public static void main(String[] args) <br>
{ <br>
// Prepare the standalone infra structure (not needed when
running inside Eclipse) <br>
IManagedContainer container = ContainerUtil.createContainer();
// Create a wiring container <br>
Net4jUtil.prepareContainer(container); // Prepare the Net4j
kernel <br>
JVMUtil.prepareContainer(container); // Prepare the JVM
transport <br>
CDOServerUtil.prepareContainer(container); // Prepare the CDO
server <br>
CDOUtil.prepareContainer(container, false); // Prepare the CDO
client <br>
// Start the transport and create a repository <br>
JVMUtil.getAcceptor(container, "default"); // Start the JVM
transport <br>
CDOServerUtil.addRepository(container, createRepository()); //
Start a CDO respository <br>
// Establish a communications connection and open a session
with the repository <br>
IConnector connector = JVMUtil.getConnector(container,
"default"); // Open a JVM connection <br>
CDOSession session = CDOUtil.openSession(connector,
REPOSITORY_NAME, true);// Open a CDO session <br>
CDOTransaction transaction = session.openTransaction();// Open a
CDO transaction <br>
Resource resource =
transaction.createResource("/my/big/resource");// Create a new EMF
resource <br>
// Work normally with the EMF resource <br>
for (int i = 0; i < 100; i++) <br>
{ <br>
EObject inputModel = getInputModel(i); <br>
resource.getContents().add(inputModel); <br>
transaction.commit(); <br>
} <br>
session.close(); <br>
} <br>
private static IRepository createRepository() <br>
{ <br>
Map<String, String> props = new HashMap<String,
String>(); <br>
props.put(IRepository.PROP_SUPPORTING_REVISION_DELTAS, "true"); <br>
props.put(IRepository.PROP_CURRENT_LRU_CAPACITY, "10000"); <br>
props.put(IRepository.PROP_REVISED_LRU_CAPACITY, "10000"); <br>
return CDOServerUtil.createRepository(REPOSITORY_NAME,
createStore(), props); <br>
} <br>
private static IStore createStore() <br>
{ <br>
// You might want to create an IDBStore here instead if memory
is an issue! <br>
return StoreUtil.createMEMStore(); <br>
} <br>
private static EObject getInputModel(int i) <br>
{ <br>
throw new UnsupportedOperationException("Load you model here"); <br>
} <br>
} <br>
<br>
It's also possible to create and wire the various components yourself,
but using a managed container with factories is easier. If running in
Eclipse even the container is created and configured automatically.
There's also a way to do the configuration via XML. Please tell me if
you need more infos. <br>
<br>
Regards, <br>
Eike Stepper <br>
---- <br>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://wiki.eclipse.org/CDO">http://wiki.eclipse.org/CDO</a> <br>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://wiki.eclipse.org/Net4j">http://wiki.eclipse.org/Net4j</a>
<br>
<br>
<br>
Eike Stepper schrieb: <br>
<blockquote type="cite">Hi Stefan, <br>
<br>
I don't know if Teneo can do this, Martin will want to explain that. <br>
<br>
CDO can do it. With CDO the modeled instances internally are not
interconnected with java reference handles. This way unneeded instances
can be completely garbage collected and transparently reloaded later
on. You don't need to write extra code for that to function. <br>
<br>
If you plan to construct the output model programmatically everything
should work fine. If you plan to read your input models for example
from xml documents they should of course be small enough to fit into
memory. The resulting CDO repository resource(s) would have to remain
within the repository until you have enough memory on client side that
you can load them completely into an XML resource to save them to a
file again. <br>
<br>
If you plan to do your conversion within one VM (that is, not through a
networked remote repository) you can gain a lot of system resources
(memory, performance, ...) if you decide to use a Net4j IJVMConnector
instead of the usual ITCPConnector. And you can use an embedded Derby
Database to further optimize, but I guess that performance is not the
issue. If client memory gets low, it could even be better to use a
remote database server. <br>
<br>
Regards, <br>
Eike Stepper <br>
---- <br>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://wiki.eclipse.org/CDO">http://wiki.eclipse.org/CDO</a> <br>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://wiki.eclipse.org/Net4j">http://wiki.eclipse.org/Net4j</a>
<br>
<br>
<br>
<br>
<br>
<br>
Stefan Winkler schrieb: <br>
<blockquote type="cite">Hi all, <br>
<br>
I am working on a project which will have to deal with quite large
(both broad and deep) models. The models themselves are only read or
created (out of other models) but not modified themselves. <br>
<br>
Regarding the creation task: this will be something like a custom model
transformation: an algorithm will read one model and produce another. <br>
<br>
Problem is: I expect the models to be too big to fit into memory, so I
need a solution which allows only parts of models to be loaded and
these parts to be unloaded, when they are not being used anymore. In
terms of EMF, I want only parts of Resources to be loaded. <br>
<br>
Ed Merks pointed out that CDO or Teneo could help here as they use lazy
loading strategies. So far I had a look into Teneo/Hibernate and I like
the interface so far. But: While I understand that Teneo supports lazy
loading of EMF Objects, does Teneo (or Hibernate behind this) have
support to unload objects that are no longer needed? <br>
<br>
E.g. <br>
<br>
A <br>
| <br>
+--- B <br>
| <br>
+---- C <br>
| <br>
+----- D <br>
<br>
When I start at A (B,C,D are proxies, because policy says so), I access
B,C and D. Is it possible to unload C and D and make C Proxy again so
that C and D get garbage collected? <br>
<br>
<br>
If Teneo does not support this, does CDO? <br>
<br>
<br>
Thanks in advance! <br>
<br>
Cheers, <br>
<br>
Stefan <br>
<br>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
<br>
<br>
</body>
</html>
|
|
|
| Re: [Teneo/CDO] Managing huge models / unloading parts of models [message #108183 is a reply to message #108165] |
Fri, 25 January 2008 12:27 |
| Eclipse User |
|
|
|
Originally posted by: stepper.sympedia.de
This is a multi-part message in MIME format.
--------------090109090003060102050408
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit
Hi Stefan,
Comments below...
Stefan Winkler schrieb:
> Hi Eike,
>
> thanks for the code snippet.
>
> Could you again clarify a few things:
>
> When you write:
> |
>
> // Prepare the standalone infra structure (not needed when running inside Eclipse)
>
> |do you mean "not needed if the code is running in Eclipse" or "not
> needed if a (separate) server is already running in Eclipse".
I mean that, if you're running within Eclipse, you can use
org.eclipse.net4j.util.container.IPluginContainer.INSTANCE which is a
managed container that is configured through extension points. Then you
don't need to decide which factories to use. Just deploy the plugins and
use the container to retrieve the connector.
> Because: I checked out all.psf and as far as I can see the internat
> server is only started, if it is run as an own product.
> Simply installing the plugins and features into my existing Eclipse
> IDE does not start the server, right?
No and yes ;-)
The server is not started automatically just because its code is deployed.
My example code in fact doesn't start a real server node at all, but
rather starts a CDO repository in the client (programmatically).
Usually a dedicated server node is started as an Eclipse Application or
an Eclipse Product. But that's just a matter of convenience.
>
>
> |// You might want to create an IDBStore here instead if memory is an issue!
>
> |How is this done (I'm going to use MySQL as a backend database)?
> There is no StoreUtil.createIDBStore :-(
That's just because the plugin of StoreUtil has no dependency on the
plugin of the IDBStore.
You can create an IDBStore by calling
new DBStore(...)
There's currently no public API to directly create instances of DBStore
(enhancement request?).
Usually the whole repository is configured through an XML file like the
following:
<?xml version="1.0" encoding="UTF-8"?>
<cdoServer>
<acceptor type="tcp" listenAddr="0.0.0.0" port="2036">
<!--
<negotiator type="challenge" description="/temp/users.db"/>
-->
</acceptor>
<repository name="repo1">
<property name="overrideUUID"
value="1ff5d226-b1f0-40fb-aba2-0c31b38c764f"/>
<property name="supportingAudits" value="true"/>
<property name="verifyingRevisions" value="false"/>
<property name="currentLRUCapacity" value="10000"/>
<property name="revisedLRUCapacity" value="100"/>
<store type="db">
<!-- type: horizontal | vertical | <any user-contributed
type> -->
<mappingStrategy type="horizontal">
<!-- ONE_TABLE_PER_REFERENCE | ONE_TABLE_PER_CLASS |
ONE_TABLE_PER_PACKAGE | ONE_TABLE_PER_REPOSITORY | LIKE_ATTRIBUTES -->
<property name="toManyReferences"
value="ONE_TABLE_PER_REFERENCE"/>
<!-- LIKE_ATTRIBUTES | LIKE_TO_MANY_REFERENCES -->
<property name="toOneReferences" value="LIKE_ATTRIBUTES"/>
<!-- MODEL | STRATEGY -->
<property name="mappingPrecedence" value="MODEL"/>
</mappingStrategy>
<!--<dbAdapter name="derby"/>
<dataSource class="org.apache.derby.jdbc.EmbeddedDataSource"
databaseName="/temp/cdodb1"
createDatabase="create"/> -->
<dbAdapter name="derby"/>
<dataSource class="org.apache.derby.jdbc.ClientDataSource"
databaseName="cdodb1"
createDatabase="create"/>
<!--<dbAdapter name="hsqldb"/>
<dataSource
class="org.eclipse.net4j.db.hsqldb.HSQLDBDataSource"
database="jdbc:hsqldb:mem:cdodb1"
user="sa"/>-->
<!--<dbAdapter name="mysql"/>
<dataSource
class="com.mysql.jdbc.jdbc2.optional.MysqlDataSource"
url="jdbc:mysql://localhost/cdodb1"
user="sa"/>-->
</store>
</repository>
</cdoServer>
You could use code like the following to start the server:
| File configFile = OMPlatform.INSTANCE.getConfigFile("cdo-server.xml");
RepositoryConfigurator repositoryConfigurator = *new *RepositoryConfigurator(IPluginContainer.INSTANCE);
IRepository[] repositories = repositoryConfigurator.configure(configFile);
Net4jConfigurator net4jConfigurator = *new *Net4jConfigurator(IPluginContainer.INSTANCE);
IAcceptor[] acceptors = net4jConfigurator.configure(configFile);
// Work with the repositories
*for *(IAcceptor acceptor : acceptors)
{
LifecycleUtil.deactivate(acceptor);
}
*for *(IRepository repository : repositories)
{
LifecycleUtil.deactivate(repository);
}|
I know I'll have to write lots of wiki pages, but I hope this helps.
Please don't hesitate to ask further questions!
Regards,
Eike Stepper
----
http://wiki.eclipse.org/CDO
http://wiki.eclipse.org/Net4j
>
> Thanks for your support,
>
> Stefan
>
>
>
> Eike Stepper schrieb:
>> I found a nice plugin to format the java code to HTML:
>>
>> |*package *org.eclipse.emf.cdo.tests;
>>
>> *import *org.eclipse.emf.cdo.CDOSession;
>> *import *org.eclipse.emf.cdo.CDOTransaction;
>> *import *org.eclipse.emf.cdo.server.CDOServerUtil;
>> *import *org.eclipse.emf.cdo.server.IRepository;
>> *import *org.eclipse.emf.cdo.server.IStore;
>> *import *org.eclipse.emf.cdo.server.StoreUtil;
>> *import *org.eclipse.emf.cdo.util.CDOUtil;
>>
>> *import *org.eclipse.net4j.Net4jUtil;
>> *import *org.eclipse.net4j.connector.IConnector;
>> *import *org.eclipse.net4j.jvm.JVMUtil;
>> *import *org.eclipse.net4j.util.container.ContainerUtil;
>> *import *org.eclipse.net4j.util.container.IManagedContainer;
>>
>> *import *org.eclipse.emf.ecore.EObject;
>> *import *org.eclipse.emf.ecore.resource.Resource;
>>
>> *import *java.util.HashMap;
>> *import *java.util.Map;
>>
>> /**
>> * @author Eike Stepper
>> */
>> *public class *ImportXML
>> {
>> *private static final *String REPOSITORY_NAME = "repo1";
>>
>> *public static **void *main(String[] args)
>> {
>> // Prepare the standalone infra structure (not needed when running inside Eclipse)
>> IManagedContainer container = ContainerUtil.createContainer(); // Create a wiring container
>> Net4jUtil.prepareContainer(container); // Prepare the Net4j kernel
>> JVMUtil.prepareContainer(container); // Prepare the JVM transport
>> CDOServerUtil.prepareContainer(container); // Prepare the CDO server
>> CDOUtil.prepareContainer(container, *false*); // Prepare the CDO client
>>
>> // Start the transport and create a repository
>> JVMUtil.getAcceptor(container, "default"); // Start the JVM transport
>> CDOServerUtil.addRepository(container, createRepository()); // Start a CDO respository
>>
>> // Establish a communications connection and open a session with the repository
>> IConnector connector = JVMUtil.getConnector(container, "default"); // Open a JVM connection
>> CDOSession session = CDOUtil.openSession(connector, REPOSITORY_NAME, *true*);// Open a CDO session
>> CDOTransaction transaction = session.openTransaction();// Open a CDO transaction
>> Resource resource = transaction.createResource("/my/big/resource");// Create a new EMF resource
>>
>> // Work normally with the EMF resource
>> *for *(*int *i = 0; i < 100; i++)
>> {
>> EObject inputModel = getInputModel(i);
>> resource.getContents().add(inputModel);
>> transaction.commit();
>> }
>>
>> session.close();
>> }
>>
>> *private static *IRepository createRepository()
>> {
>> Map<String, String> props = *new *HashMap<String, String>();
>> props.put(IRepository.PROP_SUPPORTING_REVISION_DELTAS, "true");
>> props.put(IRepository.PROP_CURRENT_LRU_CAPACITY, "10000");
>> props.put(IRepository.PROP_REVISED_LRU_CAPACITY, "10000");
>> *return *CDOServerUtil.createRepository(REPOSITORY_NAME, createStore(), props);
>> }
>>
>> *private static *IStore createStore()
>> {
>> // You might want to create an IDBStore here instead if memory is an issue!
>> *return *StoreUtil.createMEMStore();
>> }
>>
>> *private static *EObject getInputModel(*int *i)
>> {
>> *throw new *UnsupportedOperationException("Load you model here");
>> }
>> }|
>>
>>
>> Regards,
>> Eike Stepper
>> ----
>> http://wiki.eclipse.org/CDO
>> http://wiki.eclipse.org/Net4j
>>
>>
>>
>> Eike Stepper schrieb:
>>> I've written a small example how to set up a standalone client and
>>> server within one JVM:
>>>
>>> package org.eclipse.emf.cdo.tests;
>>> import org.eclipse.emf.cdo.CDOSession;
>>> import org.eclipse.emf.cdo.CDOTransaction;
>>> import org.eclipse.emf.cdo.server.CDOServerUtil;
>>> import org.eclipse.emf.cdo.server.IRepository;
>>> import org.eclipse.emf.cdo.server.IStore;
>>> import org.eclipse.emf.cdo.server.StoreUtil;
>>> import org.eclipse.emf.cdo.util.CDOUtil;
>>> import org.eclipse.net4j.Net4jUtil;
>>> import org.eclipse.net4j.connector.IConnector;
>>> import org.eclipse.net4j.jvm.JVMUtil;
>>> import org.eclipse.net4j.util.container.ContainerUtil;
>>> import org.eclipse.net4j.util.container.IManagedContainer;
>>> import org.eclipse.emf.ecore.EObject;
>>> import org.eclipse.emf.ecore.resource.Resource;
>>> /**
>>> * @author Eike Stepper
>>> */
>>> public class ImportXML
>>> {
>>> private static final String REPOSITORY_NAME = "repo1";
>>> public static void main(String[] args)
>>> {
>>> // Prepare the standalone infra structure (not needed when
>>> running inside Eclipse)
>>> IManagedContainer container =
>>> ContainerUtil.createContainer(); // Create a wiring container
>>> Net4jUtil.prepareContainer(container); // Prepare the Net4j
>>> kernel
>>> JVMUtil.prepareContainer(container); // Prepare the JVM
>>> transport
>>> CDOServerUtil.prepareContainer(container); // Prepare the CDO
>>> server
>>> CDOUtil.prepareContainer(container, false); // Prepare the
>>> CDO client
>>> // Start the transport and create a repository
>>> JVMUtil.getAcceptor(container, "default"); // Start the JVM
>>> transport
>>> CDOServerUtil.addRepository(container, createRepository());
>>> // Start a CDO respository
>>> // Establish a communications connection and open a session
>>> with the repository
>>> IConnector connector = JVMUtil.getConnector(container,
>>> "default"); // Open a JVM connection
>>> CDOSession session = CDOUtil.openSession(connector,
>>> REPOSITORY_NAME, true);// Open a CDO session
>>> CDOTransaction transaction = session.openTransaction();//
>>> Open a CDO transaction
>>> Resource resource =
>>> transaction.createResource("/my/big/resource");// Create a new EMF
>>> resource
>>> // Work normally with the EMF resource
>>> for (int i = 0; i < 100; i++)
>>> {
>>> EObject inputModel = getInputModel(i);
>>> resource.getContents().add(inputModel);
>>> transaction.commit();
>>> }
>>> session.close();
>>> }
>>> private static IRepository createRepository()
>>> {
>>> Map<String, String> props = new HashMap<String, String>();
>>> props.put(IRepository.PROP_SUPPORTING_REVISION_DELTAS, "true");
>>> props.put(IRepository.PROP_CURRENT_LRU_CAPACITY, "10000");
>>> props.put(IRepository.PROP_REVISED_LRU_CAPACITY, "10000");
>>> return CDOServerUtil.createRepository(REPOSITORY_NAME,
>>> createStore(), props);
>>> }
>>> private static IStore createStore()
>>> {
>>> // You might want to create an IDBStore here instead if
>>> memory is an issue!
>>> return StoreUtil.createMEMStore();
>>> }
>>> private static EObject getInputModel(int i)
>>> {
>>> throw new UnsupportedOperationException("Load you model here");
>>> }
>>> }
>>>
>>> It's also possible to create and wire the various components
>>> yourself, but using a managed container with factories is easier. If
>>> running in Eclipse even the container is created and configured
>>> automatically. There's also a way to do the configuration via XML.
>>> Please tell me if you need more infos.
>>>
>>> Regards,
>>> Eike Stepper
>>> ----
>>> http://wiki.eclipse.org/CDO
>>> http://wiki.eclipse.org/Net4j
>>>
>>>
>>> Eike Stepper schrieb:
>>>> Hi Stefan,
>>>>
>>>> I don't know if Teneo can do this, Martin will want to explain that.
>>>>
>>>> CDO can do it. With CDO the modeled instances internally are not
>>>> interconnected with java reference handles. This way unneeded
>>>> instances can be completely garbage collected and transparently
>>>> reloaded later on. You don't need to write extra code for that to
>>>> function.
>>>>
>>>> If you plan to construct the output model programmatically
>>>> everything should work fine. If you plan to read your input models
>>>> for example from xml documents they should of course be small
>>>> enough to fit into memory. The resulting CDO repository resource(s)
>>>> would have to remain within the repository until you have enough
>>>> memory on client side that you can load them completely into an XML
>>>> resource to save them to a file again.
>>>>
>>>> If you plan to do your conversion within one VM (that is, not
>>>> through a networked remote repository) you can gain a lot of system
>>>> resources (memory, performance, ...) if you decide to use a Net4j
>>>> IJVMConnector instead of the usual ITCPConnector. And you can use
>>>> an embedded Derby Database to further optimize, but I guess that
>>>> performance is not the issue. If client memory gets low, it could
>>>> even be better to use a remote database server.
>>>>
>>>> Regards,
>>>> Eike Stepper
>>>> ----
>>>> http://wiki.eclipse.org/CDO
>>>> http://wiki.eclipse.org/Net4j
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Stefan Winkler schrieb:
>>>>> Hi all,
>>>>>
>>>>> I am working on a project which will have to deal with quite large
>>>>> (both broad and deep) models. The models themselves are only read
>>>>> or created (out of other models) but not modified themselves.
>>>>>
>>>>> Regarding the creation task: this will be something like a custom
>>>>> model transformation: an algorithm will read one model and produce
>>>>> another.
>>>>>
>>>>> Problem is: I expect the models to be too big to fit into memory,
>>>>> so I need a solution which allows only parts of models to be
>>>>> loaded and these parts to be unloaded, when they are not being
>>>>> used anymore. In terms of EMF, I want only parts of Resources to
>>>>> be loaded.
>>>>>
>>>>> Ed Merks pointed out that CDO or Teneo could help here as they use
>>>>> lazy loading strategies. So far I had a look into Teneo/Hibernate
>>>>> and I like the interface so far. But: While I understand that
>>>>> Teneo supports lazy loading of EMF Objects, does Teneo (or
>>>>> Hibernate behind this) have support to unload objects that are no
>>>>> longer needed?
>>>>>
>>>>> E.g.
>>>>>
>>>>> A
>>>>> |
>>>>> +--- B
>>>>> |
>>>>> +---- C
>>>>> |
>>>>> +----- D
>>>>>
>>>>> When I start at A (B,C,D are proxies, because policy says so), I
>>>>> access B,C and D. Is it possible to unload C and D and make C
>>>>> Proxy again so that C and D get garbage collected?
>>>>>
>>>>>
>>>>> If Teneo does not support this, does CDO?
>>>>>
>>>>>
>>>>> Thanks in advance!
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Stefan
>>>>>
>
>
--------------090109090003060102050408
Content-Type: text/html; charset=ISO-8859-15
Content-Transfer-Encoding: 8bit
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Stefan,<br>
<br>
Comments below...<br>
<br>
<br>
Stefan Winkler schrieb:
<blockquote cite="mid:fncfah$p7e$1@build.eclipse.org" type="cite">
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
<title></title>
Hi Eike,<br>
<br>
thanks for the code snippet.<br>
<br>
Could you again clarify a few things:<br>
<br>
When you write:<br>
<code><font color="#ffffff"><br>
|
|
|
| [CDO] Repository inside Eclipse [message #108372 is a reply to message #108183] |
Sat, 26 January 2008 13:44 |
Stefan Winkler
Messages: 307
Registered: July 2009
Location: Germany |
Senior Member |
|
|
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Eike,<br>
<br>
ok, this is what I'm doing now and it seems to work (Thunderbird does
not like to insert Java2Html code from clipboard :-( ):<br>
<pre>public static void run() throws Exception {
IManagedContainer container = IPluginContainer.INSTANCE;
JVMUtil.getAcceptor(container, "default");
URL url = Activator.getDefault().getBundle().getEntry("/configuration/cdo-config.xml ");
url = FileLocator.toFileURL(url);
IRepository r = new RepositoryConfigurator(container).configure(new File(url.toURI()))[0];
String repositoryName = r.getName();
CDOServerUtil.addRepository(container, r);
IConnector connector = JVMUtil.getConnector(container, "default");
CDOSession session = CDOUtil.openSession(connector, repositoryName, true);
CDOTransaction transaction = session.openTransaction();
Resource resource = transaction.createResource("/my/big/resource");
for (int i = 0; i < 100; i++) {
EObject inputModel = getInputModel(i);
resource.getContents().add(inputModel);
transaction.commit();
}
session.close();
LifecycleUtil.deactivate(r);
}
</pre>
This seems to be doing what I want, so far. <br>
<br>
Now some advanced questions:<br>
<br>
1.<br>
Is your debug facility configurable? I only get one logging message
stating that a legacy-something is not available. I'd like to see a bit
more of what is going on behind the scenes...<br>
<br>
2.<br>
Basically my application connsists of two parts. One is the
computation-intensive part (reading a huge model and generating the new
one).<br>
The other one is a (rather) small model, which is kept in a viewer and
which the user can select and edit.<br>
<br>
So regarding transactions, the first part seems easy: start a
transaction, read the one model, create the other, commit transaction,
fine.<br>
But how do I deal with the second case - or more generally: what
happens to a resource, when the transaction is committed?<br>
<br>
Can I do something like<br>
<pre> // in the view initialization:
CDOView view = session.openView();
Resource r = view.getResource("/some/path");
editingDomain.getResourceSet().getResources().add(r);
[...]
// when the user creates an element in the model (Action handler) -
// myEObject is contained in Resource r
CDOTransaction transaction = session.openTransaction(editingDomain.getResourceSet());
myEObject.getChildren().add(newObject);
transaction.commit();
</pre>
<br>
3. <br>
How do the resource paths map? Are these just some kind of "ID" for
resources so that <br>
transaction.createResource("/some/path");<br>
always returns equivalent resources and<br>
transaction.createResource("/some/path/other");<br>
returns a different one which has nothing to do with the first one?<br>
<br>
If so, do inter-resource-references in the same repository work?<br>
<br>
4.<br>
Does a session time out? Or can I start a session when the application
starts up and close it, when it shuts down?<br>
<br>
Some commennts below ...<br>
<br>
<br>
<br>
<br>
Eike Stepper schrieb::
<blockquote cite="mid:fnckjs$bua$1@build.eclipse.org" type="cite">
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
<blockquote cite="mid:fncfah$p7e$1@build.eclipse.org" type="cite">
<code><font color="#3f7f5f"> // You might want to create an IDBStore here instea d if memory is an issue! <br>
<br>
</font></code>How is this done (I'm going to use MySQL as a backend
database)?<br>
There is no StoreUtil.createIDBStore :-(<br>
</blockquote>
That's just because the plugin of StoreUtil has no dependency on the
plugin of the IDBStore.<br>
You can create an IDBStore by calling <br>
<br>
new DBStore(...)<br>
<br>
There's currently no public API to directly create instances of DBStore
(enhancement request?).<br>
</blockquote>
Filed: <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=216668"
title="NEW - Add public API to support creation of IDBStore programmatically">Bug
216668</a>
<blockquote cite="mid:fnckjs$bua$1@build.eclipse.org" type="cite">I
know I'll have to write lots of wiki pages, but I hope this helps.
Please don't hesitate to ask further questions!<br>
</blockquote>
You'll regret your last sentence ;-))<br>
<br>
Cheers,<br>
Stefan<br>
<br>
</body>
</html>
|
|
|
| Re: [CDO] Repository inside Eclipse [message #108403 is a reply to message #108372] |
Sat, 26 January 2008 13:51 |
| Eclipse User |
|
|
|
Originally posted by: merks.ca.ibm.com
This is a multi-part message in MIME format.
--------------010602050003010906040501
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit
Stefann,
With regard to Java2Html and Thunderbird, the trick is to use
"Insert->HTML...", paste the contents in there, and then hit OK.
Stefann Winkler wrote:
> Hi Eike,
>
> ok, this is what I'm doing now and it seems to work (Thunderbird does
> not like to insert Java2Html code from clipboard :-( ):
> public static void run() throws Exception {
>
> IManagedContainer container = IPluginContainer.INSTANCE;
>
> JVMUtil.getAcceptor(container, "default");
>
> URL url = Activator.getDefault().getBundle().getEntry("/configuration/cdo-config.xml ");
> url = FileLocator.toFileURL(url);
>
> IRepository r = new RepositoryConfigurator(container).configure(new File(url.toURI()))[0];
> String repositoryName = r.getName();
> CDOServerUtil.addRepository(container, r);
>
> IConnector connector = JVMUtil.getConnector(container, "default");
> CDOSession session = CDOUtil.openSession(connector, repositoryName, true);
> CDOTransaction transaction = session.openTransaction();
>
> Resource resource = transaction.createResource("/my/big/resource");
>
> for (int i = 0; i < 100; i++) {
> EObject inputModel = getInputModel(i);
> resource.getContents().add(inputModel);
> transaction.commit();
> }
> session.close();
> LifecycleUtil.deactivate(r);
> }
>
>
> This seems to be doing what I want, so far.
>
> Now some advanced questions:
>
> 1.
> Is your debug facility configurable? I only get one logging message
> stating that a legacy-something is not available. I'd like to see a
> bit more of what is going on behind the scenes...
>
> 2.
> Basically my application connsists of two parts. One is the
> computation-intensive part (reading a huge model and generating the
> new one).
> The other one is a (rather) small model, which is kept in a viewer and
> which the user can select and edit.
>
> So regarding transactions, the first part seems easy: start a
> transaction, read the one model, create the other, commit transaction,
> fine.
> But how do I deal with the second case - or more generally: what
> happens to a resource, when the transaction is committed?
>
> Can I do something like
> // in the view initialization:
> CDOView view = session.openView();
> Resource r = view.getResource("/some/path");
> editingDomain.getResourceSet().getResources().add(r);
>
> [...]
>
> // when the user creates an element in the model (Action handler) -
> // myEObject is contained in Resource r
> CDOTransaction transaction = session.openTransaction(editingDomain.getResourceSet());
> myEObject.getChildren().add(newObject);
> transaction.commit();
>
>
> 3.
> How do the resource paths map? Are these just some kind of "ID" for
> resources so that
> transaction.createResource("/some/path");
> always returns equivalent resources and
> transaction.createResource("/some/path/other");
> returns a different one which has nothing to do with the first one?
>
> If so, do inter-resource-references in the same repository work?
>
> 4.
> Does a session time out? Or can I start a session when the application
> starts up and close it, when it shuts down?
>
> Some commennts below ...
>
>
>
>
> Eike Stepper schrieb::
>>>
>>> |// You might want to create an IDBStore here instead if memory is an issue!
>>>
>>> |How is this done (I'm going to use MySQL as a backend database)?
>>> There is no StoreUtil.createIDBStore :-(
>> That's just because the plugin of StoreUtil has no dependency on the
>> plugin of the IDBStore.
>> You can create an IDBStore by calling
>>
>> new DBStore(...)
>>
>> There's currently no public API to directly create instances of
>> DBStore (enhancement request?).
> Filed: Bug 216668 <https://bugs.eclipse.org/bugs/show_bug.cgi?id=216668>
>> I know I'll have to write lots of wiki pages, but I hope this helps.
>> Please don't hesitate to ask further questions!
> You'll regret your last sentence ;-))
>
> Cheers,
> Stefan
>
--------------010602050003010906040501
Content-Type: text/html; charset=ISO-8859-15
Content-Transfer-Encoding: 8bit
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Stefann,<br>
<br>
With regard to Java2Html and Thunderbird, the trick is to use
"Insert->HTML...", paste the contents in there, and then hit OK.<br>
<br>
Stefann Winkler wrote:
<blockquote cite="mid:fnfdgp$vo1$1@build.eclipse.org" type="cite">
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
<title></title>
Hi Eike,<br>
<br>
ok, this is what I'm doing now and it seems to work (Thunderbird does
not like to insert Java2Html code from clipboard :-( ):<br>
<pre>public static void run() throws Exception {
IManagedContainer container = IPluginContainer.INSTANCE;
JVMUtil.getAcceptor(container, "default");
URL url = Activator.getDefault().getBundle().getEntry("/configuration/cdo-config.xml ");
url = FileLocator.toFileURL(url);
IRepository r = new RepositoryConfigurator(container).configure(new File(url.toURI()))[0];
String repositoryName = r.getName();
CDOServerUtil.addRepository(container, r);
IConnector connector = JVMUtil.getConnector(container, "default");
CDOSession session = CDOUtil.openSession(connector, repositoryName, true);
CDOTransaction transaction = session.openTransaction();
Resource resource = transaction.createResource("/my/big/resource");
for (int i = 0; i < 100; i++) {
EObject inputModel = getInputModel(i);
resource.getContents().add(inputModel);
transaction.commit();
}
session.close();
LifecycleUtil.deactivate(r);
}
</pre>
This seems to be doing what I want, so far. <br>
<br>
Now some advanced questions:<br>
<br>
1.<br>
Is your debug facility configurable? I only get one logging message
stating that a legacy-something is not available. I'd like to see a bit
more of what is going on behind the scenes...<br>
<br>
2.<br>
Basically my application connsists of two parts. One is the
computation-intensive part (reading a huge model and generating the new
one).<br>
The other one is a (rather) small model, which is kept in a viewer and
which the user can select and edit.<br>
<br>
So regarding transactions, the first part seems easy: start a
transaction, read the one model, create the other, commit transaction,
fine.<br>
But how do I deal with the second case - or more generally: what
happens to a resource, when the transaction is committed?<br>
<br>
Can I do something like<br>
<pre> // in the view initialization:
CDOView view = session.openView();
Resource r = view.getResource("/some/path");
editingDomain.getResourceSet().getResources().add(r);
[...]
// when the user creates an element in the model (Action handler) -
// myEObject is contained in Resource r
CDOTransaction transaction = session.openTransaction(editingDomain.getResourceSet());
myEObject.getChildren().add(newObject);
transaction.commit();
</pre>
<br>
3. <br>
How do the resource paths map? Are these just some kind of "ID" for
resources so that <br>
transaction.createResource("/some/path");<br>
always returns equivalent resources and<br>
transaction.createResource("/some/path/other");<br>
returns a different one which has nothing to do with the first one?<br>
<br>
If so, do inter-resource-references in the same repository work?<br>
<br>
4.<br>
Does a session time out? Or can I start a session when the application
starts up and close it, when it shuts down?<br>
<br>
Some commennts below ...<br>
<br>
<br>
<br>
<br>
Eike Stepper schrieb::
<blockquote cite="mid:fnckjs$bua$1@build.eclipse.org" type="cite">
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
<blockquote cite="mid:fncfah$p7e$1@build.eclipse.org" type="cite">
|
|
|
| Re: [CDO] Repository inside Eclipse [message #108420 is a reply to message #108372] |
Sat, 26 January 2008 14:56 |
| Eclipse User |
|
|
|
Originally posted by: stepper.sympedia.de
This is a multi-part message in MIME format.
--------------060606020400040400010700
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit
Hi Stefan,
Comments below...
Stefann Winkler schrieb:
> Hi Eike,
>
> ok, this is what I'm doing now and it seems to work (Thunderbird does
> not like to insert Java2Html code from clipboard :-( ):
> public static void run() throws Exception {
>
> IManagedContainer container = IPluginContainer.INSTANCE;
>
> JVMUtil.getAcceptor(container, "default");
>
> URL url = Activator.getDefault().getBundle().getEntry("/configuration/cdo-config.xml ");
> url = FileLocator.toFileURL(url);
>
> IRepository r = new RepositoryConfigurator(container).configure(new File(url.toURI()))[0];
> String repositoryName = r.getName();
> CDOServerUtil.addRepository(container, r);
>
> IConnector connector = JVMUtil.getConnector(container, "default");
> CDOSession session = CDOUtil.openSession(connector, repositoryName, true);
> CDOTransaction transaction = session.openTransaction();
>
> Resource resource = transaction.createResource("/my/big/resource");
>
> for (int i = 0; i < 100; i++) {
> EObject inputModel = getInputModel(i);
> resource.getContents().add(inputModel);
> transaction.commit();
>
It could eventually be faster to commit the transaction only once after
the for loop.
> }
> session.close();
> LifecycleUtil.deactivate(r);
> }
>
Don't forget to close() or LifecycleUtil.deactivate() the connector and
later the acceptor.
>
> This seems to be doing what I want, so far.
Yes, your code looks fine.
>
> Now some advanced questions:
>
> 1.
> Is your debug facility configurable? I only get one logging message
> stating that a legacy-something is not available. I'd like to see a
> bit more of what is going on behind the scenes...
The OM (Operations&Maintenance) tracing facility integrates with the
Eclipse/OSGi Tracing facility if you run within Eclipse/OSGi. You can
use the Tracing tab in the launch config to configure the details. Or,
if you run headless, you can use the .options file:
http://wiki.eclipse.org/FAQ_How_do_I_use_the_platform_debug_ tracing_facility%3F#Turning_on_debug_tracing
Each Net4j and CDO bundle has an own .options file which shows the
possible tracing entities. The whole OM tracing facility can also work
standalone without change.
>
> 2.
> Basically my application connsists of two parts. One is the
> computation-intensive part (reading a huge model and generating the
> new one).
> The other one is a (rather) small model, which is kept in a viewer and
> which the user can select and edit.
>
> So regarding transactions, the first part seems easy: start a
> transaction, read the one model, create the other, commit transaction,
> fine.
> But how do I deal with the second case - or more generally: what
> happens to a resource, when the transaction is committed?
>
> Can I do something like
> // in the view initialization:
> CDOView view = session.openView();
> Resource r = view.getResource("/some/path");
> editingDomain.getResourceSet().getResources().add(r);
>
The last statement seems wrong to me. because it will detach the
resource from the ResourceSet that is internally created on
session.openView(). See my explanation below...
>
> [...]
>
> // when the user creates an element in the model (Action handler) -
> // myEObject is contained in Resource r
> CDOTransaction transaction = session.openTransaction(editingDomain.getResourceSet());
> myEObject.getChildren().add(newObject);
> transaction.commit();
>
Some explanation:
In CDO an EMF ResourceSet is associated with a CDOView. CDOTransaction
is a special CDOView. You could eventually say CDOView is a read-only
transaction. Each ResourceSet can only have a single CDOView (or
CDOTransaction) associated at any point in time. If you call
session.openTransaction() a new ResourceSet is created internally. If
you call session.openTransaction(myResourceSet) a given ResourceSet is
used for that transaction. But only if the given ResourceSet doesn't
already have another view/transaction associated. BTW. this association
happens internally via a ResourceSet adapter.
If there is a chance that the model will be modified, you should open a
transaction from the beginning and use it for reads and writes later on.
If the model is nonetheless *not* changed in a transaction it doesn't
cost you much to have a transaction instead of a read-only view.
But even if you open several views and transactions on the same session,
they will share all the object state as long as they're not modified.
Model state is stored/managed in the session itself. Only change deltas
are stored in the transactions.
>
> 3.
> How do the resource paths map? Are these just some kind of "ID" for
> resources so that
> transaction.createResource("/some/path");
> always returns equivalent resources and
> transaction.createResource("/some/path/other");
> returns a different one which has nothing to do with the first one?
Yes. In each CDOObject has a CDOID (long integer number) which is
automatically assigned and can not be changed. The CDOID of an object is
unique within the associated repository. Since a CDOResource is also an
EObject and a CDOObject it has such a CDOID, too.
The CDOResources are the entry points into the model tree of the
repository and the resource paths are a more meaningful means (than
CDOIDs) to enter this tree.
>
> If so, do inter-resource-references in the same repository work?
Yes, without any exemption. Including inter-resource-moves.
>
> 4.
> Does a session time out? Or can I start a session when the application
> starts up and close it, when it shuts down?
It doesn't time out unless the underlying connector/channel doesn't time
out. Both the TCPConnector and JVMConnector don't time out automatically.
>
> Some commennts below ...
>
>
>
>
> Eike Stepper schrieb::
>>>
>>> |// You might want to create an IDBStore here instead if memory is an issue!
>>>
>>> |How is this done (I'm going to use MySQL as a backend database)?
>>> There is no StoreUtil.createIDBStore :-(
>> That's just because the plugin of StoreUtil has no dependency on the
>> plugin of the IDBStore.
>> You can create an IDBStore by calling
>>
>> new DBStore(...)
>>
>> There's currently no public API to directly create instances of
>> DBStore (enhancement request?).
> Filed: Bug 216668 <https://bugs.eclipse.org/bugs/show_bug.cgi?id=216668>
Thx ;-)
>> I know I'll have to write lots of wiki pages, but I hope this helps.
>> Please don't hesitate to ask further questions!
> You'll regret your last sentence ;-))
I bet that not!! ;-)
Regards,
Eike Stepper
----
http://wiki.eclipse.org/CDO
http://wiki.eclipse.org/Net4j
--------------060606020400040400010700
Content-Type: text/html; charset=ISO-8859-15
Content-Transfer-Encoding: 8bit
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Stefan,<br>
<br>
Comments below...<br>
<br>
<br>
Stefann Winkler schrieb:
<blockquote cite="mid:fnfdgp$vo1$1@build.eclipse.org" type="cite">
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
<title></title>
Hi Eike,<br>
<br>
ok, this is what I'm doing now and it seems to work (Thunderbird does
not like to insert Java2Html code from clipboard :-( ):<br>
<pre>public static void run() throws Exception {
IManagedContainer container = IPluginContainer.INSTANCE;
JVMUtil.getAcceptor(container, "default");
URL url = Activator.getDefault().getBundle().getEntry("/configuration/cdo-config.xml ");
url = FileLocator.toFileURL(url);
IRepository r = new RepositoryConfigurator(container).configure(new File(url.toURI()))[0];
String repositoryName = r.getName();
CDOServerUtil.addRepository(container, r);
IConnector connector = JVMUtil.getConnector(container, "default");
CDOSession session = CDOUtil.openSession(connector, repositoryName, true);
CDOTransaction transaction = session.openTransaction();
Resource resource = transaction.createResource("/my/big/resource");
for (int i = 0; i < 100; i++) {
EObject inputModel = getInputModel(i);
resource.getContents().add(inputModel);
transaction.commit();
</pre>
</blockquote>
It could eventually be faster to commit the transaction only once after
the for loop.<br>
<br>
<blockquote cite="mid:fnfdgp$vo1$1@build.eclipse.org" type="cite">
<pre> }
session.close();
LifecycleUtil.deactivate(r);
}
</pre>
</blockquote>
Don't forget to close() or LifecycleUtil.deactivate() the connector and
later the acceptor.<br>
<br>
<blockquote cite="mid:fnfdgp$vo1$1@build.eclipse.org" type="cite">
<pre>
</pre>
This seems to be doing what I want, so far. <br>
</blockquote>
Yes, your code looks fine.<br>
<br>
<blockquote cite="mid:fnfdgp$vo1$1@build.eclipse.org" type="cite"><br>
Now some advanced questions:<br>
<br>
1.<br>
Is your debug facility configurable? I only get one logging message
stating that a legacy-something is not available. I'd like to see a bit
more of what is going on behind the scenes...<br>
</blockquote>
The OM (Operations&Maintenance) tracing facility integrates with
the Eclipse/OSGi Tracing facility if you run within Eclipse/OSGi. You
can use the Tracing tab in the launch config to configure the details.
Or, if you run headless, you can use the .options file:<br>
<pre id="line198"><span class="attribute-value"> <a class="moz-txt-link-freetext" href=" http://wiki.eclipse.org/FAQ_How_do_I_use_the_platform_debug_ tracing_facility%3F#Turning_on_debug_tracing"> http://wiki.eclipse.org/FAQ_How_do_I_use_the_platform_debug_ tracing_facility%3F#Turning_on_debug_tracing</a>
</span></pre>
Each Net4j and CDO bundle has an own .options file which shows the
possible tracing entities. The whole OM tracing facility can also work
standalone without change.<br>
<br>
<blockquote cite="mid:fnfdgp$vo1$1@build.eclipse.org" type="cite"><br>
2.<br>
Basically my application connsists of two parts. One is the
computation-intensive part (reading a huge model and generating the new
one).<br>
The other one is a (rather) small model, which is kept in a viewer and
which the user can select and edit.<br>
<br>
So regarding transactions, the first part seems easy: start a
transaction, read the one model, create the other, commit transaction,
fine.<br>
But how do I deal with the second case - or more generally: what
happens to a resource, when the transaction is committed?<br>
<br>
Can I do something like<br>
<pre> // in the view initialization:
CDOView view = session.openView();
Resource r = view.getResource("/some/path");
editingDomain.getResourceSet().getResources().add(r);
</pre>
</blockquote>
The last statement seems wrong to me. because it will detach the
resource from the ResourceSet that is internally created on
session.openView(). See my explanation below...<br>
<br>
<blockquote cite="mid:fnfdgp$vo1$1@build.eclipse.org" type="cite">
<pre>
[...]
// when the user creates an element in the model (Action handler) -
// myEObject is contained in Resource r
CDOTransaction transaction = session.openTransaction(editingDomain.getResourceSet());
myEObject.getChildren().add(newObject);
transaction.commit();
</pre>
</blockquote>
Some explanation:<br>
In CDO an EMF ResourceSet is associated with a CDOView. CDOTransaction
is a special CDOView. You could eventually say CDOView is a read-only
transaction. Each ResourceSet can only have a single CDOView (or
CDOTransaction) associated at any point in time. If you call
session.openTransaction() a new ResourceSet is created internally. If
you call session.openTransaction(myResourceSet) a given ResourceSet is
used for that transaction. But only if the given ResourceSet doesn't
already have another view/transaction associated. BTW. this association
happens internally via a ResourceSet adapter.<br>
<br>
If there is a chance that the model will be modified, you should open a
transaction from the beginning and use it for reads and writes later
on. If the model is nonetheless *not* changed in a transaction it
doesn't cost you much to have a transaction instead of a read-only
view. <br>
<br>
But even if you open several views and transactions on the same
session, they will share all the object state as long as they're not
modified. Model state is stored/managed in the session itself. Only
change deltas are stored in the transactions.<br>
<br>
<blockquote cite="mid:fnfdgp$vo1$1@build.eclipse.org" type="cite"><br>
3. <br>
How do the resource paths map? Are these just some kind of "ID" for
resources so that <br>
transaction.createResource("/some/path");<br>
always returns equivalent resources and<br>
transaction.createResource("/some/path/other");<br>
returns a different one which has nothing to do with the first one?<br>
</blockquote>
Yes. In each CDOObject has a CDOID (long integer number) which is
automatically assigned and can not be changed. The CDOID of an object
is unique within the associated repository. Since a CDOResource is also
an EObject and a CDOObject it has such a CDOID, too.<br>
<br>
The CDOResources are the entry points into the model tree of the
repository and the resource paths are a more meaningful means (than
CDOIDs) to enter this tree.<br>
<br>
<blockquote cite="mid:fnfdgp$vo1$1@build.eclipse.org" type="cite"><br>
If so, do inter-resource-references in the same repository work?<br>
</blockquote>
Yes, without any exemption. Including inter-resource-moves.<br>
<br>
<blockquote cite="mid:fnfdgp$vo1$1@build.eclipse.org" type="cite"><br>
4.<br>
Does a session time out? Or can I start a session when the application
starts up and close it, when it shuts down?<br>
</blockquote>
It doesn't time out unless the underlying connector/channel doesn't
time out. Both the TCPConnector and JVMConnector don't time out
automatically.<br>
<br>
<blockquote cite="mid:fnfdgp$vo1$1@build.eclipse.org" type="cite"><br>
Some commennts below ...<br>
<br>
<br>
<br>
<br>
Eike Stepper schrieb::
<blockquote cite="mid:fnckjs$bua$1@build.eclipse.org" type="cite">
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
<blockquote cite="mid:fncfah$p7e$1@build.eclipse.org" type="cite">
|
|
|
| Re: [CDO] Repository inside Eclipse [message #108435 is a reply to message #108403] |
Sat, 26 January 2008 14:57 |
| Eclipse User |
|
|
|
Originally posted by: stepper.sympedia.de
This is a multi-part message in MIME format.
--------------030503030205020509070901
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit
Ed Merks schrieb:
> Stefann,
>
> With regard to Java2Html and Thunderbird, the trick is to use
> "Insert->HTML...", paste the contents in there, and then hit OK.
Yes, it's a really cute and small tool ;-)
Regards,
Eike Stepper
----
http://wiki.eclipse.org/CDO
http://wiki.eclipse.org/Net4j
>
> Stefann Winkler wrote:
>> Hi Eike,
>>
>> ok, this is what I'm doing now and it seems to work (Thunderbird does
>> not like to insert Java2Html code from clipboard :-( ):
>> public static void run() throws Exception {
>>
>> IManagedContainer container = IPluginContainer.INSTANCE;
>>
>> JVMUtil.getAcceptor(container, "default");
>>
>> URL url = Activator.getDefault().getBundle().getEntry("/configuration/cdo-config.xml ");
>> url = FileLocator.toFileURL(url);
>>
>> IRepository r = new RepositoryConfigurator(container).configure(new File(url.toURI()))[0];
>> String repositoryName = r.getName();
>> CDOServerUtil.addRepository(container, r);
>>
>> IConnector connector = JVMUtil.getConnector(container, "default");
>> CDOSession session = CDOUtil.openSession(connector, repositoryName, true);
>> CDOTransaction transaction = session.openTransaction();
>>
>> Resource resource = transaction.createResource("/my/big/resource");
>>
>> for (int i = 0; i < 100; i++) {
>> EObject inputModel = getInputModel(i);
>> resource.getContents().add(inputModel);
>> transaction.commit();
>> }
>> session.close();
>> LifecycleUtil.deactivate(r);
>> }
>>
>>
>> This seems to be doing what I want, so far.
>>
>> Now some advanced questions:
>>
>> 1.
>> Is your debug facility configurable? I only get one logging message
>> stating that a legacy-something is not available. I'd like to see a
>> bit more of what is going on behind the scenes...
>>
>> 2.
>> Basically my application connsists of two parts. One is the
>> computation-intensive part (reading a huge model and generating the
>> new one).
>> The other one is a (rather) small model, which is kept in a viewer
>> and which the user can select and edit.
>>
>> So regarding transactions, the first part seems easy: start a
>> transaction, read the one model, create the other, commit
>> transaction, fine.
>> But how do I deal with the second case - or more generally: what
>> happens to a resource, when the transaction is committed?
>>
>> Can I do something like
>> // in the view initialization:
>> CDOView view = session.openView();
>> Resource r = view.getResource("/some/path");
>> editingDomain.getResourceSet().getResources().add(r);
>>
>> [...]
>>
>> // when the user creates an element in the model (Action handler) -
>> // myEObject is contained in Resource r
>> CDOTransaction transaction = session.openTransaction(editingDomain.getResourceSet());
>> myEObject.getChildren().add(newObject);
>> transaction.commit();
>>
>>
>> 3.
>> How do the resource paths map? Are these just some kind of "ID" for
>> resources so that
>> transaction.createResource("/some/path");
>> always returns equivalent resources and
>> transaction.createResource("/some/path/other");
>> returns a different one which has nothing to do with the first one?
>>
>> If so, do inter-resource-references in the same repository work?
>>
>> 4.
>> Does a session time out? Or can I start a session when the
>> application starts up and close it, when it shuts down?
>>
>> Some commennts below ...
>>
>>
>>
>>
>> Eike Stepper schrieb::
>>>>
>>>> |// You might want to create an IDBStore here instead if memory is an issue!
>>>>
>>>> |How is this done (I'm going to use MySQL as a backend database)?
>>>> There is no StoreUtil.createIDBStore :-(
>>> That's just because the plugin of StoreUtil has no dependency on the
>>> plugin of the IDBStore.
>>> You can create an IDBStore by calling
>>>
>>> new DBStore(...)
>>>
>>> There's currently no public API to directly create instances of
>>> DBStore (enhancement request?).
>> Filed: Bug 216668 <https://bugs.eclipse.org/bugs/show_bug.cgi?id=216668>
>>> I know I'll have to write lots of wiki pages, but I hope this helps.
>>> Please don't hesitate to ask further questions!
>> You'll regret your last sentence ;-))
>>
>> Cheers,
>> Stefan
>>
>
--------------030503030205020509070901
Content-Type: text/html; charset=ISO-8859-15
Content-Transfer-Encoding: 8bit
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Ed Merks schrieb:
<blockquote cite="mid:fnfdtt$105$2@build.eclipse.org" type="cite">
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
Stefann,<br>
<br>
With regard to Java2Html and Thunderbird, the trick is to use
"Insert->HTML...", paste the contents in there, and then hit OK.<br>
</blockquote>
Yes, it's a really cute and small tool ;-)<br>
<br>
Regards,<br>
Eike Stepper<br>
----<br>
<a class="moz-txt-link-freetext" href="http://wiki.eclipse.org/CDO">http://wiki.eclipse.org/CDO</a><br>
<a class="moz-txt-link-freetext" href="http://wiki.eclipse.org/Net4j">http://wiki.eclipse.org/Net4j</a><br>
<br>
<br>
<blockquote cite="mid:fnfdtt$105$2@build.eclipse.org" type="cite"><br>
Stefann Winkler wrote:
<blockquote cite="mid:fnfdgp$vo1$1@build.eclipse.org" type="cite">
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
<title></title>
Hi Eike,<br>
<br>
ok, this is what I'm doing now and it seems to work (Thunderbird does
not like to insert Java2Html code from clipboard :-( ):<br>
<pre>public static void run() throws Exception {
IManagedContainer container = IPluginContainer.INSTANCE;
JVMUtil.getAcceptor(container, "default");
URL url = Activator.getDefault().getBundle().getEntry("/configuration/cdo-config.xml ");
url = FileLocator.toFileURL(url);
IRepository r = new RepositoryConfigurator(container).configure(new File(url.toURI()))[0];
String repositoryName = r.getName();
CDOServerUtil.addRepository(container, r);
IConnector connector = JVMUtil.getConnector(container, "default");
CDOSession session = CDOUtil.openSession(connector, repositoryName, true);
CDOTransaction transaction = session.openTransaction();
Resource resource = transaction.createResource("/my/big/resource");
for (int i = 0; i < 100; i++) {
EObject inputModel = getInputModel(i);
resource.getContents().add(inputModel);
transaction.commit();
}
session.close();
LifecycleUtil.deactivate(r);
}
</pre>
This seems to be doing what I want, so far. <br>
<br>
Now some advanced questions:<br>
<br>
1.<br>
Is your debug facility configurable? I only get one logging message
stating that a legacy-something is not available. I'd like to see a bit
more of what is going on behind the scenes...<br>
<br>
2.<br>
Basically my application connsists of two parts. One is the
computation-intensive part (reading a huge model and generating the new
one).<br>
The other one is a (rather) small model, which is kept in a viewer and
which the user can select and edit.<br>
<br>
So regarding transactions, the first part seems easy: start a
transaction, read the one model, create the other, commit transaction,
fine.<br>
But how do I deal with the second case - or more generally: what
happens to a resource, when the transaction is committed?<br>
<br>
Can I do something like<br>
<pre> // in the view initialization:
CDOView view = session.openView();
Resource r = view.getResource("/some/path");
editingDomain.getResourceSet().getResources().add(r);
[...]
// when the user creates an element in the model (Action handler) -
// myEObject is contained in Resource r
CDOTransaction transaction = session.openTransaction(editingDomain.getResourceSet());
myEObject.getChildren().add(newObject);
transaction.commit();
</pre>
<br>
3. <br>
How do the resource paths map? Are these just some kind of "ID" for
resources so that <br>
transaction.createResource("/some/path");<br>
always returns equivalent resources and<br>
transaction.createResource("/some/path/other");<br>
returns a different one which has nothing to do with the first one?<br>
<br>
If so, do inter-resource-references in the same repository work?<br>
<br>
4.<br>
Does a session time out? Or can I start a session when the application
starts up and close it, when it shuts down?<br>
<br>
Some commennts below ...<br>
<br>
<br>
<br>
<br>
Eike Stepper schrieb::
<blockquote cite="mid:fnckjs$bua$1@build.eclipse.org" type="cite">
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
<blockquote cite="mid:fncfah$p7e$1@build.eclipse.org" type="cite">
|
|
|
| Re: [CDO] Repository inside Eclipse [message #108505 is a reply to message #108372] |
Sun, 27 January 2008 09:56 |
| Eclipse User |
|
|
|
Originally posted by: stepper.sympedia.de
This is a multi-part message in MIME format.
--------------040406070502030204040607
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit
Hi Stefan,
I forgot to comment on the trace output with the "legacy-something".
Originally CDO was designed to operate on models that are generated
dedicatedly for CDO, i.e. their genmodel has been modified as per
description in http://wiki.eclipse.org/Preparing_EMF_Models_for_CDO . I
call these models "native" (with respect to CDO).
For various reasons there are models that can't be native. For instance
their genmodel is not available or some of their genmodel properties
can't be set to CDO native values (Root Extends Class, ...). The Ecore
meta meta model can't be regenerated natively because of resulting
circular dependencies between EMF and CDO. I have invented an AspectJ
based approach to make CDO operate on these models as well. I call these
woven models "legacy" (with respect to CDO). The CDO Weaver can be used
to convert most generated models to legacy models. Alll the
depenpendencies of these models have to be converted to legacy models as
well. The Ecore meta meta model is a special dependency of all other
models and if Ecore itself is not converted to a legacy model I say the
"legacy-system is unavailable". It might be less confusing if this info
message would only be logged if there are other legacy models that
needed a converted Ecore model but it is not very fast to determine this
case.
BTW. native models are to be preferred over legacy models whenever
possible because they're much more efficient!
Regards,
Eike Stepper
----
http://wiki.eclipse.org/CDO
http://wiki.eclipse.org/Net4j
Stefann Winkler schrieb:
> Hi Eike,
>
> ok, this is what I'm doing now and it seems to work (Thunderbird does
> not like to insert Java2Html code from clipboard :-( ):
> public static void run() throws Exception {
>
> IManagedContainer container = IPluginContainer.INSTANCE;
>
> JVMUtil.getAcceptor(container, "default");
>
> URL url = Activator.getDefault().getBundle().getEntry("/configuration/cdo-config.xml ");
> url = FileLocator.toFileURL(url);
>
> IRepository r = new RepositoryConfigurator(container).configure(new File(url.toURI()))[0];
> String repositoryName = r.getName();
> CDOServerUtil.addRepository(container, r);
>
> IConnector connector = JVMUtil.getConnector(container, "default");
> CDOSession session = CDOUtil.openSession(connector, repositoryName, true);
> CDOTransaction transaction = session.openTransaction();
>
> Resource resource = transaction.createResource("/my/big/resource");
>
> for (int i = 0; i < 100; i++) {
> EObject inputModel = getInputModel(i);
> resource.getContents().add(inputModel);
> transaction.commit();
> }
> session.close();
> LifecycleUtil.deactivate(r);
> }
>
>
> This seems to be doing what I want, so far.
>
> Now some advanced questions:
>
> 1.
> Is your debug facility configurable? I only get one logging message
> stating that a legacy-something is not available. I'd like to see a
> bit more of what is going on behind the scenes...
>
> 2.
> Basically my application connsists of two parts. One is the
> computation-intensive part (reading a huge model and generating the
> new one).
> The other one is a (rather) small model, which is kept in a viewer and
> which the user can select and edit.
>
> So regarding transactions, the first part seems easy: start a
> transaction, read the one model, create the other, commit transaction,
> fine.
> But how do I deal with the second case - or more generally: what
> happens to a resource, when the transaction is committed?
>
> Can I do something like
> // in the view initialization:
> CDOView view = session.openView();
> Resource r = view.getResource("/some/path");
> editingDomain.getResourceSet().getResources().add(r);
>
> [...]
>
> // when the user creates an element in the model (Action handler) -
> // myEObject is contained in Resource r
> CDOTransaction transaction = session.openTransaction(editingDomain.getResourceSet());
> myEObject.getChildren().add(newObject);
> transaction.commit();
>
>
> 3.
> How do the resource paths map? Are these just some kind of "ID" for
> resources so that
> transaction.createResource("/some/path");
> always returns equivalent resources and
> transaction.createResource("/some/path/other");
> returns a different one which has nothing to do with the first one?
>
> If so, do inter-resource-references in the same repository work?
>
> 4.
> Does a session time out? Or can I start a session when the application
> starts up and close it, when it shuts down?
>
> Some commennts below ...
>
>
>
>
> Eike Stepper schrieb::
>>>
>>> |// You might want to create an IDBStore here instead if memory is an issue!
>>>
>>> |How is this done (I'm going to use MySQL as a backend database)?
>>> There is no StoreUtil.createIDBStore :-(
>> That's just because the plugin of StoreUtil has no dependency on the
>> plugin of the IDBStore.
>> You can create an IDBStore by calling
>>
>> new DBStore(...)
>>
>> There's currently no public API to directly create instances of
>> DBStore (enhancement request?).
> Filed: Bug 216668 <https://bugs.eclipse.org/bugs/show_bug.cgi?id=216668>
>> I know I'll have to write lots of wiki pages, but I hope this helps.
>> Please don't hesitate to ask further questions!
> You'll regret your last sentence ;-))
>
> Cheers,
> Stefan
>
--------------040406070502030204040607
Content-Type: text/html; charset=ISO-8859-15
Content-Transfer-Encoding: 8bit
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Stefan,<br>
<br>
I forgot to comment on the trace output with the "legacy-something".<br>
<br>
Originally CDO was designed to operate on models that are generated
dedicatedly for CDO, i.e. their genmodel has been modified as per
description in <a class="moz-txt-link-freetext" href="http://wiki.eclipse.org/Preparing_EMF_Models_for_CDO">http://wiki.eclipse.org/Preparing_EMF_Models_for_CDO</a> . I
call these models "native" (with respect to CDO).<br>
<br>
For various reasons there are models that can't be native. For instance
their genmodel is not available or some of their genmodel properties
can't be set to CDO native values (Root Extends Class, ...). The Ecore
meta meta model can't be regenerated natively because of resulting
circular dependencies between EMF and CDO. I have invented an AspectJ
based approach to make CDO operate on these models as well. I call
these woven models "legacy" (with respect to CDO). The CDO Weaver can
be used to convert most generated models to legacy models. Alll the
depenpendencies of these models have to be converted to legacy models
as well. The Ecore meta meta model is a special dependency of all other
models and if Ecore itself is not converted to a legacy model I say the
"legacy-system is unavailable". It might be less confusing if this info
message would only be logged if there are other legacy models that
needed a converted Ecore model but it is not very fast to determine
this case.<br>
<br>
BTW. native models are to be preferred over legacy models whenever
possible because they're much more efficient!<br>
<br>
Regards,<br>
Eike Stepper<br>
----<br>
<a class="moz-txt-link-freetext" href="http://wiki.eclipse.org/CDO">http://wiki.eclipse.org/CDO</a><br>
<a class="moz-txt-link-freetext" href="http://wiki.eclipse.org/Net4j">http://wiki.eclipse.org/Net4j</a><br>
<br>
<br>
<br>
Stefann Winkler schrieb:
<blockquote cite="mid:fnfdgp$vo1$1@build.eclipse.org" type="cite">
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
<title></title>
Hi Eike,<br>
<br>
ok, this is what I'm doing now and it seems to work (Thunderbird does
not like to insert Java2Html code from clipboard :-( ):<br>
<pre>public static void run() throws Exception {
IManagedContainer container = IPluginContainer.INSTANCE;
JVMUtil.getAcceptor(container, "default");
URL url = Activator.getDefault().getBundle().getEntry("/configuration/cdo-config.xml ");
url = FileLocator.toFileURL(url);
IRepository r = new RepositoryConfigurator(container).configure(new File(url.toURI()))[0];
String repositoryName = r.getName();
CDOServerUtil.addRepository(container, r);
IConnector connector = JVMUtil.getConnector(container, "default");
CDOSession session = CDOUtil.openSession(connector, repositoryName, true);
CDOTransaction transaction = session.openTransaction();
Resource resource = transaction.createResource("/my/big/resource");
for (int i = 0; i < 100; i++) {
EObject inputModel = getInputModel(i);
resource.getContents().add(inputModel);
transaction.commit();
}
session.close();
LifecycleUtil.deactivate(r);
}
</pre>
This seems to be doing what I want, so far. <br>
<br>
Now some advanced questions:<br>
<br>
1.<br>
Is your debug facility configurable? I only get one logging message
stating that a legacy-something is not available. I'd like to see a bit
more of what is going on behind the scenes...<br>
<br>
2.<br>
Basically my application connsists of two parts. One is the
computation-intensive part (reading a huge model and generating the new
one).<br>
The other one is a (rather) small model, which is kept in a viewer and
which the user can select and edit.<br>
<br>
So regarding transactions, the first part seems easy: start a
transaction, read the one model, create the other, commit transaction,
fine.<br>
But how do I deal with the second case - or more generally: what
happens to a resource, when the transaction is committed?<br>
<br>
Can I do something like<br>
<pre> // in the view initialization:
CDOView view = session.openView();
Resource r = view.getResource("/some/path");
editingDomain.getResourceSet().getResources().add(r);
[...]
// when the user creates an element in the model (Action handler) -
// myEObject is contained in Resource r
CDOTransaction transaction = session.openTransaction(editingDomain.getResourceSet());
myEObject.getChildren().add(newObject);
transaction.commit();
</pre>
<br>
3. <br>
How do the resource paths map? Are these just some kind of "ID" for
resources so that <br>
transaction.createResource("/some/path");<br>
always returns equivalent resources and<br>
transaction.createResource("/some/path/other");<br>
returns a different one which has nothing to do with the first one?<br>
<br>
If so, do inter-resource-references in the same repository work?<br>
<br>
4.<br>
Does a session time out? Or can I start a session when the application
starts up and close it, when it shuts down?<br>
<br>
Some commennts below ...<br>
<br>
<br>
<br>
<br>
Eike Stepper schrieb::
<blockquote cite="mid:fnckjs$bua$1@build.eclipse.org" type="cite">
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
<blockquote cite="mid:fncfah$p7e$1@build.eclipse.org" type="cite">
|
|
|
| Re: [CDO] Repository inside Eclipse [message #108634 is a reply to message #108420] |
Mon, 28 January 2008 14:07 |
Stefan Winkler
Messages: 307
Registered: July 2009
Location: Germany |
Senior Member |
|
|
Hi Eike,
Comments below ...
>> Now some advanced questions:
>>
>>
>> 2.
>> Basically my application connsists of two parts. One is the
>> computation-intensive part (reading a huge model and generating the
>> new one).
>> The other one is a (rather) small model, which is kept in a viewer
>> and which the user can select and edit.
>>
>> So regarding transactions, the first part seems easy: start a
>> transaction, read the one model, create the other, commit
>> transaction, fine.
>> But how do I deal with the second case - or more generally: what
>> happens to a resource, when the transaction is committed?
>>
>> Can I do something like
>> // in the view initialization:
>> CDOView view = session.openView();
>> Resource r = view.getResource("/some/path");
>> editingDomain.getResourceSet().getResources().add(r);
>>
> The last statement seems wrong to me. because it will detach the
> resource from the ResourceSet that is internally created on
> session.openView(). See my explanation below...
>
>>
>> [...]
>>
>> // when the user creates an element in the model (Action handler) -
>> // myEObject is contained in Resource r
>> CDOTransaction transaction = session.openTransaction(editingDomain.getResourceSet());
>> myEObject.getChildren().add(newObject);
>> transaction.commit();
>>
> Some explanation:
> In CDO an EMF ResourceSet is associated with a CDOView. CDOTransaction
> is a special CDOView. You could eventually say CDOView is a read-only
> transaction. Each ResourceSet can only have a single CDOView (or
> CDOTransaction) associated at any point in time. If you call
> session.openTransaction() a new ResourceSet is created internally. If
> you call session.openTransaction(myResourceSet) a given ResourceSet is
> used for that transaction. But only if the given ResourceSet doesn't
> already have another view/transaction associated. BTW. this
> association happens internally via a ResourceSet adapter.
>
> If there is a chance that the model will be modified, you should open
> a transaction from the beginning and use it for reads and writes later
> on. If the model is nonetheless *not* changed in a transaction it
> doesn't cost you much to have a transaction instead of a read-only view.
>
> But even if you open several views and transactions on the same
> session, they will share all the object state as long as they're not
> modified. Model state is stored/managed in the session itself. Only
> change deltas are stored in the transactions.
But the transaction does timeout (Transaction.setCommitTimeout()) or
does the timeout refer to the commit() operation itself?
Is it possible to do multiple commits in one transaction - like
transaction = session.openTransaction();
// do something for 4 hours
transaction.commit()
// do something else for another 6 hours
transaction.commit()
or does transaction.commit() lead to a transaction state where only
transaction.close() is possible?
So essentially, could I open the session and transaction in my
PluginActivator.start() and close them in stop() so that I have a
transaction running as long as the application for the interactive stuff?
If not, how would I realize an interactive application in which the user
is shown a tree (of the model) that he can work with interactively?
Cheers,
Stefan
|
|
|
| Re: [CDO] Repository inside Eclipse [message #108847 is a reply to message #108634] |
Tue, 29 January 2008 08:22 |
| Eclipse User |
|
|
|
Originally posted by: stepper.sympedia.de
Hi Stefan ,
Comments below ...
Stefan Winkler schrieb:
> [stuff deleted]
> But the transaction does timeout (Transaction.setCommitTimeout()) or
> does the timeout refer to the commit() operation itself?
Yes, the commit timeout does refer to the commit operation ;-)
>
> Is it possible to do multiple commits in one transaction - like
>
> transaction = session.openTransaction();
> // do something for 4 hours
> transaction.commit()
> // do something else for another 6 hours
> transaction.commit()
Yes, if you committed a transaction it returns to an "empty" state (no
change deltas) and can be used like before. It behaves similar to a JDBC
Connection with autoCommitMode==false. A better name than simply
transaction could have been transactional view...
>
> or does transaction.commit() lead to a transaction state where only
> transaction.close() is possible?
No, see above.
>
> So essentially, could I open the session and transaction in my
> PluginActivator.start() and close them in stop() so that I have a
> transaction running as long as the application for the interactive stuff?
Yes, I can imagine that this is possible.
>
> If not, how would I realize an interactive application in which the
> user is shown a tree (of the model) that he can work with interactively?
I'd say much the same way as with plain EMF. From the point on where you
have a CDOTransaction instance, which you can use to retrieve/commit the
ResourceSet, everything behaves normally. The only additional task is,
like you suspect, the management of the CDOTransaction instance,
including the needed dependencies (IConnector, ...).
BTW. There's a difference in the semantics of the CDOResource.save()
method. Since the scope of a CDOTransaction is always a whole
ResourceSet you can't save a single CDOResource (and keep local changes
to other CDOResources in the same ResourceSet/CDOTransaction). Thus
save() of a single CDOResource delegates to CDOTransaction.commit()
which will in turn "save" all the CDOResources in that ResourceSet.
Regards,
Eike Stepper
----
http://wiki.eclipse.org/CDO
http://wiki.eclipse.org/Net4j
|
|
|
| Re: [Teneo/CDO] Managing huge models / unloading parts of models [message #614061 is a reply to message #107855] |
Wed, 23 January 2008 14:39 |
|
Hi Stefan,
I don't know if Teneo can do this, Martin will want to explain that.
CDO can do it. With CDO the modeled instances internally are not
interconnected with java reference handles. This way unneeded instances
can be completely garbage collected and transparently reloaded later on.
You don't need to write extra code for that to function.
If you plan to construct the output model programmatically everything
should work fine. If you plan to read your input models for example from
xml documents they should of course be small enough to fit into memory.
The resulting CDO repository resource(s) would have to remain within the
repository until you have enough memory on client side that you can load
them completely into an XML resource to save them to a file again.
If you plan to do your conversion within one VM (that is, not through a
networked remote repository) you can gain a lot of system resources
(memory, performance, ...) if you decide to use a Net4j IJVMConnector
instead of the usual ITCPConnector. And you can use an embedded Derby
Database to further optimize, but I guess that performance is not the
issue. If client memory gets low, it could even be better to use a
remote database server.
Regards,
Eike Stepper
----
http://wiki.eclipse.org/CDO
http://wiki.eclipse.org/Net4j
Stefan Winkler schrieb:
> Hi all,
>
> I am working on a project which will have to deal with quite large
> (both broad and deep) models. The models themselves are only read or
> created (out of other models) but not modified themselves.
>
> Regarding the creation task: this will be something like a custom
> model transformation: an algorithm will read one model and produce
> another.
>
> Problem is: I expect the models to be too big to fit into memory, so I
> need a solution which allows only parts of models to be loaded and
> these parts to be unloaded, when they are not being used anymore. In
> terms of EMF, I want only parts of Resources to be loaded.
>
> Ed Merks pointed out that CDO or Teneo could help here as they use
> lazy loading strategies. So far I had a look into Teneo/Hibernate and
> I like the interface so far. But: While I understand that Teneo
> supports lazy loading of EMF Objects, does Teneo (or Hibernate behind
> this) have support to unload objects that are no longer needed?
>
> E.g.
>
> A
> |
> +--- B
> |
> +---- C
> |
> +----- D
>
> When I start at A (B,C,D are proxies, because policy says so), I
> access B,C and D. Is it possible to unload C and D and make C Proxy
> again so that C and D get garbage collected?
>
>
> If Teneo does not support this, does CDO?
>
>
> Thanks in advance!
>
> Cheers,
>
> Stefan
>
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
| Re: [Teneo/CDO] Managing huge models / unloading parts of models [message #614062 is a reply to message #107870] |
Wed, 23 January 2008 15:26 |
Stefan Winkler
Messages: 307
Registered: July 2009
Location: Germany |
Senior Member |
|
|
Hi Eike,
thanks for your reply.
> I don't know if Teneo can do this, Martin will want to explain that.
Some research about Hibernate revealed that in Hibernate you are
supposed to use Sessions for scoping. So in order to unload things, you
should load these things before in a separate session and close and
dispose that session if no longer needed.
Martin, can you confirm that?
> CDO can do it. With CDO the modeled instances internally are not
> interconnected with java reference handles. This way unneeded instances
> can be completely garbage collected and transparently reloaded later on.
> You don't need to write extra code for that to function.
>
> If you plan to construct the output model programmatically everything
> should work fine. If you plan to read your input models for example from
> xml documents they should of course be small enough to fit into memory.
> The resulting CDO repository resource(s) would have to remain within the
> repository until you have enough memory on client side that you can load
> them completely into an XML resource to save them to a file again.
All models are stored in the database back-end and processed from there.
(Importing into the database is another issue and it is clear that
reading XMI for importing into the database requires enough memory to
hold the XMI resource data.)
> If you plan to do your conversion within one VM (that is, not through a
> networked remote repository) you can gain a lot of system resources
> (memory, performance, ...) if you decide to use a Net4j IJVMConnector
> instead of the usual ITCPConnector. And you can use an embedded Derby
> Database to further optimize, but I guess that performance is not the
> issue. If client memory gets low, it could even be better to use a
> remote database server.
What I am doing is more of a research prototype, so performance is not
an issue. I also don't need a client-server architecture or distributed
databases/repositories. So my architecture simply looks like this
[ DB to store repository ] <-{1}---------{1}-> [Eclipse-based client]
Is there any hello world tutorial with a simple scenario like this one
(using the IJVMConnector as you said)?
The tutorial nodes in the wiki are still quite empty ... ;-)
Cheers,
Stefan
|
|
|
| Re: [Teneo/CDO] Managing huge models / unloading parts of models [message #614063 is a reply to message #107870] |
Wed, 23 January 2008 15:27 |
|
I've written a small example how to set up a standalone client and
server within one JVM:
package org.eclipse.emf.cdo.tests;
import org.eclipse.emf.cdo.CDOSession;
import org.eclipse.emf.cdo.CDOTransaction;
import org.eclipse.emf.cdo.server.CDOServerUtil;
import org.eclipse.emf.cdo.server.IRepository;
import org.eclipse.emf.cdo.server.IStore;
import org.eclipse.emf.cdo.server.StoreUtil;
import org.eclipse.emf.cdo.util.CDOUtil;
import org.eclipse.net4j.Net4jUtil;
import org.eclipse.net4j.connector.IConnector;
import org.eclipse.net4j.jvm.JVMUtil;
import org.eclipse.net4j.util.container.ContainerUtil;
import org.eclipse.net4j.util.container.IManagedContainer;
import org.eclipse.emf.ecore.EObject;
import org.eclipse.emf.ecore.resource.Resource;
/**
* @author Eike Stepper
*/
public class ImportXML
{
private static final String REPOSITORY_NAME = "repo1";
public static void main(String[] args)
{
// Prepare the standalone infra structure (not needed when
running inside Eclipse)
IManagedContainer container = ContainerUtil.createContainer();
// Create a wiring container
Net4jUtil.prepareContainer(container); // Prepare the Net4j kernel
JVMUtil.prepareContainer(container); // Prepare the JVM transport
CDOServerUtil.prepareContainer(container); // Prepare the CDO server
CDOUtil.prepareContainer(container, false); // Prepare the CDO
client
// Start the transport and create a repository
JVMUtil.getAcceptor(container, "default"); // Start the JVM
transport
CDOServerUtil.addRepository(container, createRepository()); //
Start a CDO respository
// Establish a communications connection and open a session with
the repository
IConnector connector = JVMUtil.getConnector(container,
"default"); // Open a JVM connection
CDOSession session = CDOUtil.openSession(connector,
REPOSITORY_NAME, true);// Open a CDO session
CDOTransaction transaction = session.openTransaction();// Open a
CDO transaction
Resource resource =
transaction.createResource("/my/big/resource");// Create a new EMF resource
// Work normally with the EMF resource
for (int i = 0; i < 100; i++)
{
EObject inputModel = getInputModel(i);
resource.getContents().add(inputModel);
transaction.commit();
}
session.close();
}
private static IRepository createRepository()
{
Map<String, String> props = new HashMap<String, String>();
props.put(IRepository.PROP_SUPPORTING_REVISION_DELTAS, "true");
props.put(IRepository.PROP_CURRENT_LRU_CAPACITY, "10000");
props.put(IRepository.PROP_REVISED_LRU_CAPACITY, "10000");
return CDOServerUtil.createRepository(REPOSITORY_NAME,
createStore(), props);
}
private static IStore createStore()
{
// You might want to create an IDBStore here instead if memory
is an issue!
return StoreUtil.createMEMStore();
}
private static EObject getInputModel(int i)
{
throw new UnsupportedOperationException("Load you model here");
}
}
It's also possible to create and wire the various components yourself,
but using a managed container with factories is easier. If running in
Eclipse even the container is created and configured automatically.
There's also a way to do the configuration via XML. Please tell me if
you need more infos.
Regards,
Eike Stepper
----
http://wiki.eclipse.org/CDO
http://wiki.eclipse.org/Net4j
Eike Stepper schrieb:
> Hi Stefan,
>
> I don't know if Teneo can do this, Martin will want to explain that.
>
> CDO can do it. With CDO the modeled instances internally are not
> interconnected with java reference handles. This way unneeded
> instances can be completely garbage collected and transparently
> reloaded later on. You don't need to write extra code for that to
> function.
>
> If you plan to construct the output model programmatically everything
> should work fine. If you plan to read your input models for example
> from xml documents they should of course be small enough to fit into
> memory. The resulting CDO repository resource(s) would have to remain
> within the repository until you have enough memory on client side that
> you can load them completely into an XML resource to save them to a
> file again.
>
> If you plan to do your conversion within one VM (that is, not through
> a networked remote repository) you can gain a lot of system resources
> (memory, performance, ...) if you decide to use a Net4j IJVMConnector
> instead of the usual ITCPConnector. And you can use an embedded Derby
> Database to further optimize, but I guess that performance is not the
> issue. If client memory gets low, it could even be better to use a
> remote database server.
>
> Regards,
> Eike Stepper
> ----
> http://wiki.eclipse.org/CDO
> http://wiki.eclipse.org/Net4j
>
>
>
>
>
> Stefan Winkler schrieb:
>> Hi all,
>>
>> I am working on a project which will have to deal with quite large
>> (both broad and deep) models. The models themselves are only read or
>> created (out of other models) but not modified themselves.
>>
>> Regarding the creation task: this will be something like a custom
>> model transformation: an algorithm will read one model and produce
>> another.
>>
>> Problem is: I expect the models to be too big to fit into memory, so
>> I need a solution which allows only parts of models to be loaded and
>> these parts to be unloaded, when they are not being used anymore. In
>> terms of EMF, I want only parts of Resources to be loaded.
>>
>> Ed Merks pointed out that CDO or Teneo could help here as they use
>> lazy loading strategies. So far I had a look into Teneo/Hibernate and
>> I like the interface so far. But: While I understand that Teneo
>> supports lazy loading of EMF Objects, does Teneo (or Hibernate behind
>> this) have support to unload objects that are no longer needed?
>>
>> E.g.
>>
>> A
>> |
>> +--- B
>> |
>> +---- C
>> |
>> +----- D
>>
>> When I start at A (B,C,D are proxies, because policy says so), I
>> access B,C and D. Is it possible to unload C and D and make C Proxy
>> again so that C and D get garbage collected?
>>
>>
>> If Teneo does not support this, does CDO?
>>
>>
>> Thanks in advance!
>>
>> Cheers,
>>
>> Stefan
>>
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
| Re: [Teneo/CDO] Managing huge models / unloading parts of models [message #614065 is a reply to message #107897] |
Wed, 23 January 2008 15:30 |
|
Hmm, the format is ruined but say, 1 Minute to deliver this example? ;-)
Cheers
/Eike
Eike Stepper schrieb:
> I've written a small example how to set up a standalone client and
> server within one JVM:
>
> package org.eclipse.emf.cdo.tests;
> import org.eclipse.emf.cdo.CDOSession;
> import org.eclipse.emf.cdo.CDOTransaction;
> import org.eclipse.emf.cdo.server.CDOServerUtil;
> import org.eclipse.emf.cdo.server.IRepository;
> import org.eclipse.emf.cdo.server.IStore;
> import org.eclipse.emf.cdo.server.StoreUtil;
> import org.eclipse.emf.cdo.util.CDOUtil;
> import org.eclipse.net4j.Net4jUtil;
> import org.eclipse.net4j.connector.IConnector;
> import org.eclipse.net4j.jvm.JVMUtil;
> import org.eclipse.net4j.util.container.ContainerUtil;
> import org.eclipse.net4j.util.container.IManagedContainer;
> import org.eclipse.emf.ecore.EObject;
> import org.eclipse.emf.ecore.resource.Resource;
> /**
> * @author Eike Stepper
> */
> public class ImportXML
> {
> private static final String REPOSITORY_NAME = "repo1";
> public static void main(String[] args)
> {
> // Prepare the standalone infra structure (not needed when
> running inside Eclipse)
> IManagedContainer container = ContainerUtil.createContainer();
> // Create a wiring container
> Net4jUtil.prepareContainer(container); // Prepare the Net4j kernel
> JVMUtil.prepareContainer(container); // Prepare the JVM transport
> CDOServerUtil.prepareContainer(container); // Prepare the CDO
> server
> CDOUtil.prepareContainer(container, false); // Prepare the CDO
> client
> // Start the transport and create a repository
> JVMUtil.getAcceptor(container, "default"); // Start the JVM
> transport
> CDOServerUtil.addRepository(container, createRepository()); //
> Start a CDO respository
> // Establish a communications connection and open a session
> with the repository
> IConnector connector = JVMUtil.getConnector(container,
> "default"); // Open a JVM connection
> CDOSession session = CDOUtil.openSession(connector,
> REPOSITORY_NAME, true);// Open a CDO session
> CDOTransaction transaction = session.openTransaction();// Open
> a CDO transaction
> Resource resource =
> transaction.createResource("/my/big/resource");// Create a new EMF
> resource
> // Work normally with the EMF resource
> for (int i = 0; i < 100; i++)
> {
> EObject inputModel = getInputModel(i);
> resource.getContents().add(inputModel);
> transaction.commit();
> }
> session.close();
> }
> private static IRepository createRepository()
> {
> Map<String, String> props = new HashMap<String, String>();
> props.put(IRepository.PROP_SUPPORTING_REVISION_DELTAS, "true");
> props.put(IRepository.PROP_CURRENT_LRU_CAPACITY, "10000");
> props.put(IRepository.PROP_REVISED_LRU_CAPACITY, "10000");
> return CDOServerUtil.createRepository(REPOSITORY_NAME,
> createStore(), props);
> }
> private static IStore createStore()
> {
> // You might want to create an IDBStore here instead if memory
> is an issue!
> return StoreUtil.createMEMStore();
> }
> private static EObject getInputModel(int i)
> {
> throw new UnsupportedOperationException("Load you model here");
> }
> }
>
> It's also possible to create and wire the various components yourself,
> but using a managed container with factories is easier. If running in
> Eclipse even the container is created and configured automatically.
> There's also a way to do the configuration via XML. Please tell me if
> you need more infos.
>
> Regards,
> Eike Stepper
> ----
> http://wiki.eclipse.org/CDO
> http://wiki.eclipse.org/Net4j
>
>
> Eike Stepper schrieb:
>> Hi Stefan,
>>
>> I don't know if Teneo can do this, Martin will want to explain that.
>>
>> CDO can do it. With CDO the modeled instances internally are not
>> interconnected with java reference handles. This way unneeded
>> instances can be completely garbage collected and transparently
>> reloaded later on. You don't need to write extra code for that to
>> function.
>>
>> If you plan to construct the output model programmatically everything
>> should work fine. If you plan to read your input models for example
>> from xml documents they should of course be small enough to fit into
>> memory. The resulting CDO repository resource(s) would have to remain
>> within the repository until you have enough memory on client side
>> that you can load them completely into an XML resource to save them
>> to a file again.
>>
>> If you plan to do your conversion within one VM (that is, not through
>> a networked remote repository) you can gain a lot of system resources
>> (memory, performance, ...) if you decide to use a Net4j IJVMConnector
>> instead of the usual ITCPConnector. And you can use an embedded Derby
>> Database to further optimize, but I guess that performance is not the
>> issue. If client memory gets low, it could even be better to use a
>> remote database server.
>>
>> Regards,
>> Eike Stepper
>> ----
>> http://wiki.eclipse.org/CDO
>> http://wiki.eclipse.org/Net4j
>>
>>
>>
>>
>>
>> Stefan Winkler schrieb:
>>> Hi all,
>>>
>>> I am working on a project which will have to deal with quite large
>>> (both broad and deep) models. The models themselves are only read or
>>> created (out of other models) but not modified themselves.
>>>
>>> Regarding the creation task: this will be something like a custom
>>> model transformation: an algorithm will read one model and produce
>>> another.
>>>
>>> Problem is: I expect the models to be too big to fit into memory, so
>>> I need a solution which allows only parts of models to be loaded and
>>> these parts to be unloaded, when they are not being used anymore. In
>>> terms of EMF, I want only parts of Resources to be loaded.
>>>
>>> Ed Merks pointed out that CDO or Teneo could help here as they use
>>> lazy loading strategies. So far I had a look into Teneo/Hibernate
>>> and I like the interface so far. But: While I understand that Teneo
>>> supports lazy loading of EMF Objects, does Teneo (or Hibernate
>>> behind this) have support to unload objects that are no longer needed?
>>>
>>> E.g.
>>>
>>> A
>>> |
>>> +--- B
>>> |
>>> +---- C
>>> |
>>> +----- D
>>>
>>> When I start at A (B,C,D are proxies, because policy says so), I
>>> access B,C and D. Is it possible to unload C and D and make C Proxy
>>> again so that C and D get garbage collected?
>>>
>>>
>>> If Teneo does not support this, does CDO?
>>>
>>>
>>> Thanks in advance!
>>>
>>> Cheers,
>>>
>>> Stefan
>>>
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
| Re: [Teneo/CDO] Managing huge models / unloading parts of models [message #614073 is a reply to message #107884] |
Thu, 24 January 2008 10:18 |
Martin Taal
Messages: 5468
Registered: July 2009 |
Senior Member |
|
|
Hi Stefan,
You can also evict individual objects from the hibernate sesion by doing session.evict(object).
However, you would also need to re-proxy objects but I am not sure how to accomplish that.
Hibernate has proxies but you would need to dive into the hibernate internals to implement
reproxying of objects. I checked and there are some opportunities to look at (afaics implement your
own proxyfactory which returns re-proxyable proxies).
You can also try re-proxying through EMF but I am not sure how well this works with hibernate.
gr. Martin
Stefan Winkler wrote:
> Hi Eike,
>
> thanks for your reply.
>
>> I don't know if Teneo can do this, Martin will want to explain that.
> Some research about Hibernate revealed that in Hibernate you are
> supposed to use Sessions for scoping. So in order to unload things, you
> should load these things before in a separate session and close and
> dispose that session if no longer needed.
>
> Martin, can you confirm that?
>
>> CDO can do it. With CDO the modeled instances internally are not
>> interconnected with java reference handles. This way unneeded
>> instances can be completely garbage collected and transparently
>> reloaded later on. You don't need to write extra code for that to
>> function.
>>
>> If you plan to construct the output model programmatically everything
>> should work fine. If you plan to read your input models for example
>> from xml documents they should of course be small enough to fit into
>> memory. The resulting CDO repository resource(s) would have to remain
>> within the repository until you have enough memory on client side that
>> you can load them completely into an XML resource to save them to a
>> file again.
>
> All models are stored in the database back-end and processed from there.
> (Importing into the database is another issue and it is clear that
> reading XMI for importing into the database requires enough memory to
> hold the XMI resource data.)
>
>> If you plan to do your conversion within one VM (that is, not through
>> a networked remote repository) you can gain a lot of system resources
>> (memory, performance, ...) if you decide to use a Net4j IJVMConnector
>> instead of the usual ITCPConnector. And you can use an embedded Derby
>> Database to further optimize, but I guess that performance is not the
>> issue. If client memory gets low, it could even be better to use a
>> remote database server.
>
> What I am doing is more of a research prototype, so performance is not
> an issue. I also don't need a client-server architecture or distributed
> databases/repositories. So my architecture simply looks like this
>
> [ DB to store repository ] <-{1}---------{1}-> [Eclipse-based client]
>
> Is there any hello world tutorial with a simple scenario like this one
> (using the IJVMConnector as you said)?
> The tutorial nodes in the wiki are still quite empty ... ;-)
>
> Cheers,
> Stefan
--
With Regards, Martin Taal
Springsite/Elver.org
Office: Hardwareweg 4, 3821 BV Amersfoort
Postal: Nassaulaan 7, 3941 EC Doorn
The Netherlands
Tel: +31 (0)84 420 2397
Fax: +31 (0)84 225 9307
Mail: mtaal@springsite.com - mtaal@elver.org
Web: www.springsite.com - www.elver.org
|
|
|
| Re: [Teneo/CDO] Managing huge models / unloading parts of models [message #614075 is a reply to message #107897] |
Thu, 24 January 2008 11:05 |
|
This is a multi-part message in MIME format.
--------------040801030404030100020306
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit
I found a nice plugin to format the java code to HTML:
|*package *org.eclipse.emf.cdo.tests;
*import *org.eclipse.emf.cdo.CDOSession;
*import *org.eclipse.emf.cdo.CDOTransaction;
*import *org.eclipse.emf.cdo.server.CDOServerUtil;
*import *org.eclipse.emf.cdo.server.IRepository;
*import *org.eclipse.emf.cdo.server.IStore;
*import *org.eclipse.emf.cdo.server.StoreUtil;
*import *org.eclipse.emf.cdo.util.CDOUtil;
*import *org.eclipse.net4j.Net4jUtil;
*import *org.eclipse.net4j.connector.IConnector;
*import *org.eclipse.net4j.jvm.JVMUtil;
*import *org.eclipse.net4j.util.container.ContainerUtil;
*import *org.eclipse.net4j.util.container.IManagedContainer;
*import *org.eclipse.emf.ecore.EObject;
*import *org.eclipse.emf.ecore.resource.Resource;
*import *java.util.HashMap;
*import *java.util.Map;
/**
* @author Eike Stepper
*/
*public class *ImportXML
{
*private static final *String REPOSITORY_NAME = "repo1";
*public static **void *main(String[] args)
{
// Prepare the standalone infra structure (not needed when running inside Eclipse)
IManagedContainer container = ContainerUtil.createContainer(); // Create a wiring container
Net4jUtil.prepareContainer(container); // Prepare the Net4j kernel
JVMUtil.prepareContainer(container); // Prepare the JVM transport
CDOServerUtil.prepareContainer(container); // Prepare the CDO server
CDOUtil.prepareContainer(container, *false*); // Prepare the CDO client
// Start the transport and create a repository
JVMUtil.getAcceptor(container, "default"); // Start the JVM transport
CDOServerUtil.addRepository(container, createRepository()); // Start a CDO respository
// Establish a communications connection and open a session with the repository
IConnector connector = JVMUtil.getConnector(container, "default"); // Open a JVM connection
CDOSession session = CDOUtil.openSession(connector, REPOSITORY_NAME, *true*);// Open a CDO session
CDOTransaction transaction = session.openTransaction();// Open a CDO transaction
Resource resource = transaction.createResource("/my/big/resource");// Create a new EMF resource
// Work normally with the EMF resource
*for *(*int *i = 0; i < 100; i++)
{
EObject inputModel = getInputModel(i);
resource.getContents().add(inputModel);
transaction.commit();
}
session.close();
}
*private static *IRepository createRepository()
{
Map<String, String> props = *new *HashMap<String, String>();
props.put(IRepository.PROP_SUPPORTING_REVISION_DELTAS, "true");
props.put(IRepository.PROP_CURRENT_LRU_CAPACITY, "10000");
props.put(IRepository.PROP_REVISED_LRU_CAPACITY, "10000");
*return *CDOServerUtil.createRepository(REPOSITORY_NAME, createStore(), props);
}
*private static *IStore createStore()
{
// You might want to create an IDBStore here instead if memory is an issue!
*return *StoreUtil.createMEMStore();
}
*private static *EObject getInputModel(*int *i)
{
*throw new *UnsupportedOperationException("Load you model here");
}
}|
Regards,
Eike Stepper
----
http://wiki.eclipse.org/CDO
http://wiki.eclipse.org/Net4j
Eike Stepper schrieb:
> I've written a small example how to set up a standalone client and
> server within one JVM:
>
> package org.eclipse.emf.cdo.tests;
> import org.eclipse.emf.cdo.CDOSession;
> import org.eclipse.emf.cdo.CDOTransaction;
> import org.eclipse.emf.cdo.server.CDOServerUtil;
> import org.eclipse.emf.cdo.server.IRepository;
> import org.eclipse.emf.cdo.server.IStore;
> import org.eclipse.emf.cdo.server.StoreUtil;
> import org.eclipse.emf.cdo.util.CDOUtil;
> import org.eclipse.net4j.Net4jUtil;
> import org.eclipse.net4j.connector.IConnector;
> import org.eclipse.net4j.jvm.JVMUtil;
> import org.eclipse.net4j.util.container.ContainerUtil;
> import org.eclipse.net4j.util.container.IManagedContainer;
> import org.eclipse.emf.ecore.EObject;
> import org.eclipse.emf.ecore.resource.Resource;
> /**
> * @author Eike Stepper
> */
> public class ImportXML
> {
> private static final String REPOSITORY_NAME = "repo1";
> public static void main(String[] args)
> {
> // Prepare the standalone infra structure (not needed when
> running inside Eclipse)
> IManagedContainer container = ContainerUtil.createContainer();
> // Create a wiring container
> Net4jUtil.prepareContainer(container); // Prepare the Net4j kernel
> JVMUtil.prepareContainer(container); // Prepare the JVM transport
> CDOServerUtil.prepareContainer(container); // Prepare the CDO
> server
> CDOUtil.prepareContainer(container, false); // Prepare the CDO
> client
> // Start the transport and create a repository
> JVMUtil.getAcceptor(container, "default"); // Start the JVM
> transport
> CDOServerUtil.addRepository(container, createRepository()); //
> Start a CDO respository
> // Establish a communications connection and open a session
> with the repository
> IConnector connector = JVMUtil.getConnector(container,
> "default"); // Open a JVM connection
> CDOSession session = CDOUtil.openSession(connector,
> REPOSITORY_NAME, true);// Open a CDO session
> CDOTransaction transaction = session.openTransaction();// Open
> a CDO transaction
> Resource resource =
> transaction.createResource("/my/big/resource");// Create a new EMF
> resource
> // Work normally with the EMF resource
> for (int i = 0; i < 100; i++)
> {
> EObject inputModel = getInputModel(i);
> resource.getContents().add(inputModel);
> transaction.commit();
> }
> session.close();
> }
> private static IRepository createRepository()
> {
> Map<String, String> props = new HashMap<String, String>();
> props.put(IRepository.PROP_SUPPORTING_REVISION_DELTAS, "true");
> props.put(IRepository.PROP_CURRENT_LRU_CAPACITY, "10000");
> props.put(IRepository.PROP_REVISED_LRU_CAPACITY, "10000");
> return CDOServerUtil.createRepository(REPOSITORY_NAME,
> createStore(), props);
> }
> private static IStore createStore()
> {
> // You might want to create an IDBStore here instead if memory
> is an issue!
> return StoreUtil.createMEMStore();
> }
> private static EObject getInputModel(int i)
> {
> throw new UnsupportedOperationException("Load you model here");
> }
> }
>
> It's also possible to create and wire the various components yourself,
> but using a managed container with factories is easier. If running in
> Eclipse even the container is created and configured automatically.
> There's also a way to do the configuration via XML. Please tell me if
> you need more infos.
>
> Regards,
> Eike Stepper
> ----
> http://wiki.eclipse.org/CDO
> http://wiki.eclipse.org/Net4j
>
>
> Eike Stepper schrieb:
>> Hi Stefan,
>>
>> I don't know if Teneo can do this, Martin will want to explain that.
>>
>> CDO can do it. With CDO the modeled instances internally are not
>> interconnected with java reference handles. This way unneeded
>> instances can be completely garbage collected and transparently
>> reloaded later on. You don't need to write extra code for that to
>> function.
>>
>> If you plan to construct the output model programmatically everything
>> should work fine. If you plan to read your input models for example
>> from xml documents they should of course be small enough to fit into
>> memory. The resulting CDO repository resource(s) would have to remain
>> within the repository until you have enough memory on client side
>> that you can load them completely into an XML resource to save them
>> to a file again.
>>
>> If you plan to do your conversion within one VM (that is, not through
>> a networked remote repository) you can gain a lot of system resources
>> (memory, performance, ...) if you decide to use a Net4j IJVMConnector
>> instead of the usual ITCPConnector. And you can use an embedded Derby
>> Database to further optimize, but I guess that performance is not the
>> issue. If client memory gets low, it could even be better to use a
>> remote database server.
>>
>> Regards,
>> Eike Stepper
>> ----
>> http://wiki.eclipse.org/CDO
>> http://wiki.eclipse.org/Net4j
>>
>>
>>
>>
>>
>> Stefan Winkler schrieb:
>>> Hi all,
>>>
>>> I am working on a project which will have to deal with quite large
>>> (both broad and deep) models. The models themselves are only read or
>>> created (out of other models) but not modified themselves.
>>>
>>> Regarding the creation task: this will be something like a custom
>>> model transformation: an algorithm will read one model and produce
>>> another.
>>>
>>> Problem is: I expect the models to be too big to fit into memory, so
>>> I need a solution which allows only parts of models to be loaded and
>>> these parts to be unloaded, when they are not being used anymore. In
>>> terms of EMF, I want only parts of Resources to be loaded.
>>>
>>> Ed Merks pointed out that CDO or Teneo could help here as they use
>>> lazy loading strategies. So far I had a look into Teneo/Hibernate
>>> and I like the interface so far. But: While I understand that Teneo
>>> supports lazy loading of EMF Objects, does Teneo (or Hibernate
>>> behind this) have support to unload objects that are no longer needed?
>>>
>>> E.g.
>>>
>>> A
>>> |
>>> +--- B
>>> |
>>> +---- C
>>> |
>>> +----- D
>>>
>>> When I start at A (B,C,D are proxies, because policy says so), I
>>> access B,C and D. Is it possible to unload C and D and make C Proxy
>>> again so that C and D get garbage collected?
>>>
>>>
>>> If Teneo does not support this, does CDO?
>>>
>>>
>>> Thanks in advance!
>>>
>>> Cheers,
>>>
>>> Stefan
>>>
--------------040801030404030100020306
Content-Type: text/html; charset=ISO-8859-15
Content-Transfer-Encoding: 8bit
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
I found a nice plugin to format the java code to HTML:<br>
<br>
<title></title>
<style type="text/css">
<!--code { font-family: Courier New, Courier; font-size: 10pt; margin: 0px; }-->
</style>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<!-- ======================================================== -->
<!-- = Java Sourcecode to HTML automatically converted code = --><!-- = Java2Html Converter 5.0 [2006-02-26] by Markus Gebhard markus@jave.de = -->
<!-- = Further information: http://www.java2html.de = -->
<div class="java" align="left">
<table bgcolor="#ffffff" border="0" cellpadding="3" cellspacing="0">
<tbody>
<tr>
<!-- start source code --> <td align="left" nowrap="nowrap"
valign="top"> <code><font color="#7f0055"><b>package
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
| Re: [Teneo/CDO] Managing huge models / unloading parts of models [message #614076 is a reply to message #108068] |
Thu, 24 January 2008 12:11 |
Ed Merks
Messages: 33264
Registered: July 2009 |
Senior Member |
|
|
This is a multi-part message in MIME format.
--------------020000080300010401020708
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit
Eike,
Oooo. That's just so pretty! Where did you find it?
Eike Stepper wrote:
> I found a nice plugin to format the java code to HTML:
>
> |*package *org.eclipse.emf.cdo.tests;
>
> *import *org.eclipse.emf.cdo.CDOSession;
> *import *org.eclipse.emf.cdo.CDOTransaction;
> *import *org.eclipse.emf.cdo.server.CDOServerUtil;
> *import *org.eclipse.emf.cdo.server.IRepository;
> *import *org.eclipse.emf.cdo.server.IStore;
> *import *org.eclipse.emf.cdo.server.StoreUtil;
> *import *org.eclipse.emf.cdo.util.CDOUtil;
>
> *import *org.eclipse.net4j.Net4jUtil;
> *import *org.eclipse.net4j.connector.IConnector;
> *import *org.eclipse.net4j.jvm.JVMUtil;
> *import *org.eclipse.net4j.util.container.ContainerUtil;
> *import *org.eclipse.net4j.util.container.IManagedContainer;
>
> *import *org.eclipse.emf.ecore.EObject;
> *import *org.eclipse.emf.ecore.resource.Resource;
>
> *import *java.util.HashMap;
> *import *java.util.Map;
>
> /**
> * @author Eike Stepper
> */
> *public class *ImportXML
> {
> *private static final *String REPOSITORY_NAME = "repo1";
>
> *public static **void *main(String[] args)
> {
> // Prepare the standalone infra structure (not needed when running inside Eclipse)
> IManagedContainer container = ContainerUtil.createContainer(); // Create a wiring container
> Net4jUtil.prepareContainer(container); // Prepare the Net4j kernel
> JVMUtil.prepareContainer(container); // Prepare the JVM transport
> CDOServerUtil.prepareContainer(container); // Prepare the CDO server
> CDOUtil.prepareContainer(container, *false*); // Prepare the CDO client
>
> // Start the transport and create a repository
> JVMUtil.getAcceptor(container, "default"); // Start the JVM transport
> CDOServerUtil.addRepository(container, createRepository()); // Start a CDO respository
>
> // Establish a communications connection and open a session with the repository
> IConnector connector = JVMUtil.getConnector(container, "default"); // Open a JVM connection
> CDOSession session = CDOUtil.openSession(connector, REPOSITORY_NAME, *true*);// Open a CDO session
> CDOTransaction transaction = session.openTransaction();// Open a CDO transaction
> Resource resource = transaction.createResource("/my/big/resource");// Create a new EMF resource
>
> // Work normally with the EMF resource
> *for *(*int *i = 0; i < 100; i++)
> {
> EObject inputModel = getInputModel(i);
> resource.getContents().add(inputModel);
> transaction.commit();
> }
>
> session.close();
> }
>
> *private static *IRepository createRepository()
> {
> Map<String, String> props = *new *HashMap<String, String>();
> props.put(IRepository.PROP_SUPPORTING_REVISION_DELTAS, "true");
> props.put(IRepository.PROP_CURRENT_LRU_CAPACITY, "10000");
> props.put(IRepository.PROP_REVISED_LRU_CAPACITY, "10000");
> *return *CDOServerUtil.createRepository(REPOSITORY_NAME, createStore(), props);
> }
>
> *private static *IStore createStore()
> {
> // You might want to create an IDBStore here instead if memory is an issue!
> *return *StoreUtil.createMEMStore();
> }
>
> *private static *EObject getInputModel(*int *i)
> {
> *throw new *UnsupportedOperationException("Load you model here");
> }
> }|
>
>
> Regards,
> Eike Stepper
> ----
> http://wiki.eclipse.org/CDO
> http://wiki.eclipse.org/Net4j
>
>
>
> Eike Stepper schrieb:
>> I've written a small example how to set up a standalone client and
>> server within one JVM:
>>
>> package org.eclipse.emf.cdo.tests;
>> import org.eclipse.emf.cdo.CDOSession;
>> import org.eclipse.emf.cdo.CDOTransaction;
>> import org.eclipse.emf.cdo.server.CDOServerUtil;
>> import org.eclipse.emf.cdo.server.IRepository;
>> import org.eclipse.emf.cdo.server.IStore;
>> import org.eclipse.emf.cdo.server.StoreUtil;
>> import org.eclipse.emf.cdo.util.CDOUtil;
>> import org.eclipse.net4j.Net4jUtil;
>> import org.eclipse.net4j.connector.IConnector;
>> import org.eclipse.net4j.jvm.JVMUtil;
>> import org.eclipse.net4j.util.container.ContainerUtil;
>> import org.eclipse.net4j.util.container.IManagedContainer;
>> import org.eclipse.emf.ecore.EObject;
>> import org.eclipse.emf.ecore.resource.Resource;
>> /**
>> * @author Eike Stepper
>> */
>> public class ImportXML
>> {
>> private static final String REPOSITORY_NAME = "repo1";
>> public static void main(String[] args)
>> {
>> // Prepare the standalone infra structure (not needed when
>> running inside Eclipse)
>> IManagedContainer container = ContainerUtil.createContainer();
>> // Create a wiring container
>> Net4jUtil.prepareContainer(container); // Prepare the Net4j
>> kernel
>> JVMUtil.prepareContainer(container); // Prepare the JVM transport
>> CDOServerUtil.prepareContainer(container); // Prepare the CDO
>> server
>> CDOUtil.prepareContainer(container, false); // Prepare the CDO
>> client
>> // Start the transport and create a repository
>> JVMUtil.getAcceptor(container, "default"); // Start the JVM
>> transport
>> CDOServerUtil.addRepository(container, createRepository()); //
>> Start a CDO respository
>> // Establish a communications connection and open a session
>> with the repository
>> IConnector connector = JVMUtil.getConnector(container,
>> "default"); // Open a JVM connection
>> CDOSession session = CDOUtil.openSession(connector,
>> REPOSITORY_NAME, true);// Open a CDO session
>> CDOTransaction transaction = session.openTransaction();// Open
>> a CDO transaction
>> Resource resource =
>> transaction.createResource("/my/big/resource");// Create a new EMF
>> resource
>> // Work normally with the EMF resource
>> for (int i = 0; i < 100; i++)
>> {
>> EObject inputModel = getInputModel(i);
>> resource.getContents().add(inputModel);
>> transaction.commit();
>> }
>> session.close();
>> }
>> private static IRepository createRepository()
>> {
>> Map<String, String> props = new HashMap<String, String>();
>> props.put(IRepository.PROP_SUPPORTING_REVISION_DELTAS, "true");
>> props.put(IRepository.PROP_CURRENT_LRU_CAPACITY, "10000");
>> props.put(IRepository.PROP_REVISED_LRU_CAPACITY, "10000");
>> return CDOServerUtil.createRepository(REPOSITORY_NAME,
>> createStore(), props);
>> }
>> private static IStore createStore()
>> {
>> // You might want to create an IDBStore here instead if memory
>> is an issue!
>> return StoreUtil.createMEMStore();
>> }
>> private static EObject getInputModel(int i)
>> {
>> throw new UnsupportedOperationException("Load you model here");
>> }
>> }
>>
>> It's also possible to create and wire the various components
>> yourself, but using a managed container with factories is easier. If
>> running in Eclipse even the container is created and configured
>> automatically. There's also a way to do the configuration via XML.
>> Please tell me if you need more infos.
>>
>> Regards,
>> Eike Stepper
>> ----
>> http://wiki.eclipse.org/CDO
>> http://wiki.eclipse.org/Net4j
>>
>>
>> Eike Stepper schrieb:
>>> Hi Stefan,
>>>
>>> I don't know if Teneo can do this, Martin will want to explain that.
>>>
>>> CDO can do it. With CDO the modeled instances internally are not
>>> interconnected with java reference handles. This way unneeded
>>> instances can be completely garbage collected and transparently
>>> reloaded later on. You don't need to write extra code for that to
>>> function.
>>>
>>> If you plan to construct the output model programmatically
>>> everything should work fine. If you plan to read your input models
>>> for example from xml documents they should of course be small enough
>>> to fit into memory. The resulting CDO repository resource(s) would
>>> have to remain within the repository until you have enough memory on
>>> client side that you can load them completely into an XML resource
>>> to save them to a file again.
>>>
>>> If you plan to do your conversion within one VM (that is, not
>>> through a networked remote repository) you can gain a lot of system
>>> resources (memory, performance, ...) if you decide to use a Net4j
>>> IJVMConnector instead of the usual ITCPConnector. And you can use an
>>> embedded Derby Database to further optimize, but I guess that
>>> performance is not the issue. If client memory gets low, it could
>>> even be better to use a remote database server.
>>>
>>> Regards,
>>> Eike Stepper
>>> ----
>>> http://wiki.eclipse.org/CDO
>>> http://wiki.eclipse.org/Net4j
>>>
>>>
>>>
>>>
>>>
>>> Stefan Winkler schrieb:
>>>> Hi all,
>>>>
>>>> I am working on a project which will have to deal with quite large
>>>> (both broad and deep) models. The models themselves are only read
>>>> or created (out of other models) but not modified themselves.
>>>>
>>>> Regarding the creation task: this will be something like a custom
>>>> model transformation: an algorithm will read one model and produce
>>>> another.
>>>>
>>>> Problem is: I expect the models to be too big to fit into memory,
>>>> so I need a solution which allows only parts of models to be loaded
>>>> and these parts to be unloaded, when they are not being used
>>>> anymore. In terms of EMF, I want only parts of Resources to be loaded.
>>>>
>>>> Ed Merks pointed out that CDO or Teneo could help here as they use
>>>> lazy loading strategies. So far I had a look into Teneo/Hibernate
>>>> and I like the interface so far. But: While I understand that Teneo
>>>> supports lazy loading of EMF Objects, does Teneo (or Hibernate
>>>> behind this) have support to unload objects that are no longer needed?
>>>>
>>>> E.g.
>>>>
>>>> A
>>>> |
>>>> +--- B
>>>> |
>>>> +---- C
>>>> |
>>>> +----- D
>>>>
>>>> When I start at A (B,C,D are proxies, because policy says so), I
>>>> access B,C and D. Is it possible to unload C and D and make C Proxy
>>>> again so that C and D get garbage collected?
>>>>
>>>>
>>>> If Teneo does not support this, does CDO?
>>>>
>>>>
>>>> Thanks in advance!
>>>>
>>>> Cheers,
>>>>
>>>> Stefan
>>>>
--------------020000080300010401020708
Content-Type: text/html; charset=ISO-8859-15
Content-Transfer-Encoding: 8bit
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Eike,<br>
<br>
Oooo. That's just so pretty!
Ed Merks
Professional Support: https://www.macromodeling.com/
|
|
|
| Re: [Teneo/CDO] Managing huge models / unloading parts of models [message #614077 is a reply to message #108084] |
Thu, 24 January 2008 13:03 |
|
This is a multi-part message in MIME format.
--------------060906030706060808070601
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit
Ed Merks schrieb:
> Eike,
>
> Oooo. That's just so pretty! Where did you find it?
Cute, isn't it?
Look at http://www.java2html.de/eclipse.html
Cheers
/Eike
>
>
> Eike Stepper wrote:
>> I found a nice plugin to format the java code to HTML:
>>
>> |*package *org.eclipse.emf.cdo.tests;
>>
>> *import *org.eclipse.emf.cdo.CDOSession;
>> *import *org.eclipse.emf.cdo.CDOTransaction;
>> *import *org.eclipse.emf.cdo.server.CDOServerUtil;
>> *import *org.eclipse.emf.cdo.server.IRepository;
>> *import *org.eclipse.emf.cdo.server.IStore;
>> *import *org.eclipse.emf.cdo.server.StoreUtil;
>> *import *org.eclipse.emf.cdo.util.CDOUtil;
>>
>> *import *org.eclipse.net4j.Net4jUtil;
>> *import *org.eclipse.net4j.connector.IConnector;
>> *import *org.eclipse.net4j.jvm.JVMUtil;
>> *import *org.eclipse.net4j.util.container.ContainerUtil;
>> *import *org.eclipse.net4j.util.container.IManagedContainer;
>>
>> *import *org.eclipse.emf.ecore.EObject;
>> *import *org.eclipse.emf.ecore.resource.Resource;
>>
>> *import *java.util.HashMap;
>> *import *java.util.Map;
>>
>> /**
>> * @author Eike Stepper
>> */
>> *public class *ImportXML
>> {
>> *private static final *String REPOSITORY_NAME = "repo1";
>>
>> *public static **void *main(String[] args)
>> {
>> // Prepare the standalone infra structure (not needed when running inside Eclipse)
>> IManagedContainer container = ContainerUtil.createContainer(); // Create a wiring container
>> Net4jUtil.prepareContainer(container); // Prepare the Net4j kernel
>> JVMUtil.prepareContainer(container); // Prepare the JVM transport
>> CDOServerUtil.prepareContainer(container); // Prepare the CDO server
>> CDOUtil.prepareContainer(container, *false*); // Prepare the CDO client
>>
>> // Start the transport and create a repository
>> JVMUtil.getAcceptor(container, "default"); // Start the JVM transport
>> CDOServerUtil.addRepository(container, createRepository()); // Start a CDO respository
>>
>> // Establish a communications connection and open a session with the repository
>> IConnector connector = JVMUtil.getConnector(container, "default"); // Open a JVM connection
>> CDOSession session = CDOUtil.openSession(connector, REPOSITORY_NAME, *true*);// Open a CDO session
>> CDOTransaction transaction = session.openTransaction();// Open a CDO transaction
>> Resource resource = transaction.createResource("/my/big/resource");// Create a new EMF resource
>>
>> // Work normally with the EMF resource
>> *for *(*int *i = 0; i < 100; i++)
>> {
>> EObject inputModel = getInputModel(i);
>> resource.getContents().add(inputModel);
>> transaction.commit();
>> }
>>
>> session.close();
>> }
>>
>> *private static *IRepository createRepository()
>> {
>> Map<String, String> props = *new *HashMap<String, String>();
>> props.put(IRepository.PROP_SUPPORTING_REVISION_DELTAS, "true");
>> props.put(IRepository.PROP_CURRENT_LRU_CAPACITY, "10000");
>> props.put(IRepository.PROP_REVISED_LRU_CAPACITY, "10000");
>> *return *CDOServerUtil.createRepository(REPOSITORY_NAME, createStore(), props);
>> }
>>
>> *private static *IStore createStore()
>> {
>> // You might want to create an IDBStore here instead if memory is an issue!
>> *return *StoreUtil.createMEMStore();
>> }
>>
>> *private static *EObject getInputModel(*int *i)
>> {
>> *throw new *UnsupportedOperationException("Load you model here");
>> }
>> }|
>>
>>
>> Regards,
>> Eike Stepper
>> ----
>> http://wiki.eclipse.org/CDO
>> http://wiki.eclipse.org/Net4j
>>
>>
>>
>> Eike Stepper schrieb:
>>> I've written a small example how to set up a standalone client and
>>> server within one JVM:
>>>
>>> package org.eclipse.emf.cdo.tests;
>>> import org.eclipse.emf.cdo.CDOSession;
>>> import org.eclipse.emf.cdo.CDOTransaction;
>>> import org.eclipse.emf.cdo.server.CDOServerUtil;
>>> import org.eclipse.emf.cdo.server.IRepository;
>>> import org.eclipse.emf.cdo.server.IStore;
>>> import org.eclipse.emf.cdo.server.StoreUtil;
>>> import org.eclipse.emf.cdo.util.CDOUtil;
>>> import org.eclipse.net4j.Net4jUtil;
>>> import org.eclipse.net4j.connector.IConnector;
>>> import org.eclipse.net4j.jvm.JVMUtil;
>>> import org.eclipse.net4j.util.container.ContainerUtil;
>>> import org.eclipse.net4j.util.container.IManagedContainer;
>>> import org.eclipse.emf.ecore.EObject;
>>> import org.eclipse.emf.ecore.resource.Resource;
>>> /**
>>> * @author Eike Stepper
>>> */
>>> public class ImportXML
>>> {
>>> private static final String REPOSITORY_NAME = "repo1";
>>> public static void main(String[] args)
>>> {
>>> // Prepare the standalone infra structure (not needed when
>>> running inside Eclipse)
>>> IManagedContainer container =
>>> ContainerUtil.createContainer(); // Create a wiring container
>>> Net4jUtil.prepareContainer(container); // Prepare the Net4j
>>> kernel
>>> JVMUtil.prepareContainer(container); // Prepare the JVM
>>> transport
>>> CDOServerUtil.prepareContainer(container); // Prepare the CDO
>>> server
>>> CDOUtil.prepareContainer(container, false); // Prepare the
>>> CDO client
>>> // Start the transport and create a repository
>>> JVMUtil.getAcceptor(container, "default"); // Start the JVM
>>> transport
>>> CDOServerUtil.addRepository(container, createRepository());
>>> // Start a CDO respository
>>> // Establish a communications connection and open a session
>>> with the repository
>>> IConnector connector = JVMUtil.getConnector(container,
>>> "default"); // Open a JVM connection
>>> CDOSession session = CDOUtil.openSession(connector,
>>> REPOSITORY_NAME, true);// Open a CDO session
>>> CDOTransaction transaction = session.openTransaction();//
>>> Open a CDO transaction
>>> Resource resource =
>>> transaction.createResource("/my/big/resource");// Create a new EMF
>>> resource
>>> // Work normally with the EMF resource
>>> for (int i = 0; i < 100; i++)
>>> {
>>> EObject inputModel = getInputModel(i);
>>> resource.getContents().add(inputModel);
>>> transaction.commit();
>>> }
>>> session.close();
>>> }
>>> private static IRepository createRepository()
>>> {
>>> Map<String, String> props = new HashMap<String, String>();
>>> props.put(IRepository.PROP_SUPPORTING_REVISION_DELTAS, "true");
>>> props.put(IRepository.PROP_CURRENT_LRU_CAPACITY, "10000");
>>> props.put(IRepository.PROP_REVISED_LRU_CAPACITY, "10000");
>>> return CDOServerUtil.createRepository(REPOSITORY_NAME,
>>> createStore(), props);
>>> }
>>> private static IStore createStore()
>>> {
>>> // You might want to create an IDBStore here instead if
>>> memory is an issue!
>>> return StoreUtil.createMEMStore();
>>> }
>>> private static EObject getInputModel(int i)
>>> {
>>> throw new UnsupportedOperationException("Load you model here");
>>> }
>>> }
>>>
>>> It's also possible to create and wire the various components
>>> yourself, but using a managed container with factories is easier. If
>>> running in Eclipse even the container is created and configured
>>> automatically. There's also a way to do the configuration via XML.
>>> Please tell me if you need more infos.
>>>
>>> Regards,
>>> Eike Stepper
>>> ----
>>> http://wiki.eclipse.org/CDO
>>> http://wiki.eclipse.org/Net4j
>>>
>>>
>>> Eike Stepper schrieb:
>>>> Hi Stefan,
>>>>
>>>> I don't know if Teneo can do this, Martin will want to explain that.
>>>>
>>>> CDO can do it. With CDO the modeled instances internally are not
>>>> interconnected with java reference handles. This way unneeded
>>>> instances can be completely garbage collected and transparently
>>>> reloaded later on. You don't need to write extra code for that to
>>>> function.
>>>>
>>>> If you plan to construct the output model programmatically
>>>> everything should work fine. If you plan to read your input models
>>>> for example from xml documents they should of course be small
>>>> enough to fit into memory. The resulting CDO repository resource(s)
>>>> would have to remain within the repository until you have enough
>>>> memory on client side that you can load them completely into an XML
>>>> resource to save them to a file again.
>>>>
>>>> If you plan to do your conversion within one VM (that is, not
>>>> through a networked remote repository) you can gain a lot of system
>>>> resources (memory, performance, ...) if you decide to use a Net4j
>>>> IJVMConnector instead of the usual ITCPConnector. And you can use
>>>> an embedded Derby Database to further optimize, but I guess that
>>>> performance is not the issue. If client memory gets low, it could
>>>> even be better to use a remote database server.
>>>>
>>>> Regards,
>>>> Eike Stepper
>>>> ----
>>>> http://wiki.eclipse.org/CDO
>>>> http://wiki.eclipse.org/Net4j
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Stefan Winkler schrieb:
>>>>> Hi all,
>>>>>
>>>>> I am working on a project which will have to deal with quite large
>>>>> (both broad and deep) models. The models themselves are only read
>>>>> or created (out of other models) but not modified themselves.
>>>>>
>>>>> Regarding the creation task: this will be something like a custom
>>>>> model transformation: an algorithm will read one model and produce
>>>>> another.
>>>>>
>>>>> Problem is: I expect the models to be too big to fit into memory,
>>>>> so I need a solution which allows only parts of models to be
>>>>> loaded and these parts to be unloaded, when they are not being
>>>>> used anymore. In terms of EMF, I want only parts of Resources to
>>>>> be loaded.
>>>>>
>>>>> Ed Merks pointed out that CDO or Teneo could help here as they use
>>>>> lazy loading strategies. So far I had a look into Teneo/Hibernate
>>>>> and I like the interface so far. But: While I understand that
>>>>> Teneo supports lazy loading of EMF Objects, does Teneo (or
>>>>> Hibernate behind this) have support to unload objects that are no
>>>>> longer needed?
>>>>>
>>>>> E.g.
>>>>>
>>>>> A
>>>>> |
>>>>> +--- B
>>>>> |
>>>>> +---- C
>>>>> |
>>>>> +----- D
>>>>>
>>>>> When I start at A (B,C,D are proxies, because policy says so), I
>>>>> access B,C and D. Is it possible to unload C and D and make C
>>>>> Proxy again so that C and D get garbage collected?
>>>>>
>>>>>
>>>>> If Teneo does not support this, does CDO?
>>>>>
>>>>>
>>>>> Thanks in advance!
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Stefan
>>>>>
>
--------------060906030706060808070601
Content-Type: text/html; charset=ISO-8859-15
Content-Transfer-Encoding: 8bit
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Ed Merks schrieb:
<blockquote cite="mid:fn9v9l$fsp$1@build.eclipse.org" type="cite">
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
Eike,<br>
<br>
Oooo. That's just so pretty!
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
| Re: [Teneo/CDO] Managing huge models / unloading parts of models [message #614081 is a reply to message #108068] |
Fri, 25 January 2008 10:57 |
Stefan Winkler
Messages: 307
Registered: July 2009
Location: Germany |
Senior Member |
|
|
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Eike,<br>
<br>
thanks for the code snippet.<br>
<br>
Could you again clarify a few things:<br>
<br>
When you write:<br>
<code><font color="#ffffff"><br>
</font><font color="#3f7f5f"> // Prepare the standalone infra structure (not needed when running inside Eclipse) </font><br>
<br>
</code>do you mean "not needed if the code is running in Eclipse" or
"not needed if a (separate) server is already running in Eclipse".<br>
Because: I checked out all.psf and as far as I can see the internat
server is only started, if it is run as an own product.<br>
Simply installing the plugins and features into my existing Eclipse IDE
does not start the server, right?<br>
<br>
<code><font color="#ffffff"></font><font color="#3f7f5f"> // You might want to create an IDBStore here instea d if memory is an issue! <br>
<br>
</font></code>How is this done (I'm going to use MySQL as a backend
database)?<br>
There is no StoreUtil.createIDBStore :-(<br>
<br>
Thanks for your support,<br>
<br>
Stefan<br>
<br>
<br>
<br>
Eike Stepper schrieb:
<blockquote cite="mid:fn9re7$q6m$1@build.eclipse.org" type="cite">
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
I found a nice plugin to format the java code to HTML:<br>
<br>
<title></title>
<style type="text/css">
<!--code { font-family: Courier New, Courier; font-size: 10pt; margin: 0px; }-->
</style>
<meta http-equiv="Content-Type" content="text/html; ">
<!-- ======================================================== -->
<!-- = Java Sourcecode to HTML automatically converted code = --><!-- = Java2Html Converter 5.0 [2006-02-26] by Markus Gebhard markus@jave.de = --><!-- = Further information: http://www.java2html.de = -->
<div class="java" align="left">
<table bgcolor="#ffffff" border="0" cellpadding="3" cellspacing="0">
<tbody>
<tr>
<!-- start source code --> <td align="left" nowrap="nowrap"
valign="top"> <code><font color="#7f0055"><b>package </b></font><font
color="#000000">org.eclipse.emf.cdo.tests;</font><br>
<br>
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.emf.cdo.CDOSession;</font><br >
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.emf.cdo.CDOTransaction;</font ><br>
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.emf.cdo.server.CDOServerUtil;</font ><br>
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.emf.cdo.server.IRepository;</font ><br>
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.emf.cdo.server.IStore;</font> <br>
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.emf.cdo.server.StoreUtil;</font ><br>
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.emf.cdo.util.CDOUtil;</font><br >
<br>
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.net4j.Net4jUtil;</font><br>
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.net4j.connector.IConnector;</font ><br>
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.net4j.jvm.JVMUtil;</font><br >
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.net4j.util.container.ContainerUtil; </font><br>
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.net4j.util.container.IManagedContainer; </font><br>
<br>
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.emf.ecore.EObject;</font><br >
<font color="#7f0055"><b>import </b></font><font color="#000000">org.eclipse.emf.ecore.resource.Resource;</font ><br>
<br>
<font color="#7f0055"><b>import </b></font><font color="#000000">java.util.HashMap;</font><br>
<font color="#7f0055"><b>import </b></font><font color="#000000">java.util.Map;</font><br>
<br>
<font color="#3f5fbf">/**</font><br>
<font color="#ffffff"> </font><font color="#3f5fbf">* </font><font
color="#7f9fbf">@author </font><font color="#3f5fbf">Eike Stepper</font><br>
<font color="#ffffff"> </font><font color="#3f5fbf">*/</font><br>
<font color="#7f0055"><b>public class </b></font><font
color="#000000">ImportXML</font><br>
<font color="#000000">{</font><br>
<font color="#ffffff"> </font><font color="#7f0055"><b>private static final </b></font><font
color="#000000">String REPOSITORY_NAME = </font><font color="#2a00ff">"repo1"</font><font
color="#000000">;</font><br>
<br>
<font color="#ffffff"> </font><font color="#7f0055"><b>public static </b></font><font
color="#7f0055"><b>void </b></font><font color="#000000">main</font><font
color="#000000">(</font><font color="#000000">String</font><font
color="#000000">[] </font><font color="#000000">args</font><font
color="#000000">)</font><br>
<font color="#ffffff"> </font><font color="#000000">{</font><br>
<font color="#ffffff"> </font><font color="#3f7f5f"> // Prepare the standalone infra structure (not needed when running inside Eclipse) </font><br>
<font color="#ffffff"> </font><font color="#000000"> IManagedContainer container = ContainerUtil.createContain er </font><font
color="#000000">()</font><font color="#000000">; </font><font
color="#3f7f5f">// Create a wiring container</font><br >
<font color="#ffffff"> </font><font color="#000000">Net4jUtil.prepareContainer</font><font
color="#000000">(</font><font color="#000000">container</font><font
color="#000000">)</font><font color="#000000">; </font><font
color="#3f7f5f">// Prepare the Net4j kernel</font><br >
<font color="#ffffff"> </font><font color="#000000">JVMUtil.prepareContainer</font><font
color="#000000">(</font><font color="#000000">container</font><font
color="#000000">)</font><font color="#000000">; </font><font
color="#3f7f5f">// Prepare the JVM transport</font><br >
<font color="#ffffff"> </font><font color="#000000">CDOServerUtil.prepareContainer</font><font
color="#000000">(</font><font color="#000000">container</font><font
color="#000000">)</font><font color="#000000">; </font><font
color="#3f7f5f">// Prepare the CDO server</font><br>
<font color="#ffffff"> </font><font color="#000000">CDOUtil.prepareContainer</font><font
color="#000000">(</font><font color="#000000">container, </font><font
color="#7f0055"><b>false</b></font><font color="#000000">)</font><font
color="#000000">; </font><font color="#3f7f5f">// Prepare the CDO client</font><br>
<br>
<font color="#ffffff"> </font><font color="#3f7f5f">// Start the transport and create a repository </font><br>
<font color="#ffffff"> </font><font color="#000000">JVMUtil.getAcceptor</font><font
color="#000000">(</font><font color="#000000">container, </font><font
color="#2a00ff">"default"</font><font color="#000000">)</font><font
color="#000000">; </font><font color="#3f7f5f">// Start the JVM transport</font><br >
<font color="#ffffff"> </font><font color="#000000">CDOServerUtil.addRepository</font><font
color="#000000">(</font><font color="#000000">container, createRepository</font><font
color="#000000">())</font><font color="#000000">; </font><font
color="#3f7f5f">// Start a CDO respository</font><br >
<br>
<font color="#ffffff"> </font><font color="#3f7f5f"> // Establish a communications connection and open a session with the repository </font><br>
<font color="#ffffff"> </font><font color="#000000">IConnector connector = JVMUtil.getConnector </font><font
color="#000000">(</font><font color="#000000">container, </font><font
color="#2a00ff">"default"</font><font color="#000000">)</font><font
color="#000000">; </font><font color="#3f7f5f">// Open a JVM connection</font><br>
<font color="#ffffff"> </font><font color="#000000">CDOSession session = CDOUtil.openSession </font><font
color="#000000">(</font><font color="#000000">connector, REPOSITORY_NAME, </font><font
color="#7f0055"><b>true</b></font><font color="#000000">)</font><font
color="#000000">;</font><font color="#3f7f5f">// Open a CDO session</font><br>
<font color="#ffffff"> </font><font color="#000000">CDOTransaction transaction = session.openTransaction </font><font
color="#000000">()</font><font color="#000000">;</font><font
color="#3f7f5f">// Open a CDO transaction</font><br>
<font color="#ffffff"> </font><font color="#000000">Resource resource = transaction.createResource </font><font
color="#000000">(</font><font color="#2a00ff">"/my/big/resource"</font><font
color="#000000">)</font><font color="#000000">;</font><font
color="#3f7f5f">// Create a new EMF resource</font><br >
<br>
<font color="#ffffff"> </font><font color="#3f7f5f">// Work normally with the EMF resource </font><br>
<font color="#ffffff"> </font><font color="#7f0055"><b>for </b></font><font
color="#000000">(</font><font color="#7f0055"><b>int </b></font><font
color="#000000">i = </font><font color="#990000">0</font><font
color="#000000">; i < </font><font color="#990000">100</font><font
color="#000000">; i++</font><font color="#000000">)</font><br>
<font color="#ffffff"> </font><font color="#000000">{</font><br>
<font color="#ffffff"> </font><font color="#000000">EObject inputModel = getInputModel</font ><font
color="#000000">(</font><font color="#000000">i</font><font
color="#000000">)</font><font color="#000000">;</font><br>
<font color="#ffffff"> </font><font color="#000000">resource.getContents</font><font
color="#000000">()</font><font color="#000000">.add</font><font
color="#000000">(</font><font color="#000000">inputModel</font><font
color="#000000">)</font><font color="#000000">;</font><br>
<font color="#ffffff"> </font><font color="#000000">transaction.commit</font><font
color="#000000">()</font><font color="#000000">;</font><br>
<font color="#ffffff"> </font><font color="#000000">}</font><br>
<br>
<font color="#ffffff"> </font><font color="#000000">session.close</font><font
color="#000000">()</font><font color="#000000">;</font><br>
<font color="#ffffff"> </font><font color="#000000">}</font><br>
<br>
<font color="#ffffff"> </font><font color="#7f0055"><b>private static </b></font><font
color="#000000">IRepository createRepository</font><font
color="#000000">()</font><br>
<font color="#ffffff"> </font><font color="#000000">{</font><br>
<font color="#ffffff"> </font><font color="#000000">Map<String, String> props = </font><font
color="#7f0055"><b>new </b></font><font color="#000000">HashMap<String, String></font ><font
color="#000000">()</font><font color="#000000">;</font><br>
<font color="#ffffff"> </font><font color="#000000">props.put</font><font
color="#000000">(</font><font color="#000000">IRepository.PROP_SUPPORTING_REVISION_DELTAS, </font><font
color="#2a00ff">"true"</font><font color="#000000">)</font><font
color="#000000">;</font><br>
<font color="#ffffff"> </font><font color="#000000">props.put</font><font
color="#000000">(</font><font color="#000000">IRepository.PROP_CURRENT_LRU_CAPACITY, </font ><font
color="#2a00ff">"10000"</font><font color="#000000">)</font><font
color="#000000">;</font><br>
<font color="#ffffff"> </font><font color="#000000">props.put</font><font
color="#000000">(</font><font color="#000000">IRepository.PROP_REVISED_LRU_CAPACITY, </font ><font
color="#2a00ff">"10000"</font><font color="#000000">)</font><font
color="#000000">;</font><br>
<font color="#ffffff"> </font><font color="#7f0055"><b>return </b></font><font
color="#000000">CDOServerUtil.createRepository</font><font
color="#000000">(</font><font color="#000000">REPOSITORY_NAME, createStore</font><font
color="#000000">()</font><font color="#000000">, props</font><font
color="#000000">)</font><font color="#000000">;</font><br>
<font color="#ffffff"> </font><font color="#000000">}</font><br>
<br>
<font color="#ffffff"> </font><font color="#7f0055"><b>private static </b></font><font
color="#000000">IStore createStore</font><font color="#000000">()</font><br>
<font color="#ffffff"> </font><font color="#000000">{</font><br>
<font color="#ffffff"> </font><font color="#3f7f5f"> // You might want to create an IDBStore here instea d if memory is an issue! </font><br>
<font color="#ffffff"> </font><font color="#7f0055"><b>return </b></font><font
color="#000000">StoreUtil.createMEMStore</font><font color="#000000">()</font><font
color="#000000">;</font><br>
<font color="#ffffff"> </font><font color="#000000">}</font><br>
<br>
<font color="#ffffff"> </font><font color="#7f0055"><b>private static </b></font><font
color="#000000">EObject getInputModel</font><font color="#000000">(</font><font
color="#7f0055"><b>int </b></font><font color="#000000">i</font><font
color="#000000">)</font><br>
<font color="#ffffff"> </font><font color="#000000">{</font><br>
<font color="#ffffff"> </font><font color="#7f0055"><b>throw new </b></font><font
color="#000000">UnsupportedOperationException</font><font
color="#000000">(</font><font color="#2a00ff">"Load you model here"</font><font
color="#000000">)</font><font color="#000000">;</font><br>
<font color="#ffffff"> </font><font color="#000000">}</font><br>
<font color="#000000">}</font></code> </td>
<!-- end source code --> </tr>
</tbody>
</table>
</div>
<!-- = END of automatically generated HTML code = -->
<!-- ======================================================== --><br>
Regards,<br>
Eike Stepper<br>
----<br>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://wiki.eclipse.org/CDO">http://wiki.eclipse.org/CDO</a><br>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://wiki.eclipse.org/Net4j">http://wiki.eclipse.org/Net4j</a><br>
<br>
<br>
<br>
Eike Stepper schrieb:
<blockquote cite="mid:fn7mcv$nfg$2@build.eclipse.org" type="cite">I've
written a small example how to set up a standalone client and server
within one JVM: <br>
<br>
package org.eclipse.emf.cdo.tests; <br>
import org.eclipse.emf.cdo.CDOSession; <br>
import org.eclipse.emf.cdo.CDOTransaction; <br>
import org.eclipse.emf.cdo.server.CDOServerUtil; <br>
import org.eclipse.emf.cdo.server.IRepository; <br>
import org.eclipse.emf.cdo.server.IStore; <br>
import org.eclipse.emf.cdo.server.StoreUtil; <br>
import org.eclipse.emf.cdo.util.CDOUtil; <br>
import org.eclipse.net4j.Net4jUtil; <br>
import org.eclipse.net4j.connector.IConnector; <br>
import org.eclipse.net4j.jvm.JVMUtil; <br>
import org.eclipse.net4j.util.container.ContainerUtil; <br>
import org.eclipse.net4j.util.container.IManagedContainer; <br>
import org.eclipse.emf.ecore.EObject; <br>
import org.eclipse.emf.ecore.resource.Resource; <br>
/** <br>
* @author Eike Stepper <br>
*/ <br>
public class ImportXML <br>
{ <br>
private static final String REPOSITORY_NAME = "repo1"; <br>
public static void main(String[] args) <br>
{ <br>
// Prepare the standalone infra structure (not needed when
running inside Eclipse) <br>
IManagedContainer container = ContainerUtil.createContainer();
// Create a wiring container <br>
Net4jUtil.prepareContainer(container); // Prepare the Net4j
kernel <br>
JVMUtil.prepareContainer(container); // Prepare the JVM
transport <br>
CDOServerUtil.prepareContainer(container); // Prepare the CDO
server <br>
CDOUtil.prepareContainer(container, false); // Prepare the CDO
client <br>
// Start the transport and create a repository <br>
JVMUtil.getAcceptor(container, "default"); // Start the JVM
transport <br>
CDOServerUtil.addRepository(container, createRepository()); //
Start a CDO respository <br>
// Establish a communications connection and open a session
with the repository <br>
IConnector connector = JVMUtil.getConnector(container,
"default"); // Open a JVM connection <br>
CDOSession session = CDOUtil.openSession(connector,
REPOSITORY_NAME, true);// Open a CDO session <br>
CDOTransaction transaction = session.openTransaction();// Open a
CDO transaction <br>
Resource resource =
transaction.createResource("/my/big/resource");// Create a new EMF
resource <br>
// Work normally with the EMF resource <br>
for (int i = 0; i < 100; i++) <br>
{ <br>
EObject inputModel = getInputModel(i); <br>
resource.getContents().add(inputModel); <br>
transaction.commit(); <br>
} <br>
session.close(); <br>
} <br>
private static IRepository createRepository() <br>
{ <br>
Map<String, String> props = new HashMap<String,
String>(); <br>
props.put(IRepository.PROP_SUPPORTING_REVISION_DELTAS, "true"); <br>
props.put(IRepository.PROP_CURRENT_LRU_CAPACITY, "10000"); <br>
props.put(IRepository.PROP_REVISED_LRU_CAPACITY, "10000"); <br>
return CDOServerUtil.createRepository(REPOSITORY_NAME,
createStore(), props); <br>
} <br>
private static IStore createStore() <br>
{ <br>
// You might want to create an IDBStore here instead if memory
is an issue! <br>
return StoreUtil.createMEMStore(); <br>
} <br>
private static EObject getInputModel(int i) <br>
{ <br>
throw new UnsupportedOperationException("Load you model here"); <br>
} <br>
} <br>
<br>
It's also possible to create and wire the various components yourself,
but using a managed container with factories is easier. If running in
Eclipse even the container is created and configured automatically.
There's also a way to do the configuration via XML. Please tell me if
you need more infos. <br>
<br>
Regards, <br>
Eike Stepper <br>
---- <br>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://wiki.eclipse.org/CDO">http://wiki.eclipse.org/CDO</a> <br>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://wiki.eclipse.org/Net4j">http://wiki.eclipse.org/Net4j</a>
<br>
<br>
<br>
Eike Stepper schrieb: <br>
<blockquote type="cite">Hi Stefan, <br>
<br>
I don't know if Teneo can do this, Martin will want to explain that. <br>
<br>
CDO can do it. With CDO the modeled instances internally are not
interconnected with java reference handles. This way unneeded instances
can be completely garbage collected and transparently reloaded later
on. You don't need to write extra code for that to function. <br>
<br>
If you plan to construct the output model programmatically everything
should work fine. If you plan to read your input models for example
from xml documents they should of course be small enough to fit into
memory. The resulting CDO repository resource(s) would have to remain
within the repository until you have enough memory on client side that
you can load them completely into an XML resource to save them to a
file again. <br>
<br>
If you plan to do your conversion within one VM (that is, not through a
networked remote repository) you can gain a lot of system resources
(memory, performance, ...) if you decide to use a Net4j IJVMConnector
instead of the usual ITCPConnector. And you can use an embedded Derby
Database to further optimize, but I guess that performance is not the
issue. If client memory gets low, it could even be better to use a
remote database server. <br>
<br>
Regards, <br>
Eike Stepper <br>
---- <br>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://wiki.eclipse.org/CDO">http://wiki.eclipse.org/CDO</a> <br>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://wiki.eclipse.org/Net4j">http://wiki.eclipse.org/Net4j</a>
<br>
<br>
<br>
<br>
<br>
<br>
Stefan Winkler schrieb: <br>
<blockquote type="cite">Hi all, <br>
<br>
I am working on a project which will have to deal with quite large
(both broad and deep) models. The models themselves are only read or
created (out of other models) but not modified themselves. <br>
<br>
Regarding the creation task: this will be something like a custom model
transformation: an algorithm will read one model and produce another. <br>
<br>
Problem is: I expect the models to be too big to fit into memory, so I
need a solution which allows only parts of models to be loaded and
these parts to be unloaded, when they are not being used anymore. In
terms of EMF, I want only parts of Resources to be loaded. <br>
<br>
Ed Merks pointed out that CDO or Teneo could help here as they use lazy
loading strategies. So far I had a look into Teneo/Hibernate and I like
the interface so far. But: While I understand that Teneo supports lazy
loading of EMF Objects, does Teneo (or Hibernate behind this) have
support to unload objects that are no longer needed? <br>
<br>
E.g. <br>
<br>
A <br>
| <br>
+--- B <br>
| <br>
+---- C <br>
| <br>
+----- D <br>
<br>
When I start at A (B,C,D are proxies, because policy says so), I access
B,C and D. Is it possible to unload C and D and make C Proxy again so
that C and D get garbage collected? <br>
<br>
<br>
If Teneo does not support this, does CDO? <br>
<br>
<br>
Thanks in advance! <br>
<br>
Cheers, <br>
<br>
Stefan <br>
<br>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
<br>
<br>
</body>
</html>
|
|
|
| Re: [Teneo/CDO] Managing huge models / unloading parts of models [message #614082 is a reply to message #108165] |
Fri, 25 January 2008 12:27 |
|
This is a multi-part message in MIME format.
--------------090109090003060102050408
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit
Hi Stefan,
Comments below...
Stefan Winkler schrieb:
> Hi Eike,
>
> thanks for the code snippet.
>
> Could you again clarify a few things:
>
> When you write:
> |
>
> // Prepare the standalone infra structure (not needed when running inside Eclipse)
>
> |do you mean "not needed if the code is running in Eclipse" or "not
> needed if a (separate) server is already running in Eclipse".
I mean that, if you're running within Eclipse, you can use
org.eclipse.net4j.util.container.IPluginContainer.INSTANCE which is a
managed container that is configured through extension points. Then you
don't need to decide which factories to use. Just deploy the plugins and
use the container to retrieve the connector.
> Because: I checked out all.psf and as far as I can see the internat
> server is only started, if it is run as an own product.
> Simply installing the plugins and features into my existing Eclipse
> IDE does not start the server, right?
No and yes ;-)
The server is not started automatically just because its code is deployed.
My example code in fact doesn't start a real server node at all, but
rather starts a CDO repository in the client (programmatically).
Usually a dedicated server node is started as an Eclipse Application or
an Eclipse Product. But that's just a matter of convenience.
>
>
> |// You might want to create an IDBStore here instead if memory is an issue!
>
> |How is this done (I'm going to use MySQL as a backend database)?
> There is no StoreUtil.createIDBStore :-(
That's just because the plugin of StoreUtil has no dependency on the
plugin of the IDBStore.
You can create an IDBStore by calling
new DBStore(...)
There's currently no public API to directly create instances of DBStore
(enhancement request?).
Usually the whole repository is configured through an XML file like the
following:
<?xml version="1.0" encoding="UTF-8"?>
<cdoServer>
<acceptor type="tcp" listenAddr="0.0.0.0" port="2036">
<!--
<negotiator type="challenge" description="/temp/users.db"/>
-->
</acceptor>
<repository name="repo1">
<property name="overrideUUID"
value="1ff5d226-b1f0-40fb-aba2-0c31b38c764f"/>
<property name="supportingAudits" value="true"/>
<property name="verifyingRevisions" value="false"/>
<property name="currentLRUCapacity" value="10000"/>
<property name="revisedLRUCapacity" value="100"/>
<store type="db">
<!-- type: horizontal | vertical | <any user-contributed
type> -->
<mappingStrategy type="horizontal">
<!-- ONE_TABLE_PER_REFERENCE | ONE_TABLE_PER_CLASS |
ONE_TABLE_PER_PACKAGE | ONE_TABLE_PER_REPOSITORY | LIKE_ATTRIBUTES -->
<property name="toManyReferences"
value="ONE_TABLE_PER_REFERENCE"/>
<!-- LIKE_ATTRIBUTES | LIKE_TO_MANY_REFERENCES -->
<property name="toOneReferences" value="LIKE_ATTRIBUTES"/>
<!-- MODEL | STRATEGY -->
<property name="mappingPrecedence" value="MODEL"/>
</mappingStrategy>
<!--<dbAdapter name="derby"/>
<dataSource class="org.apache.derby.jdbc.EmbeddedDataSource"
databaseName="/temp/cdodb1"
createDatabase="create"/> -->
<dbAdapter name="derby"/>
<dataSource class="org.apache.derby.jdbc.ClientDataSource"
databaseName="cdodb1"
createDatabase="create"/>
<!--<dbAdapter name="hsqldb"/>
<dataSource
class="org.eclipse.net4j.db.hsqldb.HSQLDBDataSource"
database="jdbc:hsqldb:mem:cdodb1"
user="sa"/>-->
<!--<dbAdapter name="mysql"/>
<dataSource
class="com.mysql.jdbc.jdbc2.optional.MysqlDataSource"
url="jdbc:mysql://localhost/cdodb1"
user="sa"/>-->
</store>
</repository>
</cdoServer>
You could use code like the following to start the server:
| File configFile = OMPlatform.INSTANCE.getConfigFile("cdo-server.xml");
RepositoryConfigurator repositoryConfigurator = *new *RepositoryConfigurator(IPluginContainer.INSTANCE);
IRepository[] repositories = repositoryConfigurator.configure(configFile);
Net4jConfigurator net4jConfigurator = *new *Net4jConfigurator(IPluginContainer.INSTANCE);
IAcceptor[] acceptors = net4jConfigurator.configure(configFile);
// Work with the repositories
*for *(IAcceptor acceptor : acceptors)
{
LifecycleUtil.deactivate(acceptor);
}
*for *(IRepository repository : repositories)
{
LifecycleUtil.deactivate(repository);
}|
I know I'll have to write lots of wiki pages, but I hope this helps.
Please don't hesitate to ask further questions!
Regards,
Eike Stepper
----
http://wiki.eclipse.org/CDO
http://wiki.eclipse.org/Net4j
>
> Thanks for your support,
>
> Stefan
>
>
>
> Eike Stepper schrieb:
>> I found a nice plugin to format the java code to HTML:
>>
>> |*package *org.eclipse.emf.cdo.tests;
>>
>> *import *org.eclipse.emf.cdo.CDOSession;
>> *import *org.eclipse.emf.cdo.CDOTransaction;
>> *import *org.eclipse.emf.cdo.server.CDOServerUtil;
>> *import *org.eclipse.emf.cdo.server.IRepository;
>> *import *org.eclipse.emf.cdo.server.IStore;
>> *import *org.eclipse.emf.cdo.server.StoreUtil;
>> *import *org.eclipse.emf.cdo.util.CDOUtil;
>>
>> *import *org.eclipse.net4j.Net4jUtil;
>> *import *org.eclipse.net4j.connector.IConnector;
>> *import *org.eclipse.net4j.jvm.JVMUtil;
>> *import *org.eclipse.net4j.util.container.ContainerUtil;
>> *import *org.eclipse.net4j.util.container.IManagedContainer;
>>
>> *import *org.eclipse.emf.ecore.EObject;
>> *import *org.eclipse.emf.ecore.resource.Resource;
>>
>> *import *java.util.HashMap;
>> *import *java.util.Map;
>>
>> /**
>> * @author Eike Stepper
>> */
>> *public class *ImportXML
>> {
>> *private static final *String REPOSITORY_NAME = "repo1";
>>
>> *public static **void *main(String[] args)
>> {
>> // Prepare the standalone infra structure (not needed when running inside Eclipse)
>> IManagedContainer container = ContainerUtil.createContainer(); // Create a wiring container
>> Net4jUtil.prepareContainer(container); // Prepare the Net4j kernel
>> JVMUtil.prepareContainer(container); // Prepare the JVM transport
>> CDOServerUtil.prepareContainer(container); // Prepare the CDO server
>> CDOUtil.prepareContainer(container, *false*); // Prepare the CDO client
>>
>> // Start the transport and create a repository
>> JVMUtil.getAcceptor(container, "default"); // Start the JVM transport
>> CDOServerUtil.addRepository(container, createRepository()); // Start a CDO respository
>>
>> // Establish a communications connection and open a session with the repository
>> IConnector connector = JVMUtil.getConnector(container, "default"); // Open a JVM connection
>> CDOSession session = CDOUtil.openSession(connector, REPOSITORY_NAME, *true*);// Open a CDO session
>> CDOTransaction transaction = session.openTransaction();// Open a CDO transaction
>> Resource resource = transaction.createResource("/my/big/resource");// Create a new EMF resource
>>
>> // Work normally with the EMF resource
>> *for *(*int *i = 0; i < 100; i++)
>> {
>> EObject inputModel = getInputModel(i);
>> resource.getContents().add(inputModel);
>> transaction.commit();
>> }
>>
>> session.close();
>> }
>>
>> *private static *IRepository createRepository()
>> {
>> Map<String, String> props = *new *HashMap<String, String>();
>> props.put(IRepository.PROP_SUPPORTING_REVISION_DELTAS, "true");
>> props.put(IRepository.PROP_CURRENT_LRU_CAPACITY, "10000");
>> props.put(IRepository.PROP_REVISED_LRU_CAPACITY, "10000");
>> *return *CDOServerUtil.createRepository(REPOSITORY_NAME, createStore(), props);
>> }
>>
>> *private static *IStore createStore()
>> {
>> // You might want to create an IDBStore here instead if memory is an issue!
>> *return *StoreUtil.createMEMStore();
>> }
>>
>> *private static *EObject getInputModel(*int *i)
>> {
>> *throw new *UnsupportedOperationException("Load you model here");
>> }
>> }|
>>
>>
>> Regards,
>> Eike Stepper
>> ----
>> http://wiki.eclipse.org/CDO
>> http://wiki.eclipse.org/Net4j
>>
>>
>>
>> Eike Stepper schrieb:
>>> I've written a small example how to set up a standalone client and
>>> server within one JVM:
>>>
>>> package org.eclipse.emf.cdo.tests;
>>> import org.eclipse.emf.cdo.CDOSession;
>>> import org.eclipse.emf.cdo.CDOTransaction;
>>> import org.eclipse.emf.cdo.server.CDOServerUtil;
>>> import org.eclipse.emf.cdo.server.IRepository;
>>> import org.eclipse.emf.cdo.server.IStore;
>>> import org.eclipse.emf.cdo.server.StoreUtil;
>>> import org.eclipse.emf.cdo.util.CDOUtil;
>>> import org.eclipse.net4j.Net4jUtil;
>>> import org.eclipse.net4j.connector.IConnector;
>>> import org.eclipse.net4j.jvm.JVMUtil;
>>> import org.eclipse.net4j.util.container.ContainerUtil;
>>> import org.eclipse.net4j.util.container.IManagedContainer;
>>> import org.eclipse.emf.ecore.EObject;
>>> import org.eclipse.emf.ecore.resource.Resource;
>>> /**
>>> * @author Eike Stepper
>>> */
>>> public class ImportXML
>>> {
>>> private static final String REPOSITORY_NAME = "repo1";
>>> public static void main(String[] args)
>>> {
>>> // Prepare the standalone infra structure (not needed when
>>> running inside Eclipse)
>>> IManagedContainer container =
>>> ContainerUtil.createContainer(); // Create a wiring container
>>> Net4jUtil.prepareContainer(container); // Prepare the Net4j
>>> kernel
>>> JVMUtil.prepareContainer(container); // Prepare the JVM
>>> transport
>>> CDOServerUtil.prepareContainer(container); // Prepare the CDO
>>> server
>>> CDOUtil.prepareContainer(container, false); // Prepare the
>>> CDO client
>>> // Start the transport and create a repository
>>> JVMUtil.getAcceptor(container, "default"); // Start the JVM
>>> transport
>>> CDOServerUtil.addRepository(container, createRepository());
>>> // Start a CDO respository
>>> // Establish a communications connection and open a session
>>> with the repository
>>> IConnector connector = JVMUtil.getConnector(container,
>>> "default"); // Open a JVM connection
>>> CDOSession session = CDOUtil.openSession(connector,
>>> REPOSITORY_NAME, true);// Open a CDO session
>>> CDOTransaction transaction = session.openTransaction();//
>>> Open a CDO transaction
>>> Resource resource =
>>> transaction.createResource("/my/big/resource");// Create a new EMF
>>> resource
>>> // Work normally with the EMF resource
>>> for (int i = 0; i < 100; i++)
>>> {
>>> EObject inputModel = getInputModel(i);
>>> resource.getContents().add(inputModel);
>>> transaction.commit();
>>> }
>>> session.close();
>>> }
>>> private static IRepository createRepository()
>>> {
>>> Map<String, String> props = new HashMap<String, String>();
>>> props.put(IRepository.PROP_SUPPORTING_REVISION_DELTAS, "true");
>>> props.put(IRepository.PROP_CURRENT_LRU_CAPACITY, "10000");
>>> props.put(IRepository.PROP_REVISED_LRU_CAPACITY, "10000");
>>> return CDOServerUtil.createRepository(REPOSITORY_NAME,
>>> createStore(), props);
>>> }
>>> private static IStore createStore()
>>> {
>>> // You might want to create an IDBStore here instead if
>>> memory is an issue!
>>> return StoreUtil.createMEMStore();
>>> }
>>> private static EObject getInputModel(int i)
>>> {
>>> throw new UnsupportedOperationException("Load you model here");
>>> }
>>> }
>>>
>>> It's also possible to create and wire the various components
>>> yourself, but using a managed container with factories is easier. If
>>> running in Eclipse even the container is created and configured
>>> automatically. There's also a way to do the configuration via XML.
>>> Please tell me if you need more infos.
>>>
>>> Regards,
>>> Eike Stepper
>>> ----
>>> http://wiki.eclipse.org/CDO
>>> http://wiki.eclipse.org/Net4j
>>>
>>>
>>> Eike Stepper schrieb:
>>>> Hi Stefan,
>>>>
>>>> I don't know if Teneo can do this, Martin will want to explain that.
>>>>
>>>> CDO can do it. With CDO the modeled instances internally are not
>>>> interconnected with java reference handles. This way unneeded
>>>> instances can be completely garbage collected and transparently
>>>> reloaded later on. You don't need to write extra code for that to
>>>> function.
>>>>
>>>> If you plan to construct the output model programmatically
>>>> everything should work fine. If you plan to read your input models
>>>> for example from xml documents they should of course be small
>>>> enough to fit into memory. The resulting CDO repository resource(s)
>>>> would have to remain within the repository until you have enough
>>>> memory on client side that you can load them completely into an XML
>>>> resource to save them to a file again.
>>>>
>>>> If you plan to do your conversion within one VM (that is, not
>>>> through a networked remote repository) you can gain a lot of system
>>>> resources (memory, performance, ...) if you decide to use a Net4j
>>>> IJVMConnector instead of the usual ITCPConnector. And you can use
>>>> an embedded Derby Database to further optimize, but I guess that
>>>> performance is not the issue. If client memory gets low, it could
>>>> even be better to use a remote database server.
>>>>
>>>> Regards,
>>>> Eike Stepper
>>>> ----
>>>> http://wiki.eclipse.org/CDO
>>>> http://wiki.eclipse.org/Net4j
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Stefan Winkler schrieb:
>>>>> Hi all,
>>>>>
>>>>> I am working on a project which will have to deal with quite large
>>>>> (both broad and deep) models. The models themselves are only read
>>>>> or created (out of other models) but not modified themselves.
>>>>>
>>>>> Regarding the creation task: this will be something like a custom
>>>>> model transformation: an algorithm will read one model and produce
>>>>> another.
>>>>>
>>>>> Problem is: I expect the models to be too big to fit into memory,
>>>>> so I need a solution which allows only parts of models to be
>>>>> loaded and these parts to be unloaded, when they are not being
>>>>> used anymore. In terms of EMF, I want only parts of Resources to
>>>>> be loaded.
>>>>>
>>>>> Ed Merks pointed out that CDO or Teneo could help here as they use
>>>>> lazy loading strategies. So far I had a look into Teneo/Hibernate
>>>>> and I like the interface so far. But: While I understand that
>>>>> Teneo supports lazy loading of EMF Objects, does Teneo (or
>>>>> Hibernate behind this) have support to unload objects that are no
>>>>> longer needed?
>>>>>
>>>>> E.g.
>>>>>
>>>>> A
>>>>> |
>>>>> +--- B
>>>>> |
>>>>> +---- C
>>>>> |
>>>>> +----- D
>>>>>
>>>>> When I start at A (B,C,D are proxies, because policy says so), I
>>>>> access B,C and D. Is it possible to unload C and D and make C
>>>>> Proxy again so that C and D get garbage collected?
>>>>>
>>>>>
>>>>> If Teneo does not support this, does CDO?
>>>>>
>>>>>
>>>>> Thanks in advance!
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Stefan
>>>>>
>
>
--------------090109090003060102050408
Content-Type: text/html; charset=ISO-8859-15
Content-Transfer-Encoding: 8bit
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Stefan,<br>
<br>
Comments below...<br>
<br>
<br>
Stefan Winkler schrieb:
<blockquote cite="mid:fncfah$p7e$1@build.eclipse.org" type="cite">
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
<title></title>
Hi Eike,<br>
<br>
thanks for the code snippet.<br>
<br>
Could you again clarify a few things:<br>
<br>
When you write:<br>
<code><font color="#ffffff"><br>
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
| [CDO] Repository inside Eclipse [message #614095 is a reply to message #108183] |
Sat, 26 January 2008 13:44 |
Stefan Winkler
Messages: 307
Registered: July 2009
Location: Germany |
Senior Member |
|
|
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Eike,<br>
<br>
ok, this is what I'm doing now and it seems to work (Thunderbird does
not like to insert Java2Html code from clipboard :-( ):<br>
<pre>public static void run() throws Exception {
IManagedContainer container = IPluginContainer.INSTANCE;
JVMUtil.getAcceptor(container, "default");
URL url = Activator.getDefault().getBundle().getEntry("/configuration/cdo-config.xml ");
url = FileLocator.toFileURL(url);
IRepository r = new RepositoryConfigurator(container).configure(new File(url.toURI()))[0];
String repositoryName = r.getName();
CDOServerUtil.addRepository(container, r);
IConnector connector = JVMUtil.getConnector(container, "default");
CDOSession session = CDOUtil.openSession(connector, repositoryName, true);
CDOTransaction transaction = session.openTransaction();
Resource resource = transaction.createResource("/my/big/resource");
for (int i = 0; i < 100; i++) {
EObject inputModel = getInputModel(i);
resource.getContents().add(inputModel);
transaction.commit();
}
session.close();
LifecycleUtil.deactivate(r);
}
</pre>
This seems to be doing what I want, so far. <br>
<br>
Now some advanced questions:<br>
<br>
1.<br>
Is your debug facility configurable? I only get one logging message
stating that a legacy-something is not available. I'd like to see a bit
more of what is going on behind the scenes...<br>
<br>
2.<br>
Basically my application connsists of two parts. One is the
computation-intensive part (reading a huge model and generating the new
one).<br>
The other one is a (rather) small model, which is kept in a viewer and
which the user can select and edit.<br>
<br>
So regarding transactions, the first part seems easy: start a
transaction, read the one model, create the other, commit transaction,
fine.<br>
But how do I deal with the second case - or more generally: what
happens to a resource, when the transaction is committed?<br>
<br>
Can I do something like<br>
<pre> // in the view initialization:
CDOView view = session.openView();
Resource r = view.getResource("/some/path");
editingDomain.getResourceSet().getResources().add(r);
[...]
// when the user creates an element in the model (Action handler) -
// myEObject is contained in Resource r
CDOTransaction transaction = session.openTransaction(editingDomain.getResourceSet());
myEObject.getChildren().add(newObject);
transaction.commit();
</pre>
<br>
3. <br>
How do the resource paths map? Are these just some kind of "ID" for
resources so that <br>
transaction.createResource("/some/path");<br>
always returns equivalent resources and<br>
transaction.createResource("/some/path/other");<br>
returns a different one which has nothing to do with the first one?<br>
<br>
If so, do inter-resource-references in the same repository work?<br>
<br>
4.<br>
Does a session time out? Or can I start a session when the application
starts up and close it, when it shuts down?<br>
<br>
Some commennts below ...<br>
<br>
<br>
<br>
<br>
Eike Stepper schrieb::
<blockquote cite="mid:fnckjs$bua$1@build.eclipse.org" type="cite">
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
<blockquote cite="mid:fncfah$p7e$1@build.eclipse.org" type="cite">
<code><font color="#3f7f5f"> // You might want to create an IDBStore here instea d if memory is an issue! <br>
<br>
</font></code>How is this done (I'm going to use MySQL as a backend
database)?<br>
There is no StoreUtil.createIDBStore :-(<br>
</blockquote>
That's just because the plugin of StoreUtil has no dependency on the
plugin of the IDBStore.<br>
You can create an IDBStore by calling <br>
<br>
new DBStore(...)<br>
<br>
There's currently no public API to directly create instances of DBStore
(enhancement request?).<br>
</blockquote>
Filed: <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=216668"
title="NEW - Add public API to support creation of IDBStore programmatically">Bug
216668</a>
<blockquote cite="mid:fnckjs$bua$1@build.eclipse.org" type="cite">I
know I'll have to write lots of wiki pages, but I hope this helps.
Please don't hesitate to ask further questions!<br>
</blockquote>
You'll regret your last sentence ;-))<br>
<br>
Cheers,<br>
Stefan<br>
<br>
</body>
</html>
|
|
|
| Re: [CDO] Repository inside Eclipse [message #614097 is a reply to message #108372] |
Sat, 26 January 2008 13:51 |
Ed Merks
Messages: 33264
Registered: July 2009 |
Senior Member |
|
|
This is a multi-part message in MIME format.
--------------010602050003010906040501
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit
Stefann,
With regard to Java2Html and Thunderbird, the trick is to use
"Insert->HTML...", paste the contents in there, and then hit OK.
Stefann Winkler wrote:
> Hi Eike,
>
> ok, this is what I'm doing now and it seems to work (Thunderbird does
> not like to insert Java2Html code from clipboard :-( ):
> public static void run() throws Exception {
>
> IManagedContainer container = IPluginContainer.INSTANCE;
>
> JVMUtil.getAcceptor(container, "default");
>
> URL url = Activator.getDefault().getBundle().getEntry("/configuration/cdo-config.xml ");
> url = FileLocator.toFileURL(url);
>
> IRepository r = new RepositoryConfigurator(container).configure(new File(url.toURI()))[0];
> String repositoryName = r.getName();
> CDOServerUtil.addRepository(container, r);
>
> IConnector connector = JVMUtil.getConnector(container, "default");
> CDOSession session = CDOUtil.openSession(connector, repositoryName, true);
> CDOTransaction transaction = session.openTransaction();
>
> Resource resource = transaction.createResource("/my/big/resource");
>
> for (int i = 0; i < 100; i++) {
> EObject inputModel = getInputModel(i);
> resource.getContents().add(inputModel);
> transaction.commit();
> }
> session.close();
> LifecycleUtil.deactivate(r);
> }
>
>
> This seems to be doing what I want, so far.
>
> Now some advanced questions:
>
> 1.
> Is your debug facility configurable? I only get one logging message
> stating that a legacy-something is not available. I'd like to see a
> bit more of what is going on behind the scenes...
>
> 2.
> Basically my application connsists of two parts. One is the
> computation-intensive part (reading a huge model and generating the
> new one).
> The other one is a (rather) small model, which is kept in a viewer and
> which the user can select and edit.
>
> So regarding transactions, the first part seems easy: start a
> transaction, read the one model, create the other, commit transaction,
> fine.
> But how do I deal with the second case - or more generally: what
> happens to a resource, when the transaction is committed?
>
> Can I do something like
> // in the view initialization:
> CDOView view = session.openView();
> Resource r = view.getResource("/some/path");
> editingDomain.getResourceSet().getResources().add(r);
>
> [...]
>
> // when the user creates an element in the model (Action handler) -
> // myEObject is contained in Resource r
> CDOTransaction transaction = session.openTransaction(editingDomain.getResourceSet());
> myEObject.getChildren().add(newObject);
> transaction.commit();
>
>
> 3.
> How do the resource paths map? Are these just some kind of "ID" for
> resources so that
> transaction.createResource("/some/path");
> always returns equivalent resources and
> transaction.createResource("/some/path/other");
> returns a different one which has nothing to do with the first one?
>
> If so, do inter-resource-references in the same repository work?
>
> 4.
> Does a session time out? Or can I start a session when the application
> starts up and close it, when it shuts down?
>
> Some commennts below ...
>
>
>
>
> Eike Stepper schrieb::
>>>
>>> |// You might want to create an IDBStore here instead if memory is an issue!
>>>
>>> |How is this done (I'm going to use MySQL as a backend database)?
>>> There is no StoreUtil.createIDBStore :-(
>> That's just because the plugin of StoreUtil has no dependency on the
>> plugin of the IDBStore.
>> You can create an IDBStore by calling
>>
>> new DBStore(...)
>>
>> There's currently no public API to directly create instances of
>> DBStore (enhancement request?).
> Filed: Bug 216668 <https://bugs.eclipse.org/bugs/show_bug.cgi?id=216668>
>> I know I'll have to write lots of wiki pages, but I hope this helps.
>> Please don't hesitate to ask further questions!
> You'll regret your last sentence ;-))
>
> Cheers,
> Stefan
>
--------------010602050003010906040501
Content-Type: text/html; charset=ISO-8859-15
Content-Transfer-Encoding: 8bit
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Stefann,<br>
<br>
With regard to Java2Html and Thunderbird, the trick is to use
"Insert->HTML...", paste the contents in there, and then hit OK.<br>
<br>
Stefann Winkler wrote:
<blockquote cite="mid:fnfdgp$vo1$1@build.eclipse.org" type="cite">
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
<title></title>
Hi Eike,<br>
<br>
ok, this is what I'm doing now and it seems to work (Thunderbird does
not like to insert Java2Html code from clipboard :-( ):<br>
<pre>public static void run() throws Exception {
IManagedContainer container = IPluginContainer.INSTANCE;
JVMUtil.getAcceptor(container, "default");
URL url = Activator.getDefault().getBundle().getEntry("/configuration/cdo-config.xml ");
url = FileLocator.toFileURL(url);
IRepository r = new RepositoryConfigurator(container).configure(new File(url.toURI()))[0];
String repositoryName = r.getName();
CDOServerUtil.addRepository(container, r);
IConnector connector = JVMUtil.getConnector(container, "default");
CDOSession session = CDOUtil.openSession(connector, repositoryName, true);
CDOTransaction transaction = session.openTransaction();
Resource resource = transaction.createResource("/my/big/resource");
for (int i = 0; i < 100; i++) {
EObject inputModel = getInputModel(i);
resource.getContents().add(inputModel);
transaction.commit();
}
session.close();
LifecycleUtil.deactivate(r);
}
</pre>
This seems to be doing what I want, so far. <br>
<br>
Now some advanced questions:<br>
<br>
1.<br>
Is your debug facility configurable? I only get one logging message
stating that a legacy-something is not available. I'd like to see a bit
more of what is going on behind the scenes...<br>
<br>
2.<br>
Basically my application connsists of two parts. One is the
computation-intensive part (reading a huge model and generating the new
one).<br>
The other one is a (rather) small model, which is kept in a viewer and
which the user can select and edit.<br>
<br>
So regarding transactions, the first part seems easy: start a
transaction, read the one model, create the other, commit transaction,
fine.<br>
But how do I deal with the second case - or more generally: what
happens to a resource, when the transaction is committed?<br>
<br>
Can I do something like<br>
<pre> // in the view initialization:
CDOView view = session.openView();
Resource r = view.getResource("/some/path");
editingDomain.getResourceSet().getResources().add(r);
[...]
// when the user creates an element in the model (Action handler) -
// myEObject is contained in Resource r
CDOTransaction transaction = session.openTransaction(editingDomain.getResourceSet());
myEObject.getChildren().add(newObject);
transaction.commit();
</pre>
<br>
3. <br>
How do the resource paths map? Are these just some kind of "ID" for
resources so that <br>
transaction.createResource("/some/path");<br>
always returns equivalent resources and<br>
transaction.createResource("/some/path/other");<br>
returns a different one which has nothing to do with the first one?<br>
<br>
If so, do inter-resource-references in the same repository work?<br>
<br>
4.<br>
Does a session time out? Or can I start a session when the application
starts up and close it, when it shuts down?<br>
<br>
Some commennts below ...<br>
<br>
<br>
<br>
<br>
Eike Stepper schrieb::
<blockquote cite="mid:fnckjs$bua$1@build.eclipse.org" type="cite">
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
<blockquote cite="mid:fncfah$p7e$1@build.eclipse.org" type="cite">
Ed Merks
Professional Support: https://www.macromodeling.com/
|
|
|
| Re: [CDO] Repository inside Eclipse [message #614098 is a reply to message #108372] |
Sat, 26 January 2008 14:56 |
|
This is a multi-part message in MIME format.
--------------060606020400040400010700
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit
Hi Stefan,
Comments below...
Stefann Winkler schrieb:
> Hi Eike,
>
> ok, this is what I'm doing now and it seems to work (Thunderbird does
> not like to insert Java2Html code from clipboard :-( ):
> public static void run() throws Exception {
>
> IManagedContainer container = IPluginContainer.INSTANCE;
>
> JVMUtil.getAcceptor(container, "default");
>
> URL url = Activator.getDefault().getBundle().getEntry("/configuration/cdo-config.xml ");
> url = FileLocator.toFileURL(url);
>
> IRepository r = new RepositoryConfigurator(container).configure(new File(url.toURI()))[0];
> String repositoryName = r.getName();
> CDOServerUtil.addRepository(container, r);
>
> IConnector connector = JVMUtil.getConnector(container, "default");
> CDOSession session = CDOUtil.openSession(connector, repositoryName, true);
> CDOTransaction transaction = session.openTransaction();
>
> Resource resource = transaction.createResource("/my/big/resource");
>
> for (int i = 0; i < 100; i++) {
> EObject inputModel = getInputModel(i);
> resource.getContents().add(inputModel);
> transaction.commit();
>
It could eventually be faster to commit the transaction only once after
the for loop.
> }
> session.close();
> LifecycleUtil.deactivate(r);
> }
>
Don't forget to close() or LifecycleUtil.deactivate() the connector and
later the acceptor.
>
> This seems to be doing what I want, so far.
Yes, your code looks fine.
>
> Now some advanced questions:
>
> 1.
> Is your debug facility configurable? I only get one logging message
> stating that a legacy-something is not available. I'd like to see a
> bit more of what is going on behind the scenes...
The OM (Operations&Maintenance) tracing facility integrates with the
Eclipse/OSGi Tracing facility if you run within Eclipse/OSGi. You can
use the Tracing tab in the launch config to configure the details. Or,
if you run headless, you can use the .options file:
http://wiki.eclipse.org/FAQ_How_do_I_use_the_platform_debug_ tracing_facility%3F#Turning_on_debug_tracing
Each Net4j and CDO bundle has an own .options file which shows the
possible tracing entities. The whole OM tracing facility can also work
standalone without change.
>
> 2.
> Basically my application connsists of two parts. One is the
> computation-intensive part (reading a huge model and generating the
> new one).
> The other one is a (rather) small model, which is kept in a viewer and
> which the user can select and edit.
>
> So regarding transactions, the first part seems easy: start a
> transaction, read the one model, create the other, commit transaction,
> fine.
> But how do I deal with the second case - or more generally: what
> happens to a resource, when the transaction is committed?
>
> Can I do something like
> // in the view initialization:
> CDOView view = session.openView();
> Resource r = view.getResource("/some/path");
> editingDomain.getResourceSet().getResources().add(r);
>
The last statement seems wrong to me. because it will detach the
resource from the ResourceSet that is internally created on
session.openView(). See my explanation below...
>
> [...]
>
> // when the user creates an element in the model (Action handler) -
> // myEObject is contained in Resource r
> CDOTransaction transaction = session.openTransaction(editingDomain.getResourceSet());
> myEObject.getChildren().add(newObject);
> transaction.commit();
>
Some explanation:
In CDO an EMF ResourceSet is associated with a CDOView. CDOTransaction
is a special CDOView. You could eventually say CDOView is a read-only
transaction. Each ResourceSet can only have a single CDOView (or
CDOTransaction) associated at any point in time. If you call
session.openTransaction() a new ResourceSet is created internally. If
you call session.openTransaction(myResourceSet) a given ResourceSet is
used for that transaction. But only if the given ResourceSet doesn't
already have another view/transaction associated. BTW. this association
happens internally via a ResourceSet adapter.
If there is a chance that the model will be modified, you should open a
transaction from the beginning and use it for reads and writes later on.
If the model is nonetheless *not* changed in a transaction it doesn't
cost you much to have a transaction instead of a read-only view.
But even if you open several views and transactions on the same session,
they will share all the object state as long as they're not modified.
Model state is stored/managed in the session itself. Only change deltas
are stored in the transactions.
>
> 3.
> How do the resource paths map? Are these just some kind of "ID" for
> resources so that
> transaction.createResource("/some/path");
> always returns equivalent resources and
> transaction.createResource("/some/path/other");
> returns a different one which has nothing to do with the first one?
Yes. In each CDOObject has a CDOID (long integer number) which is
automatically assigned and can not be changed. The CDOID of an object is
unique within the associated repository. Since a CDOResource is also an
EObject and a CDOObject it has such a CDOID, too.
The CDOResources are the entry points into the model tree of the
repository and the resource paths are a more meaningful means (than
CDOIDs) to enter this tree.
>
> If so, do inter-resource-references in the same repository work?
Yes, without any exemption. Including inter-resource-moves.
>
> 4.
> Does a session time out? Or can I start a session when the application
> starts up and close it, when it shuts down?
It doesn't time out unless the underlying connector/channel doesn't time
out. Both the TCPConnector and JVMConnector don't time out automatically.
>
> Some commennts below ...
>
>
>
>
> Eike Stepper schrieb::
>>>
>>> |// You might want to create an IDBStore here instead if memory is an issue!
>>>
>>> |How is this done (I'm going to use MySQL as a backend database)?
>>> There is no StoreUtil.createIDBStore :-(
>> That's just because the plugin of StoreUtil has no dependency on the
>> plugin of the IDBStore.
>> You can create an IDBStore by calling
>>
>> new DBStore(...)
>>
>> There's currently no public API to directly create instances of
>> DBStore (enhancement request?).
> Filed: Bug 216668 <https://bugs.eclipse.org/bugs/show_bug.cgi?id=216668>
Thx ;-)
>> I know I'll have to write lots of wiki pages, but I hope this helps.
>> Please don't hesitate to ask further questions!
> You'll regret your last sentence ;-))
I bet that not!! ;-)
Regards,
Eike Stepper
----
http://wiki.eclipse.org/CDO
http://wiki.eclipse.org/Net4j
--------------060606020400040400010700
Content-Type: text/html; charset=ISO-8859-15
Content-Transfer-Encoding: 8bit
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Stefan,<br>
<br>
Comments below...<br>
<br>
<br>
Stefann Winkler schrieb:
<blockquote cite="mid:fnfdgp$vo1$1@build.eclipse.org" type="cite">
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
<title></title>
Hi Eike,<br>
<br>
ok, this is what I'm doing now and it seems to work (Thunderbird does
not like to insert Java2Html code from clipboard :-( ):<br>
<pre>public static void run() throws Exception {
IManagedContainer container = IPluginContainer.INSTANCE;
JVMUtil.getAcceptor(container, "default");
URL url = Activator.getDefault().getBundle().getEntry("/configuration/cdo-config.xml ");
url = FileLocator.toFileURL(url);
IRepository r = new RepositoryConfigurator(container).configure(new File(url.toURI()))[0];
String repositoryName = r.getName();
CDOServerUtil.addRepository(container, r);
IConnector connector = JVMUtil.getConnector(container, "default");
CDOSession session = CDOUtil.openSession(connector, repositoryName, true);
CDOTransaction transaction = session.openTransaction();
Resource resource = transaction.createResource("/my/big/resource");
for (int i = 0; i < 100; i++) {
EObject inputModel = getInputModel(i);
resource.getContents().add(inputModel);
transaction.commit();
</pre>
</blockquote>
It could eventually be faster to commit the transaction only once after
the for loop.<br>
<br>
<blockquote cite="mid:fnfdgp$vo1$1@build.eclipse.org" type="cite">
<pre> }
session.close();
LifecycleUtil.deactivate(r);
}
</pre>
</blockquote>
Don't forget to close() or LifecycleUtil.deactivate() the connector and
later the acceptor.<br>
<br>
<blockquote cite="mid:fnfdgp$vo1$1@build.eclipse.org" type="cite">
<pre>
</pre>
This seems to be doing what I want, so far. <br>
</blockquote>
Yes, your code looks fine.<br>
<br>
<blockquote cite="mid:fnfdgp$vo1$1@build.eclipse.org" type="cite"><br>
Now some advanced questions:<br>
<br>
1.<br>
Is your debug facility configurable? I only get one logging message
stating that a legacy-something is not available. I'd like to see a bit
more of what is going on behind the scenes...<br>
</blockquote>
The OM (Operations&Maintenance) tracing facility integrates with
the Eclipse/OSGi Tracing facility if you run within Eclipse/OSGi. You
can use the Tracing tab in the launch config to configure the details.
Or, if you run headless, you can use the .options file:<br>
<pre id="line198"><span class="attribute-value"> <a class="moz-txt-link-freetext" href=" http://wiki.eclipse.org/FAQ_How_do_I_use_the_platform_debug_ tracing_facility%3F#Turning_on_debug_tracing"> http://wiki.eclipse.org/FAQ_How_do_I_use_the_platform_debug_ tracing_facility%3F#Turning_on_debug_tracing</a>
</span></pre>
Each Net4j and CDO bundle has an own .options file which shows the
possible tracing entities. The whole OM tracing facility can also work
standalone without change.<br>
<br>
<blockquote cite="mid:fnfdgp$vo1$1@build.eclipse.org" type="cite"><br>
2.<br>
Basically my application connsists of two parts. One is the
computation-intensive part (reading a huge model and generating the new
one).<br>
The other one is a (rather) small model, which is kept in a viewer and
which the user can select and edit.<br>
<br>
So regarding transactions, the first part seems easy: start a
transaction, read the one model, create the other, commit transaction,
fine.<br>
But how do I deal with the second case - or more generally: what
happens to a resource, when the transaction is committed?<br>
<br>
Can I do something like<br>
<pre> // in the view initialization:
CDOView view = session.openView();
Resource r = view.getResource("/some/path");
editingDomain.getResourceSet().getResources().add(r);
</pre>
</blockquote>
The last statement seems wrong to me. because it will detach the
resource from the ResourceSet that is internally created on
session.openView(). See my explanation below...<br>
<br>
<blockquote cite="mid:fnfdgp$vo1$1@build.eclipse.org" type="cite">
<pre>
[...]
// when the user creates an element in the model (Action handler) -
// myEObject is contained in Resource r
CDOTransaction transaction = session.openTransaction(editingDomain.getResourceSet());
myEObject.getChildren().add(newObject);
transaction.commit();
</pre>
</blockquote>
Some explanation:<br>
In CDO an EMF ResourceSet is associated with a CDOView. CDOTransaction
is a special CDOView. You could eventually say CDOView is a read-only
transaction. Each ResourceSet can only have a single CDOView (or
CDOTransaction) associated at any point in time. If you call
session.openTransaction() a new ResourceSet is created internally. If
you call session.openTransaction(myResourceSet) a given ResourceSet is
used for that transaction. But only if the given ResourceSet doesn't
already have another view/transaction associated. BTW. this association
happens internally via a ResourceSet adapter.<br>
<br>
If there is a chance that the model will be modified, you should open a
transaction from the beginning and use it for reads and writes later
on. If the model is nonetheless *not* changed in a transaction it
doesn't cost you much to have a transaction instead of a read-only
view. <br>
<br>
But even if you open several views and transactions on the same
session, they will share all the object state as long as they're not
modified. Model state is stored/managed in the session itself. Only
change deltas are stored in the transactions.<br>
<br>
<blockquote cite="mid:fnfdgp$vo1$1@build.eclipse.org" type="cite"><br>
3. <br>
How do the resource paths map? Are these just some kind of "ID" for
resources so that <br>
transaction.createResource("/some/path");<br>
always returns equivalent resources and<br>
transaction.createResource("/some/path/other");<br>
returns a different one which has nothing to do with the first one?<br>
</blockquote>
Yes. In each CDOObject has a CDOID (long integer number) which is
automatically assigned and can not be changed. The CDOID of an object
is unique within the associated repository. Since a CDOResource is also
an EObject and a CDOObject it has such a CDOID, too.<br>
<br>
The CDOResources are the entry points into the model tree of the
repository and the resource paths are a more meaningful means (than
CDOIDs) to enter this tree.<br>
<br>
<blockquote cite="mid:fnfdgp$vo1$1@build.eclipse.org" type="cite"><br>
If so, do inter-resource-references in the same repository work?<br>
</blockquote>
Yes, without any exemption. Including inter-resource-moves.<br>
<br>
<blockquote cite="mid:fnfdgp$vo1$1@build.eclipse.org" type="cite"><br>
4.<br>
Does a session time out? Or can I start a session when the application
starts up and close it, when it shuts down?<br>
</blockquote>
It doesn't time out unless the underlying connector/channel doesn't
time out. Both the TCPConnector and JVMConnector don't time out
automatically.<br>
<br>
<blockquote cite="mid:fnfdgp$vo1$1@build.eclipse.org" type="cite"><br>
Some commennts below ...<br>
<br>
<br>
<br>
<br>
Eike Stepper schrieb::
<blockquote cite="mid:fnckjs$bua$1@build.eclipse.org" type="cite">
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
<blockquote cite="mid:fncfah$p7e$1@build.eclipse.org" type="cite">
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
| Re: [CDO] Repository inside Eclipse [message #614099 is a reply to message #108403] |
Sat, 26 January 2008 14:57 |
|
This is a multi-part message in MIME format.
--------------030503030205020509070901
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit
Ed Merks schrieb:
> Stefann,
>
> With regard to Java2Html and Thunderbird, the trick is to use
> "Insert->HTML...", paste the contents in there, and then hit OK.
Yes, it's a really cute and small tool ;-)
Regards,
Eike Stepper
----
http://wiki.eclipse.org/CDO
http://wiki.eclipse.org/Net4j
>
> Stefann Winkler wrote:
>> Hi Eike,
>>
>> ok, this is what I'm doing now and it seems to work (Thunderbird does
>> not like to insert Java2Html code from clipboard :-( ):
>> public static void run() throws Exception {
>>
>> IManagedContainer container = IPluginContainer.INSTANCE;
>>
>> JVMUtil.getAcceptor(container, "default");
>>
>> URL url = Activator.getDefault().getBundle().getEntry("/configuration/cdo-config.xml ");
>> url = FileLocator.toFileURL(url);
>>
>> IRepository r = new RepositoryConfigurator(container).configure(new File(url.toURI()))[0];
>> String repositoryName = r.getName();
>> CDOServerUtil.addRepository(container, r);
>>
>> IConnector connector = JVMUtil.getConnector(container, "default");
>> CDOSession session = CDOUtil.openSession(connector, repositoryName, true);
>> CDOTransaction transaction = session.openTransaction();
>>
>> Resource resource = transaction.createResource("/my/big/resource");
>>
>> for (int i = 0; i < 100; i++) {
>> EObject inputModel = getInputModel(i);
>> resource.getContents().add(inputModel);
>> transaction.commit();
>> }
>> session.close();
>> LifecycleUtil.deactivate(r);
>> }
>>
>>
>> This seems to be doing what I want, so far.
>>
>> Now some advanced questions:
>>
>> 1.
>> Is your debug facility configurable? I only get one logging message
>> stating that a legacy-something is not available. I'd like to see a
>> bit more of what is going on behind the scenes...
>>
>> 2.
>> Basically my application connsists of two parts. One is the
>> computation-intensive part (reading a huge model and generating the
>> new one).
>> The other one is a (rather) small model, which is kept in a viewer
>> and which the user can select and edit.
>>
>> So regarding transactions, the first part seems easy: start a
>> transaction, read the one model, create the other, commit
>> transaction, fine.
>> But how do I deal with the second case - or more generally: what
>> happens to a resource, when the transaction is committed?
>>
>> Can I do something like
>> // in the view initialization:
>> CDOView view = session.openView();
>> Resource r = view.getResource("/some/path");
>> editingDomain.getResourceSet().getResources().add(r);
>>
>> [...]
>>
>> // when the user creates an element in the model (Action handler) -
>> // myEObject is contained in Resource r
>> CDOTransaction transaction = session.openTransaction(editingDomain.getResourceSet());
>> myEObject.getChildren().add(newObject);
>> transaction.commit();
>>
>>
>> 3.
>> How do the resource paths map? Are these just some kind of "ID" for
>> resources so that
>> transaction.createResource("/some/path");
>> always returns equivalent resources and
>> transaction.createResource("/some/path/other");
>> returns a different one which has nothing to do with the first one?
>>
>> If so, do inter-resource-references in the same repository work?
>>
>> 4.
>> Does a session time out? Or can I start a session when the
>> application starts up and close it, when it shuts down?
>>
>> Some commennts below ...
>>
>>
>>
>>
>> Eike Stepper schrieb::
>>>>
>>>> |// You might want to create an IDBStore here instead if memory is an issue!
>>>>
>>>> |How is this done (I'm going to use MySQL as a backend database)?
>>>> There is no StoreUtil.createIDBStore :-(
>>> That's just because the plugin of StoreUtil has no dependency on the
>>> plugin of the IDBStore.
>>> You can create an IDBStore by calling
>>>
>>> new DBStore(...)
>>>
>>> There's currently no public API to directly create instances of
>>> DBStore (enhancement request?).
>> Filed: Bug 216668 <https://bugs.eclipse.org/bugs/show_bug.cgi?id=216668>
>>> I know I'll have to write lots of wiki pages, but I hope this helps.
>>> Please don't hesitate to ask further questions!
>> You'll regret your last sentence ;-))
>>
>> Cheers,
>> Stefan
>>
>
--------------030503030205020509070901
Content-Type: text/html; charset=ISO-8859-15
Content-Transfer-Encoding: 8bit
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Ed Merks schrieb:
<blockquote cite="mid:fnfdtt$105$2@build.eclipse.org" type="cite">
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
Stefann,<br>
<br>
With regard to Java2Html and Thunderbird, the trick is to use
"Insert->HTML...", paste the contents in there, and then hit OK.<br>
</blockquote>
Yes, it's a really cute and small tool ;-)<br>
<br>
Regards,<br>
Eike Stepper<br>
----<br>
<a class="moz-txt-link-freetext" href="http://wiki.eclipse.org/CDO">http://wiki.eclipse.org/CDO</a><br>
<a class="moz-txt-link-freetext" href="http://wiki.eclipse.org/Net4j">http://wiki.eclipse.org/Net4j</a><br>
<br>
<br>
<blockquote cite="mid:fnfdtt$105$2@build.eclipse.org" type="cite"><br>
Stefann Winkler wrote:
<blockquote cite="mid:fnfdgp$vo1$1@build.eclipse.org" type="cite">
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
<title></title>
Hi Eike,<br>
<br>
ok, this is what I'm doing now and it seems to work (Thunderbird does
not like to insert Java2Html code from clipboard :-( ):<br>
<pre>public static void run() throws Exception {
IManagedContainer container = IPluginContainer.INSTANCE;
JVMUtil.getAcceptor(container, "default");
URL url = Activator.getDefault().getBundle().getEntry("/configuration/cdo-config.xml ");
url = FileLocator.toFileURL(url);
IRepository r = new RepositoryConfigurator(container).configure(new File(url.toURI()))[0];
String repositoryName = r.getName();
CDOServerUtil.addRepository(container, r);
IConnector connector = JVMUtil.getConnector(container, "default");
CDOSession session = CDOUtil.openSession(connector, repositoryName, true);
CDOTransaction transaction = session.openTransaction();
Resource resource = transaction.createResource("/my/big/resource");
for (int i = 0; i < 100; i++) {
EObject inputModel = getInputModel(i);
resource.getContents().add(inputModel);
transaction.commit();
}
session.close();
LifecycleUtil.deactivate(r);
}
</pre>
This seems to be doing what I want, so far. <br>
<br>
Now some advanced questions:<br>
<br>
1.<br>
Is your debug facility configurable? I only get one logging message
stating that a legacy-something is not available. I'd like to see a bit
more of what is going on behind the scenes...<br>
<br>
2.<br>
Basically my application connsists of two parts. One is the
computation-intensive part (reading a huge model and generating the new
one).<br>
The other one is a (rather) small model, which is kept in a viewer and
which the user can select and edit.<br>
<br>
So regarding transactions, the first part seems easy: start a
transaction, read the one model, create the other, commit transaction,
fine.<br>
But how do I deal with the second case - or more generally: what
happens to a resource, when the transaction is committed?<br>
<br>
Can I do something like<br>
<pre> // in the view initialization:
CDOView view = session.openView();
Resource r = view.getResource("/some/path");
editingDomain.getResourceSet().getResources().add(r);
[...]
// when the user creates an element in the model (Action handler) -
// myEObject is contained in Resource r
CDOTransaction transaction = session.openTransaction(editingDomain.getResourceSet());
myEObject.getChildren().add(newObject);
transaction.commit();
</pre>
<br>
3. <br>
How do the resource paths map? Are these just some kind of "ID" for
resources so that <br>
transaction.createResource("/some/path");<br>
always returns equivalent resources and<br>
transaction.createResource("/some/path/other");<br>
returns a different one which has nothing to do with the first one?<br>
<br>
If so, do inter-resource-references in the same repository work?<br>
<br>
4.<br>
Does a session time out? Or can I start a session when the application
starts up and close it, when it shuts down?<br>
<br>
Some commennts below ...<br>
<br>
<br>
<br>
<br>
Eike Stepper schrieb::
<blockquote cite="mid:fnckjs$bua$1@build.eclipse.org" type="cite">
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
<blockquote cite="mid:fncfah$p7e$1@build.eclipse.org" type="cite">
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
| Re: [CDO] Repository inside Eclipse [message #614104 is a reply to message #108372] |
Sun, 27 January 2008 09:56 |
|
This is a multi-part message in MIME format.
--------------040406070502030204040607
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit
Hi Stefan,
I forgot to comment on the trace output with the "legacy-something".
Originally CDO was designed to operate on models that are generated
dedicatedly for CDO, i.e. their genmodel has been modified as per
description in http://wiki.eclipse.org/Preparing_EMF_Models_for_CDO . I
call these models "native" (with respect to CDO).
For various reasons there are models that can't be native. For instance
their genmodel is not available or some of their genmodel properties
can't be set to CDO native values (Root Extends Class, ...). The Ecore
meta meta model can't be regenerated natively because of resulting
circular dependencies between EMF and CDO. I have invented an AspectJ
based approach to make CDO operate on these models as well. I call these
woven models "legacy" (with respect to CDO). The CDO Weaver can be used
to convert most generated models to legacy models. Alll the
depenpendencies of these models have to be converted to legacy models as
well. The Ecore meta meta model is a special dependency of all other
models and if Ecore itself is not converted to a legacy model I say the
"legacy-system is unavailable". It might be less confusing if this info
message would only be logged if there are other legacy models that
needed a converted Ecore model but it is not very fast to determine this
case.
BTW. native models are to be preferred over legacy models whenever
possible because they're much more efficient!
Regards,
Eike Stepper
----
http://wiki.eclipse.org/CDO
http://wiki.eclipse.org/Net4j
Stefann Winkler schrieb:
> Hi Eike,
>
> ok, this is what I'm doing now and it seems to work (Thunderbird does
> not like to insert Java2Html code from clipboard :-( ):
> public static void run() throws Exception {
>
> IManagedContainer container = IPluginContainer.INSTANCE;
>
> JVMUtil.getAcceptor(container, "default");
>
> URL url = Activator.getDefault().getBundle().getEntry("/configuration/cdo-config.xml ");
> url = FileLocator.toFileURL(url);
>
> IRepository r = new RepositoryConfigurator(container).configure(new File(url.toURI()))[0];
> String repositoryName = r.getName();
> CDOServerUtil.addRepository(container, r);
>
> IConnector connector = JVMUtil.getConnector(container, "default");
> CDOSession session = CDOUtil.openSession(connector, repositoryName, true);
> CDOTransaction transaction = session.openTransaction();
>
> Resource resource = transaction.createResource("/my/big/resource");
>
> for (int i = 0; i < 100; i++) {
> EObject inputModel = getInputModel(i);
> resource.getContents().add(inputModel);
> transaction.commit();
> }
> session.close();
> LifecycleUtil.deactivate(r);
> }
>
>
> This seems to be doing what I want, so far.
>
> Now some advanced questions:
>
> 1.
> Is your debug facility configurable? I only get one logging message
> stating that a legacy-something is not available. I'd like to see a
> bit more of what is going on behind the scenes...
>
> 2.
> Basically my application connsists of two parts. One is the
> computation-intensive part (reading a huge model and generating the
> new one).
> The other one is a (rather) small model, which is kept in a viewer and
> which the user can select and edit.
>
> So regarding transactions, the first part seems easy: start a
> transaction, read the one model, create the other, commit transaction,
> fine.
> But how do I deal with the second case - or more generally: what
> happens to a resource, when the transaction is committed?
>
> Can I do something like
> // in the view initialization:
> CDOView view = session.openView();
> Resource r = view.getResource("/some/path");
> editingDomain.getResourceSet().getResources().add(r);
>
> [...]
>
> // when the user creates an element in the model (Action handler) -
> // myEObject is contained in Resource r
> CDOTransaction transaction = session.openTransaction(editingDomain.getResourceSet());
> myEObject.getChildren().add(newObject);
> transaction.commit();
>
>
> 3.
> How do the resource paths map? Are these just some kind of "ID" for
> resources so that
> transaction.createResource("/some/path");
> always returns equivalent resources and
> transaction.createResource("/some/path/other");
> returns a different one which has nothing to do with the first one?
>
> If so, do inter-resource-references in the same repository work?
>
> 4.
> Does a session time out? Or can I start a session when the application
> starts up and close it, when it shuts down?
>
> Some commennts below ...
>
>
>
>
> Eike Stepper schrieb::
>>>
>>> |// You might want to create an IDBStore here instead if memory is an issue!
>>>
>>> |How is this done (I'm going to use MySQL as a backend database)?
>>> There is no StoreUtil.createIDBStore :-(
>> That's just because the plugin of StoreUtil has no dependency on the
>> plugin of the IDBStore.
>> You can create an IDBStore by calling
>>
>> new DBStore(...)
>>
>> There's currently no public API to directly create instances of
>> DBStore (enhancement request?).
> Filed: Bug 216668 <https://bugs.eclipse.org/bugs/show_bug.cgi?id=216668>
>> I know I'll have to write lots of wiki pages, but I hope this helps.
>> Please don't hesitate to ask further questions!
> You'll regret your last sentence ;-))
>
> Cheers,
> Stefan
>
--------------040406070502030204040607
Content-Type: text/html; charset=ISO-8859-15
Content-Transfer-Encoding: 8bit
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Stefan,<br>
<br>
I forgot to comment on the trace output with the "legacy-something".<br>
<br>
Originally CDO was designed to operate on models that are generated
dedicatedly for CDO, i.e. their genmodel has been modified as per
description in <a class="moz-txt-link-freetext" href="http://wiki.eclipse.org/Preparing_EMF_Models_for_CDO">http://wiki.eclipse.org/Preparing_EMF_Models_for_CDO</a> . I
call these models "native" (with respect to CDO).<br>
<br>
For various reasons there are models that can't be native. For instance
their genmodel is not available or some of their genmodel properties
can't be set to CDO native values (Root Extends Class, ...). The Ecore
meta meta model can't be regenerated natively because of resulting
circular dependencies between EMF and CDO. I have invented an AspectJ
based approach to make CDO operate on these models as well. I call
these woven models "legacy" (with respect to CDO). The CDO Weaver can
be used to convert most generated models to legacy models. Alll the
depenpendencies of these models have to be converted to legacy models
as well. The Ecore meta meta model is a special dependency of all other
models and if Ecore itself is not converted to a legacy model I say the
"legacy-system is unavailable". It might be less confusing if this info
message would only be logged if there are other legacy models that
needed a converted Ecore model but it is not very fast to determine
this case.<br>
<br>
BTW. native models are to be preferred over legacy models whenever
possible because they're much more efficient!<br>
<br>
Regards,<br>
Eike Stepper<br>
----<br>
<a class="moz-txt-link-freetext" href="http://wiki.eclipse.org/CDO">http://wiki.eclipse.org/CDO</a><br>
<a class="moz-txt-link-freetext" href="http://wiki.eclipse.org/Net4j">http://wiki.eclipse.org/Net4j</a><br>
<br>
<br>
<br>
Stefann Winkler schrieb:
<blockquote cite="mid:fnfdgp$vo1$1@build.eclipse.org" type="cite">
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
<title></title>
Hi Eike,<br>
<br>
ok, this is what I'm doing now and it seems to work (Thunderbird does
not like to insert Java2Html code from clipboard :-( ):<br>
<pre>public static void run() throws Exception {
IManagedContainer container = IPluginContainer.INSTANCE;
JVMUtil.getAcceptor(container, "default");
URL url = Activator.getDefault().getBundle().getEntry("/configuration/cdo-config.xml ");
url = FileLocator.toFileURL(url);
IRepository r = new RepositoryConfigurator(container).configure(new File(url.toURI()))[0];
String repositoryName = r.getName();
CDOServerUtil.addRepository(container, r);
IConnector connector = JVMUtil.getConnector(container, "default");
CDOSession session = CDOUtil.openSession(connector, repositoryName, true);
CDOTransaction transaction = session.openTransaction();
Resource resource = transaction.createResource("/my/big/resource");
for (int i = 0; i < 100; i++) {
EObject inputModel = getInputModel(i);
resource.getContents().add(inputModel);
transaction.commit();
}
session.close();
LifecycleUtil.deactivate(r);
}
</pre>
This seems to be doing what I want, so far. <br>
<br>
Now some advanced questions:<br>
<br>
1.<br>
Is your debug facility configurable? I only get one logging message
stating that a legacy-something is not available. I'd like to see a bit
more of what is going on behind the scenes...<br>
<br>
2.<br>
Basically my application connsists of two parts. One is the
computation-intensive part (reading a huge model and generating the new
one).<br>
The other one is a (rather) small model, which is kept in a viewer and
which the user can select and edit.<br>
<br>
So regarding transactions, the first part seems easy: start a
transaction, read the one model, create the other, commit transaction,
fine.<br>
But how do I deal with the second case - or more generally: what
happens to a resource, when the transaction is committed?<br>
<br>
Can I do something like<br>
<pre> // in the view initialization:
CDOView view = session.openView();
Resource r = view.getResource("/some/path");
editingDomain.getResourceSet().getResources().add(r);
[...]
// when the user creates an element in the model (Action handler) -
// myEObject is contained in Resource r
CDOTransaction transaction = session.openTransaction(editingDomain.getResourceSet());
myEObject.getChildren().add(newObject);
transaction.commit();
</pre>
<br>
3. <br>
How do the resource paths map? Are these just some kind of "ID" for
resources so that <br>
transaction.createResource("/some/path");<br>
always returns equivalent resources and<br>
transaction.createResource("/some/path/other");<br>
returns a different one which has nothing to do with the first one?<br>
<br>
If so, do inter-resource-references in the same repository work?<br>
<br>
4.<br>
Does a session time out? Or can I start a session when the application
starts up and close it, when it shuts down?<br>
<br>
Some commennts below ...<br>
<br>
<br>
<br>
<br>
Eike Stepper schrieb::
<blockquote cite="mid:fnckjs$bua$1@build.eclipse.org" type="cite">
<meta content="text/html;charset=ISO-8859-15"
http-equiv="Content-Type">
<blockquote cite="mid:fncfah$p7e$1@build.eclipse.org" type="cite">
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
| Re: [CDO] Repository inside Eclipse [message #614113 is a reply to message #108420] |
Mon, 28 January 2008 14:07 |
Stefan Winkler
Messages: 307
Registered: July 2009
Location: Germany |
Senior Member |
|
|
Hi Eike,
Comments below ...
>> Now some advanced questions:
>>
>>
>> 2.
>> Basically my application connsists of two parts. One is the
>> computation-intensive part (reading a huge model and generating the
>> new one).
>> The other one is a (rather) small model, which is kept in a viewer
>> and which the user can select and edit.
>>
>> So regarding transactions, the first part seems easy: start a
>> transaction, read the one model, create the other, commit
>> transaction, fine.
>> But how do I deal with the second case - or more generally: what
>> happens to a resource, when the transaction is committed?
>>
>> Can I do something like
>> // in the view initialization:
>> CDOView view = session.openView();
>> Resource r = view.getResource("/some/path");
>> editingDomain.getResourceSet().getResources().add(r);
>>
> The last statement seems wrong to me. because it will detach the
> resource from the ResourceSet that is internally created on
> session.openView(). See my explanation below...
>
>>
>> [...]
>>
>> // when the user creates an element in the model (Action handler) -
>> // myEObject is contained in Resource r
>> CDOTransaction transaction = session.openTransaction(editingDomain.getResourceSet());
>> myEObject.getChildren().add(newObject);
>> transaction.commit();
>>
> Some explanation:
> In CDO an EMF ResourceSet is associated with a CDOView. CDOTransaction
> is a special CDOView. You could eventually say CDOView is a read-only
> transaction. Each ResourceSet can only have a single CDOView (or
> CDOTransaction) associated at any point in time. If you call
> session.openTransaction() a new ResourceSet is created internally. If
> you call session.openTransaction(myResourceSet) a given ResourceSet is
> used for that transaction. But only if the given ResourceSet doesn't
> already have another view/transaction associated. BTW. this
> association happens internally via a ResourceSet adapter.
>
> If there is a chance that the model will be modified, you should open
> a transaction from the beginning and use it for reads and writes later
> on. If the model is nonetheless *not* changed in a transaction it
> doesn't cost you much to have a transaction instead of a read-only view.
>
> But even if you open several views and transactions on the same
> session, they will share all the object state as long as they're not
> modified. Model state is stored/managed in the session itself. Only
> change deltas are stored in the transactions.
But the transaction does timeout (Transaction.setCommitTimeout()) or
does the timeout refer to the commit() operation itself?
Is it possible to do multiple commits in one transaction - like
transaction = session.openTransaction();
// do something for 4 hours
transaction.commit()
// do something else for another 6 hours
transaction.commit()
or does transaction.commit() lead to a transaction state where only
transaction.close() is possible?
So essentially, could I open the session and transaction in my
PluginActivator.start() and close them in stop() so that I have a
transaction running as long as the application for the interactive stuff?
If not, how would I realize an interactive application in which the user
is shown a tree (of the model) that he can work with interactively?
Cheers,
Stefan
|
|
|
| Re: [CDO] Repository inside Eclipse [message #614129 is a reply to message #108634] |
Tue, 29 January 2008 08:22 |
|
Hi Stefan ,
Comments below ...
Stefan Winkler schrieb:
> [stuff deleted]
> But the transaction does timeout (Transaction.setCommitTimeout()) or
> does the timeout refer to the commit() operation itself?
Yes, the commit timeout does refer to the commit operation ;-)
>
> Is it possible to do multiple commits in one transaction - like
>
> transaction = session.openTransaction();
> // do something for 4 hours
> transaction.commit()
> // do something else for another 6 hours
> transaction.commit()
Yes, if you committed a transaction it returns to an "empty" state (no
change deltas) and can be used like before. It behaves similar to a JDBC
Connection with autoCommitMode==false. A better name than simply
transaction could have been transactional view...
>
> or does transaction.commit() lead to a transaction state where only
> transaction.close() is possible?
No, see above.
>
> So essentially, could I open the session and transaction in my
> PluginActivator.start() and close them in stop() so that I have a
> transaction running as long as the application for the interactive stuff?
Yes, I can imagine that this is possible.
>
> If not, how would I realize an interactive application in which the
> user is shown a tree (of the model) that he can work with interactively?
I'd say much the same way as with plain EMF. From the point on where you
have a CDOTransaction instance, which you can use to retrieve/commit the
ResourceSet, everything behaves normally. The only additional task is,
like you suspect, the management of the CDOTransaction instance,
including the needed dependencies (IConnector, ...).
BTW. There's a difference in the semantics of the CDOResource.save()
method. Since the scope of a CDOTransaction is always a whole
ResourceSet you can't save a single CDOResource (and keep local changes
to other CDOResources in the same ResourceSet/CDOTransaction). Thus
save() of a single CDOResource delegates to CDOTransaction.commit()
which will in turn "save" all the CDOResources in that ResourceSet.
Regards,
Eike Stepper
----
http://wiki.eclipse.org/CDO
http://wiki.eclipse.org/Net4j
Cheers
/Eike
----
http://www.esc-net.de
http://thegordian.blogspot.com
http://twitter.com/eikestepper
|
|
|
Goto Forum:
Current Time: Sat Dec 21 16:21:34 GMT 2024
Powered by FUDForum. Page generated in 0.10905 seconds |
] 
Search
Help
Register
Login
Home
