Eclipse Community Forums: EMF » Memory disposal during unload()

Home » Modeling » EMF » Memory disposal during unload()(Intended control for garbage collection)

Memory disposal during unload() [message #1855896]

Wed, 09 November 2022 02:08

Eclipse User

It seems that the default implementation of Resource does not free up any memory when it is unloaded(). I wonder if this method is the correct injection point for such controls. I have looked at the EMF book, articles and code but found no definitive answer. Maybe I am misreading the intent of unload all together, and it is only to remove the costly notifiers, rather than the data held by the object itself.
Can anybody point me in the right direction, please?

Re: Memory disposal during unload() [message #1855898 is a reply to message #1855896]

Wed, 09 November 2022 03:00

Eclipse User

I'm not sure what you are expecting to see in terms of "freeing up memory". Java uses garbage collection so memory is freed when there are no more references. Generally an unload will increase memory consumption because org.eclipse.emf.ecore.resource.impl.ResourceImpl.unloaded(InternalEObject) converts all objects to proxies. If there are no reference to any of the objects in the resource, everything will be garbage collected immediately or later when all such references have been eliminated, e.g., all referenced proxies have re-resolved...

Re: Memory disposal during unload() [message #1855899 is a reply to message #1855898]

Wed, 09 November 2022 04:18

Eclipse User

Hi

Models are clearly graphs and so there are prolific reference cycles making a successful GC difficult as part of a live system, but easy at end of application.

EMF has a number of global registries from which just one reference can lock all your models into memory. If you use ResourceSets carefully many of these references can be localized, however since you use Epsilon you should be aware that Epsilon over-uses the global facilities. Tracking the reference down can be quite time consuming since the one 'bad' reference is hidden by numerous parasitic references. Ensuring that you set stale references, particularly on the stack, to null 'redundantly' and stepping out of the current debugger stack frame can help reduce the confusion of references from the debugger's working copy of the stack frame.

Xtext introduced org.eclipse.xtext.testing.GlobalRegistries to create a memento of the start of JUnit test state so that it could be restored at the end of the test. Very useful if you lack time to manage removes accurately. Unfortunately it was never upgraded to support the EAnnotationValidator.Registry.INSTANCE, so you may prefer org.eclipse.ocl.examples.test.xtext.GlobalRegistries2.

Unloading Resources is particularly hard since there are often eAdapters() that need to be respected. This seems to be a point of design challenge/confusion. IIRC CDO found Resource unloading unsatisfactory and so does its own diligent unload. The Pivot OCL makes sure to clear eAdapters. In general ensuring that every adapter is transitively notified of bye-bye can be very costly when only GC needs to be done (except for specialized adapters that hold general resources that really need releasing).

Regards

Ed Willink

Re: Memory disposal during unload() [message #1855900 is a reply to message #1855898]

Wed, 09 November 2022 04:23

Eclipse User

Thank you for your response and sorry for being vague. I am quite familiar with garbage collection and the various classes of weak references available. I was expecting that unload would possibly drop (set null) all the attribute information of an EObject, as this can be reconstructed by loading from the stream, as identifiers should be stable. Depending on the size of these, this could be a relevant savings. Then there would be various degrees of trimming the structure, by tree depth, by use frequency of features, etc.

At the heart of it is that I am simply trying to understand when and why you would unload a resource, other then to disable the notifiers. With the default implementation, as you have stated, unloading uses more memory, not less, as the contents structures use regular references and hence will prevent garbage collection. Even if a client does not hold a reference to any EObject in a reference, the memory is still held, even after an unload.

Or am I missing anything in the memory management here?

[Updated on: Wed, 09 November 2022 04:24] by Moderator

Re: Memory disposal during unload() [message #1855902 is a reply to message #1855900]

Wed, 09 November 2022 04:36

Eclipse User

w.r.t. Epsilon's use of static features and EMF's global registries, I agree that there is unfortunately not very much detailed control. Part of my exploration of EMF's resource management system is to decide what control facilities would be useful to implement as workflow steps in our build system to properly configure and inspect EMF's mechanisms, and adapt them to Epsilon. For this I need to understand what the intentions of design, capabilities and limits are. I will read up on how CDO has varied the architecture. I can imagine that this would be the missing piece of the puzzle for me.

In case you are wondering why I am after this, here is the
article that describes how we use DevOps toolchains in Agile model-driven engineering.

[Updated on: Wed, 09 November 2022 05:13] by Moderator

Re: Memory disposal during unload() [message #1855907 is a reply to message #1855902]

Wed, 09 November 2022 06:30

Eclipse User

Some folks do specialize the unloading to unset the references between the objects resource such that parts of the graph can be more easily garbage collected.

Suppose in an editor you've loaded a resource that references other resources such that you have a bunch of resources in your resource set. Now, someone goes and modifies one of those resources manually (perhaps doing a Git pull). The editor now needs to reload each modified resource and before it can reload the a it must unload it. All the objects them become proxies and those can be resolved to the newly loaded replacements.

Re: Memory disposal during unload() [message #1855908 is a reply to message #1855907]

Wed, 09 November 2022 06:40

Eclipse User

Aha! That is very illuminating. This is effectively a means to refresh the state of a resource. I had never understood it that way. I will include that in my description. Thanks a lot for that example.

Re: Memory disposal during unload() [message #1855912 is a reply to message #1855908]

Wed, 09 November 2022 07:42

Eclipse User

Hi Joern

I started reading your article. In principle I concur wholeheatedly, but I stopped reading when it became clear that you build upon Ant.

The Eclipse support for Ant is quite dreadful and so I gave up on Ant over ten years ago. Once I was using Xtext whose sibling MWE2 is vastly superior I found that I could debug. MWE2 is far from perfect but it's in a different league compared to Ant.

One of your suggestions earlier that EMF has not progressed beyond toy applications is very misinformed. Ed M can give you details of many large scale IBM usages. The usage of Xtext and so EMF by the, particularly German, automotive industry hardly corresponds to toy usage. One of the changes for Eclipse 4 was to use models defined by EMF. Eclipse itself is hardly a toy. So quite the contrary, EMF has so many large usages that EMF has a quite different problem; any change in behavior that results from a bug fix has to be carefully compromised against the impact on legacy users.

Regards

Ed Willink

Re: Memory disposal during unload() [message #1855926 is a reply to message #1855912]

Thu, 10 November 2022 00:31

Eclipse User

Hi Ed,

Thanks for having a look. I guess as I stated in earlier discussion, I can run MWE2 from ant, but not the other way around. Note that I am quite aware of it and its role. My working history includes being in charge of the model-driven bus for a large German automotive. And that involved MWE2 and its use. You may find some posts regarding its limitations in automated use if you dig through the history around 2013. I would nod to "far from perfect" as an assessment at least at that point in time. In fact, its usage was so poor that we went around it and wrote things in gradle to achieve reasonable behaviour. W.r.t. EMF not progressing beyond toy applications I guess I am saying that its industry relevance is quite limited and that, in large part, is due to Eclipse's intractability. This extends to EMF transitively. If I propose to use Eclipse as a basis for our work, I do that so we benefit from EMF's positive properties in the Eclipse component context. My developers would much rather I ditch it today and use Epsilon and Gradle only. The thinking I receive in mde net is much the same. So either the joint investment of EMF so far is made to pay, or we can throw it in and rewrite in TypeScript. Which is, by the way, seriously suggested. Use mine (i.e. xText) is not enough. I am very happy for the frame EMF has provided. It and you must be credited for removing the clunk of MOF out of the equation and so it and you succeeded, where JMI failed. Now my view would be to make sure that it becomes an industry engineering solution, rather than a click-click IDE tool or an academic pass time that will be ignored in the trenches. And for that I need reuse, and for that, at this stage, I will accept ant.

Best wishes,
JG

[Updated on: Thu, 10 November 2022 00:34] by Moderator

Previous Topic:	Problems to open a XMI file using the Sample Reflective Ecore Model Editor
Next Topic:	Xcore validation fails in Maven Release Build

Goto Forum:

-=] Back to Top [=-

Current Time: Tue Jul 01 08:01:13 EDT 2025

.:: Contact :: Home ::.

Breadcrumbs

Sign up to our Newsletter