| Re: [epsilon-dev] IModel / CachedModel enhancements proposal |
Hi Dimitris, Yes you’re right. I was going on the assumption that a well-optimised modelling technology would support lazy loading of elements and having IModel support such efficiencies would be useful to exploit them where available as opposed to forcing them to use a less efficient collection-based approach. For non-lazy drivers / modelling technologies, the default implementations would provide a convenient fallback. I was also working on the assumption that EMF provides some sort of lazy loading facility given that allContentsFromModel in AbstractEmfModel effectively goes through a TreeIterator and adds all of the elements to a collection; though I’m not sure if this is purely an API decision which gives the impression of lazy loading and is really backed by an already-populated collection / caches or whether model elements are loaded on-demand. Thanks, Sina From: Dimitris Kolovos Hi Sina, For iterateAllOf* to be more efficient than getAllOf*, modelling technologies underpinning EMC drivers should natively support lazy loading of model elements, so in terms of current drivers I suspect that this would mostly benefit the Hawk driver. Does this sound right? Cheers, Dimitris On Mon, 7 Jan 2019 at 15:01, Sina Madani <sinadoom@xxxxxxxxxxxxxx> wrote: > > Hi all, > > > > I should also mention that an iterator-based approach could also be applied to first-order operations. Currently org.eclipse.epsilon.eol.execute.operations.declarative.* requires the target to be a collection, including model element types. With an iterator-based approach, we could perform queries and transformations one element at a time in a lazy manner. Of course the parallel variants would have to be a Spliterator or collection but an iterator-based approach should, at least in theory, be more efficient (both in time and memory). A stream-based approach could also allow for lazy evaluation at the model-level as well, so even though we can use getAllOfKind().stream() now it eagerly fetches all elements and we stream a collection, whereas with a Spliterator-based approach we can make the EMC-level calls lazy as well. > > > > Just an idea. Perhaps it would be worth prototyping and if the implementation turns out to be elegant without many issues and there are notable gains we could integrate it into master branch. I would be happy to try this out and report back. > > > > Thanks, > > Sina > > > > From: Sina Madani > Sent: 07 January 2019 11:53 > To: Dimitris Kolovos > Cc: Antonio Garcia-Dominguez; Epislon Project developer discussions; horacio.hoyos@xxxxxxxxxx; basp91@xxxxxxxxx; jonathan.co@xxxxxxxxxx > Subject: Re: IModel / CachedModel enhancements proposal > > > > Hi Dimitris, > > > > I think iterateAllOfKind would be useful in rule-based languages like EVL which just use the result in a for loop. Presumably an iterator-based approach would be more memory-efficient since elements can be "thrown away" once processed rather than being persisted because of references in a collection. That said, the iterateContentsFromModel would only make sense if caching is disabled. > > > > Regarding isLoaded, I suppose it is useful for standalone applications / users of the Epsilon API. > > > > Thanks, > > Sina > > > > On Mon, 7 Jan 2019, 08:17 Dimitris Kolovos <dimitris.kolovos@xxxxxxxxxx wrote: > > Hi Sina, > > Do we have concrete motivating scenarios for isLoaded() and iterateAllOf*? > > Cheers, > Dimitris > > > On Thu, 3 Jan 2019 at 17:11, Antonio Garcia-Dominguez > <agarcdomi@xxxxxxxxx> wrote: > > > > Hi Sina, > > > > That's good to hear - we agree on extended IModel interface + default implementation in Model, then? > > > > I also agree that the Stream/Iterator produced from those *allOf* methods should create an immutable view. Deleting elements should be done through the usual mechanisms - we should have the iterator throw an UnsupportedOperationException if somebody tries to remove through it. > > > > Everyone else - what do you think? > > > > Kind regards, > > Antonio > > > > > > On Thu, 3 Jan 2019 at 17:43, Sina Madani <sinadoom@xxxxxxxxxxxxxx> wrote: > >> > >> Hi Antonio, > >> > >> > >> > >> Thank you for your detailed thoughts. I agree we should probably put isLoaded() into Model or CachedModel as it’s implementation-specific. Regarding Iterator#remove, I think both the stream-based and iterator-based methods should return immutable values for consistency. I’m not sure if getAllOf*() returns a mutable collection but IMO it’s bad practice to be directly mutating such collections anyway. > >> > >> > >> > >> Thanks, > >> > >> Sina > >> > >> > >> > >> From: Antonio Garcia-Dominguez > >> Sent: 03 January 2019 15:35 > >> To: Sina Madani > >> Cc: Epislon Project developer discussions; dimitris.kolovos@xxxxxxxxxx; horacio.hoyos@xxxxxxxxxx; basp91@xxxxxxxxx; jonathan.co@xxxxxxxxxx > >> Subject: Re: IModel / CachedModel enhancements proposal > >> > >> > >> > >> Hi Sina, > >> > >> > >> > >> Happy New Year as well! Let's see: > >> > >> > >> > >> * Having an isLoaded() method would be good, but it's a bit strange to have a default implementation that always returns true. We have a base Model class for most of our models - would it make sense to have a default "proper" implementation there? > >> > >> > >> > >> * Having a way to iterate through the contents of a model (as opposed to an eager construction of a collection) would be good - in fact, in Hawk I recently implemented an optimisation that "faked" a collection this way - works great for the typical .all.first queries (obviously, .all.size still triggers a full iteration). Still, given that Stream is a richer API than Iterator and that BaseStream has an .iterator() method to produce an iterator, it might be better to have a default implementation of iterateAllOf* based on streamAllOf* rather than the other way around, unless there is some issue in supporting Iterator#remove from a converted stream. > >> > >> > >> > >> BTW, do we want to support Iterator#remove at all from iterateAllContentsFromModel()? > >> > >> > >> > >> Kind regards, > >> > >> Antonio > >> > >> > >> > >> On Tue, 1 Jan 2019 at 16:59, Sina Madani <sinadoom@xxxxxxxxxxxxxx> wrote: > >> > >> Hi all and a Happy New Year! > >> > >> > >> > >> I thought I would suggest a couple of additions to IModel or CachedModel (which would of course be non-abstract methods to avoid breaking existing implementations) for discussion. > >> > >> > >> > >> default boolean isLoaded() { > >> > >> return true; > >> > >> } > >> > >> > >> > >> Jon, Horacio, Betty and I were discussing the idea of being able to detect whether a model has already been loaded. Currently there isn’t a general way to do this, so in some implementations a call to load() may unnecessary reload the model. CachedModel may override this to return allContentsAreCached if caching is enabled, for example. > >> > >> > >> > >> protected Iterator<ModelElementType> iterateAllContentsFromModel() { > >> > >> return allContentsFromModel().iterator(); > >> > >> } > >> > >> > >> > >> The second idea was proposed by Horacio (albeit not in this exact form) which is presumably motivated by the efficiency of iterators compared to collections. Iterators can produce values lazily (EMF already uses TreeIterator, for example), whilst Collections must be populated eagerly even if only part of the collection is used. Implementations could therefore override this method to provide lazy model iteration which would be more efficient. This method could then be used to replace some allContentsFromModel invocations. We could then also add similar variants for getAllOf* (where * = Kind/Type), e.g.: > >> > >> > >> > >> default Iterator<?> iterateAllOf*(String type) { > >> > >> return getAllOf*(type).iterator(); > >> > >> } > >> > >> > >> > >> Of course this would be overridden by CachedModel to use the iterator-based allContentsFromModel. > >> > >> Additionally, I propose a complimentary pair of methods for getAllOf* as follows: > >> > >> > >> > >> default Stream<?> streamAllOf*(String type) { > >> > >> return getAllOf*(type).stream(); > >> > >> } > >> > >> > >> > >> With the associated streamAllOf*FromModel() as well in the case of CachedModel. This would also allow for lazy model iteration but with a richer API since many queries and transformations can be performed using Streams both lazily and in parallel if desired. Models that implement iterateAllContentsFromModel and iterateAllOf* can then make use of this to create a stream as follows: > >> > >> > >> > >> public Stream<?> streamAllOf*(String type) { > >> > >> return StreamSupport.stream( > >> > >> Spliterators.spliteratorUnknownSize(iterateAllOf*(type), 0), > >> > >> false > >> > >> ); > >> > >> } > >> > >> > >> > >> Please feel free to add your thoughts / suggestions. > >> > >> > >> > >> Thanks, > >> > >> Sina > >> > >> > >> > >> > >> -- > >> > >> Antonio Garcia-Dominguez > >> > >> > > > > > > > > -- > > Antonio Garcia-Dominguez > > > > -- > Dimitris Kolovos > Professor of Software Engineering > Department of Computer Science > University of York > http://www.cs.york.ac.uk/~dkolovos > > EMAIL DISCLAIMER http://www.york.ac.uk/docs/disclaimer/email.htm > > -- Dimitris Kolovos Professor of Software Engineering Department of Computer Science University of York http://www.cs.york.ac.uk/~dkolovos EMAIL DISCLAIMER http://www.york.ac.uk/docs/disclaimer/email.htm |