Re: [incquery-dev] PSystem/Builder refactor - part 2

[Ujhelyi Zoltán <ujhelyiz@xxxxxxxxxx>]

Hi,

as promised, I have continued with the refactor, and today I have managed to create a working version with the inverted dependencies and the updated code generator in review https://git.eclipse.org/r/#/c/21050/ Of course it depends on the previous review, but if someone tries to check it out, Gerrit will handle it correctly.

Basically, most things are working, but there are some rough edges/caveats.

Changelog:
 * incquery.runtime does not know anything about the pattern models of Xtext
   * incquery.runtime is Xtext independent (or at least should be :) )
   * There is no injection in runtime
   * All PSystem building code is moved into patternlanguage.emf project (they should still be eclipse and UI independent)
     * Fixes some issues that our users have also seen
     * Test cases will not require UI (except they are testing the UI of course)
 * JvmModelInferrer now emits PSystem construction code
   * Of course generated code needs to be regenerated
   * For a sample, you could look at the file https://git.eclipse.org/r/#/c/21050/3/tests/org.eclipse.incquery.testing.queries/src-gen/org/eclipse/incquery/testing/queries/util/CorrespondingRecordInMatchSetRecordQuerySpecification.java
 * Generated code updates
   * There are no more XExpressionEvaluator classes and extensions anymore. Everything required from there is added to the generated Query Specification class.
   * From private patterns query specification is generated, but
     * It is not accessible outside the corresponding util package
     * No extension is generated to plugin.xml
     * It cannot be used to instantiate a matcher (throws unsupportedoperationexception)
   * globaleiqmodel.xmi is not used and updated anymore (most important benefit for me :) )
 * Query Explorer received a large-scale refactoring of its inner classes. TODO: extensive testing required
 
Known issues:
 * The update path is not clear and filled with exceptions. By manually cleaning the projects, the system gets into a consistent state, but during the first build there might be a lot of exceptions thrown.
   * I am planning to create a project migrator.
 * The clean phase of the build is not completely implemented - this will need some experimenting but is coming up right now
   * Java classes are updated correctly
   * Extensions are not removed (but at least I have seen no extension duplication)
   * Exported packages are not removed (maybe this is not required?)
 * The Pattern Registry was not updated - if nothing comes up I would suggest removing it entirely, because:
   * The interesting features (e.g. dynamic group building) were not implemented because of lack of time
   * The base design was focused on managing different versions of the same patterns, and providing consistent views about it. As of now, it is much easier to provide these views via the SpecificationBuilder mechanism; so a revisit of the design seems necessary to me.
   * And practically no-one is using pattern registry (the only known use in a project can trivially be replaced with the QuerySpecificationRegistry)
 * The registration of injectors (e.g. the ones used in test environment) needs some work. They are simpler than before, but some experimentation is still needed (such issues were causing the issues in the first changesets of the review).

Next week I will be working on clean support and bug fixing to make this piece of code mergeable into master. 

Cheers,
Zoli
-- Zoltán Ujhelyi
https://www.inf.mit.bme.hu/en/members/ujhelyiz

Fault Tolerant Systems Research Group
Budapest University of Technology and Economics

On 2014.01.07., at 19:42, Ujhelyi Zoltán <ujhelyiz@xxxxxxxxxx> wrote:

> Hi,
> 
> today I have achieved a working version of the "pattern-less" IncQuery runtime. The changes were aimed at avoiding instantiating stuff in runtime with Pattern objects, instead IQuerySpecification objects are used now throughout the system. Some testing/review would be very helpful (yes, it is a pretty big change set, but this was the smallest self-contained set I could come up with), as the change is quite big - on the other hand, I do not consider this stable, and any decision is open to debate.
> 
> The changes are pushed into Gerrit: https://git.eclipse.org/r/#/c/20346/
> 
> A more detailed changelog
> * A new builder for query specifications is created with SpecificationBuilder (in runtime project)
>  * This class maintains a set of Pattern->QuerySpecification mapping (patterns are represented with fully qualified names), and encapsulates the older EPMToPSystem classes functionality).
>  * The class also encompasses the previously engine-level validations (it is the last component that knows the original pattern objects)
> * An interface named PQuery is introduced, that became the super interface of IQuerySpecification. These two interfaces can be used for accessing query metadata.
>  * It is possible that matchers/matches currently maintain metainformation that is not forwarded to the corresponding specification, but that is something I have missed.
>  * Additionally, among these lines https://bugs.eclipse.org/bugs/show_bug.cgi?id=423279 has also been fixed in these changes.
> * All query specifications needs to be regenerated. Some internal changes require this.
> * The changes were having some rippling effect throughout the codebase. Luckily, only minor modifications were required, with the following exceptions:
>  * The entire Viewers API was having patterns as its central element. This needed some heavy refactorings - it seemed to work for me, but needs extensive testing.
>  * The Query Explorer also needs some experimenting. Seems working, but some rough edges may remain.
> 
> Known issues
> * Private patterns are not supported in generated code for now. This is a major regression, but it makes no sense to fix it at this stage. I will not merge these changes into master until the code generator has been correctly updated to also support these cases.
> * Generated code still relies on patterns. It is bound to be changed, but I was aiming for a testable version.
> 
> After incorporating anything you might come up related to this changeset, I will continue with
> 1) Making the runtime project indeed Xtext-independent
> 2) Updating the generator to use this new API in the generated code
> 3) After the generator is correctly updated, globaleiq.xmi will be removed
> 
> Cheers,
> Zoli
> 
> 
> PS.: I have an additional commit (as a different change set) pushed to gerrit: https://git.eclipse.org/r/#/c/20345/ - this deals with extension duplication in Kepler that came forth deterministicly after the refactoring, but I am not confident enough to release it into master. If you check out the other review into your eclipse instance, this will also be included by default.
> -- Zoltán Ujhelyi
> https://www.inf.mit.bme.hu/en/members/ujhelyiz
> 
> Fault Tolerant Systems Research Group
> Budapest University of Technology and Economics
> 
> _______________________________________________
> incquery-dev mailing list
> incquery-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/incquery-dev