Hi Antonio, Thank you for the clarification. However I should note that there are no API or design changes (aside from a couple of things in EVL internals) – only additions and efficiency improvements. The efficiency improvements should also benefit all Epsilon languages, not just EVL, since not all changes in the branch are concurrency related. Nevertheless the changes are admittedly numerous, especially given the jump from Java 6 to Java 8. I was just hoping to get some feedback from users and see if there are any bugs, but I presume the changes can still go into interim? Furthermore the tests have now been decoupled from the CLI plugins so there are no additional dependencies or libraries. I know running Epsilon without Eclipse is something a few of us are interested in and I have been working with Horacio to incorporate a fully-featured command line interface for running Epsilon, but I doubt that this will be ready in time for the 1.5 release. Thanks, Sina Hi Sina, I believe the reason for not putting the parallel bits in 1.5.0 is this change in design for erl.engine? I agree with it - I would not do any large changes in design right before a new release. Let's do 1.5.0 and then we can refactor that :-). On 13 April 2018 at 14:53, arcanefoam@xxxxxxxxx <arcanefoam@xxxxxxxxx> wrote: Hi Sina, The only comments I can make right now (without looking into the code) is that org.eclipse.epsilon.erl.engine.launch should be decoupled from the EMC. For this, a new interface could be provided (e.g. IModelCli) that adds a couple of methods that EMC drivers can decide to implement to be usable form the cli interface. The second one is that from what you showed me the use of reflection in the cli interface seems a bit of a stretch (again, I would need to look with more detail at what you did). Perhaps we can reuse the work I did on the standalone execution to use it as a backbone for the cli interface. Drop me an email so we can schedule a meeting to look further into these. Hi all, As some of you may know, I have been working on parallelising EVL for many months now. After extensive development, testing, refinement and benchmarking, I believe the project is of production-ready maturity. The changes made in this branch are extensive, so I will try to summarise them below. Please let me know if you have any suggestions and/or objections to this request. New Projects- org.eclipse.epsilon.launch - allows for standalone launching of Epsilon programs from command line. Uses org.apache.commons.cli_1.4, but this is not on Orbit, so not included in the parallel-epsilon branch. This provides a command-line argument parser as well as infrastructure for running programs, getting results and profiling.
- org.eclipse.epsilon.erl.engine.launch - an extension of the base org.eclipse.epsilon.launch project which allows to run ERL programs standalone (i.e. without Eclipse) through command-line (i.e. an executable JAR file). The required inputs are path to script, model and metamodel (currently only EMF models are supported). The module can be chosen by appending the package name after "org.eclipse.epsilon." in the "-module" option, so for example to run ETL one would add "-module etl.EtlModule" (note that ETL must be on classpath for this to work). Additional arguments to the module can also be passed and will be inferred through reflection. For example, to pass an int argument of value 4 to the constructor of the module, simply add "int=4". To profile execution time, use "-profile" option. To show results, use "-results".
- org.eclipse.epsilon.evl.engine.launch - extends org.eclipse.epsilon.erl.engine.launch for EVL.
- org.eclipse.epsilon.profiling.test - test for new classes added to profiling (see below).
- org.eclipse.epsilon.erl.engine.test - Infrastructure for thorough equivalence testing. The "ErlEquivalenceTests" base class is particularly complex but greatly reduces the repetitive code required to perform equivalence tests between different modules. The class has been documented and is mainly there for testing parallel ERL extensions.
- org.eclipse.epsilon.common - Extensions of commonly used java.util.function interfaces which allow throwing of checked exceptions. Concurrency utilities. Additional methods in CollectionUtil. Ability to get CPU model for Linux and Windows systems in OperatingSystem. "BaseDelegate" allows for a class to have a parent, and to look up items in the parent and also merge items from or into parent (see FrameStack example below).
- org.eclipse.epsilon.profiling - two new classes: "ProfileDiagnostic" and "BenchmarkUtils" for standalone command-line profiling. These measure execution time and memory usage, with adjustable units for memory (i.e. can convert between bytes, MB, GB etc.). Facilities are provided for benchmarking any type of code passed as a Runnable, Function, Supplier, Consumer and their checked variants.
- org.eclipse.epsilon.eol.engine - functional interfaces which can throw EolRuntimeException.
- org.eclipse.epsilon.erl.engine - Concurrency utilities for parallel execution. This includes an interface "IErlContextParallel" along with skeletal implementation, as well as a custom "ErlExecutorService" interface (and a ThreadPoolExecutor implementation).
- org.eclipse.epsilon.evl.engine - Parallel implementations of EVL, building on the concurrency framework introduced in ERL. These execution strategies are implemented as extensions of the abstract "EvlModuleParallel" and reside in the org.eclipse.epsilon.evl.concurrent package. Currently the best two strategies are "EvlModuleParallelElements" and "EvlModuleParallelStaged". Also a shallow static analysis class "EvlGraph" can order constraints in an EvlModule based on satisfies calls, so that those depended on are executed first.
- org.eclipse.epsilon.evl.engine.test.acceptance - Add many new tests to exercise all features of EVL, as well as new scripts, models and metamodels. Added equivalence tests between original EvlModule and the parallel ones. Infrastructure built on top of org.eclipse.epsilon.erl.engine.test.
- Many collections in EOL (namely ArrayLists) are now sized to avoid wasting memory.
- Cached operations map assigned in build if applicable, more efficient checks, implemented as a synchronized WeakHashMap.
- AbstractModuleElement has equals and hashCode.
- StackTraceManager uses ArrayDeque instead of Stack (which is a legacy class and is extremely slow because it is synchronized). Performance of all Epsilon programs should be noticeably better as a result.
- FrameStack (and FrameStackRegion) can now be toggled as concurrent (and have a parent). FrameStacks (and regions) can now be merged - typically used for merging concurrent FrameStack into the single-threaded parent.
- EolModule asyncStatementQueue is now a Queue instead of ArrayList, leading to simpler code.
- MultiMap has concurrent version (toggled with boolean).
- CachedModel has concurrent version (again toggled with boolean, default is true).
- Parallel getAllOfKindFromModel/getAllOfTypeFromModel for large EMF models.
- Concurrent AbstractEmfModel (toggled with boolean).
- OperationContributorRegistry has concurrent version (toggled with boolean) and like with FrameStack, can have parents and be merged.
- ExecutorFactory has concurrent version (toggled with boolean). As with FrameStack, can have parents and ExecutionListeners can be merged.
- EolRuntimeException's message can now be cached if desired, to capture the AST stack trace at a particular point in time for later reporting (useful for parallel execution exception handling semantics). A recursive method to find an EolRuntimeException from a Throwable is also added.
- IEvlContext has new methods for grouping unsatisfied constraints, sorting based on number of failures. The UnsatisfiedConstraints is now a Set instead of a List as it should have been, since it makes no sense to have duplicate unsatisfied constraints.
- Constraints are only added to the ConstraintTrace if they are the target of a dependency (works in conjunction with new Set<Constraint> constraintDependedOn).
- equals and hashCode for EVL data structures - used for equivalence testing.
- EvlModule.executeImpl returns Set<UnsatisfiedConstraint> instead of null.
- Moved Execution logic for checking all constraints out of ConstraintContext and into EvlModule.
- Concurrent data structures in EVL (with serial thread confinement pattern where appropriate). Includes ConstraintTrace and Set<UnsatisfiedConstraint>.
- dt support for Parallel EVL. Currently implemented as check box in "Advanced" tab.Used EvlModuleParallelElements with Runtime.getRuntime.availableProcessors() number of threads.
- Various other small optimisations and minor refactoring in EOL.
I am also aware that running Epsilon standalone (i.e. without Eclipse/GUI) is something that a few people are interested in. I would be happy to collaborate with anyone on this, perhaps by enhancing the functionality of the new CLI/reflection-based org.eclipse.epsilon.erl.engine.launch project. Finally I'd like to note that despite the number of changes, every attempt has been made to ensure backwards compatibility, so for example in places where a concurrent data structure is used, with the exception of CachedModel the default is the original non-concurrent structure. The only "breaking" change is that EVL's results are now a Set<UnsatisfiedConstraint> instead of a List<UnsatisfiedConstraint>; though it is difficult to think of a scenario where index-based access and/or ordering would be required for EVL results. Furthermore all tests are passing (with the exception of EGL-related tests on Windows, which presumably was never used as a test platform). Thanks, Sina Madani _______________________________________________
epsilon-dev mailing list
epsilon-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/epsilon-dev
_______________________________________________
epsilon-dev mailing list
epsilon-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/epsilon-dev
-- Antonio Garcia-Dominguez |
