[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: [linuxtools-dev] Tmf: a model to support new trace analysis types
|
Hi Genevieve and Alex
Sorry for getting so late into the discussion, but I was on vacation and
had to catch up with work.
Before doing big changes and additions to the framework we need to look
what is already available and we need to understand the goals of the
analysis type.
[1] Event Based Analysis
Currently, we already have a kind of analysis framework and which is
based on ITmfDataRequest/ITmfEventRequest and ITmfDataProvider.
Currently there is only one ITmfDataProvider implementation which is the
TmfTrace. The request are sent to the ITmfDataProvider and the data
provider provides data in form of ITmfEvent. The actual analysis is done
in the request, mainly in handleData(). So by implementing different
requests classes you can do all kinds of analysis. This is currently the
core of the framework and different types of (I call it) analysis
requests exists, e.g. histogram request, request for building state
systems (Kernel state system, statistics), sequence diagram requests,
search requests, filter request and more.
The Event request framework allows for waiting till a request is
finished (request.waitForCompletion()). This enables to run multiple
requests in sequence.
The event request framework supports also coalescing of requests so that
a trace is read once for multiple analysis. This means, when reading an
event it will be dispatched to all analysis requests that are running
(coalesced) at the same time. This is a very powerful mechanism, that
allows for excellent performance.
With this event request framework it is also possible to cascade
different implementation of ITmfDataProvider. This can be useful for
creating synthetic events for trace abstraction. Let's say an analysis
creates a request to get synthetic events (that implements ITmfEvent)
from a SyntheticEventProvider (that implements ITmfDataProvider). The
SyntheticEventProvider on the other hand queries other ITmfDataProvider
(e.g. TmfTrace) for events and aggregates them to a synthetic event and
forwards to the analysis. This had been done in the legacy LTTng Eclipse
integration (sorry for bringing the legacy LTTng up :-)), but the
concept could be reused. Of course, there might be some tweaking to do
in the framework to improve the concept of cascading of data providers.
[2] Goals of Analysis type extension point
From all the ongoing discussions, the above event based analysis
doesn't cover all use cases. There seems to be a need for having the
possibility to
[2.1] run analysis modules self-contained
[2.2] run analysis modules sequentially, that means an analysis type
needs to be finished before the next one starts.
[2.3] decouple the analysis from the views (and other output)
[2.4] have a better way of triggering analysis e.g. from a menu action
instead of running it when a trace is opened.
[2.5] better visualize what analysis is available for a given trace type
I think some of the goals can be achieved by the current framework.
[2.1] can be done with Event Requests.
[2.2] can also be done with the current framework. Let's say a
multi-step analysis needs run analysis module A, B and C sequentially.
So, the multi-step analysis sends a request for A to the trace and waits
for it's completion. After that, it executes analysis B, waits for B to
finish and then executes C. After C is finished it notifies the views to
update their data.
[2.3] is the responsibility of the each analysis and view. For that a
Model-View-Control (MVC) design is needed. In the current available
analysis we only implemented it to a certain extend. The control part of
the MVC design for most of the analysis views is integrated into the
view itself. To fully decouple the analysis and data model generation
from the view, there needs to be a proper notification mechanism
defined, so that the view (or other output means) can hook to and are
notified by any data model changes. I think, this is not necessary part
of the analysis type definition itself, it needs to be addressed during
implementation of the analysis module.
[2.4] and [2.5] I agree with that. With the multiplication of analysis
modules and views the framework needs to improve. Currently, all the
analysis are related to a trace type and are within views. Using
perspectives we try to group which analysis is active at a time. All the
analysis are triggered using the TMF signal framework for synchronizing
the views. When a trace is opened, all open views and analysis modules
(e.g. state system) start gathering the data and start the analysis.
When a new active time range is selected, all relevant open views update
their data to the active time range. Same for the current selected time.
When the active trace is closed the views clear their content.
But we need more. Maybe an analysis should be triggered by a menu action
(see contribution under review https://git.eclipse.org/r/#/c/13788/ for
the plotting engine). There could be a preference page where a list of
analysis to be executed when a trace opens is defined. I think this is
what you tried to show in the initial email with new project explorer
structure. I think for this purpose an extension point can provide the
framework.
[3] Design approach
I think it's a good idea have some code and prototypes available so that
users can try it and give comments. Often discussions in meetings and
mailing lists can do only so much. Some new ideas, problems and concerns
are coming during the implementation and when running the features. The
existing TMF framework has gone through several design stages and
modifications. "We improve as we go". You've already have some proposal
on Gerrit which is very good and will help us to understand. However,
this is only the framework part. The new analysis type is not used yet.
So it's hard to see the benefit and give comments from a user point of
view. So, I think as a next step would be to have a concrete example. I
would suggest to use one of the existing analysis modules (e.g. kernel
state system, statistics, histogram etc). The new analysis type needs to
work with the existing modules. Otherwise there is something wrong and
we'll end-up with different ways of doing analysis in the TMF framework
and that will get messy in the long run.
[4] Some other comments
[4.1] I just want to remind everybody that TMF is a framework for
analysis and viewing of any type of traces. When designing a new concept
in the framework (e.g. analysis type) we need to keep in mind that it
has to fit with all potential trace types.
[4.2] Currently there are generic analysis types (e.g. statistics,
histogram) that work for all trace types. I don't think we should
display them in the project explorer as analysis type for each of the
traces.
[4.3] There was some back and forth discussion about having 2 extension
points, one for the analysis type and one for the views. Is the views
extension point just to have a way to display the available views in the
project explorer under the analysis type and be able to open it from
there? I think, there should be, instead, a way to register a view to
the analysis and by doing that it can be easily displayed and accessed
from the project explorer. This will allow to add plug-ins that provide
views for existing analysis types.
[4.4] The trace type extension point is in the tmf.ui. So, the analysis
type extension point might need to be moved there, too, to avoid
references from tmf.core to tmf.ui. If we use the trace type extension
in the analysis type handling then we need it. The trace type extension
defines the trace class implementation, event class implementation and
it might be beneficial to have access to this information. Just to keep
in mind.
[4.5] command-line mode: If you want to use the RCP to trigger analysis
from command-line without opening the GUI, I have to mention that the
RCP application will use the GUI plug-ins even if now Display is opened.
The RCP requires
org.eclipse.ui. To use only the core plug-ins of the TMF/LTTng another
startup procedure is needed. It's probably possible (see JUnit test).
However, it would be another application.
[4.6] I haven't seen an comments about handling of concurrently running
analysis. This will have an impact of performance and we need to
consider that in the design of the analysis type framework. As much as
possible, we should use the coalescing mechanism of the event request
framework. But, I can see that here is a need to improve on that. In the
framework, the coalescing is done during generation of analysis
requests. But it would be beneficial to join an ongoing request so that
the number that a trace is read is minimal. BTW, a prototype for this
feature of coalescing is currently implemented within our team.
That's from me right now. Any comments?
Best Regards
Bernd
On 07/31/2013 03:03 PM, Geneviève Bastien wrote:
Hi Alex,
On 07/31/2013 01:39 PM, Alexandre Montplaisir wrote:
On 13-07-31 11:52 AM, Geneviève Bastien wrote:
Hi there,
Here is a first patch for review of this feature:
https://git.eclipse.org/r/#/c/14935/
Alexandre, I know you're not hot at the idea of adding extension
points. I'd like to know why because I don't see how else to do it
than with those 2 extension points (I tried...).
Hehe, it's not that I don't like extension points per se. But more often
than not, extension points are used where plain polymorphism could do
the same job, in a simpler way.
There's still some discussion to do around this, before we dive head-on
into refactoring the whole code base... Personally, I find the whole
"generic pluggable analysis modules with inter-dependencies and such" to
be waaaay over-engineered, at this stage. We should start with something
small, and as we discover new use cases we can iterate on it.
Here's what I had in mind, which is smaller in scope, but I think can
cover the use cases you and the people at Poly are trying to do. Of
course, this is just an idea among others, perhaps the end result will
be something in between ;)
- Have the "analysis" become the new "view". So replace the View
extension point with an Analysis one.
- An analysis specifies which trace type it supports (so the
framework
knows, from a given trace, which analyzes are available).
- The analysis determines which outputs it has (views, command line
printouts, files, etc.)
- The analysis controls all its data-gathering requirements. So stuff
like building state systems, doing requests, etc. If needed it can
dictate the order in which it runs those (wait for a query to be
finished before starting the next one, etc.)
Actually, that's about exactly how it's been implemented ;-) No code
refactoring at all (or else a little line here and there) and a number
of the lines of this commits are taken by comments and file headers,
it's really not that big a change!
Here's the problematic:
* Analysis are core concepts, they can be executed from command line,
and have default outputs, like text outputs, .dot files for graph,
etc. Therefore analysis need to be in the .core plugins (and .core
plugins must not depend on UI stuff)
I would still put the Analysis in the .ui plugins. Command-line output
is still a UI. If you have absolutely no output at the end, then why run
an analysis at all?
Really? I guess my idea of the .ui plugins is "that big thing
containing all heavy eclipse user interface stuff", so I don't really
see a simple text output as being a UI. I guess I'll wait to see how
the command line will work out before worrying about it.
* Views are associated with an analysis, but analysis couldn't care
less about the views. So there shouldn't be a getter/setter for the
list of views.
I disagree. I prefer having the views "inside" the analysis. By having
generic analyzes and generic views at the same time, it increases the
"degrees of freedom" of the system, and everything ends up being more
confusing. Not sure if the gain is really worth this added complexity.
This is where polymorphism could do the job, without the need for an
extension point. If you want to add a view to an existing analysis, you
can just extend it and implement your view as part of your new extended
analysis.
I guess we'll need at least a third opinion on this. I think the
little added complexity is worth it to keep modularity. Anyway, I'll
leave it as it is for now, with a list of string ids of views. Some of
the things Florian is implementing these days may bring us an
interesting use case to see how to go about this.
Looking forward to some in person discussions on this topic ;-)
Geneviève
_______________________________________________
linuxtools-dev mailing list
linuxtools-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/linuxtools-dev
--
This Communication is Confidential. We only send and receive email on
the basis of the terms set out at www.ericsson.com/email_disclaimer
