Re: [mat-dev] Clear only-softly-retained objects by default?

["Kevin Grigorenko" <kevin.grigorenko@xxxxxxxxxx>]

Sorry, I clicked
send too early by accident. Completing the email:

> A possible
idea, presumably clearing weak references etc

One potential problem I thought of is that an only-softly-reachable object
with an
associated ReferenceQueue may still require reference queue processing
before
being cleaned up, so we may want to exclude such objects. It's unclear
what the
semantics are if the SoftReference referent has a finalize method. WeakReference
is more explicit that an only weakly reachable object will be declared
finalizable.
So we might want to only perform this clean up for objects without a finalize
method.

> There isn't really a concept of 
> soft reference in the snapshot API, so changing it there is pushing
down 
> implementation specifics.

Interesting, ok, that does complicate this.

> If you really wanted to just operate on those 182530 objects you could
> select those objects and do an Export Snapshot to export it as a new
HPROF 
> file. That works, even if the original is DTFJ, though when you reparse
> the dump you won't get the DTFJ specific information.

One idea would
be to allow creating a new DTFJ dump given that the
format is now
open source, although this won't be trivial since a core dump
is specific to
each OS.

> I think we
should check that the reports are giving directly useful 
> information
- everyone runs Leak Suspects, but who runs Top Components?

My observation
is that most non-power users don't run reports such as
Top Components
and instead primarily use Leak Suspects, the dominator tree,
histogram, and
threads views.

> Should the
empty collections report say exactly how many bytes would be 
> saved if
the empty collections were replaced by null references, and 
> created when
needed. Should the low fill ratio collections say how many 
> bytes would
be saved if they were trimmed to size? Are there more queries 
> that are
useful, such as: length 0 arrays (replace by global singleton, or 
> allocation
on need?); length 1 - if always one and not modifier, replace 
> by direct
reference; duplicate objects by shallow comparison.

I've used these
queries sometimes and when I need them, they are extremely
valuable.

> Are users
finding all the capabilities they need? Are they reading the 
> help? Are
they clicking on help links?

My observation
is that the general answer is no except for power users.

> Is Leak Suspects
report understandable?

My observation
is that most even non-power users understand it pretty well
although I have
heard of some confusion, so it's worth thinking about 
clarifying it.

> Do we need
help links inside the 
> report? If
there are steps we can suggest then either automate them, or 
> consider
longer cheat sheets with choices about where to go next.

That might be
useful although some users only use the headless leak
suspect report,
so the more that can be explained in text the better,
in my opinion.

--
Kevin Grigorenko
IBM App Platform SWAT




From:
       "Kevin
Grigorenko" <kevin.grigorenko@xxxxxxxxxx>
To:
       Memory
Analyzer Dev list <mat-dev@xxxxxxxxxxx>
Date:
       05/02/2021
03:20 PM
Subject:
       [EXTERNAL]
Re: [mat-dev] Clear only-softly-retained objects by default?
Sent
by:        "mat-dev"
<mat-dev-bounces@xxxxxxxxxxx>

---

Hey Andrew, > A possible
idea, presumably clearing weak references etc One potential problem I thought
of is that an only-softly-reachable object with an associated ReferenceQueue
may still require reference queue processing before being 
Hey Andrew,

> A possible idea, presumably clearing weak references etc

One potential problem I thought of is that an only-softly-reachable object
with an
associated ReferenceQueue may still require reference queue processing
before
being cleaned up, so we may want to exclude such objects. It's unclear
what the
semantics are if the SoftReference referent has a finalize method. WeakReference
is more explicit that an only weakly reachable object will be declared
finalizable.
So we might want to only perform this clean up for objects without a finalize
method.

> There isn't really a concept of 
> soft reference in the snapshot API, so changing it there is pushing
down 
> implementation specifics.

Interesting, ok, that does complicate this.

> If you really wanted to just operate on those 182530 objects you could
> select those objects and do an Export Snapshot to export it as a new
HPROF 
file. That works, even if the original is DTFJ, though when you reparse
the dump you won't get the DTFJ specific information.

--
Kevin Grigorenko
IBM App Platform SWAT




From:        Andrew
Johnson <andrew_johnson@xxxxxxxxxx>
To:        Memory
Analyzer Dev list <mat-dev@xxxxxxxxxxx>
Date:        04/29/2021
04:08 AM
Subject:        [EXTERNAL]
Re: [mat-dev] Clear only-softly-retained objects by default?
Sent by:        "mat-dev"
<mat-dev-bounces@xxxxxxxxxxx>

---

> An interesting idea came up of clearing only softly-retained objects
> by default when parsing a dump in the same way that MAT clears 
> unreachable objects by default, and having a similar preference of
> "Keep only-softly-retained objects". Thoughts? I have seen
this 
> issue cause confusion for a customer at least one time before.
> 
> --
> Kevin Grigorenko
A possible idea, presumably clearing weak references etc. also. In terms
of phases - the garbage collection is done in org.eclipse.mat.parser which
has little knowledge of soft references. There isn't really a concept of
soft reference in the snapshot API, so changing it there is pushing down
implementation specifics.

Soft references are handled by code in org.eclipse.mat.api, such as: 
References Statistics
Path to GC Roots
Merge Shortest Path to GC Root
Component Report
Customized Retained Set

I think we need to explain some of these more. To find objects which are
all strongly retained, you might think the customized retained set would
help. Actually that serves a different purpose:

"The custom retained set comprises all objects included in the normal
retained set. Additionally, it adds objects which are (a) reachable from
the root set and (b) would be garbage collected if the references defined
by the excludes would not exist.
In terms of set operations, one could put it this way: the customized 
retained set contains all objects which are unreachable if all references
to the root set and all references defined by the excludes are removed.
From the resulting set, the objects of the retained set derived only 
through the excludes are removed."

I've only just worked out how to find the set of objects which are 
retained without soft references:

merge_shortest_paths  .*  -excludes java.lang.ref.SoftReference:referent
-groupby FROM_OBJECTS_BY_CLASS

In a sample dump with 187950 objects this gives 182530 objects.
The only softly retained report gives 5420 objects, so that matches.

If you really wanted to just operate on those 182530 objects you could
select those objects and do an Export Snapshot to export it as a new HPROF
file. That works, even if the original is DTFJ, though when you reparse
the dump you won't get the DTFJ specific information.

I think we should check that the reports are giving directly useful 
information - everyone runs Leak Suspects, but who runs Top Components?
Should the empty collections report say exactly how many bytes would be
saved if the empty collections were replaced by null references, and 
created when needed. Should the low fill ratio collections say how many
bytes would be saved if they were trimmed to size? Are there more queries
that are useful, such as: length 0 arrays (replace by global singleton,
or 
allocation on need?); length 1 - if always one and not modifier, replace
by direct reference; duplicate objects by shallow comparison.

Are users finding all the capabilities they need? Are they reading the
help? Are they clicking on help links? Would having 'Tip of the Day' help?
Is Leak Suspects report understandable? Do we need help links inside the
report? If there are steps we can suggest then either automate them, or
consider longer cheat sheets with choices about where to go next.

--
Andrew Johnson


Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU

_______________________________________________
mat-dev mailing list
mat-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/mat-dev



_______________________________________________
mat-dev mailing list
mat-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://www.eclipse.org/mailman/listinfo/mat-dev