Skip to main content
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [adoptium.incubator-dev] Some ideas about pre-initialization
Hi Yang,

On Tue, Apr 27, 2021 at 7:45 PM Yi Yang <qingfeng.yy@xxxxxxxxxxxxxxx> wrote:
Hi Jiangli,

Thanks for your quick reply, your concerns are very reasonable. Let me clarify them one by one.

1. Goal
Alignment of goals is important. Our goal is to make Java applications start faster as possible as they could, actually that's what the project name(jdk11u-fast-startup-incubator) means. Based on this goal, any specific techniques such as CDS-based heap archiving and AOT are welcome, it seems that we are both interested in CDS-based heap archive.

2. Milestones and Roadmap
I suggest following the upstream release cycle, i.e. release a new build every 3 months. In the single release cycle, we can set up some feasible milestones. For example, the first milestone could be: Backport preliminary patches from upstream and prepare an initial patch(http://cr.openjdk.java.net/~jiangli/Leyden/general_class_pre_initialization_core_mechanism/general_class_pre_init_patch_1.) for later development.

3. Code review and feature/bug tracking
I find neither any bug tracking systems nor find code review and commit rules in eclipse foundation(Please let me know if it indeed exists), so I propose using the following format(which is similar to JDK PR format):

#GITHUB-ISSUE-ID: Title of patch
e.g.
#377: Backport some stuff...

Just IMHO, any other formats are welcome.

Thanks, please see the other responses to Sanhong's email. 


4. Technical details

> Agreed. I've backported that in our JDK 11 codebase. We can backport that for the JDK 11 based collab repo.

I'd like to backport JDK-8206009 to our collaboration repo as my first patch. Thank you for comparing and listing some preliminary patches in https://github.com/jianglizhou/adoptium-startup-incubator-docs/wiki/Backports-Needed-for-General-Java-Class-Pre-initialization-Work. Later, I will file an Github issue that has corresponding name with JBS issue name to mark a backport-patch is working in progress.

Thanks! The list is based on the order of our internal backports for JDK 11. It helps to avoid merge conflicts and reduces manual work by doing the backports in the order. There are a few bugs in the lists that should be backported to the OpenJDK 11 update repo (for community benefits). We can sync from the OpenJDK 11 update repo when they are backported. I can work on those.
 

> For #1, #2 and $4, those are addressed (or partially addressed) by http://cr.openjdk.java.net/~jiangli/Leyden/general_class_pre_initialization_core_mechanism/general_class_pre_init_patch_1. It implements my Java Class Pre-resolution, and Pre-initialization design proposal. Is everyone okay with taking our patch and collaborating on top of it? We can follow a review process that's agreed for the collab repo.

Very great! Taking that patch as an initial patch could drive further development.

> OpenJDK has some work done in that area. We can evaluate the benefits and explore.
> That sounds like a longer term goal and has the dependency on actual usages of ShenandoahGC/ZGC. 

Yes, they are just brain storm. In my internal practice, I don't implement partial pre-initialization, I find most time-consuming classes contain at least one lambda field in its subgraph. But with your patch, it's possible to pre-initialize partial class, so this might not so important.

Thanks,
Yang

Best,
Jiangli 
------------------------------------------------------------------
Sender:Jiangli Zhou<jianglizhou@xxxxxxxxxx>
Date:2021/04/28 10:26:40
Recipient:Yi Yang<qingfeng.yy@xxxxxxxxxxxxxxx>; adoptium.incubator developer discussions<adoptium.incubator-dev@xxxxxxxxxxx>
Theme:Re: [adoptium.incubator-dev] Some ideas about pre-initialization

Hi, 

On Sun, Apr 25, 2021 at 6:36 PM Jiangli Zhou <jianglizhou@xxxxxxxxxx> wrote:
Hi Yi Yang and everyone,

Thanks a lot for creating the incubator project and initiating the discussions (catching up after some slowdown on my side for a couple of days from last week)! Truly excited about the collaboration and the potential it may achieve. A few high level thoughts before diving deep into the technical discussions.

Goal
The startup incubator project has similar traits (or subset of those) as the OpenJDK Leyden project, which targets for producing a static image for optimized startup time, slow-to-peak performance and runtime memory footprint. The static image may include pre-generated class metadata, Java heap objects and compiled code. It would be good to have that broader goal in mind when we collaborate. Our JDK 11 based collaboration here may help innovations in OpenJDK head and fuel inspirations for Leyden.

Milestones and Roadmap
You brought up the question about future roadmap. It would be good to identify the achievable milestones. Based on our previous discussions, we both are interested in improving OpenJDK CDS (Class Data Sharing) based heap archiving for more general class pre-initialization as a short-term milestone. Based on my past experiences from AOT work on JavaME CVM, pre-resolving constant pool references (pre-initialization assists pre-resolving) help generate more optimal ahead-of-time compiled code. Are there other ideas that we can explore beyond those? It's a good idea to document the milestones and roadmap in https://github.com/adoptium/jdk11u-fast-startup-incubator/wiki.

Code review and feature/bug tracking
Which bug system can be used to help track features and bugs for our JDK 11 based collaboration here? Are there established Eclipse Foundation code review and commit rules that we can follow when collaborating using the jdk11u-fast-startup-incubator repository?

Please see more comments inlined below. 

On Thu, Apr 22, 2021 at 3:50 AM Yi Yang <qingfeng.yy@xxxxxxxxxxxxxxx> wrote:
Hi all,

There I have some ideas about pre-initialization:

1. Backports

As discussed with @jiangli before,
> The current general class pre-initialization work is based on JDK 11 with some backports from OpenJDK 12 & 13, particularly with the backport for https://bugs.openjdk.java.net/browse/JDK-8206009. Majority of the needed backports have already landed in OpenJDK 11. We still need to backport JDK-8206009 to jdk11u-dev-class-preprocessing.
This seems to be a good starting point, I can try to backport it. 

Yes, the majority of the needed backports have already landed in OpenJDK 11.
 
Besides, do you think there is any prerequisite patch that we must backport right now? 

I'll run a full comparison against OpenJDK 11 and send out the list of needed additional backports tomorrow or the day after tomorrow.


Best,
Jiangli
 
 
If not, we can consider developing new features until we need to backport some patches in later development.

2. New features
1) Allow pre-initializing fields whose types are primitives
2) Remove hard-coded archivable_static_fields, accepts users inputs, either a safe fields/klass list or fields/klass that annotated with @Preserve or whatever you like
3) In HeapShared::initialize_from_archived_subgraph, find the klass in hashmap instead of an array, this had been done in upstream

Agreed. I've backported that in our JDK 11 codebase. We can backport that for the JDK 11 based collab repo.
 
4) Safety check. Checking if given input is valid

For #1, #2 and $4, those are addressed (or partially addressed) by http://cr.openjdk.java.net/~jiangli/Leyden/general_class_pre_initialization_core_mechanism/general_class_pre_init_patch_1. It implements my Java Class Pre-resolution, and Pre-initialization design proposal. Is everyone okay with taking our patch and collaborating on top of it? We can follow a review process that's agreed for the collab repo.

There are additional bug fixes and enhancements on top of that.
 
5) Allow pre-initializing lambda fields

OpenJDK has some work done in that area. We can evaluate the benefits and explore.
 
6) More region-based GC supports(ShenandoahGC/ZGC)

That sounds like a longer term goal and has the dependency on actual usages of ShenandoahGC/ZGC. 
 

IMHO, I prefer item 2) since other items require removing hard-wired code and accepting users' input as their preconditions.

Agreed.

Your static analyzer proposal for checking the 'safty' of pre-initialization was interesting to us. Could you provide a timeline for that?

Best,
Jiangli


Do you have any ideas about the future roadmap? I'm new to pre-initialization, anything that is not listed above can also be raised. I'm very glad to hear about your ideas/feedback, any input is appreciated!

Best Regards,
Yang
_______________________________________________
adoptium.incubator-dev mailing list
adoptium.incubator-dev@xxxxxxxxxxx
To unsubscribe from this list, visit https://accounts.eclipse.org

Back to the top