Re: [equinox-dev] Re: Patch for RepoOptimizer

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]

Re: [equinox-dev] Re: Patch for RepoOptimizer

From: Jeff McAffer <Jeff_McAffer@xxxxxxxxxx>
Date: Wed, 24 Oct 2007 22:42:39 -0400
Delivered-to: equinox-dev@xxxxxxxxxxx
List-archive: <https://dev.eclipse.org/mailman/listinfo/equinox-dev>
List-help: <mailto:equinox-dev-request@eclipse.org?subject=help>
List-subscribe: <https://dev.eclipse.org/mailman/listinfo/equinox-dev>, <mailto:equinox-dev-request@eclipse.org?subject=subscribe>
List-unsubscribe: <https://dev.eclipse.org/mailman/listinfo/equinox-dev>, <mailto:equinox-dev-request@eclipse.org?subject=unsubscribe>

Ok. So for now I am thinking that the optimizers are just independent. When/if we see a common pattern we can look at generalizing and putting some infrastructure. So far the only win I've seen is the potential for caching if you are doing multiple optimzations.

Having said that, if we go for different bundles for each optmizer, the current one should be renamed to be a pack200 optimizer. Should I do that?

Jeff

Stefan Liebig <Stefan.Liebig@xxxxxxxxxxxx>
Sent by: equinox-dev-bounces@xxxxxxxxxxx

10/24/2007 02:43 AM

Please respond to
Equinox development mailing list <equinox-dev@xxxxxxxxxxx>

To	Equinox development mailing list <equinox-dev@xxxxxxxxxxx>
cc
Subject	Re: [equinox-dev] Re: Patch for RepoOptimizer

Yes, that´s right. There is not that much common code. It is mainly parsing the command line arguments. Dealing with different types of inputs (special parameters for the delta thing) is surely easier if it is not generalized but done by each optimizer individually.
Should each optimizer (or strongly related optimizers) have it´s on bundle?

Jeff McAffer wrote:

<moving this thread to equinox-dev>

I guess I wonder if there is much value in generalizing the optimizer structure. For example, what is the difference between running the pack200 optimizer then running the delta optimizer and running one app and spec'ing multiple optimizers? I am concerned actually that the different optimzers will require significantly different inputs (e.g., the delta optimizer is going to need some baselines etc). As yet there does not appear to be much common code to be shared.

So for now I was thinking of just having separate optimizers...

One place that we might be able to win is if we could somehow coordinate the dowload of the "canonical" artifact form (e.g., the unadulterated JAR). It would be boring to do several optimizations remotely and have the artifact downloaded multiple times.
Sorry, I did not get the general picture here. Could you please rephrase it?

Stefan

What do you think?

Jeff

Stefan Liebig <Stefan.Liebig@xxxxxxxxxxxx>

10/18/2007 09:43 AM

To	Jeff McAffer/Ottawa/IBM@IBMCA
cc
Subject	Patch for RepoOptimizer

Hi Jeff, I tried to generalize the handling of ´optimizers´. The idea is that this forms the basis for extending the Optimizer with new IOptimizer impls. Maybe we have to change a few names. Test works as before. Tell me your opinion. Tschüß, Stefan ### Eclipse Workspace Patch 1.0 #P org.eclipse.equinox.p2.repositoryoptimizer Index: src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Pack200Step.java =================================================================== RCS file: /cvsroot/eclipse/equinox-incubator/provisioning/org.eclipse.equinox.p2.repositoryoptimizer/src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Pack200Step.java,v retrieving revision 1.1 diff -u -r1.1 Pack200Step.java --- src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Pack200Step.java 17 Oct 2007 03:47:55 -0000 1.1 +++ src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Pack200Step.java 18 Oct 2007 13:34:47 -0000 @@ -90,7 +90,7 @@ // So before closing, run unpack and write the unpacked result to the destination performPack(); super.close(); - // TODO need to get real status here but curently the JAR processor does not give + // TODO need to get real status here but currently the JAR processor does not give // any reasonable return status if (status == null) status = Status.OK_STATUS; Index: src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Optimizer.java =================================================================== RCS file: /cvsroot/eclipse/equinox-incubator/provisioning/org.eclipse.equinox.p2.repositoryoptimizer/src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Optimizer.java,v retrieving revision 1.1 diff -u -r1.1 Optimizer.java --- src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Optimizer.java 17 Oct 2007 03:47:55 -0000 1.1 +++ src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Optimizer.java 18 Oct 2007 13:34:47 -0000 @@ -8,69 +8,33 @@ ******************************************************************************/ package org.eclipse.equinox.internal.p2.repositoryoptimizer; -import java.io.IOException; -import java.io.OutputStream; -import org.eclipse.core.runtime.NullProgressMonitor; -import org.eclipse.equinox.p2.artifact.repository.*; -import org.eclipse.equinox.p2.artifact.repository.processing.*; +import java.util.List; +import org.eclipse.equinox.p2.artifact.repository.IArtifactRepository; import org.eclipse.equinox.p2.metadata.IArtifactKey; public class Optimizer { private IArtifactRepository repository; + private List optimizers; - public Optimizer(IArtifactRepository repository) { + public Optimizer(IArtifactRepository repository, List optimizers) { this.repository = repository; + this.optimizers = optimizers; + + for (int i = 0; i < optimizers.size(); i++) { + IOptimizer optimizer = (IOptimizer) optimizers.get(i); + optimizer.setArtifactRepository(repository); + } } public void run() { IArtifactKey[] keys = repository.getArtifactKeys(); for (int i = 0; i < keys.length; i++) { IArtifactKey key = keys[i]; - if (!key.getClassifier().equals("plugin")) - continue; - IArtifactDescriptor[] descriptors = repository.getArtifactDescriptors(key); - IArtifactDescriptor complete = null; - boolean optimized = false; - for (int j = 0; j < descriptors.length; j++) { - IArtifactDescriptor descriptor = descriptors[j]; - if (descriptor.getProcessingSteps().length == 0) - complete = descriptor; - optimized |= isOptimized(descriptor); + for (int j = 0; j < optimizers.size(); j++) { + IOptimizer optimizer = (IOptimizer) optimizers.get(j); + optimizer.optimize(key); } - if (!optimized) - optimize(complete); } } - private void optimize(IArtifactDescriptor descriptor) { - ArtifactDescriptor newDescriptor = new ArtifactDescriptor(descriptor); - ProcessingStepDescriptor[] steps = new ProcessingStepDescriptor[] {new ProcessingStepDescriptor("org.eclipse.equinox.p2.repositoryoptimizer.Pack200Unpacker", null, true)}; - newDescriptor.setProcessingSteps(steps); - OutputStream repositoryStream = null; - try { - repositoryStream = repository.getOutputStream(newDescriptor); - - // Add in all the processing steps needed to optimize (e.g., pack200, ...) - ProcessingStepHandler handler = new ProcessingStepHandler(); - OutputStream destination = handler.link(new ProcessingStep[] {new Pack200Step()}, repositoryStream, null); - - // Do the actual work by asking the repo to get the artifact and put it in the destination. - repository.getArtifact(descriptor, destination, new NullProgressMonitor()); - } finally { - if (repositoryStream != null) - try { - repositoryStream.close(); - } catch (IOException e) { - // TODO Auto-generated catch block - e.printStackTrace(); - } - } - } - - private boolean isOptimized(IArtifactDescriptor descriptor) { - // TODO this is a hack for now. Really we should inspect the steps to see - // if the optimization(s) we are doing here have already been done - return descriptor.getProcessingSteps().length > 0; - } - } Index: src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Application.java =================================================================== RCS file: /cvsroot/eclipse/equinox-incubator/provisioning/org.eclipse.equinox.p2.repositoryoptimizer/src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Application.java,v retrieving revision 1.1 diff -u -r1.1 Application.java --- src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Application.java 17 Oct 2007 03:47:55 -0000 1.1 +++ src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Application.java 18 Oct 2007 13:34:47 -0000 @@ -9,7 +9,7 @@ package org.eclipse.equinox.internal.p2.repositoryoptimizer; import java.net.URL; -import java.util.Map; +import java.util.*; import org.eclipse.equinox.app.IApplication; import org.eclipse.equinox.app.IApplicationContext; import org.eclipse.equinox.p2.artifact.repository.IArtifactRepository; @@ -22,12 +22,13 @@ public class Application implements IApplication { private URL artifactRepositoryLocation; + private List optimizers = new ArrayList(); public Object start(IApplicationContext context) throws Exception { Map args = context.getArguments(); initializeFromArguments((String[]) args.get("application.args")); IArtifactRepository repository = setupRepository(artifactRepositoryLocation); - new Optimizer(repository).run(); + new Optimizer(repository, optimizers).run(); return null; } @@ -47,8 +48,8 @@ return; for (int i = 0; i < args.length; i++) { // check for args without parameters (i.e., a flag arg) - // if (args[i].equals("-pack")) - // pack = true; + if (args[i].equalsIgnoreCase("-pack200")) + optimizers.add(new Pack200Optimizer()); // check for args with parameters. If we are at the last argument or // if the next one has a '-' as the first character, then we can't have @@ -57,7 +58,7 @@ continue; String arg = args[++i]; - if (args[i - 1].equalsIgnoreCase("-artifactRepository") | args[i - 1].equalsIgnoreCase("-ar")) + if (args[i - 1].equalsIgnoreCase("-artifactRepository") || args[i - 1].equalsIgnoreCase("-ar")) artifactRepositoryLocation = new URL(arg); } } Index: src/org/eclipse/equinox/internal/p2/repositoryoptimizer/IOptimizer.java =================================================================== RCS file: src/org/eclipse/equinox/internal/p2/repositoryoptimizer/IOptimizer.java diff -N src/org/eclipse/equinox/internal/p2/repositoryoptimizer/IOptimizer.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/internal/p2/repositoryoptimizer/IOptimizer.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,22 @@ +/******************************************************************************* + * Copyright (c) 2007 compeople AG and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + *http://www.eclipse.org/legal/epl-v10.html+ * + * Contributors: + * compeople AG (Stefan Liebig) - initial API and implementation + *******************************************************************************/ +package org.eclipse.equinox.internal.p2.repositoryoptimizer; + +import org.eclipse.equinox.p2.artifact.repository.IArtifactRepository; +import org.eclipse.equinox.p2.metadata.IArtifactKey; + +public interface IOptimizer { + + void setArtifactRepository(IArtifactRepository repository); + + void optimize(IArtifactKey key); + +} Index: src/org/eclipse/equinox/internal/p2/repositoryoptimizer/AbstractOptimizer.java =================================================================== RCS file: src/org/eclipse/equinox/internal/p2/repositoryoptimizer/AbstractOptimizer.java diff -N src/org/eclipse/equinox/internal/p2/repositoryoptimizer/AbstractOptimizer.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/internal/p2/repositoryoptimizer/AbstractOptimizer.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,32 @@ +/******************************************************************************* + * Copyright (c) 2007 compeople AG and others. + * All rights reserved. This program and the accompanying materials + * are made available under the terms of the Eclipse Public License v1.0 + * which accompanies this distribution, and is available at + *http://www.eclipse.org/legal/epl-v10.html+ * + * Contributors: + * compeople AG (Stefan Liebig) - initial API and implementation + *******************************************************************************/ +package org.eclipse.equinox.internal.p2.repositoryoptimizer; + +import org.eclipse.equinox.p2.artifact.repository.IArtifactRepository; + +/** + * Forms a common root for various <code>IOptimizers</code>. + * An <code>IOptimizers</code> does not need to implement this but doing so + * eases writing of <code>IOptimizers</code>. + */ +public abstract class AbstractOptimizer { + + protected IArtifactRepository repository; + + public AbstractOptimizer() { + super(); + } + + public void setArtifactRepository(IArtifactRepository repository) { + this.repository = repository; + } + +} Index: src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Pack200Optimizer.java =================================================================== RCS file: src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Pack200Optimizer.java diff -N src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Pack200Optimizer.java --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ src/org/eclipse/equinox/internal/p2/repositoryoptimizer/Pack200Optimizer.java 1 Jan 1970 00:00:00 -0000 @@ -0,0 +1,72 @@ +/******************************************************************************* +* Copyright (c) 2007 compeople AG and others. +* All rights reserved. This program and the accompanying materials +* are made available under the terms of the Eclipse Public License v1.0 +* which accompanies this distribution, and is available at +*http://www.eclipse.org/legal/epl-v10.html+* +* Contributors: +* compeople AG (Stefan Liebig) - initial API and implementation +* IBM - continuing development +*******************************************************************************/ +package org.eclipse.equinox.internal.p2.repositoryoptimizer; + +import java.io.IOException; +import java.io.OutputStream; +import org.eclipse.core.runtime.NullProgressMonitor; +import org.eclipse.equinox.p2.artifact.repository.ArtifactDescriptor; +import org.eclipse.equinox.p2.artifact.repository.IArtifactDescriptor; +import org.eclipse.equinox.p2.artifact.repository.processing.*; +import org.eclipse.equinox.p2.metadata.IArtifactKey; + +public class Pack200Optimizer extends AbstractOptimizer implements IOptimizer { + + public void optimize(IArtifactKey key) { + if (!key.getClassifier().equals("plugin")) + return; + + IArtifactDescriptor[] descriptors = repository.getArtifactDescriptors(key); + IArtifactDescriptor complete = null; + boolean optimized = false; + for (int j = 0; j < descriptors.length; j++) { + IArtifactDescriptor descriptor = descriptors[j]; + if (descriptor.getProcessingSteps().length == 0) + complete = descriptor; + optimized |= isOptimized(descriptor); + } + if (!optimized) + optimize(complete); + } + + private void optimize(IArtifactDescriptor descriptor) { + ArtifactDescriptor newDescriptor = new ArtifactDescriptor(descriptor); + ProcessingStepDescriptor[] steps = new ProcessingStepDescriptor[] {new ProcessingStepDescriptor("org.eclipse.equinox.p2.repositoryoptimizer.Pack200Unpacker", null, true)}; + newDescriptor.setProcessingSteps(steps); + OutputStream repositoryStream = null; + try { + repositoryStream = repository.getOutputStream(newDescriptor); + + // Add in all the processing steps needed to optimize (e.g., pack200, ...) + ProcessingStepHandler handler = new ProcessingStepHandler(); + OutputStream destination = handler.link(new ProcessingStep[] {new Pack200Step()}, repositoryStream, null); + + // Do the actual work by asking the repo to get the artifact and put it in the destination. + repository.getArtifact(descriptor, destination, new NullProgressMonitor()); + } finally { + if (repositoryStream != null) + try { + repositoryStream.close(); + } catch (IOException e) { + // TODO Auto-generated catch block + e.printStackTrace(); + } + } + } + + private boolean isOptimized(IArtifactDescriptor descriptor) { + // TODO this is a hack for now. Really we should inspect the steps to see + // if the optimization(s) we are doing here have already been done + return descriptor.getProcessingSteps().length > 0; + } + +} ### Eclipse Workspace Patch 1.0 #P org.eclipse.equinox.p2.tests Index: src/org/eclipse/equinox/p2/tests/artifact/repository/optimizer/PackRepositoryTest.java =================================================================== RCS file: /cvsroot/eclipse/equinox-incubator/provisioning/org.eclipse.equinox.p2.tests/src/org/eclipse/equinox/p2/tests/artifact/repository/optimizer/PackRepositoryTest.java,v retrieving revision 1.1 diff -u -r1.1 PackRepositoryTest.java --- src/org/eclipse/equinox/p2/tests/artifact/repository/optimizer/PackRepositoryTest.java 17 Oct 2007 21:12:34 -0000 1.1 +++ src/org/eclipse/equinox/p2/tests/artifact/repository/optimizer/PackRepositoryTest.java 18 Oct 2007 13:38:41 -0000 @@ -2,10 +2,13 @@ import java.io.*; import java.net.URL; +import java.util.ArrayList; +import java.util.List; import junit.framework.TestCase; import org.eclipse.core.runtime.NullProgressMonitor; import org.eclipse.equinox.internal.p2.metadata.ArtifactKey; import org.eclipse.equinox.internal.p2.repositoryoptimizer.Optimizer; +import org.eclipse.equinox.internal.p2.repositoryoptimizer.Pack200Optimizer; import org.eclipse.equinox.p2.artifact.repository.*; import org.eclipse.equinox.p2.core.helpers.ServiceHelper; import org.eclipse.equinox.p2.metadata.IArtifactKey; @@ -49,8 +52,9 @@ IArtifactKey key = new ArtifactKey("eclipse", "plugin", "org.eclipse.equinox.prov.engine", new Version("0.1.0.200709241631")); IArtifactDescriptor[] descriptors = repository.getArtifactDescriptors(key); assertTrue("Artifact Descriptor for engine missing", descriptors.length == 1); - - new Optimizer(repository).run(); + List optimizers = new ArrayList(); + optimizers.add(new Pack200Optimizer()); + new Optimizer(repository, optimizers).run(); descriptors = repository.getArtifactDescriptors(key); assertTrue("Optimization was a no-op", descriptors.length == 2);

_______________________________________________ equinox-dev mailing listequinox-dev@xxxxxxxxxxxhttps://dev.eclipse.org/mailman/listinfo/equinox-dev
_______________________________________________ equinox-dev mailing list equinox-dev@xxxxxxxxxxx https://dev.eclipse.org/mailman/listinfo/equinox-dev

Follow-Ups:
- Re: [equinox-dev] Re: Patch for RepoOptimizer
  - From: Stefan Liebig

References:
- Re: [equinox-dev] Re: Patch for RepoOptimizer
  - From: Stefan Liebig

Prev by Date: Re: [equinox-dev] [prov] Framework Admin test
Next by Date: [equinox-dev] [p2] Upgraded to ECF 1.2
Previous by thread: Re: [equinox-dev] Re: Patch for RepoOptimizer
Next by thread: Re: [equinox-dev] Re: Patch for RepoOptimizer
Index(es):
- Date
- Thread

Breadcrumbs