Maven Reloaded Jason van Zyl jason@sonatype.com http://twitter.com/jvanzyl Why Maven 3.0 Maven 2.0 Components lacked a degree of encapsulation and layering. We didn't know what extension points we would need to be honest. Code base in many places is very difficult to understand and therefore very hard to get more people involved. We actually have more contributors to 3.x already. Fundamental problems with many of the sub-systems we just couldn't have foreseen without extensive use. Maven 3.0 We really tried to understand from users what they needed and then fixed and implemented their requests. The number one concern was backward compatibility. Maven 3.x is backward compatible. Fewer modules, code has been reduced. We hope the code base will be simpler so that more people can get involved. Heavy overhauling of many of the internals to make way for future features. Maven 3.x is really the only path forward for Maven. A performance framework is being used to track any non-functional regressions. A massive increase in the integration tests in the core and for core plugins. Again protection for users, and helps other developers learn and get involved. Desire lines The path usually represents the shortest or most easily navigated route between an origin and destination. The width and amount of erosion of the line represents the amount of demand. The term was coined by Gaston Bachelard in his book The Poetics of Space. Desire lines can usually be found as shortcuts where constructed pathways take a circuitous route. Responding to invitations for improvement Responding to under utilization of what exists Sonatype Books Maven: The Definitive Guide Repository Management with Nexus Developing with Eclipse and Maven Maven Handbook (focusing on OSGi so far) All available @ http://www.sonatype.com/documentation/books All books will be available for on-demand printing @ Lulu @ cost. Backward Compatibility We must ensure that plugins and reports written against the Maven 2.0.x APIs remain viable in 3.0. We don't want people rewriting their plugins or having to change their projects at all. We must also ensure that POMs of version 4.0.0 are supported in 3.x along with the behavior currently experienced. We are relying heavily on our integrations tests right now but as we move forward the work that Benjamin is doing on the project builder with model-builder will help us to accommodate different versions of a POM, and different formats we decide to support. The model-builder code has no limitation with respect to formats. We can support XML, or any source that anyone can dream up. These implementations may find use outside of Maven. For example someone might build something with the Maven Embedder, JRuby, and Mercury to create a JRuby-based system. The same could be done for Groovy. We have managed to keep almost everything backward compatible and we will go so far as to provide an isolated execution environment that can contain older versions of the plugin API so that everything can work. We'll see as we progress toward 3.0 GA if this is necessary. This primarily revolves around plugins that require the old artifact resolution code which is not entirely compatible with the new artifact resolution code at the API level. We tried to preserve behavior but there's not much we could do about some of the APIs. Integration Testing An enormous amount of time and energy has gone into improving the the integration tests for Maven. The integration tests are the gatekeeper and will let us determine that we have a Maven 3.x that is compatible for Maven 2.x projects. We now have 506 integration tests that will help protect us as we move forward. All core Maven plugins now have ITs and we have started branching out into the Mojo project to create plugin ITs there as well. We have a good pattern so we are encouraging anyone writing a Maven plugin to create ITs so that we can help ensure we don't break people in the future. Refactored project builder Pluggable ModelReader i.e. the source of the information is irrelevant provided we get a Model object back from the reader: Scripted POM formats (pom.{rb|groovy|py}) & terse formats like YAML Mechanism to extend POMs in a compositional way: Mixins: the first implementation of a mixin will be a capability, and the first capability will a standard release toolchain Keep hold of non-mutated model & track contributions to the effective POM POM refactoring in M2Eclipse Polyglot Maven Polyglot Maven DSLs Groovy Ruby Scala Python DSLs will have access lifecycle extensions points DSLs will be able to modify the execution plan DSLs will be able to decorate the lifecycle Queryable lifecycle The most important change internally in Maven 3.x. You can actually query Maven for the complete execution plan before it happens. We know about all dependencies, plugins, versions and configurations. We have the complete view of all metadata because the resolution of metadata is completely separate from the resolution of actual artifacts. Symmetric plugin parameters will be necessary to know how the model might change during execution. We need to know before hand where a plugin will generate a source directory versus having to execute the lifecycle and examine the compile source roots. This also applies to plugins that would add resources to a project. Incremental build support We have better integration with m2eclipse to provide deadly fast execution in Eclipse where most of the phases in the default Maven lifecycle don't need to execute. Most projects really only need the dependency management aspects of Maven and a few plugins to execute. Of the plugins that execute Maven should not operate on all resources in a project for an operation when only a single resource has changed. We have only modified three plugins and the mechanism is working well working well. We have modified: maven-resources-plugin: generated resources modello-maven-plugin: generated sources plexus-component-metadata: annotation processing producing resources Lifecycle extension points Clear extension points for processing at the beginning and end of a lifecycle. For our Tycho OSGi integration we need access to the resolved dependencies in Maven form before the lifecycle executed so that we could adjust the versions with real OSGi resolution. Aggregator mojos bound to the lifecycle have been deprecated. This practice can produce some very strange results, and isn't really the right solution for many of the problems it attempts to solve. They really need to run in a clear location after the lifecycle has executed. Pre-build Pre-build Lifecycle EP Build Lifecycle Build Lifecycle Post-build Post-build Lifecycle EP Tycho Tycho Tycho attempts to be a complete replacement for PDE headless build. Tycho already uses Eclipse/OSGi metadata to resolve project dependencies by OSGi rules and injects these dependencies into maven project model dynamically, at build time. Tycho uses JDT to make sure modularity rules are applied. Tycho uses the OSGi state resolver is used to make sure the resolution that occurs during build-time matches what you will need at runtime. Tycho supports bundle, fragment, feature and update site projects, as well as RCP applications. Tycho knows how to run JUnit test plugins using the OSGi runtime. Tycho has support for P2 repositories, Update Sites and Maven repositories. Embedding The Maven 3.x codebase is currently being used in all the major IDEs m2eclipse IDEA Netbeans Maven can now be integrated in any build or development tools and lot easier then it ever could. Now that we have successfully integrated Maven 3.x into M2Eclipse we are attempting to embed Maven 3.x in Nexus and Hudson. A huge number of internal changes have been made to support extremely tight and efficient integration with Maven in embedded environments. Maven Shell Maven Shell Make-like reactor support Archetype integration: interactively create new projects Hudson integration: creating jobs, querying results Nexus integration: provisioning Maven settings, searching, creating repositories Error & integrity reporting Much improved error reporting where we will provide links to each identifiable problem we know of. There are currently 42 common things that can go wrong. Don't allow builds where versions come from non-project sources like local settings and CLI parameters Don't allow builds where versions come from profiles that have to be activated manually Refactored plugin manager Generalizing and extracting the plugin manager capabilities to make it available as a standalone component Download artifacts from artifact repositories Create isolated classloaders based on the artifacts downloaded XStream-like configuration mechanism Create isolated execution environments We will use these features to make sure we can support all previous versions of Maven plugins Plugin extension points A plugin extension point must have access to everything in the execution context ExecutionRequest MavenSession Project and its lineage Plugin authors would provide known sites of extension. There may be any number of extensions points depending on what you're doing: For the OSGi example we need an explicit place to modify manifest entries Adding license information could just be something done at an extension point of the packaging plugins Manipulation of web.xml in a WAR, or manipulation of the libraries ... a simpler WAR overlay system We need specification about ordering and guidelines where extension points could be manipulating the same resources Maven War Plugin Extension Extension Extension Maven War Plugin Subclass Subclass Subclass Jetty Client & SAT4J New repository and transport layer Developed by Greg, Jan, and Jesse from Webtide (the Jetty people) Super fast transport -- async client with connection pooling and parallelization Atomic downloads and deployments (with Nexus) Full SSL support Full PGP support WebDAV client built-in Full support for ranges using a pseudo boolean solver -- SAT4J Designed, implemented and tested without POMs. Mercury can be embedded and will allow any domain specific application to retrieve, and deploy artifacts from any type of Repository: GEMs, P2, and Maven repositories are possible Extensible reporting Create a completely separate system for reporting. It should not be part of Maven's core and should really live in its own world. Different reporting solutions should be supported: not just the maven-site-plugin solution which is completely coupled to Doxia. Maven 3.x will not be coupled to Doxia. An entirely separate execution environment is required for reporting. A data production model for build information is more useful then the document production model. This is where we could completely mesh with Hudson so that Maven reports could be used to produce useful trending information. This approach would also be useful for quality reporting systems like Sonar. Maven Maven Standard Site Generation & Reporting Site Generation & Reporting Data-centric Reporting Sonar or Hudson Reporting Guice + Peaberry Plexus has a long but quiet history. Started as an offshoot of the Apache Avalon project but it has outlived its usefulness. Plexus started at approximately the same time as Spring but Spring didn't yet support many features Maven needed. It does now, but Guice is closer to what Plexus is. Guice will provide the new mode of DI in Maven and Peaberry will allow us to run transparently within OSGi. Peaberry is the Guice equivalent of Spring dm. Suck less. Use Maven. What questions do you have?