Thursday, July 3, 2008

Maven not downloading latest snapshots or releases

Ever had issues with maven not downloading the latest snapshots when you know for a fact new snapshots are available? Or your CI environment just deployed a new release (2.0), but when another Hudson job builds, maven does not download the latest 2.0 release artifact. Want an automated solution so you don't have to manually delete the artifacts from your local repository just so maven will download the latest?

Force maven to download latest snapshots
Our company uses Hudson for our automated CI environments. Our project basically has two jobs. The first job checks out and builds HEAD when modified and deploys SNAPSHOT WARs to our companies maven2 repository (artifactory). The second job, which builds nightly, uses maven to download the SNAPSHOT WARs from artifactory, creates an EAR, deploys it to JBoss, and runs integration tests. By default, maven will check once a day for changes to snapshots, so when our second job was triggered, maven inside hudson was not downloading the latest SNAPSHOT WARs.

The solution was to append the -U in the maven goals (run mvn -help). It stands for update-snapshots and tells maven to update all snapshots no matter what.


Force maven to download latest releases
Our next problem was when we created a branch and started creating release artifacts such as 2.0. Unfortunately the description given by maven for the -U option is incorrect (or at least in v 2.0.9), "Forces a check for updated releases and snapshots on remote repositories". As much as I tried, the -U option wouldn't work in our hudson job to force maven to download the latest non-snapshot releases.

The only current solution I know of is to use the maven-dependency-plugin and its goal purge-local-repository. So in your maven goals at some point execute mvn dependency:purge-local-repository and maven will physically delete your projects artifacts from the local repository (/home/user/.m2/repistory) and its transitive dependencies (I think). I tried setting the actTransitively to false and it didn't work for us so I just removed it. I also set verbose to true so I could see what maven deleted in Hudson's console output.



The pipes are used to separate out different goals to isolate its classpath or properties. That way we can skip tests in one run, and then run them in the next all in the same goals section.

16 comments:

  1. You can't force Maven to check for new releases because by definition, releases are supposed to be immutable. Therefore, forcing a clean of your local repo is the only way to do this...but you should evaluate your process as you should not be redeploying release artifacts.

    That said, I still purge my CI local nightly...it's a good way to catch other missing dependency errors to ensure everything is really located in the remote repositories. I also like to make sure each build is running in a separate local.

    ReplyDelete
  2. I would have to concur with previous poster, it's dangerous to have multiple versions of the (intended) same release-version. That's what snapshots are for. -How many times have you been annoyed with identical file naming although the contents aren't the same? - It would be nice to see an (even) less general maven though, that enforces stricter and newer process/release-practices. It's my belief that maven embodies a lot of knowledge at this stage and therefore will act as a teaching tool for beginners.

    ReplyDelete
  3. What is the best way to find out that a new release version is available? Let's say I depend on spring-2.5.2 and suddenly a spring-2.5.5 is available. Is there some way to make Maven check that a new version is available from some of the repositories (by reading the maven-metadata.xml) and put it on a report so that the project could choose to upgrade to the newer version.

    ReplyDelete
  4. That's a question that comes up often here as well. I think the idea is you don't want your build to automatically pull version 2.5.5 of something when you're currently using 2.5.2.

    You simply don't know whether your project will break because of something changing in one of your dependencies. Better that you consciously and deliberately specify a new version of a dependency after testing it.

    Ironically, I think this contrasts with how Maven plugins are managed - it automatically pulls down new versions and then one day your Maven build stops behaving the way it used to and you have no idea what's going on.

    As for notifications of new versions, I guess it's a person task instead of an automated machine process. i.e., check the websites, subscribe to an RSS feed etc.

    ReplyDelete
  5. @Andy,
    They fixed maven automatically pulling down core snapshot plugins in maven 2.0.9. In previous versions you should specify the version in the build pluginManagement section.

    ReplyDelete
  6. Andy,
    Yes, I definitely agree that it is a persons job to choose to upgrade to a newer version of Spring/Hibernate etc after the new versions have been tested and proved working well.

    But what I'm looking for is some kind of Maven report that tells you that a newer release version is available so that you don't have to manually monitor available releases of every dependency. Like the depgraph-plugin (http://el4j.sourceforge.net/plugins/maven-depgraph-plugin/images/depgraph.png) but possible with an extension that paints the rectangles in green when using the latest release version, yellow when using an older version with a newer available in Archiva/Artifactory... and orange for using a snapshot. This would be a fantastic tool for our team.

    ReplyDelete
  7. The tag updatePolicy in your local settings.xml is may be the solution.

    See http://maven.apache.org/ref/2.0.8/maven-settings/settings.html#class_releases

    Djedje

    ReplyDelete
  8. I think updatePolicy=always would also do the trick, but it has a big disadvantage of slowing the builds down for everyone all the time.

    ReplyDelete
  9. Update. I've basically found that the dependency:purge-local-repository doesn't really work like I expect it to.
    Therefore here is a Profile that I use which uses ant to delete a configurable directory under the local repo. This just works.
    Run using: mvn -PPurge -Dpurge.dir=com/google

    http://pastebin.com/fd69ba20

    ReplyDelete
  10. Thanks, your post helped me figure out my maven problem - much appreciated!

    ReplyDelete
  11. Thank you. That's exactly what I was looking for!

    ReplyDelete
  12. But consider the case maven is being used on hudson. Someone accidently used same artifact id and version number and did maven install.
    The modified is now in hudson .m2 and now be used in further builds which depend on it.
    How can one prevent this accidental human error. Anyway to revoke permissions of maven install on non snapshot versions (released artifacts). Any help would be appreciated.

    ReplyDelete
  13. just add always to your settings.xml file

    ReplyDelete
  14. I learned your posting! its really informative and helpful.
    getflv crack

    ReplyDelete