Artifactory 2.1.3 upgrade

I was waiting for Artifactory 2.1.3 upgrade since it includes a long-waited feature of being able to set an include and exclude patterns on virtual repositories.

Finally, I’m able to stop Maven and Artifactory from looking in external repos for our company’s snapshots updates. Setting an exclude pattern of “com/company/**” on “remote-repos” does the job. Great!

Also, I’ve started to use another neat feature (that was introduced in version 2.1) of being able to move artifacts to another repository. I’m not sure JFrog developers had my scenario in mind when they were working on it (they crafted it for staging and promotion – we’re not there yet .. ) but here it goes: from time to time it’s time to cleanup all old snapshots and start from day zero. I mean – re-build everything from scratch as if there were no yesterday’s snapshots in the company’s repository manager. Since we rely on snapshots heavily (create hundreds of them in different products) the good question is – are we still able to create all artifacts having absolutely no snapshots in the repository? Is our build order correct? Do we have any cyclic dependencies (that are perfectly hidden when they’re always yesterday’s snapshots to use)?

One way to answer this question is remove all snapshots from Artifactory and fire up the Hudson jobs that create all our products. If build order is wrong (and when there are many products built with different layers of “infrastructures” developed by different teams – it’s not that hard, really) – the job will fail. Since it has no “yesterday’s” snapshot to rely on. So it either works or it doesn’t.

I did it couple of time (still thinking of a way to do it automatically on a nightly basis – any suggestions here?) but what stopped me from doing it more frequently is I was simply afraid of deleting all old snapshots. What happens if some important snapshot was deployed once and I’ll find out I really need it after deleting it ?

Not any more since Artifactory 2.1!

After all – we have something called “Recycle Bin” since Windows 95 so I’ve created a standalone repository called “trash” that is not part of virtual repo in Artifactory (that are viewable to people in the company) and I move all old snapshots there first. That’s it!

I think it’s a critical check for any complex product since each time we do it – we find out our build order is slightly wrong and we do have cyclic dependencies here and there (with ~300 POMs spread all over SVN tree – you can’t monitor each of them).

Now, Hudson integration is coming next!

It’s a big thing, of course, but makes me give up on Maven’s deployment mechanism and rely on Hudson’s one. So far I’ve tried to keep our Hudson jobs as simple as possible without any additional steps (except invoking other jobs) so that all “build” logic will be stored in POMs for people to see it. If it’s spread between Maven and Hudson – my assumption is that it will be harder for people to see “how it works” and “what’s going on here” (questions I’m already asked a little bit too often).

But .. They way Hudson does a clean deploy to Artifactory will all metadata uploaded really makes me re-consider this approach and make our Hudson jobs “heavier”. This time it’s worth it, I think.

Some more recent Artifactory tips:

• A video session of Yoav and Fred (Artifactory authors) from Devoxx09 
     (PDF is available at JavaEdge 2009)
• How to use “artifactory.config.import.xml” file to bootstrap config data from it 
     (overriding possibly corrupted DB data)
• How to cleanup broken indices

P.S
Since version 2.1.3 Artifactory became even more stricter regarding POM’s “health status”. It’s good JFrog folks care a lot about what is stored inside Artifactory and work really hard not to allow any garbage “in”.

After making an upgrade some of our builds failed since it appears there are some pretty old POMs that have UTF-8 byte issues and Artifactory isn’t willing to serve them any more (they were already inside the storage but couldn’t be downloaded by Maven).

To fix that I had to:

• Download POMs manually from repo1
• Remove some characters in developers names XML parser was complaining 
     about (no offense, guys)
• Save them as “UTF-8 + BOM” in EditPlus
• Delete old copies that were already stored in Artifactory 
• Deploy new “fixed” copies instead

Here are the POMs that were fixed that way (updating our <dependencies> to rely on newer <version> didn’t help – those dependencies hardcoded somewhere deeper, inside Maven itself):

• org.codehaus.plexus:plexus:pom:1.0.4 (fixed version)
• org.codehaus.plexus:plexus:pom:1.0.5 (fixed version)
• plexus:plexus-root:pom:1.0.3 (fixed version)