Archive

Archive for the ‘Java’ Category

Artifactory Online – the case of distributing Groovy++

May 5, 2010 4 comments

After working with open source Artifactory version and thoroughly exploring it’s add-ons I knew it would come a moment to put my hands on its cloud solution – Artifactory Online. It just made sense to “close the loop” this way .. Тhe moment I’ve heard about online instance, running 24×7 without having to take care of anything – it sounded really, really nice.

I can’t say we spend a lot of time administering our open source version in Thomson Reuters. Quite the opposite – I only need to take it down for upgrades from time to time. But it still takes us a machine. A virtual one, of course but still – that’s CPU and memory that could be well spent somewhere else. So having online instance not only gives a peace of mind freeing everybody from taking care of one more server and one more database – it frees some hardware resources as well.

Well, a beauty of cloud computing, when it works. And Artifactory certainly does!

But my first use of Artifactory Online was for slightly different purpose – Maven support for the Groovy++ project. There was a clear need to host Groovy++ binaries in a public Maven repo.

What options are available today?

  • OSS repository hosting from Sonatype.
    It’s a good free solution but like any other free solution it only provides you so much: if you don’t mind being at mercy of other people with certain demands about how your POMs should look like – then it’s a good way to go. But I’d prefer my personal repository, where I can configure it the way I want without sharing it with other projects and asking favors. Also, bear in mind you get no security whatsoever – all your binaries would be open to everybody, anytime. And it only works for open-source projects which can be another showstopper.
  •  

  • Another option is to host a public Apache or nginx server and just make the files available following Maven’s naming conventions, like it’s done on "repo1". Not to mention the lack of security (again) – this kind of storage can be fragile to files corruptions: after all it’s just a dumb files storage, not an intelligent repo manager. You can’t use Maven to deploy artifacts and it provides no additional services, like virtual repos, artifacts searching or usage statistics.
  •  

  • Public hosting of open-source Nexus or Artifactory – it’s much better and we can finally protect it the way we want. But we still need to pay for hosting, memory usage, bandwidth usage and we now need to install and administer it on top of everything else. And put some extra protection, may be.
  •  

  • Artifactory Online. The best way to go, if you ask me. Not only it provides a cloud-based 24×7 running Artifactory instance, but it does so with all add-ons installed, so you really get it the ‘Full Monty’! It solves our original problem, to host binaries in a public Maven repo, but it doesn’t stop there, as I’ll show shortly – running a private Artifactory instance brings other advantages to your projects.

We’ve settled with last option, meet http://groovypp.artifactoryonline.com/!
The initial setup went very fast as there were very few things to take care of, actually:

  1. Registration
  2.  

  3. Creating deployment user and "settings.xml". The fact that Artifactory provides a way to generate new “settings.xml” (skip to 00:10:40) and store an encrypted passwords (00:11:55) comes in very handy:

    <servers>
        <server>
            <id>groovypp</id>
            <username>username</username>
            <password>\{ABAeqq\}pIcMooZ8G/2Y2drgC99SDw==</password> <!-- Sample -->
        </server>
    </servers>
    
  4. Instructing Maven about new repositories:

    <properties>
        <repo>http://groovypp.artifactoryonline.com/groovypp</repo>
    </properties>
    
    <distributionmanagement>
        <repository>
            <id>libs-releases-local</id>
            <url>${repo}/libs-releases-local</url>
        </repository>
    </distributionmanagement>
    
    <repositories>
        <repository>
            <id>libs-releases</id>
            <url>${repo}/libs-releases</url>
        </repository>
    </repositories>
    
    <pluginrepositories>
        <pluginrepository>
            <id>plugins-releases</id>
            <url>${repo}/plugins-releases</url>
        </pluginrepository>
    </pluginrepositories>
    

As you see, we’re using new repo not only for <distributionManagement> but as our only Maven repository.
From now on we only talk to groovypp.artifactoryonline.com/groovypp/libs-releases.
This is “virtual repository”, a “gateway” Maven will connect to for retrieving any 3-rd party library. I can now add additional Maven repositories by editing it in Artifactory, there’s no need to update the POM any more when new external repos are added to the project.

After this quick setup I ran it for the first time. I was expecting somewhat slower performance than the one we have in the office where Artifactory is running on the same network. After all, we’re talking here about remote repository running somewhere across the ocean:

But the download was pretty fast. It depends on the bandwidth, of course but I can’t say that significantly more distant repository has slowed me down. UI was very response and Maven’s filling of empty local repo was fast enough as not to notice any significant difference. Good!

Groovy++ project is now happily using Artifactory Online for several months and releases, you’re always welcome to download the latest version manually or give it a try with Maven.

What else can I say about running a private repo like that?

I think the main beauty of it is being able to “go public” in matter of minutes. No setups, no worries – you have your very own binaries storage, intelligent and secured that can be used for any purpose. That’s right, Artifactory can serve any binaries, not necessarily Maven’s artifacts. So one can store there practically anything and then secure or backup it safely.

Makes me think of various App Store services where people publish their Android / iPhone applications and enjoy the ride. That’s good, I believe “going public” should be easy for anyone today – this way creativity meets no entry barriers!

I only have a single request to Artifactory developers – an option to create aliases to existing repo. This would allow to reuse the same repository for different projects or purposes: http://projectA.artifactoryonline.com/ and http://projectB.artifactoryonline.com/ will point to the same Artifactory instance but will be used by different people.

Overall, a very pleasant experience!
Exactly what I was expecting – can’t help it but these guys never disappoint 🙂

Uploading files – multipart HTTP POST and Apache HttpClient

May 1, 2010 59 comments

I had to implement a files transferring mechanism where one machine was sending files to another using "multipart/form-data" POST request. It can be done using Apache’s Commons FileUpload and HttpClient.

The receiving part was an easy one:

<dependency>
    <groupid>commons-fileupload</groupid>
    <artifactid>commons-fileupload</artifactid>
    <version>1.2.1</version>
</dependency>

We parse an incoming request with ServletFileUpload and get a list of FileItems in return. Each FileItem is either form’s input field or a file uploaded:

if ( ServletFileUpload.isMultipartContent( request ))
{
    List<fileitem> fileItems = 
        new ServletFileUpload( new DiskFileItemFactory( 1024 * 1024, DIR )).
        parseRequest( request );

    for ( FileItem item : fileItems )
    {
        String fieldName = item.getFieldName();

        if ( item.isFormField()) { item.getString()      } // Form's input field
        else                     { item.getInputStream() } // File uploaded
    }
}

In our case, we use DiskFileItemFactory to store files larger than 1Mb in a temporary DIR. After reading file’s InputStream and storing the data in a proper storage – we need to delete the temporary copy: item.delete().

It’s the sending part that came out to be a bit trickier. Initially, I was using a simple HTML form:

<form action="http://localhost" method="post" enctype="multipart/form-data">
    <input type="file" name="file">
    <input type="text" name="paramName">
    <input type="submit" name="Submit" value="Upload File">
</form>

But then I’ve switched back to Java and assumed HttpClient will do the job.

<dependency>
    <groupid>org.apache.httpcomponents</groupid>
    <artifactid>httpclient</artifactid>
    <version>4.0.1</version>
</dependency>

Eventually, it did but it took me some time to figure out how. The problem with HttpClient is that it provides a nice tutorial and various usage examples but none of them actually mentions a word about uploading files!

I’ve figured out I need to set an instance of HttpEntity to request but it seemed like it’s going to be either a StringEntity or FileEntity but not both. How come ?! Why is it so hard to send a usual POST request with String and file parameters?

Ok, it’s Google time.

Some examples and documentation were referring to an outdated version when HttpClient was part of Apache Commons and, therefore, were of no use for me – the API has changed dramatically. Until I’ve found this example that finally saved my day. Radomir, thank you!

The solution is to use an additional Apache component – HttpMime:

<dependency>
    <groupid>org.apache.httpcomponents</groupid>
    <artifactid>httpclient</artifactid>
    <version>4.0.1</version>
</dependency>

<dependency>
    <groupid>org.apache.httpcomponents</groupid>
    <artifactid>httpmime</artifactid>
    <version>4.0.1</version>
</dependency>

and then we finally get to use a magical MultipartEntity:

HttpClient client = new DefaultHttpClient();
client.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1);

HttpPost        post   = new HttpPost( url );
MultipartEntity entity = new MultipartEntity( HttpMultipartMode.BROWSER_COMPATIBLE );

// For File parameters
entity.addPart( paramName, new FileBody((( File ) paramValue ), "application/zip" ));

// For usual String parameters
entity.addPart( paramName, new StringBody( paramValue.toString(), "text/plain", 
                                           Charset.forName( "UTF-8" )));

post.setEntity( entity );

// Here we go!
String response = EntityUtils.toString( client.execute( post ).getEntity(), "UTF-8" );

client.getConnectionManager().shutdown();

Note the use of EntityUtils for reading the response.

That’s it.

I only wish library authors were providing better support and examples for more common cases like files uploading in our case. I mean, come on, when people get to use HttpClient they either want to send a usual request or upload a file, same thing they do with browser. Am I wrong here?

Categories: Java, Web Tags: , , , , ,

Searching for classes inside *.jar files

January 13, 2010 Leave a comment

Just a quick tip about searching for classes inside *.jar files.
TC rocks! I was introduced Agent Ransack today but .. nothing beats good old Alt+F7 🙂

 

1

Categories: Java, PC Tags: ,

Books Summaries – Data Structures & Algorithms, Design Patterns, Java

August 28, 2009 Leave a comment

When preparing for Google and Microsoft interviews two years ago (that was a crazy month!) – I made summaries of some books and materials I was reading: