Home > Java, Web > Uploading files – multipart HTTP POST and Apache HttpClient

Uploading files – multipart HTTP POST and Apache HttpClient

I had to implement a files transferring mechanism where one machine was sending files to another using "multipart/form-data" POST request. It can be done using Apache’s Commons FileUpload and HttpClient.

The receiving part was an easy one:

<dependency>
    <groupid>commons-fileupload</groupid>
    <artifactid>commons-fileupload</artifactid>
    <version>1.2.1</version>
</dependency>

We parse an incoming request with ServletFileUpload and get a list of FileItems in return. Each FileItem is either form’s input field or a file uploaded:

if ( ServletFileUpload.isMultipartContent( request ))
{
    List<fileitem> fileItems = 
        new ServletFileUpload( new DiskFileItemFactory( 1024 * 1024, DIR )).
        parseRequest( request );

    for ( FileItem item : fileItems )
    {
        String fieldName = item.getFieldName();

        if ( item.isFormField()) { item.getString()      } // Form's input field
        else                     { item.getInputStream() } // File uploaded
    }
}

In our case, we use DiskFileItemFactory to store files larger than 1Mb in a temporary DIR. After reading file’s InputStream and storing the data in a proper storage – we need to delete the temporary copy: item.delete().

It’s the sending part that came out to be a bit trickier. Initially, I was using a simple HTML form:

<form action="http://localhost" method="post" enctype="multipart/form-data">
    <input type="file" name="file">
    <input type="text" name="paramName">
    <input type="submit" name="Submit" value="Upload File">
</form>

But then I’ve switched back to Java and assumed HttpClient will do the job.

<dependency>
    <groupid>org.apache.httpcomponents</groupid>
    <artifactid>httpclient</artifactid>
    <version>4.0.1</version>
</dependency>

Eventually, it did but it took me some time to figure out how. The problem with HttpClient is that it provides a nice tutorial and various usage examples but none of them actually mentions a word about uploading files!

I’ve figured out I need to set an instance of HttpEntity to request but it seemed like it’s going to be either a StringEntity or FileEntity but not both. How come ?! Why is it so hard to send a usual POST request with String and file parameters?

Ok, it’s Google time.

Some examples and documentation were referring to an outdated version when HttpClient was part of Apache Commons and, therefore, were of no use for me – the API has changed dramatically. Until I’ve found this example that finally saved my day. Radomir, thank you!

The solution is to use an additional Apache component – HttpMime:

<dependency>
    <groupid>org.apache.httpcomponents</groupid>
    <artifactid>httpclient</artifactid>
    <version>4.0.1</version>
</dependency>

<dependency>
    <groupid>org.apache.httpcomponents</groupid>
    <artifactid>httpmime</artifactid>
    <version>4.0.1</version>
</dependency>

and then we finally get to use a magical MultipartEntity:

HttpClient client = new DefaultHttpClient();
client.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1);

HttpPost        post   = new HttpPost( url );
MultipartEntity entity = new MultipartEntity( HttpMultipartMode.BROWSER_COMPATIBLE );

// For File parameters
entity.addPart( paramName, new FileBody((( File ) paramValue ), "application/zip" ));

// For usual String parameters
entity.addPart( paramName, new StringBody( paramValue.toString(), "text/plain", 
                                           Charset.forName( "UTF-8" )));

post.setEntity( entity );

// Here we go!
String response = EntityUtils.toString( client.execute( post ).getEntity(), "UTF-8" );

client.getConnectionManager().shutdown();

Note the use of EntityUtils for reading the response.

That’s it.

I only wish library authors were providing better support and examples for more common cases like files uploading in our case. I mean, come on, when people get to use HttpClient they either want to send a usual request or upload a file, same thing they do with browser. Am I wrong here?

About these ads
Categories: Java, Web Tags: , , , , ,
  1. Sanket
    May 13, 2010 at 23:34

    Can you please send me code for this ?
    I followed completely your mentioned steps.
    However, I have added login steps before actually setting MultiPartEntity and executing post request.
    At Server side, in servlet ServletFileUpload.isMultipartContent( request ) returns false!!
    Thanks in advance!

    • Sanket
      May 14, 2010 at 03:03

      I was able to resolve issue. thanks!

      • May 14, 2010 at 08:28

        Glad to hear that. Can you share the solution to this problem may be?

  2. Sanket
    May 14, 2010 at 18:21

    HttpClient uses same cookie store for below mentioned steps :
    1. Request to protected resource using HttpGet
    2. Submit j_security_check with j_username & j_password using HttpPost (FormAuthentication)
    3. Request for the same protected resource using HttpGet
    4. Request to any protected resource

    Earlier, I was missing step 3 and trying to perform step 4 directly.

    • May 15, 2010 at 00:09

      I see. Many thanks for sharing that, I or anybody else may need it one day :)

      • Imani Mwendamseke
        January 30, 2013 at 12:05

        your very right in this

  3. June 14, 2010 at 19:52

    Thanks! it was very usefull!

  4. Jasim
    June 25, 2010 at 12:08

    I’m trying this out for Android development. As you pointed, the documentation for the POSTing files using Apache HttpClient is very sparse on the net.

    Thanks very much for sharing.

    • June 25, 2010 at 18:42

      You’re most welcome! I would appreciate if you post here anything interesting you come along. Just to let other people learn from our efforts .. May I ask you what kind of Android development are you dealing with? I had a very limited Android experience in the past, before first HTC phone was even launched, it’s definitely waaay different now.

    • jw
      August 29, 2012 at 18:23

      Jasim

      Did you ever get this working with android?

  5. Josep
    June 30, 2010 at 09:38

    Thank you very much for the example. You have saved my life.

  6. August 14, 2010 at 10:55

    Hi,

    Thanks for the pointers.

    I tried to upload to a web service and even though I’ve set the Mime type for the file, everytime I uploaded a file, the Mime type somehow is lost. A debug on the server side showed that the file uploaded doesn’t have a Mime type declared.

    My coding on the android side is pretty much the same with the example you posted here. I’m using php to handle the file uploads.

    Any other pointers? Thanks.

    Best,
    Tista

    • August 14, 2010 at 13:51

      Hi Tista, I would try to sniff the network and see what HTTP headers are sent. You’ll be able then to see if MIME type was dropped on the sending or receiving side. But from what you described it looks to me the sending side is the one to blame .. After making sure where the problem is I would start digging it with a debugger or asking on a relevant forum.

      Hope it helps. Let me know what you find out, if you don’t mind.

    • jw
      August 29, 2012 at 18:02

      Tista

      did you get this working with android?

  7. Rafae
    August 27, 2010 at 13:00

    Hi, I´m trying to send a file to my server followingyour guide.
    As you can see it is almost your code.

    It seem to work properly, but my jsp detectas that it received a multipart request, but does´nt find any file in it.

    My client side is:

    File f1 = new File(“C:\\workspace\\WTP\\TestBlueViaSVN2\\WebContent\\WEB-INF\\lib\\httpmime-4.0.1.jar”);
    HttpPost httppost = new HttpPost(“myserver/myapp/jsp/endpoints/uploadDummyListener.jsp”);
    HttpClient client = new DefaultHttpClient(); client.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1);
    MultipartEntity entity = new MultipartEntity( HttpMultipartMode.BROWSER_COMPATIBLE );
    // For File parameters
    entity.addPart( “fichero”, new FileBody((f1 ), “application/zip” ));
    httppost.setEntity( entity );
    // Here we go!
    String response = EntityUtils.toString( client.execute( httppost ).getEntity(), “UTF-8″ );
    client.getConnectionManager().shutdown();
    System.out.println(response);

    At this pint it connects, and y canread the response (it is a dunny after all :) )

    My server side (JSP) is
    if ( ServletFileUpload.isMultipartContent( request ))
    {
    System.out.println(“detectamos que es multipart”);
    sesion.getServletContext().getRealPath(“./img_upload/”);
    String rutaInicial = session.getServletContext().getServletContextName();
    rutaInicial = session.getServletContext().getRealPath(“/img_upload/”);
    System.out.println(rutaInicial);
    FileItemFactory factory = new DiskFileItemFactory();
    DiskFileItemFactory diskFileItemFactory = new DiskFileItemFactory();

    // Set factory constraints
    diskFileItemFactory.setSizeThreshold(1994096);
    diskFileItemFactory.setRepository(new File(rutaInicial));
    // Create a new file upload handler
    ServletFileUpload upload = new ServletFileUpload(diskFileItemFactory);

    List fileItems = upload.parseRequest( request );

    for ( FileItem item : fileItems )
    {
    String fieldName = item.getFieldName();

    if ( item.isFormField()) { item.getString(); } // Form’s input field
    else { item.getInputStream(); } // File uploaded
    }
    }

    At this point in my jsp out put i can see that reqest is multipart, but it can´t find the file sent.

    Cabeceras:
    header: content-length = 25614
    header: content-type = multipart/form-data; boundary=eP63Lo9GN86HN0rtUXdyzmnmOUV
    oUuGAodfLAk2
    header: host = localhost:8080
    header: connection = Keep-Alive
    header: user-agent = Apache-HttpClient/4.0.1 (java 1.5)
    header: expect = 100-Continue
    Parametros:
    ahora lo intentamos con el fileupload.
    detectamos que es multipart
    C:\tomcat-6.0.18\webapps\TestBlueVia\img_upload

    Have anyone of you an idea of what i´m doing wrong?

    Thank you very much

  8. September 6, 2010 at 10:08

    I having some trouble, i think it upload but i can see it, one question, the html has 2 hidden value

    is these equivalent ?

    entity.addPart( “upload_ticket”, new StringBody(“385468-a88cd45bb4ae”));
    entity.addPart( “response_url”, new StringBody(“http://www.mysite.com/response_url.php”));

  9. lee
    October 19, 2010 at 09:18

    Hi, how should the client parse the response if it is also multipart?
    say text/html and text/plain?

    • October 19, 2010 at 19:03

      Can the response be multipart? Never heard of that, really. Can you provide some example or some online service sending such a response? I would love to take a look.

      • lee
        October 19, 2010 at 19:23

        likely non-standard. we’re told it’s somewhat a proprietary version of protocol using only multipart post for both request/response.
        (by the way any problem doing this?)

        so is there any easy trick to parse a multipart on the client side?

  10. October 19, 2010 at 23:51

    Not that I’m aware of. Need to see how it looks like first.

  11. Anirudh
    December 22, 2010 at 08:54

    How would we extract the uploaded files above ? Is there a way to directly upload files to our hard drives using http client or is it only possible through httppost or put ?

    • December 22, 2010 at 15:24

      Hello Anirudh, Extracting files is shown in the second code example where “item.getInputStream()” is reading file data as InputStream. Uploading files directly to the hard drive can only be done when a POST request is sent and some code reads it and writes to the disk. Or you can use alternative solution like Dropbox which synchronizes folders on different computers. With Dropbox all one needs to do is to cope a file to a local folder, watched by a Dropbox service. After a while this fill will appear on the second computer as well.

  12. Anirudh
    December 23, 2010 at 05:43

    Hi, Evgeny
    Thanks a ton for both our solutions. It has helped to a great extent.

  13. Anirudh
    December 23, 2010 at 05:43

    *read it as your thats a typo

  14. Patricio Echague
    March 21, 2011 at 19:37

    Thank you for the neat example!. Very useful.

  15. Matt
    April 7, 2011 at 04:02

    This is quite helpful and I was wondering how you would propose to use the multi-part entity to post to picasa – when i post a single fileentity with just the file and no meta data it works great and the picture uploads properly. When i switch to multipart i am receiving 403 statuses.

    From Google API docs : http://code.google.com/apis/picasaweb/docs/2.0/developers_guide_protocol.html#Auth

    And use the following format for the body of the POST:
    —–
    Content-Type: multipart/related; boundary=”END_OF_PART”
    Content-Length: 423478347
    MIME-version: 1.0

    Media multipart posting
    –END_OF_PART
    Content-Type: application/atom+xml

    plz-to-love-realcat.jpg
    Real cat wants attention too.

    –END_OF_PART
    Content-Type: image/jpeg

    …binary image data…
    –END_OF_PART–

    If you were able to help me move that into Java (as i know CF and am interfacing the java functions) that would accomplish this, it would be very helpful to me and to a CF project on riaforge that i am adding this functionality into.

    Thanks for any help you may have to offer,
    Matt

  16. RR
    July 18, 2011 at 14:35

    Hello :

    Thanks for your example. But I would like to ask if need upload 2 files at same time…
    What I’m doing is:

    entity.addPart(“uploadedfile”, new FileBody((file1), “application/zip”));
    entity.addPart(“uploadedfile2″, new FileBody((file2), “application/zip”));

    The “uploadedfile” and “uploadedfile2″ are the name of HTML’s file tag.
    And I did this for getting response page body:

    BufferedReader bs=null;
    bs=new BufferedReader(new InputStreamReader(client.execute(post).getEntity().getContent()));
    String s=””;
    while(s!=null)
    {
    responseBody.append(s);
    s=bs.readLine();
    }

    But only response I can get is a “0″.
    Am I doing wrong ?

    • July 21, 2011 at 07:00

      Self reply:

      Finally I did confirmed that was server side issue, the codes above I posted was workable, thanks for your samples Evgeny … :-)

      • July 21, 2011 at 11:15

        Hi, Glad it worked out for you! Sorry for not replying earlier, this blog is an abandoned one for me after I moved to my own hosting.

  17. jb1
    July 20, 2011 at 16:28

    I’m having an issue with authentication when using this. The httpclient is not sending the Authorization header when a FileBody or InputStreamBody are added. It will send it when there is only a StringBody though. If I use addHeader there is an extra CRLF that gets inserted. Any advice?

    • July 21, 2011 at 11:17

      How about trying out other clients, like Groovy HttpBuilder?

      • jb1
        July 22, 2011 at 05:45

        I got it working for what I needed using a very basic implementation with URLConnection but then switched to Socket so I could get upload progress. Unfortunate bug though.

  18. mjlks
    July 22, 2011 at 19:00

    Hello! I come from Chinese readers. What: entity.addPart (paramName, new FileBody (((File) paramValue), “application / zip”)); one of how to deal with when File is empty

  19. mjlks
    July 24, 2011 at 13:55

    Hello! Upload all parameters, how to set the specified separator

  20. navil
    July 25, 2011 at 18:05

    Hi, first is so helpfull this post, thanks to share,

    I’m using your code but i have a problem at this line
    ServletFileUpload.isMultipartContent(request)
    the form that i´m using I draw it dynamically, so when I’m using IE isMultipartContent but when i use firefox always return false.
    If there´s any suggestion about this I really appreciate.

    • July 26, 2011 at 23:34

      Hi, I would try to see in HTTP sniffer how headers sent differ in both cases. Hope that helps.

  21. Thiyagarajan
    September 19, 2011 at 13:06

    hai evgeny i designed one jsp page for how to upload a file by using streams concept. i want to upload my file by using servlet coding and also store it in a mysql page. could u please send me example regarding that

  22. October 24, 2011 at 08:31

    I cherished up to you will obtain carried out right here. The comic strip is attractive, your authored subject matter stylish. nonetheless, you command get got an impatience over that you want be delivering the following. ill without a doubt come more earlier once more as precisely the similar nearly a lot incessantly inside of case you protect this hike.

  23. Jan B
    October 27, 2011 at 22:19

    Awesome tutorial, saved me a LOT of time,

    Thanks a lot

  24. Pete S
    November 11, 2011 at 18:55

    This is what all code tutorials should look like. Concise, to-the-point and with immediately applicable examples. Thank you!

  25. avinash
    April 10, 2012 at 13:21

    good one made my day

  26. lindi
    April 26, 2012 at 18:52

    First of all I would like to say thanks for this good tutorial.

    I’m trying to use this solution with jboss 4.0.3, and I have commons-fileupload-1.2.1.jar but seam that doesn’t work.
    When I post the form with the chosen file (via debug) there are no items and no errors.
    Does someone have any idea of what could be?

    Thank you in advance.

    lindi

    • lindi
      April 26, 2012 at 18:57

      errata corrige: jboss 4.3. I’m afraid that there are some problems using commons-fileupload-1.2.1.jar with this version of jboss.

  27. Andy
    July 12, 2012 at 11:12

    Files are not getting uploaded on the server / url.
    When I am uploading some files to httpd server using the above code , I am getting status code as 301 Moved permanently.

  28. July 14, 2012 at 20:04

    Heya this is kind of of off topic but I was wondering
    if blogs use WYSIWYG editors or if you have to manually code with HTML.
    I’m starting a blog soon but have no coding knowledge so I wanted to get advice from someone with experience. Any help would be enormously appreciated!

  29. soumyajit
    August 23, 2012 at 12:30

    Hi Evgeny,
    I have gone through the code, it is very helpful. But, can I upload a document in along with the metadata (like “title”,”Description”etc) using httpclient post/put method?

  30. Yuriy
    October 5, 2012 at 10:50

    A very helpful post!! Thank you!

  31. April 27, 2013 at 14:29

    you’re really a just right webmaster. The site loading pace is amazing. It kind of feels that you’re
    doing any distinctive trick. Also, The contents are masterpiece.
    you’ve done a excellent activity in this topic!

  32. May 18, 2013 at 09:28

    Solid write up. I study something like this here at Boise State
    University. It’s definitively stimulating to know writing from other writers and observe a little bit from their source. I’d love to use some of this material on my own webpage (if you don’t mind). And of course, I’ll put up
    a link to your site at wordpress.com on my own blog.

    Kudos for posting.

  33. chetan
    January 17, 2014 at 12:07

    can u send the code please..

  34. Anand
    April 1, 2014 at 08:50

    I am getting the below exception:
    com.sun.xml.ws.server.UnsupportedMediaException: Unsupported Content-Type: multipart/form-data; boundary=ZJop7mVAk6EuTefHEjQfFlAUTv7aov_vq9uCAN Supported ones are: [text/xml]
    Any ides

  35. May 23, 2014 at 10:13

    All modern games should have some form of a social networking component as well.
    You can do their hair, make up and give them a totally different look.
    Older calculators may not display information about
    the hardware version unless its HW2 or newer.

  36. June 2, 2014 at 06:06

    Fastidious response in return of this difficulty with solid arguments and explaining everything
    regarding that.

  37. September 16, 2014 at 08:40

    Thanks, this post is very useful.

  1. November 22, 2014 at 20:54

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: