Uploading files – multipart HTTP POST and Apache HttpClient
I had to implement a files transferring mechanism where one machine was sending files to another using "multipart/form-data"
POST request. It can be done using Apache’s Commons FileUpload
and HttpClient
.
The receiving part was an easy one:
<dependency> <groupid>commons-fileupload</groupid> <artifactid>commons-fileupload</artifactid> <version>1.2.1</version> </dependency>
We parse an incoming request with ServletFileUpload
and get a list of FileItem
s
in return. Each FileItem
is either form’s input field or a file uploaded:
if ( ServletFileUpload.isMultipartContent( request )) { List<fileitem> fileItems = new ServletFileUpload( new DiskFileItemFactory( 1024 * 1024, DIR )). parseRequest( request ); for ( FileItem item : fileItems ) { String fieldName = item.getFieldName(); if ( item.isFormField()) { item.getString() } // Form's input field else { item.getInputStream() } // File uploaded } }
In our case, we use DiskFileItemFactory
to store files larger than 1Mb in a temporary DIR
. After reading file’s InputStream
and storing the data in a proper storage – we need to delete the temporary copy: item.delete()
.
It’s the sending part that came out to be a bit trickier. Initially, I was using a simple HTML form:
<form action="http://localhost" method="post" enctype="multipart/form-data"> <input type="file" name="file"> <input type="text" name="paramName"> <input type="submit" name="Submit" value="Upload File"> </form>
But then I’ve switched back to Java and assumed HttpClient
will do the job.
<dependency> <groupid>org.apache.httpcomponents</groupid> <artifactid>httpclient</artifactid> <version>4.0.1</version> </dependency>
Eventually, it did but it took me some time to figure out how. The problem with HttpClient
is that it provides a nice tutorial and various usage examples but none of them actually mentions a word about uploading files!
I’ve figured out I need to set an instance of HttpEntity
to request but it seemed like it’s going to be either a StringEntity
or FileEntity
but not both. How come ?! Why is it so hard to send a usual POST request with String and file parameters?
Ok, it’s Google time.
Some examples and documentation were referring to an outdated version when HttpClient
was part of Apache Commons and, therefore, were of no use for me – the API has changed dramatically. Until I’ve found this example that finally saved my day. Radomir, thank you!
The solution is to use an additional Apache component – HttpMime:
<dependency> <groupid>org.apache.httpcomponents</groupid> <artifactid>httpclient</artifactid> <version>4.0.1</version> </dependency> <dependency> <groupid>org.apache.httpcomponents</groupid> <artifactid>httpmime</artifactid> <version>4.0.1</version> </dependency>
and then we finally get to use a magical MultipartEntity
:
HttpClient client = new DefaultHttpClient(); client.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1); HttpPost post = new HttpPost( url ); MultipartEntity entity = new MultipartEntity( HttpMultipartMode.BROWSER_COMPATIBLE ); // For File parameters entity.addPart( paramName, new FileBody((( File ) paramValue ), "application/zip" )); // For usual String parameters entity.addPart( paramName, new StringBody( paramValue.toString(), "text/plain", Charset.forName( "UTF-8" ))); post.setEntity( entity ); // Here we go! String response = EntityUtils.toString( client.execute( post ).getEntity(), "UTF-8" ); client.getConnectionManager().shutdown();
Note the use of EntityUtils
for reading the response.
That’s it.
I only wish library authors were providing better support and examples for more common cases like files uploading in our case. I mean, come on, when people get to use HttpClient
they either want to send a usual request or upload a file, same thing they do with browser. Am I wrong here?
Can you please send me code for this ?
I followed completely your mentioned steps.
However, I have added login steps before actually setting MultiPartEntity and executing post request.
At Server side, in servlet ServletFileUpload.isMultipartContent( request ) returns false!!
Thanks in advance!
I was able to resolve issue. thanks!
Glad to hear that. Can you share the solution to this problem may be?
HttpClient uses same cookie store for below mentioned steps :
1. Request to protected resource using HttpGet
2. Submit j_security_check with j_username & j_password using HttpPost (FormAuthentication)
3. Request for the same protected resource using HttpGet
4. Request to any protected resource
Earlier, I was missing step 3 and trying to perform step 4 directly.
I see. Many thanks for sharing that, I or anybody else may need it one day 🙂
your very right in this
Thanks! it was very usefull!
I’m glad to help, Diego
I’m trying this out for Android development. As you pointed, the documentation for the POSTing files using Apache HttpClient is very sparse on the net.
Thanks very much for sharing.
You’re most welcome! I would appreciate if you post here anything interesting you come along. Just to let other people learn from our efforts .. May I ask you what kind of Android development are you dealing with? I had a very limited Android experience in the past, before first HTC phone was even launched, it’s definitely waaay different now.
Jasim
Did you ever get this working with android?
Thank you very much for the example. You have saved my life.
Wow. I’m very glad to hear that 🙂
Hi,
Thanks for the pointers.
I tried to upload to a web service and even though I’ve set the Mime type for the file, everytime I uploaded a file, the Mime type somehow is lost. A debug on the server side showed that the file uploaded doesn’t have a Mime type declared.
My coding on the android side is pretty much the same with the example you posted here. I’m using php to handle the file uploads.
Any other pointers? Thanks.
Best,
Tista
Hi Tista, I would try to sniff the network and see what HTTP headers are sent. You’ll be able then to see if MIME type was dropped on the sending or receiving side. But from what you described it looks to me the sending side is the one to blame .. After making sure where the problem is I would start digging it with a debugger or asking on a relevant forum.
Hope it helps. Let me know what you find out, if you don’t mind.
Tista
did you get this working with android?
Hi, I´m trying to send a file to my server followingyour guide.
As you can see it is almost your code.
It seem to work properly, but my jsp detectas that it received a multipart request, but does´nt find any file in it.
My client side is:
File f1 = new File(“C:\\workspace\\WTP\\TestBlueViaSVN2\\WebContent\\WEB-INF\\lib\\httpmime-4.0.1.jar”);
HttpPost httppost = new HttpPost(“myserver/myapp/jsp/endpoints/uploadDummyListener.jsp”);
HttpClient client = new DefaultHttpClient(); client.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1);
MultipartEntity entity = new MultipartEntity( HttpMultipartMode.BROWSER_COMPATIBLE );
// For File parameters
entity.addPart( “fichero”, new FileBody((f1 ), “application/zip” ));
httppost.setEntity( entity );
// Here we go!
String response = EntityUtils.toString( client.execute( httppost ).getEntity(), “UTF-8” );
client.getConnectionManager().shutdown();
System.out.println(response);
At this pint it connects, and y canread the response (it is a dunny after all 🙂 )
My server side (JSP) is
if ( ServletFileUpload.isMultipartContent( request ))
{
System.out.println(“detectamos que es multipart”);
sesion.getServletContext().getRealPath(“./img_upload/”);
String rutaInicial = session.getServletContext().getServletContextName();
rutaInicial = session.getServletContext().getRealPath(“/img_upload/”);
System.out.println(rutaInicial);
FileItemFactory factory = new DiskFileItemFactory();
DiskFileItemFactory diskFileItemFactory = new DiskFileItemFactory();
// Set factory constraints
diskFileItemFactory.setSizeThreshold(1994096);
diskFileItemFactory.setRepository(new File(rutaInicial));
// Create a new file upload handler
ServletFileUpload upload = new ServletFileUpload(diskFileItemFactory);
List fileItems = upload.parseRequest( request );
for ( FileItem item : fileItems )
{
String fieldName = item.getFieldName();
if ( item.isFormField()) { item.getString(); } // Form’s input field
else { item.getInputStream(); } // File uploaded
}
}
At this point in my jsp out put i can see that reqest is multipart, but it can´t find the file sent.
Cabeceras:
header: content-length = 25614
header: content-type = multipart/form-data; boundary=eP63Lo9GN86HN0rtUXdyzmnmOUV
oUuGAodfLAk2
header: host = localhost:8080
header: connection = Keep-Alive
header: user-agent = Apache-HttpClient/4.0.1 (java 1.5)
header: expect = 100-Continue
Parametros:
ahora lo intentamos con el fileupload.
detectamos que es multipart
C:\tomcat-6.0.18\webapps\TestBlueVia\img_upload
Have anyone of you an idea of what i´m doing wrong?
Thank you very much
I having some trouble, i think it upload but i can see it, one question, the html has 2 hidden value
is these equivalent ?
entity.addPart( “upload_ticket”, new StringBody(“385468-a88cd45bb4ae”));
entity.addPart( “response_url”, new StringBody(“http://www.mysite.com/response_url.php”));
Hi, how should the client parse the response if it is also multipart?
say text/html and text/plain?
Can the response be multipart? Never heard of that, really. Can you provide some example or some online service sending such a response? I would love to take a look.
likely non-standard. we’re told it’s somewhat a proprietary version of protocol using only multipart post for both request/response.
(by the way any problem doing this?)
so is there any easy trick to parse a multipart on the client side?
Not that I’m aware of. Need to see how it looks like first.
How would we extract the uploaded files above ? Is there a way to directly upload files to our hard drives using http client or is it only possible through httppost or put ?
Hello Anirudh, Extracting files is shown in the second code example where “item.getInputStream()” is reading file data as InputStream. Uploading files directly to the hard drive can only be done when a POST request is sent and some code reads it and writes to the disk. Or you can use alternative solution like Dropbox which synchronizes folders on different computers. With Dropbox all one needs to do is to cope a file to a local folder, watched by a Dropbox service. After a while this fill will appear on the second computer as well.
Hi, Evgeny
Thanks a ton for both our solutions. It has helped to a great extent.
*read it as your thats a typo
Thank you for the neat example!. Very useful.
Glad to hear it was helpful for you!
This is quite helpful and I was wondering how you would propose to use the multi-part entity to post to picasa – when i post a single fileentity with just the file and no meta data it works great and the picture uploads properly. When i switch to multipart i am receiving 403 statuses.
From Google API docs : http://code.google.com/apis/picasaweb/docs/2.0/developers_guide_protocol.html#Auth
And use the following format for the body of the POST:
—–
Content-Type: multipart/related; boundary=”END_OF_PART”
Content-Length: 423478347
MIME-version: 1.0
Media multipart posting
–END_OF_PART
Content-Type: application/atom+xml
plz-to-love-realcat.jpg
Real cat wants attention too.
–END_OF_PART
Content-Type: image/jpeg
…binary image data…
–END_OF_PART–
If you were able to help me move that into Java (as i know CF and am interfacing the java functions) that would accomplish this, it would be very helpful to me and to a CF project on riaforge that i am adding this functionality into.
Thanks for any help you may have to offer,
Matt
Hello :
Thanks for your example. But I would like to ask if need upload 2 files at same time…
What I’m doing is:
entity.addPart(“uploadedfile”, new FileBody((file1), “application/zip”));
entity.addPart(“uploadedfile2”, new FileBody((file2), “application/zip”));
The “uploadedfile” and “uploadedfile2″ are the name of HTML’s file tag.
And I did this for getting response page body:
BufferedReader bs=null;
bs=new BufferedReader(new InputStreamReader(client.execute(post).getEntity().getContent()));
String s=””;
while(s!=null)
{
responseBody.append(s);
s=bs.readLine();
}
But only response I can get is a “0”.
Am I doing wrong ?
Self reply:
Finally I did confirmed that was server side issue, the codes above I posted was workable, thanks for your samples Evgeny … 🙂
Hi, Glad it worked out for you! Sorry for not replying earlier, this blog is an abandoned one for me after I moved to my own hosting.
I’m having an issue with authentication when using this. The httpclient is not sending the Authorization header when a FileBody or InputStreamBody are added. It will send it when there is only a StringBody though. If I use addHeader there is an extra CRLF that gets inserted. Any advice?
How about trying out other clients, like Groovy HttpBuilder?
I got it working for what I needed using a very basic implementation with URLConnection but then switched to Socket so I could get upload progress. Unfortunate bug though.
Hello! I come from Chinese readers. What: entity.addPart (paramName, new FileBody (((File) paramValue), “application / zip”)); one of how to deal with when File is empty
Hello! Upload all parameters, how to set the specified separator
Hi, first is so helpfull this post, thanks to share,
I’m using your code but i have a problem at this line
ServletFileUpload.isMultipartContent(request)
the form that i´m using I draw it dynamically, so when I’m using IE isMultipartContent but when i use firefox always return false.
If there´s any suggestion about this I really appreciate.
Hi, I would try to see in HTTP sniffer how headers sent differ in both cases. Hope that helps.
hai evgeny i designed one jsp page for how to upload a file by using streams concept. i want to upload my file by using servlet coding and also store it in a mysql page. could u please send me example regarding that
I cherished up to you will obtain carried out right here. The comic strip is attractive, your authored subject matter stylish. nonetheless, you command get got an impatience over that you want be delivering the following. ill without a doubt come more earlier once more as precisely the similar nearly a lot incessantly inside of case you protect this hike.
Awesome tutorial, saved me a LOT of time,
Thanks a lot
This is what all code tutorials should look like. Concise, to-the-point and with immediately applicable examples. Thank you!
good one made my day
First of all I would like to say thanks for this good tutorial.
I’m trying to use this solution with jboss 4.0.3, and I have commons-fileupload-1.2.1.jar but seam that doesn’t work.
When I post the form with the chosen file (via debug) there are no items and no errors.
Does someone have any idea of what could be?
Thank you in advance.
lindi
errata corrige: jboss 4.3. I’m afraid that there are some problems using commons-fileupload-1.2.1.jar with this version of jboss.
Files are not getting uploaded on the server / url.
When I am uploading some files to httpd server using the above code , I am getting status code as 301 Moved permanently.
Heya this is kind of of off topic but I was wondering
if blogs use WYSIWYG editors or if you have to manually code with HTML.
I’m starting a blog soon but have no coding knowledge so I wanted to get advice from someone with experience. Any help would be enormously appreciated!
Hi Evgeny,
I have gone through the code, it is very helpful. But, can I upload a document in along with the metadata (like “title”,”Description”etc) using httpclient post/put method?
A very helpful post!! Thank you!
you’re really a just right webmaster. The site loading pace is amazing. It kind of feels that you’re
doing any distinctive trick. Also, The contents are masterpiece.
you’ve done a excellent activity in this topic!
Solid write up. I study something like this here at Boise State
University. It’s definitively stimulating to know writing from other writers and observe a little bit from their source. I’d love to use some of this material on my own webpage (if you don’t mind). And of course, I’ll put up
a link to your site at wordpress.com on my own blog.
Kudos for posting.
can u send the code please..
I am getting the below exception:
com.sun.xml.ws.server.UnsupportedMediaException: Unsupported Content-Type: multipart/form-data; boundary=ZJop7mVAk6EuTefHEjQfFlAUTv7aov_vq9uCAN Supported ones are: [text/xml]
Any ides
All modern games should have some form of a social networking component as well.
You can do their hair, make up and give them a totally different look.
Older calculators may not display information about
the hardware version unless its HW2 or newer.
Fastidious response in return of this difficulty with solid arguments and explaining everything
regarding that.
Thanks, this post is very useful.
ハロー。うちは10年ほど前からダイエットを行ってきました。体重は落ちてもすぐに戻ったりで結局たいして体重が落ちてません。ですが、30歳を超えると昔よりも少ない食事量でも太りやすくなってきたと感じています。そのため、今後は本気でダイエットしようと決めたんです。しかもこれから暑くなってきます。ですからダイエットブログを見て調べたんです。どんなやり方のダイエット法があるのか、こんな方法のダイエット法がおすすめとかの知識が増えました。最近は酵素を使ったサプリのダイエットが注目されていますね。体の体調も良くしながらダイエットできるようです。また、ダイエットエステなどで人気の高い痩身ですね。これも美を大切にしながら痩せる方法ですね。その痩身エステにもいろんな機械を使って効果が期待できそうです。しかもほとんどのエステには体験コースというものがあるので、まずは体験コースから試してみるといいですよ。そして食べものも大事です。また、野菜野菜が苦手な方は野菜ジュースや青汁などでしっかりと栄養を摂ってダイエットしましょう。
Good web site you have here.. It’s hard to find
quality writing like yours these days. I truly appreciate people like you!
Take care!!
However, online gaming in addition has become extremely popular to bettors
around the globe.
An intriguing discussion is worth comment. I think that you ought
to publish more about this subject, it might not be a taboo matter but
typically folks don’t speak about such subjects. To the next!
All the best!!