Forcing attachments to always download - Article

Forcing attachments to always download 19 May 2002

Knowing how a certain browser on a certain operating-system will respond to a click on a link to a file of a certain type is something of a guessing-game. We know and expect that clicking a link to an HTML will open that file in the browser. Anything else and it depends on what software the machine has installed and how it's configured. To some extent it also depends how the web server itself is configured.

Take an Excel file for example. When you click a link that ends in .xls and you have MS Office installed you may expect this to open in-line in the browser. Whether or not this happens depends on whether the browser is setup to allow it to do so. Not as simple as you may have thought. In order to fully understand the whole "saga" it's well worth reading this attempt to demystify it all.

These uncertainties in the behaviour of links can make designing applications where you want to have some files available for download very awkward. More than once I've had to include, alongside my "download" links, text along the lines of:

To download this file, right-click the link and, in Internet Explorer choose 'Save target as' in Netscape choose....."

The problem:

You're getting the picture right? Wouldn't it be nice if there were a way we could guarantee that every time a user clicked a link that said download, that that was what actually happened.

You could of course do this by fudging all files that are available for download in to Zip files. This way you can be pretty sure of the result. Not perfect though - how will you get all the files uploaded in to Zip archives for a start?

What we need is a way to tell the browser that they should always prompt the user to download the file. This is, in essence, not as hard as it may sound.

The solution:

It's not often that reading the Notes.net forum leaves me intrigued. However, when I read this post it got me thinking - can this be done? Sure enough, a quick search on Google led me to an answer and I was happy to be able to respond - even if they didn't keep their promise.

This wasn't enough though - I was curious. Surely something like this could be really useful. My initial response to the post on the forum was enough to prove a point but it wasn't very practical. It only works for text-based content that you can stream back to the browser using the agent's PrintWriter object i.e. .txt, .csv, .xml etc. What about attachments of a binary content like .exe and .gif?!

Having written the simple Java agent that could stream text back to the browser in the guise of a file of any name I was pretty much reaching as far as my Java knowledge at the time would let me.

At the same time as I was playing with these ideas I got a mail from Jon LeDrew of Newfoundland, Canada, that included some servlets that he wanted to share with us all. Not wanting to miss the oppurtunity of a knowledgeable Java brain, willing to share, I asked him his thoughts on the download code and this is about the point where he took over. I would love to say I had written all the code for this servlet myself but that would be a lie. Luckily Jon's boss decided this would be a "nice to have" for their intranet and so he could then justify the time he spent on the code. Which was nice.

While Jon sent me regular updates of the code I also passed this on the Brendon Upson who knows a hell of a lot when it comes to Java. This he will prove when he gets round to releasing his amazing Puakma web server. With his suggestions "we" could then make major improvements to the performance of the servlet and get it to the point where it was practical to use it in a live environment.

Using it:

Whether or not you carry on to the next section and see what code's involved is down to you. What you'll need to do though is see it in action. Download the files I've attached to this document. Place the database on a Notes server and put the .class servlet file in the server's Servlet directory. Open the database in the browser at its root and follow the instructions on that page.

You will notice that when you create a document and re-open it after having uploaded a file there are a pair of lists of the attachments. One set of links opens the files "normally" via the normal Domino URL access to the attachment while the other references the attached file via the servlet. To use the servlet to get to a file the syntax is similar to the following:


http://hostname/servlet/GetNotesAttachments?dbpath=dir/db.nsf
&unid=637D5FAF2F8B993580256BA9005ED9FB&filename=myphoto.jpg

Notice that the three parameters that we pass in are enough to mean that you can use the servlet to download a file from any database. That being one of the pros of using servlets. If this were an agent then in which database would it belong?

Having tested the servlet with various file types and sizes up to about 20MB in size I can see hardly any issue with performance. Download times seem to be the same as, if not quicker than, the usual method. How this translates to an environment with multiple users and downloads remains to be seen.

The code:

The key to this whole solution lies in our ability to specify the type of the content. We do this with the setContentType method of the servlet's response object.



response.setContentType("application/download");

Another essential line of code is the one that adds the field "Content-Disposition" to the response's header. With this we let the browser know it's receiving an attachment and what the default name should be for the user to save this file as. We do that with the following code:



String filename = request.getParameter("filename");

response.setHeader("Content-Disposition","attachment;filename=\"" + filename + "\"");

We then need to stream the whole file to the browser. To do this we use a BufferedInputStream object. In to this we place the contents of the file. The servlet contains a method called "getAttachment()". In to this we pass all three of the servlet's parameters in a call like below:



BufferedInputStream bis = getAttachment(unid, filename, dbpath);

Assuming that the above method is successful in getting the attachment out of the Notes document we can carry on now, using the BufferedInputStream object that's returned. The first thing we need to do is find out how big this object is. We then create a byte array that holds every byte of the attached file.



int bytesA = bis.available();

response.setContentLength(bytesA);

byte[] attachment = new byte[bytesA];

Notice in the above code that we also set the Content-Length field of the header. This is how, when you're downloading a file, the browser knows you've downloaded "X of 10.3 MB" and have X minutes remaining.

Download times..

The next important code fragment is the one that does all the work. After getting a handle on an OutputStream for the servlet we pass every byte of the attachment in to it and subsequently to the browser. Here's how:

ServletOutputStream op = response.getOutputStream();



while (true) {

 int bytesRead = bis.read(attachment, 0, attachment.length);

 if (bytesRead < 0)

  break;

 op.write(attachment, 0, bytesRead);

}

Obviously there's a lot more to it than the above snippets. If you want to know more about what's going on and see how you can tailor it to you own needs, there's a copy of the source code attached to this document and stored in the sample database. Enjoy.

Afterthoughts:

As it stands the servlet has a major problem. If a user cancels the download before it has finished then the servlet doesn't "tidy up" properly and this leads to the JVM running out of memory and the HTTP server becoming unresponsive. Obviously this isn't acceptable. As soon as I get it sorted I'll publish it again as v1.0.1 or something. Unless one of you guys can find the answer before that.

What I've also failed to mention in all of this is that there is an inherent security problem involved. Using the above URL format it doesn't take a genius to work out that they can pass in the location of any file attachment and the servlet will happily fetch it for them. What's needed is the servlet to use the getRemoteUser() method and to make sure the user isn't trying to get at files they wouldn't normally have permissions to.

Note that Internet Explorer 5.5 SP1 has problems with Content-Disposition: Attachment. Apart from that I think this is pretty much a working solution for all browsers on all platforms. Not very often you hear me saying something like that is it ;o)

Feedback

- Jesper Kiaer
- Sun 19 May 2002
Security
Hi
Just a remark..
Using getRemoteUser is not the right way to make a servlet secure "in a Domino way".
The servlet is running with the security rights of the server and not the security rights of the user.
The right way would be to:
- Use Corba to access the databases from the servlet - use SSO on the server - Use the LtpaToken to let the user log on in the servlet
In that way the servlet is running with the security rights of the user and not the server.
regards Jesper Kiaer http://www.activator.dk

Reply
1. - Jake Howlett
  - Mon 20 May 2002
  Re: Security
  Hi Jesper,
  Thanks for the feedback.
  What I was thinking of doing was the servlet's own checking on access rights depending on the user name. i.e. the servlet can do some ACL checks and document-access checks based on the username.
  Please correct me if I'm wrong, but that's the way I see it *could* work...
  Jake -codestore
  
  Reply
  
  Show the rest of this thread
  1. - Dave Meehan
    - Mon 20 May 2002
    Re: Security
    I'm not sure of the detail, but there is going to be a difference in testing the user rights against the ACL, and assigning the servlet the permissions of the user according to the ACL. Whilst the servlet is running with server rights, there is the possibility of circumventing any security tests that you code. I think we could assume that whatever Domino does internally to verify user permissions would be more robust than what you could code, and hey, why re-invent the wheel.
    Although you've chosen to implement this as a servlet, you could also implement it as an agent, for which you could set the "Run agent as web user" property, and Domino will take care of the authentication for you. Given that your example was for file downloads, an agent should cope with the traffic, unless your talking huge numbers of concurrent users.
    
    Reply
    1. - Jake Howlett
      - Mon 20 May 2002
      Agents not good
      Originally I did use agents. That was until it was pointed out to me that the agent's "PrintWriter" object has shortcomings when it comes to passing byte arrays through it. Or something like that. As I said in the article, I didn't write this code and it reaching the extent of Java "capabilities"
      Jake -codestore
      
      Reply
  2. - Jesper Kiaer
    - Mon 20 May 2002
    Re: Security
    Hi
    I still think that's the wrong way to go. You are trying to do what Domino does best: Handling the security.
    You can handle the security yourself when it's simple, but in real life it is offen not.
    In real life usernames are not used in ACLs, READERS fields ... groups are ...and worse ..offen nested groups :-(
    Controling this yourself in yourcode is a mess and you should not.
    Using my previous advice makes your servlet security wise run like an agent with "Run Agent as a Web user" checked...BUT with a performance boost! :-)
    regards Jesper Kiaer http://www.activator.dk
    
    Reply
    1. - Krasna Halopti
      - Mon 20 May 2002
      Re: Security
      YMMV, but I've found CORBA to be (a) slow, and (b) unreliable. DIIOP seems to crash every now and again on my Windows 2000, 5.0.8 system. It's a nice idea but a poor implementation.
      The other way of ensuring that you run with a user's credentials is that you open a Domino session not with the server's ID, but with the user's. I have done this successfully, using Tomcat. (I would use Domino's servlet manager only if a customer forced me to.) I create not only a server session, but a session for each user who logs in. I use my own Java-driven, form-based authentication to do this, interfacing with Domino for the actual authentication only (no reliance on domcfg.nsf form-based authentication).
      I find Lotus' default mechanisms for making this kind of thing possible sub-optimal - a bit like Domino's HTML rendering. But with some thought and the right amount of skepticism about Lotus' received wisdom about how to do things, some quite nice things *are* possible!
      
      Reply
      1. ZuuL
        
        Wed 3 Jul 2002
        
        Re: Security
        I think your all going about it the wrong way.
        The best way to stop people from exploiting that script (especially if it's just for attachments you want people to download) is just create a directory specifically for the attachments, and have it coded in your script (ie, unchangable in the variables)
        That way you they can only download files from that directory, which presumably is full of files for them to download anyway.
        Doesn't require any access checking or anything, just lateral thinking.
        -ZuuL
        
        Reply
      2. Hari Prakash
        
        Thu 29 Jun 2006
        
        Re: open dynamically created pdf file with open\sa
        Hi Can any body help me ? I am trying to open a dynamically created pdf file in a browser using servlet. I want the browser either to ask open/save option before to open the file OR it should display the complete pdf screen so that i can use the save icon to save it . Thanks
        
        Reply
- Dave Meehan
- Mon 20 May 2002
Re: error checking
I've not tried this out, I'm not setup for servlet's at the moment, but your read/write loop probably needs some error checking. Looking at the JDK docs, you might need to be using checkError() rather than relying on an exception to be thrown when there is an IO error.

Reply
1. - Krasna Halopti
  - Mon 20 May 2002
  Re: error checking
  It looks as if checkError() is available only for PrintStreams.
  It may be that different servlet engines handle underlying IOExceptions differently - which servlet engines have you tested with?
  I'm not sure why you would get a memory leak, unless the thread is stuck in the loop. I notice that you've got a bytesRead < 0 condition - why? It should be <= 0 in my view. If this were the case, your loop should exit eventually anyway, unless it blocks in op.write() or bis.read().
  
  Reply
  
  Show the rest of this thread
  1. - Jon LeDrew
    - Mon 20 May 2002
    Re: error checking
    Just curious as to why the condition should be bytesRead <= 0 instead of bytesRead < 0.
    I know the read method returns the number of bytes read or -1 if it reaches to end of the file or stream.
    I'm not questioning your idea just wondered why it makes a difference. Please enlighten.
    
    Reply
    1. - Krasna Halopti
      - Mon 20 May 2002
      Re: error checking
      I was thinking that the only reason for memory leakage would be a failure to exit the loop, and was thinking of what conditions might cause this.
      It may not be the case in practice, but I thought the read method might return zero bytes under some circumstances, and was thinking that perhaps you should exit the loop in this circumstance.
      Of course the spec for InputStream says -1 on EOF and IOError for any other error; but I'm not sure if the servlet engine would rigorously enforce this. It's possible that a socket-level IOException is caught in the ServletxxxStreams but not propagated upwards.
      The most natural way to handle an error would of course be an IOException, but if this is not being thrown then I suppose a bit of experimentation is called for!
      
      Reply
- Brian Miller
- Tue 28 May 2002
The content-type thing
Why not try using a content-type of "application/octet-stream"? That's the default for any file that isn't recognized by the browser, and it seems to start as a download the way we want it to. And, I suspect that it might get around the "content-disposition" issue.

Reply
1. - Jake Howlett
  - Tue 28 May 2002
  Good idea! Re: The content-type thing
  Hi Brian,
  Good idea. Thanks for that. Got to admit I skipped over the octet-stream part, not really appreciating what it could do.
  Anything that makes it IE proof has to be good...
  Jake
  
  Reply
2. - Lindon Flood
  - Thu 30 May 2002
  Related possibilty
  I read that it was possible to change parameters in the server's httpd.cnf file to force the browser to download by specifying octet-stream for a particular file extension.
  eg AddType .csv application/octet-stream binary 1.0 # Comma-sep value
  However, when I tried it there was no difference. Perhaps someone else will have more luck.
  
  Reply
3. - Jon LeDrew
  - Fri 31 May 2002
  Re: The content-type thing
  Anyone else heard that the content-disposition header is going to be phased out of IE without a replacement?
  Am I being incredibly thick?
  Probably!
  Jon
  
  Reply
4. - lfink
  - Tue 21 Dec 2004
  Re: The content-type thing
  When using JSPs redirecting the response once seems to help with IE. IE, Firefox download smoothly.
  jsp 1: ---- <%@ page language="java" contentType="application/zip"%> <% response.sendRedirect("download.jsp"); %> ----
  download 2: ---- <%@ page language="java" contentType="application/zip"%> <% response.setHeader("Content-Disposition","attachement; filename=test.zip"); try { // the response outputstream thing }(IOException e) { } %> ----
  
  Reply
- Giles Hinton
- Tue 4 Jun 2002
How about this - easier?
Maybe I've missed something here, but I thought a much simpler approach to download the file would be to fool the browser into thinking that what you are downloading will need to have the "Save as" box open. So why not just append this string to the end of the URL for the file in order to get it to do this?
http://Server/Path+Database/View/Document/$FILE/filename?OpenElement&FieldElemFo rmat=.exe

Reply
1. - Jake Howlett
  - Tue 4 Jun 2002
  Re: How about this - easier?
  Well it sounds good in theory but I can't get it to work.
  Have you got this to work in your own experience?
  Jake -codestore
  
  Reply
2. - Frank
  - Fri 28 Jun 2002
  Re: How about forcing a filepath/name?
  Is it possible to force a download path/filename?
  I saw a suggestion to use something like:
  response.setHeader( "Content-Disposition", "attachment; filename=\"" + localPath + filename + "\";");
  but it doesn't seem to work.... it gives to the fiel the name of the SERVLET, instead.
  If I take away the "filepath" it works in the LAST USED DIRECTORY!
  Any idea?
  
  Reply
  
  Show the rest of this thread
  1. - Mark Clifton
    - Tue 25 Feb 2003
    Re: How about forcing a filepath/name?
    I made a servlet a couple years ago that would force a download of a specific type and a specific filename... the solution that would work in all browsers was to set the url to the filename you want to download. The servlet would be an inner 'directory' to the url:
    http://host/servletname/downloadfiledirective/filename.ext?fileid=1245
    The browser would look at just the uri part (/servletname/downloadfiledirective/filename.ext) to get the filename, and some browsers would even ignore the specified content type in the response. If the filename/type is not known before the request (you just have a reference to a file), you could have a middle-man servlet to see what it is and redirect to a url including the filename.
    
    Reply
- N Wharton
- Wed 14 Aug 2002
On Stopped Java Agents...
Jake -- you mention a serious problem I've continued to have with my Domino Java agents:
"...the servlet has a major problem. If a user cancels the download before it has finished then the servlet doesn't "tidy up" properly and this leads to the JVM running out of memory and the HTTP server becoming unresponsive.
I see this when I use a java agent in Domino 5.0.8 and directly print to the browser using printwriter.println('') - like commands, and a user hits "stop" before the agent has completed running.
Have there been any good answers for that one? Have you seen any good articles on what to do about this, in general? Regards, N Wharton

Reply
1. - Jake
  - Thu 15 Aug 2002
  Re: On Stopped Java Agents...
  Unfortunately not. This is one of those things we are just going to have to accept I think.
  Jake codestore.net
  
  Reply
- Lokesh
- Fri 16 Aug 2002
Java Exceptions
Hi Jake,
Thanks for the nice utility and it helps a lot to us.
I am having one problem with the servlet. If i create a document with attachment in web and I tried to open that attachment after the save. I get page cannot be displayed and following error message in log.nsf
----------------------- 08/16/2002 03:27:00 PM Addin: Agent error message: lotus.domino.NotesException 08/16/2002 03:27:00 PM Addin: Agent error message: at lotus.domino.local.Document.getAttachment(Document.java:781) 08/16/2002 03:27:00 PM Addin: Agent error message: at GetNotesAttachment.getAttachment(GetNotesAttachment.java:127) 08/16/2002 03:27:00 PM Addin: Agent error message: at GetNotesAttachment.doGet(GetNotesAttachment.java:45) 08/16/2002 03:27:00 PM Addin: Agent error message: at javax.servlet.http.HttpServlet.service(HttpServlet.java:499) 08/16/2002 03:27:00 PM Addin: Agent error message: at javax.servlet.http.HttpServlet.service(HttpServlet.java:588) 08/16/2002 03:27:00 PM Addin: Agent error message: at lotus.domino.servlet.DominoServletInvoker.executeServlet(DominoServletInvoker.ja va:266) 08/16/2002 03:27:00 PM Addin: Agent error message: at lotus.domino.servlet.DominoServletInvoker.service(DominoServletInvoker.java:212) 08/16/2002 03:27:00 PM Addin: Agent error message: at lotus.domino.servlet.ServletManager.service(ServletManager.java:235) 08/16/2002 03:47:01 PM Addin: Agent error message: Malformed URL: null: java.lang.NullPointerException 08/16/2002 03:48:01 PM Addin: Agent error message: Malformed URL: null: java.lang.NullPointerException: 08/16/2002 03:49:18 PM Addin: Agent error message: Malformed URL: null: java.lang.NullPointerException: --------------------
I am not familiar with Java. Please let me know if you have any idea why it is happening.
I can open the attachments which are created in Lotus Notes using this servlets in browser immediatley after I saved the document.
Thank You Lokesh

Reply
1. - Lokesh
  - Wed 21 Aug 2002
  Re: Java Exceptions - Resolved
  Our Server didn't have proper rights to the document. I granted rights to that server and now it seems everything is working fine.
  Thank You
  
  Reply
  
  Show the rest of this thread
  1. - tarun
    - Mon 20 Jun 2005
    Re: Java Exceptions - Resolved
    problem while open a file. It is not working. File saved properly but not opened. Thanks
    
    Reply
- Julian Lishev
- Mon 9 Sep 2002
Generalization CGI script in Perl
I could give an example that's working in my web server (Apache) with NS4, NS6x, Mozilla, IE4, IE5, (IE5.5 not tested!) IE6 browsers. It's written in Perl.
$type = 'application/download'; #$type = 'application/octet-stream';
$name = 'some.txt'; #$name = 'some.wav'; #$name = 'some.tar.gz';
print "MIME-Type: 1.0\n"; print "X-Powered-By: WebTools/1.28\n"; print "Content-Disposition: attachment;filename=\"$name\"\n"; print "Content-Transfer-Encoding: binary\n"; print "Content-Type: ".$type.";name=\"$name\"\n\n";
It's seems to run smoothly. Regards, Julian Lishev www.proscriptum.com

Reply
- Sami
- Sun 27 Oct 2002
Resume Support
How can I make the servlet able to support resume for downloading files?
any help would be appreciated

Reply
- Tiho
- Fri 18 Jul 2003
Problems with IE 6.xx and NS 7.xx
setting "attachment;" in response.setHeader("Content-Disposition","attachment;filename=\"" + filename + "\""); leeds some problems with IE, after the file download "Access denied" is recived if one tries to access document property of the window object in JS. Instead use response.setHeader("Content-Disposition","filename=\"" + filename + "\""); NS don't work in neither case. It tries to download file with added .jsp extension (in my case) and after that did not download anything. Maybe it needs some additional header(s)?

Reply
- Ruben Cepeda
- Fri 6 Aug 2004
Thank you,
This article was helpful beyond belive. Thank you.

Reply
- belajiang
- Fri 1 Apr 2005
why i cant't download document at chinese
my name of document use chinese,i dont't open the document

Reply
- Christian Sowada
- Wed 14 Feb 2007
OutOfMemory Exception Tip
Large Attachments can cause an OutOfMemory Exception. In this case you must choose a smaller byte buffer.
Example: byte[] attachment = new byte[20000];

Reply
1. - Dietrich Willing
  - Tue 24 Apr 2007
  Re: OutOfMemory Exception Tip
  How big is big?
  
  Reply
- Vera
- Mon 7 Jan 2008
Where is the getAttachment()?
Why I can not find the method in the HttpServlet class or Servlet class?

Reply
- Gustavo Pagotto
- Mon 10 Mar 2008
Problems with "C:\" Drive space
Hi -
Just installed your solution a few weeks ago (which works very well) and I noticed that my server hard drive started running out of space. It seems that files that users downloaded are being saved on server hard drive - like a cache - at the time user does a download and it isn't being deleted after a successful download. Is it an expected behavior of this servlet or is there a way to avoid this behavior?
Thanks in advance.
Gustavo.

Reply
- satourne
- Thu 5 Feb 2009
Are you Flex ible ?
I have written a Flash that force a directdownload with FileReference and .download()
It seems someone writes article about flex in that blog. So you know there's at least two solutions ...
Geetings ^^

Reply