Why No OutputStream?! -- Creating Image Files In Java Agents
While writing this entry I'll try not to turn it in to just another long dig at Domino, but it might end up going that way. Sorry.
I'm writing a CAPTCHA plugin for a Domino site at the moment. Although I've managed to get a working solution I can't help but feel it could have been a whole lot easier to achieve were there not a couple of things missing from the basic toolkit of a Domino developer.
Creating images on the fly using Java Agents is actually a lot easier than you might think. The hard part -- at least with Domino -- is getting the resulting image to the browser.
There are several Java libraries out there for creating CAPTCHAS but they all seemed huge to me and just a little OTT. Not only that but the two or three I looked at (including the most popular JCaptcha seem to rely on you having such a thing as a user "session" that ties separate page requests together. We don't in Domino and so can't use them unless we're willing to suffer writing and deploying servlets (something I'd leave to the very end until I went down that route).
After further searching I found the source to a JSP page where somebody had written a CAPTCHA generator in less than 100 lines of code. That's what I like to see. Lean and to the point.
Not that is was a simple case of copy/paste of course. The code was written using parts of Java, such as ImageIO, which were only introduced in Java 1.4. My target was a Domino 6 server, which only has Java 1.3 (Domino 7 has Java 1.4).
After more searching I found you could write JPEG files using an undocumented package called com.sun.image.codec.jpeg in Java 1.3. The trouble here though is that there's no way to write the JPG file you've produced directly to the browser.
The main way a Java Agent writes to the browser is using the PrintWriter class, but this won't let you work with files and their inner bytes. Just text! Mikkel Heisterberg has written about this limitation on his well-worth-reading blog. He found that there's an undocumented method in Domino 7's Java called "getAgentOutputStream()" which gives us an OutputStream object. This is exactly what we need. The trouble is it's buggy and unusable. Whether it works in Domino 8 or not I don't know?
The only real way (although I hope somebody is going to correct me on this) to get the image to the browser is to save it to the server's disk first!
Saving to the server's disk is easily done and isn't a big deal, as long as you clean up after yourself, but I'd prefer to avoid that in the first place. Not only because it means you have to make sure code is signed with IDs that are allowed to run unrestriced code and the Agent itself needs to have an increased level of access, but it also means you have to make sure the Agent never gets updated by a copy signed with another ID, which would break the code. It's really quite messy.
Why, why, why can't we create an OutputStream in a Domino Java Agent!?
Skip to the 
You can subscribe to an individual
I was in the same situation and moved the whole image output part to a servlet where you dont have such limitations. with a servlet it is possible to write binary data directly to the browser, without temporary files. I use it to create a dynamic PNG background image for the cisco 7970 voip phone, updated every n minutes with latest stock quotes, charts, inbox, callers and other information from notes databases..
You're a lucky man to have such easy access to the power of servlets. I'm guessing you "own" the server?
With most of my customers I deliver an NSF (or two) and any changes I make thereafter are refreshed in to that same NSF. Most times they give my Notes ID access to their system so I can do the refresh for them. It all works fine. If I were to throw in a servlet though it gets a whole lot messier. It involves me asking customer's admin people to add to the OS and restart http etc. Any changes I make to it involve more of their time (including waiting for them to get round to it etc).
I just can't consider using servlets. If I were a developer *at* a company with direct access to the servers maybe I'd think differently...
You can encode an image to a stream of characters (via PrintWriter) and decode it in JavaScript, take a look at:
{Link}
{Link}
No outputstream is a real PITA.
By the way, com.sun.* packages should not be used as they are not supported and might disappear in other vendors JVM or in next versions of sun's one :
{Link}
I'm even surprised that theses packages are available in an IBM JVM.
Agreed it would be better to be able to return a stream directly rather than futzing with PrintWriter.
I'm guessing this won't completely do the trick for you, but in one application I've generated a file attachment without using the server's disk by creating a document, using a MIME stream to create an attachment, then redirecting to that document, which can get cleaned up after the fact.
This isn't my code, but shows some basics for generating an attachment as an element of a document.
{Link}
Hi Jake,
Cutting out the disk IO would be great.
In the code you linked to the example uses a file on the hard drive though.
The setContentFromBytes() method in Notes takes a "Stream" object in. Is that a native Notes object? Would it accept an OutputStream?
You may want to use an external provider for the images like I did in my tutorial:
{Link}
Hi Jake,
Devin Olson wrote two articles for Lotus Advisor in February and March of 2006.
"Add Advanced Human User Verification to Your IBM Lotus Domino Web Pages"
{Link}
and
"Fine-Tune Web User Access to Your IBM Lotus Domino Web Pages"
{Link}
There was supposed to be a third article in the series, but I couldn't find it just now.
You will need a subscription to access the articles and get the downloads, but it would obviously be worth it for the time it may save you.
I'd seen your post on the topic Christian and was aware of recaptha.net's offerings, but I decided to avoid building a solution based on a third-party service. I'm just not that comfortable with the idea of selling a solution that could break/vanish at any time and isn't under my control. I don't like the iframe that loads the page with their company name/link in etc.
Between me and the customer we decided to avoid the overly-annoying CAPTCHAs and go for a better balance of user-friendliness over secureness. The recaptcha.net ones seem one of the hardest to fathom to me.
Jake --
The code I linked to uses a disk file as an input stream -- I'm pretty sure any byte stream could be used and its content written as an attachment (perhaps by getting the bytes and passing them into a Notes "Stream" object).
My application code (can't post it, sorry) uses a Java PDF library to stamp an value on every page of a file, then return that result as a downloadable attachment.
My code first gets the source file as an input stream, then hands that to the PDF library, which generates an output stream containing the stamped PDF.
I then pass the output stream to the SetContentFromBytes method of NotesMimeEntity (like in the linked article).
I then close the MIMEEntities, and save the new document.
Finally, I create a URL like .../database.nsf/0/big_unid_here/$File/myattachmentname.pdf?OpenElement and redirect the browser to that URL using PrintWriter("["+url+"]")
Again, the approach has its weaknesses, but it does avoid disk I/O (and security issues on a server I don't "own").
Jake
Thanks Jake. Sounds like just what I'm after (avoiding disk writes). I tried it earlier and managed to save as MIME to the document but couldn't get to the file via the /$file/ path. I guess I need to name the MIME entity. How do I do that?
Nice name by the way! Are you a real Jake or a Jacob? If real, how old? I don't meet many of my age (33) although something tells me there will be a lot more in the generation to follow. At least in the UK.
Jake,
You are right about not having access to use servlet on domino server. I was in same situation few months ago so I used outputstream class from Julian Robichaux's site
{Link}
He has compiled very good example of outputstream class with richtext fields..take a look
Regards!
GS
Something that was on my todo list as well..
I was looking at using ReCaptcha a hosted captcha system and integrating it into domino..
{Link}
Nice office by the way, cheers, Alastair
Hello,
I am interested in what Richie is doing with 7940 Cisco phones and Notes Db's.
Would you be intrested in sharing your code / db, or pointing me you where I can fine some examples etc...
Hi Jake, I've found that:
{Link}
Maybe it helps...
Greetings from Geneva!
Alex
Hi Jake - why use an image capture? I had to write something a couple of years ago in asp classic to stop comment spam on a blog.
I generated a random 10 digit code and then request randomly 3 of characters of the code to be submitted with the form. You can see it here: {Link}
Hi Stuart. That's an interesting approach and would make more sense/be a lot easier. However the customer asked for this kind of CAPTCHA and they're always right aren't they ;o) It would be interesting to see how you could implement what you did in ASP but in Domino. I guess you'd have to also create a temporary hidden document which stored the code and the list of characters from it which need to be entered. This hidden document is always needed as a kind of "session" (see my post today for more on that).
Jake -
I looked back in the code and was reminded that $file doesn't work. Here's what does (it was right next to a commented out line with $file...) :
url = urlThisDatabase() + "/0/" + doc.getUniversalID() + "/Body/M1/" + filename;
And here's the line that encodes the content...
body.setContentFromBytes(stream, "application/pdf", MIMEEntity.ENC_IDENTITY_BINARY);
About the name, you're right -- Jake is becoming more and more popular. That said, I'm actually James -- no one in my family can explain to me how the nickname makes any sense, but it stuck...
Thanks Jake for writing a wonderful article on image creation in Java agent. Though i am a beginner in java with little knowledge. But i was able to create Image and store it on Disk.
Hi,
I found this interesting discussion, when I tried to help someone on our german notes forum to integrate the JCaptcha library.
I've come up with a completly different proposal, which consists intackling the Jcaptcha-1.0.jar library with very dirty Java.
In my own words I found out that it consists of 3 projects:
project i: generates the captcha-image, the solution of the captcha challenge and an id controlled by the user of the library.
project ii: stores captcha-image, the solution of the captcha challenge and the id in kind of a temporary storage and manages the access to this "database"
project iii: integration for all sorts of java frameworks like servlets/jsp, spring, etc.
Those parts are closely coupled. This coupling is intended by the authors of the library.
From a Domino standpoint you only need project i. You just need the captcha and the solution of the challenge. You may store and manage the access to that data in Domino. Whereas project ii) and iii) complicate the usage of jcaptcha in Domino. Unfortunatedly the authors of JCaptcha restrict the access to the solution of the captcha with Java access modifiers.
Now, you may
a) patch JCaptcha-1.0.jar by rewriting 2 of the access modifiers deep inside the projects code and recompile/repackage.
b) you may hack your way to the information by using Java reflection.
The code below demonstrates the b) path. If someone stumbles into this thread, I hope that trying out the code gives some idea about what the the heck I am talking about.
There may be other cases, when usage of one of the many Java open source libs may be easier by just analysing and if necesary even changing the open open source library itself.
In a real world scenario I would patch the open source library and not go the reflection way. Anyway: The code gives the idea, which 2 access modifiers in jcaptcha-1.0-all.jar you have to change to break the coupling of project i with ii, iii and use project i in isolation.
Anybody trying this out should put jcaptcha-1.0-all.jar, commons-collections-3.2.1.jar and commons-logging-1.1.1.jar into the class path (for example in an eclipse project).
I posted this to add another aspect to an interesting and informed discussion and because I kind of like my solution.
public class NewTest {
private static String BASE_VERZEICHNIS_FUER_BILDER = "/Users/axeljanssen/";
public static void main(String args[]) {
String id = "1";
DefaultManageableImageCaptchaService instance = new DefaultManageableImageCaptchaService();
ByteArrayOutputStream imgOutputStream = new ByteArrayOutputStream();
BufferedImage im = instance.getImageChallengeForID(id, Locale.GERMAN);
BufferedOutputStream fous = null;
try {
Class clazz = instance.getClass().getSuperclass().getSuperclass()
.getSuperclass();
System.out.println(instance.getClass().getSuperclass()
.getSuperclass().getSuperclass().getName());
// for (Field field : clazz.getDeclaredFields()) {
// System.out.println("field:" + field.getName() + ":" +
// field.getClass().getName());
// }
Field firstDoor = clazz.getDeclaredField("store");
firstDoor.setAccessible(true); // ha ha
FastHashMapCaptchaStore firstDoorOpen = (FastHashMapCaptchaStore) firstDoor
.get(instance);
Gimpy gimpy = (Gimpy) firstDoorOpen.getCaptcha(id);
Class <Gimpy>clazz2 = (Class<Gimpy>) gimpy.getClass();
Field secondDoor = clazz2.getDeclaredField("response");
secondDoor.setAccessible(true);
String ratsAreDancingInThePantry = (String) secondDoor
.get(gimpy);
System.out.println("DER CAPTCHA TEXT IST:" + ratsAreDancingInThePantry);
} catch (Exception e1) {
e1.printStackTrace();
System.out.println("ECHTER ALARM");
}
try {
ImageIO.write(im, "png", imgOutputStream);
byte[] captchaBytes = imgOutputStream.toByteArray();
fous = new BufferedOutputStream(new FileOutputStream(
BASE_VERZEICHNIS_FUER_BILDER + id + ".png"));
System.out.println("Bild wurde gespeichert in:" + BASE_VERZEICHNIS_FUER_BILDER + id + ".png");
fous.write(captchaBytes);
fous.flush();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally {
if (fous != null) {
try {
fous.close();
} catch (IOException e) {
// Exception verschlucken i.O. in tcftc Konstrukt.
}
}
}
}
}