Domino, XML-RPC and Java - A Primer - Article

Domino, XML-RPC and Java - A Primer 5 June 2003

Yesterday, for reasons I won't go in to just yet, I started looking in to using a Domino agent as an XML-RPC server. What I found was that there was a complete lack of decent documentation. Sure there were quite a few articles but they all seemed to skip over the essential example code that I required.

This article is to try and fill that gap and hopefully the next time somebody is in my shoes they will find this to be just what they were looking for.

A Little Background:

Wondering what XML-RPC is? Read on. Even if you think it has no bearing on you whatsoever, you may well be pleasantly surprised what it has to offer!

From the XML-RPC homepage:

[a] set of implementations that allow software running on disparate operating systems, running in different environments to make procedure calls over the Internet.

That's all the technical mumbo-jumbo I will bother you with. The trouble with specs like this is that they go in to great detail about the ins and outs of the procedures when a lot of us don't really care. If you're like me you just want to know what it can do and how it does it.

In short RPC is a way of using HTTP to send a call to a function on any system that will reply with the result of that method. The beauty of RPC is that, because it uses HTTP as the transport, it operates through port 80 (by default). This means that it can be used in almost all environments. Including those that use paranoid firewalls and port blocking. If you can browse the web you can use RPC!

What XML-RPC looks like:

Obviously it's XML. No surprises there. Let's look at an example. Say we want to call a function on our Domino server that finds the location of a user's mail file. The function you might in imagine in your mind as:

getUsersMailFileLocation("Jake Howlett/codestore");

In RPC's XML it would look like this:

<?xml version="1.0"?>
<methodCall>

<methodName>getUsersMailFileLocation</methodName>

<param>

<value><string>Jake Howlett/codestore</string></value>

</param>

</params>

</methodCall>

You don't need to worry about this format too much as the XML-RPC server we are going to implement in our Java agent takes care of parsing it for all the relevant data.

So how would you send this XML to Domino? You would POST it, that's how. Like so:

POST /db.nsf/xml-rpc?OpenAgent HTTP/1.1
User-Agent: My Client 1.0 
Host: mydominoserver 
Content-Type: text/xml 
Content-Length: 209

How you do this depends on what client you're using to talk to Domino with and isn't really important just yet. Let's look at what the reply from the agent might look like.

<?xml version="1.0"?>
<methodResponse>

<param>

<value><string>prominic/codestore/mail/jake_howlett.nsf</string></value>

</param>

</params>

</methodResponse>

As you can see the answer to our question lies in the middle of all this XML. The calling client would then be responsible for parsing for this data.

The HTTP headers returned by Domino would look something like:

HTTP/1.1 200 OK
Server: Lotus-Domino
Date: Thu, 05 Jun 2003 10:25:39 GMT
Content-Type: text/xml
Content-Length: 158

So, now we know what the XML looks like, we need to start thinking about how we get Domino to return something useful to our client.

Coding the Domino agent:

I have to admit that, when I started this task yesterday, I wasn't 100% sure it would be possible. The problem (or at least I though it was) being that the XML is sent to the server as a POST reqeust. Having skimmed the spec of the Apache XML-RPC server I knew that it required the XML to be passed to it as an InputStream object. Would we be able to get a handle on this data and stream it to the XML-RPC server without resorting to a servlet? Well, yes, as it turned out, we could. Let's see how:

session = getSession();

AgentContext agentContext = session.getAgentContext();

db = agentContext.getCurrentDatabase();

Document doc = agentContext.getDocumentContext();

Item xml = doc.getFirstItem("Request_Content");

InputStream is = xml.getInputStream();

The important lines are the last two. In the second to last line we declare an Item object called "xml" that is the Request_Content field of the document context. When we POST to Domino all the data is held in here. In the case of RPC it's all the XML being sent. In the next line we use the getInputStream method of the Item and, hey presto, we have the InputStream we needed.

Now we need to initiate the XML-RPC server, define the method to handle the call and pass it the request. This is the part where I got really stuck. There's a lot of documentation out there but not a whole lot for Domino and Java. I got there in the end though.

The first thing you need to do is get yourself a copy of the Apacha XML-RPC code archive. I've attached this at the bottom of the document (xmlrpc-1.1.jar). With this on your local drive, click the Edit Project button in your Java agent and add it to the agent, as below:

You now have the server at your disposal and you can make it available to your code using the following import statement:
import org.apache.xmlrpc.*;
And you can create a new server like so:
XmlRpcServer xmlrpc = new XmlRpcServer ();
Now for the tricky part and the bit that I get very confused by. You need to add a "handler" to the server. This tells it what code (i.e. object) you would like to use to process the request. Here's how:
xmlrpc.addHandler ("domino", this );
Note the use of the keyword "this". This tells the server that the methods required to process calls reside in the current object. i.e within the JavaAgent object. If your code is in another class you would add a handler like so:


xmlrpc.addHandler ("domino", new myHandlerClass() );

Using code within the same class allows us easy access to all the Notes objects defined in the agent's code. Here's the method we would add if we wanted to process a call to the procedure domino.getUsersMailFileLocation:

public String getUsersMailFileLocation(String username) throws Exception

{

Database nab = session.getDatabase("", "names.nsf");

View people = nab.getView("($VIMPeople)");

Document person = people.getDocumentByKey(username, false);

return person.getItemValueString("MailFile");

}

To call it we have to call the execute method of our XML-RPC server, passing in to it the InputStream we defined above.
byte[] result = xmlrpc.execute( is );
Then we can pass result back to the calling client as some XML. For this bit of code I have to thank Johan Känngård for his debugging skills. I don't for one moment pretend to be any kind of Java guru. Having spent hours trying to work out why it wasn't working I got in touch with Johan. He soon told me it was because I was using result.toString(). Doh!

pw.println("content-type:text/xml");

pw.println( new String(result) );

And that about covers the basic requirements of an XML-RPC server running on top of Domino. The full code of the agent is attached at the bottom of the article if you want to play with it or, god forbid, actually use it.

In summary:

Unlike most of my articles, where I try and give an example you can get your hands on, this one doesn't have anything you can play with as such. However, I wrote it with the intention of saving anybody else the trouble I went through yesterday. Next time you find yourself thinking "Maybe I could use RPC for that" you know where to come. The alternative is a case of trying to find your way through IBM's Web Services page without getting bogged down in techy-talk.

The title of this article included the word primer. I used this because I still have a way to go with this method in trying to add Blogger API functionality to the version 2 of the DomBlog Template. As I learn and progress I will share more solid (real world?) techniques with you. Hopefully then you will get an ideas as to how useful this can be. The possibilities are almost endless. Remember what I said at the beginning of this article: RPC operates through port 80 so almost anybody can use it no matter where they are.

If you do a search for "xml-rpc clients" you soon see there are lots out there, including clients for Flash and Oracle.

Addendum:

As I mentioned when I first wrote this article, it would have been made a little easier if we were using servlets rather than agents. In case you wondered what the equivalent code might look like in a servlet, Keith Nolen has kindly coded us one and I've attached it alongside the agent code.

Feedback

- Spuggy
- Thu 5 Jun 2003
SOAP
Sorry if this is a stupid question but what is the relationship with XML-RPC and SOAP?

Reply
1. - Jake Howlett
  - Thu 5 Jun 2003
  Re: SOAP
  SOAP is a much cleaner method. Ha ha, only joking.
  Don't know really. Don't care too much either. As I said in the article I am much more interested in what they can do and how they do it. That said, I guess I need to know what SOAP can offer AS well. Hang on a minute.... goes and searches....found this article which some it up nicely:
  "XML-RPC is a final specification which is less verbose and easier to implement than SOAP" http://developer.apple.com/techpubs/macosx/Networking/WebServices/1_intro_folder /chapter_1_section_5.html
  HTH Jake
  
  Reply
  
  Show the rest of this thread
  1. - Marcin
    - Thu 5 Jun 2003
    Re: SOAP
    All well and good, but in practice if you want to interoperate with .Net and WebSphere you need to talk SOAP / Web Services. (Disclaimer: I haven't had any experience with Web Services / SOAP / XML-RPC apart from reading a few articles and checking out some example code)
    Apparently Web Services aren't too hard to implement (with Domino both being a producer and consumer) in Domino 6 given all the extra XML functionality they've added to the base classes.
    I agree though from what i've seen of SOAP and what i've seen of XML-RPC, is that SOAP is pretty messy by comparison. Not to mention the one implementation of SOAP under Domino i've seen used string manipulation functions (Instr/Left/Right/etc) to parse the XML rather than the XML classes (this is the "official" Lotus example, mind you), also the response packet was generated using string concatenation.
    My theory is that they wanted to show how "easy" it is to do SOAP with Domino, so they wanted to minimise the amount of code, there's a bit more overhead loading the XML into a DOM tree and getting the information the right way.
    It saddens me because people that don't know any better will base production code off this Lotus example and a few months down the track wonder why their implementation isn't very flexible :(
    
    Reply
  2. - Brandon
    - Thu 5 Jun 2003
    Re: SOAP
    XML-RPC was originally developed by Microsoft, Developmentor and Userland and essentially provided a simple and effective way to implement distributed RPCs utilising XML. However its developers realised that it suffered from a number of flaws in areas such as extensibility, robustness and interoperability when pushed on a global scale. Having then worked on the draft specifications of SOAP, they then teamed up with IBM and pushed the final SOAP specifications before handing them over to the W3C.
    Some of the flaws inherent in XML-RPC are as follows:
    Inability to transparently extend the request/response formats. So for example if you needed to pass security credentials with your XML-RPC call you would have to modify the procedure and add one parameter.
    XML-RPC is bound to the HTTP protocol, whilst SOAP allows the use of HTTP, SMTP, JMS etc.
    XML-RPC doesn't use XML namespaces and defines its own data types and is therefore redundant given use of XML schemas.
    Although XML-RPC is XML based it provides no clean mechanism to pass an XML document within a request/response. You therefore have to revert to base64 encoding techniques etc. should this be required.
    
    Reply
    1. - Jake Howlett
      - Fri 6 Jun 2003
      Re: SOAP
      Cheers Brandon, you've turned me off XML-RPC before I even really got started ;o) I take your word that SOAP is the way aheasd. However, the only reason I started with XML-RPC is that that's the tool the Blogger.com API uses and hence the one I needed to know. The fact that it's simple also means it's an easy in-road for me to the whole Web Services arena. After a few more articles getting dirty with XML-RPC I may well clean up my act (excuse the pun) and get in to SOAP....
      
      Reply
- Justin Freeman
- Thu 5 Jun 2003
Great article
Hi Jake,
Great article! On your recommendation I downloaded and installed w.bloggar and was thrilled to see that it natively supported Postnuke (what my blog site runs). My next train of thought was "if this works for Postnuke then why not Domino". Your article has been spot on for timing and detail.
Great job! Justin

Reply
- Keith Nolen
- Mon 9 Jun 2003
Testing with wFetch listener
If you want to test this out, you need a tool to create and recieve raw HTTP requests. Microsoft has a good one (wFetch) at:
http://www.microsoft.com/serviceproviders/webhosting/HowTo/wfetch.asp

Reply
1. - Jake Howlett
  - Mon 9 Jun 2003
  Re: Testing with wFetch listener
  Thanks Keith.
  I've been using Socket Workbench (http://www.sigmasolutions.com.au/tcpiq/SWB/Info.asp) which is a little but more advanced but a pain to work with. Trust MS to deliver the goods ;o)
  Jake
  
  Reply
- Keith Nolen
- Mon 9 Jun 2003
Slight XML correction - prevent "RPC hand.." error
Great article, BTW. Very cool.
To get the agent to work you need to change the XML request a bit. Change:
<methodName>getUsersMailFileLocation</methodName>
to:
<methodName>domino.getUsersMailFileLocation</methodName>
...or else you'll get an "RPC handler object not found" Java error.

Reply
- Patrick Correya
- Thu 19 Jun 2003
xmlhttp
I have used the xmlhttp object to achieve the same result.

Reply
1. - Suraj
  - Mon 12 Apr 2004
  Re: xmlhttp
  Hi, can you help how can i receive request XML in domino if the request is coming from ASP using XMLHTTP
  
  Reply
- Marc
- Fri 5 Mar 2004
Changing Domino HTTP Headers
Is there way in pre-R6 Domino to define or set the HTTP Headers returned by a view. I have a View which returns XML (RSS) but since the Server still returns a "Content-Type: text/html" header this is causing problems. In R6 there is apparently a "Content Type" options. Any help would be appreciated!

Reply
- Steve
- Mon 11 Jul 2005
Making the RPC call
How do you go about actually posting the data to the Domino agent?

Reply
1. - Jake Howlett
  - Mon 11 Jul 2005
  Re: Making the RPC call
  You can do it using a form which has the method paramater set to POST. Depends really. There are different ways.
  
  Reply
  
  Show the rest of this thread
  1. - Steve
    - Tue 12 Jul 2005
    Re: Making the RPC call
    So the xml data would be in a specific field on the form? This causes the name of the field to be included in the Request_Content. I ask because using your example, I am getting an error "<value>java.lang.Exception: expected Element</value>"
    
    Reply