Domgle
Multi-database searching is one of those things that I'm glad I've never had to deal with. Maybe it's a lot easier than it sounds, but I imagine is isn't.
Well, here's an interesting take on the problem - Domgle, created by Julien Bottemanne
The app uses Ajax (what esle!?) to fetch search results from more than one database. The JavaScript then sorts all the results before displaying them. Here's the example form. Try a search for "a" to start with.
It's an interesting concept, but I'd prefer to see the search/sorting done server-side to avoid any accessibility issues.
Skip to the 
You can subscribe to an individual
Thanks Jake !
As i said to Joe Litton, I think Domgle is much more a good idea than a coding performance.
For us (domino developers) Ajax open doors of news functionalities. A lots of things are still waiting to be explored.
You're right for the sorting problem but doing it client side allowed us to make search queries without any agent access issues.
Sorting problem is no more a problem, I've implemented recently server-side sorting using ajax (dojo toolkit). It need some testing before publishing, but generally it works. I don't use agents for this. I'll mention other problem, session timeout, which also can be handled via Ajax.
For those who are considering their options on searching. Domino already provides two ways of searching multiple databases:
1. A "Domain Search" can be used for searching across all indexed databases or you can restrict your search to databases with a specific Database Category (which you allocate in the database properties).
2. The search site database (introduced in Release 4) is still supported and is included as a template on the server.
While sorting search results clientside using javascript is certainly possible (and works well) the problem starts when your resultsets increase in size. If your search returns 100-200 hits, its no problem, but if it returns say 5000, the time to sort increases a lot.
Glen: While the included search works, it's by no means perfect, especially concerning formatting and custom sorting...
I have been thinking about using apache lucene with Domino though, but I have never gotten around to test how complicated it would be. It would certainly be easy enough to make it read the Domino data:)
I have a suite of databases where I needed to achieve similar functionality, a little over 2 years ago now.
It uses a Lotusscript agent to get the parameters passed for search, start, count etc, it then does an FTSearch in the database and writes the results as a custom javascript array which it imports into a dhtml framework used to custom sort ans paginate the results.
The overhead in the agent doing the initial work is greatly enhanced by the fact that the resulting search data is cached in the browsers javascript array.
It takes just a second to return the results, and as each line is only about 200 characters 1000's of results take little transfer time, and processing on a FT index database is one of the fastest searches anyway.
The javascript code that gets included with the results then allow sorting and resorting of the data by columns, without another round trip to the server, so again is very fast.
It also allows muti-word searches, which is typically something the native dominosearch functionality generally doesn't like.
I see one advantage to use AJAX for multi-database search. There are no problems to manage documents with reader field.
I don"t know for "Domain Search" solution but Search Site Database didn't display this documents (perhaps this problem have been corrected in next versions).
@Glen :
Domain search is not very customisable and a bit "heavy" to manage. It also asks a lot ressources from your server.
Sorry but the search site database is not suported until release 4... Try to take controle of the html view result and you'll see.
@Bjorn :
If a search result return 5000 results you first need to change your request...
@Andy I don't understand what you referring to when describing problems with multi word searching in Notes. The search engine that Domino uses allows for a multitude of searching options, including using boolean experssions. So by using operators, parenthesis, and some search options, you can make Domino search for whatever you want. But just like the help documentation says, "Conducting a successful search can be an art".
Sean---
Hey Jake,
Just sent a demo multi database search tool to Jerry to have a look at and we are looking at collaborating in producing an article for it (downloadable version etc).
Similar to what other people have done - its an agent which does the Multi FT searches, sorts the order and then caches the result by serialising the results into a document - subsequent paging requests are instant.
I use injected JSON to allow the results to be used in any calling web page - there's no AJAX so no cross domain issues. The JSON data is a full object representation of each result so you can display how you feel fit. - I even include the URL to the database icon if that floats your boat.
The databases / forms & fields to return are all configuration documents so its completely reusable and the Lotusscript is full OO (thanks Mr Bill Buchan).
If your interested I can fire a copy across for you to have a look at.
@Julien
I agree that the Search Site DB is not very good / customisable. It's probably only still there for backward compatability.
The Domain Search results are customisable though. You can modify how results are displayed by creating your own results forms including the "ResultEntry" form that determines the formatting of an individual search result entry. However, the sorting of results are limited to relevance, date acsending/descending.
For some people this will be enough. But if you need to sort results differently and add different fields to the results then a solution like yours is great.
The point I was trying to make is that if your looking into a search solution (or any problem for that matter) don't automatically discount what Domino can already do.
We started using javascript to do this type of work about three years ago, when we ported a Lotus web search solution to SQL.
Domino natively still has a problem with record set type views of data and a lot of work has to be done to make searching large databases using paged record set type views. In SQL based back end, this is a piece of cake!
I hope with the new Hannover buld there might be some nifty features we could use to convince Domino to provide fast and efficient search result sets that are compariable to SQL + server side code based search results.
@sean, I was merely pointing out, if you pop along to IBM's Sandbox...
{Link}
and search for "text editor", you get 3-4 results, but try searching for "editor text", nothing is returned.
Why Not ? 'cus it doesn't handle it particularly well. !
...Thats all !
Jake, I agree this is best done server side.
The problem with Domino's search for me has always been that it indexes every field in the document.
If I had to do multi-database searching then I would create my own Domain Search database which contains stub copies of documents, containing only the fields I want to search on (and the source document URL). Then I'd build a view of all those documents and search those using a URL get.
No agents required (apart from scheduled ones to sync the documents) so searches would be fast and no issues with audit trail fields containing words that people end up searching for!
@Andy : that's because in the Notes syntax, following words are considered as a whole "phrase". Just as if you search (with quotes):
"text editor" and "editor text" in google.
Thus, if you search for "text and editor", or "editor and text" (without the quotes) in the sandbox, you will find 5 results in both query.
Notes search does work very well, but it's syntax is unusual and unnatural. I (re)wrote at this purprose a JS parser to mimic the google syntax. You may find it at the bottom of this blog : {Link}
I also prefer server-side processing for reducing the amout of hits, paging and sorting. What I came across were sorting-problems. There are a myriad of sorting-algos, but the one that performed the most, was not usable in LotusScript, because of out-of-stack memory errors: quicksort.
In the end, I used a Shell-Metzner sort, which gave no memory problems and performed rather good.
has anyone experience of using a google appliance {Link} to search a domino server?
@Andy and @YoGi: Mike Golding of Notestips has written some nice javascript code to search Notes with Google syntax search queries:
{Link}
I think (nearly) any Notes search, wether multi db or not, should abide to the standard search syntax of Google. It is only in Notes that I still have to add "and" operators.
@Larry : For instance, my JS parser is an enhanced version of Mike Golding's one.