Uncovering Problems With My WebSession Class
Last week I encountered one of those problems that you just can't work out, but end up kicking yourself when you do as it's such a school-boy error.
I had written an Agent to accept and log requests for files before redirecting the user to the download. The file's name was passed as a paramater. All worked fine until one of the files contained an & in the name. Although I'd done all the usual stuff of encoding the file name before calling the agent the code was still only using the part of the file name before the &.
The reason it turns out was obvious and rooted in my own code in the WebSession class. The method called URLQueryString(), which works just like its @function equivalent, had been made to work with the value of the Query_String_Decoded field as the raw data. See the problem?
If you use that class in any databases you might want to go and change the way it uses the decoded query string field and use the normal Query_String field instead. The offending line looks like this:
Let Me.query = Me.document.Query_String_Decoded(0)
This has been a painful lesson for me as I've used the code in all and everything I've worked on recently -- such is my love for it -- and have had to work my way through them all, updating each Script Library.
It all reminds me though that I need to update the original download with this and the other improvements I've since made to the class. Others have improved on it too, which is always nice to see. Tanny O'Haley has for one. As has Michel Van Der Meiren, whose customised version shows how easy it is to tailor to the needs of your own application.
Skip to the 
You can subscribe to an individual
Which is where the use of templates comes in.
Any developer worth his salt has come across old code that has bugs in it or needs to be enhanced.
I have a series of templates which cascades their inheritances through a selection of other templates. The Master Template simply contains various totally generic routines. prototype.js sits in here, as does TinyMCE and a variety of other useful libraries, shared fields, subforms, shared pages, $$Templates etc. All of my application templates inherit bits from here, which in turn is inherited by the test applications. The Live apps never inherit for safeties sake, but have their design copied from the test apps - which the end client has tested and approved.
So if you ever find you've made a bug in a routine, fix it in one spot, and run the designer task on the server. Now everything is sorted. :)
So have you added that example to your unit tests for WebSession?
;-)
I feel your pain. The thing that really gets me is that there is no equivalent to @URLDecode for LotusScript. I am reduced to having to use the @Formula via an Evaluate call. Effective, but certainly not elegant.
@Sean,
Is this the problem we ran into on Idea Jam with extended ascii chars in the username field?
@Bruce - Your memory serves you well.
There are a couple of LS functions that take the place of @URLDecode if you don't mind more lines of code.
Also, java.net.io has a URLDecoder and URLEncoder class available. You could LS2J or write your agents in straight Java if you wanted to.
Sean - did the problem you were having fall on the encoding scheme argument being supplied as "Domino"? If so what did you use to handle the odd characters or what was the final solution?
One problem using @URLDecode is that it doesn't support the & entity as a separator value in a query string. Another problem that I ran into decoding value data with @URLDecode is that it does not decode "+" characters. I ended up writing a function to make up for the limitations of @URLDecode.
Here is a problem with using @URLDecode in an LS agent. If the string is to long Domino won't handle it. Post a 100K of encoded XML to an agent. Then concat the resulting reqest_content_00.. fields into a single string then try to evaluate. Doesn't work. Next try using an LS based Decode function. Carefull it may crash your server. Why would someone need to accept this much data. Receive encoded xml containing Base64 encoded attachments silly. Looks like Java may be the only option for something like this. Of course it could be a memory problem with the decode function itself
Function DecodeContent( s As String ) As String
On Error Goto errHandler
Dim r1 As String, r2 As String
Dim p_start As Long, p_found As Long
r1 = ""
p_start = 1
p_found = Instr( p_start, s, "+", 0 )
Do While p_found > 0
r1 = r1 + Mid$( s, p_start, p_found - p_start )
r1 = r1 + " "
p_start = p_found + 1
p_found = Instr( p_start, s, "+", 0 )
Loop
r1 = r1 + Mid$( s, p_start )
p_start = 1
p_found = Instr( p_start, r1, "%", 0 )
Do While p_found > 0
r2 = r2 + Mid$( r1, p_start, p_found - p_start )
r2 = r2 + Chr$( Val( "&H" + Mid$( r1, p_found + 1, 2 )))
p_start = p_found + 3
p_found = Instr( p_start, r1, "%", 0 )
Loop
r2 = r2 + Mid$( r1, p_start )
DecodeContent = r2
varCleanup:
Exit Function
errHandler:
Msgbox "Decode error" & Error$ & " at line " & Erl & "Cur p_start:" & p_start
Resume varCleanup
End Function
I was also looking for a good way to decode strings in LotusScript. The formula above does work, but doesn't handle UTF-8 always correctly. It fails when the string to be decoded contains 2 or 3 escape sequences for a single character. A ü (u-umlaut) for instance is replaced by a %C3%BC sequence when encoded to UTF-8 with the (JavaScript) encodeURIComponent function.
Another problem with the function above is bad performance when decoding large strings due all the string concatenations (see also http://blog.lotusnotes.be/domino/archive/2008-01-14-concatenate-string-test.html)
If you use @URLDecode("utf-8", "%C3%BC") the string is correctly decoded to the original character but as Rick already pointed out, that function is limited in LotusScript. Tests showed that the limit for the input string when calling @URLDecode from LotusScript using Evaluate is 2048 characters.
To work around this I wrote a LotusScript function that uses Evaluate(@URLDecode), but splits the input string in chunks of 2048 characters (or less) and also handles the situation that the input string is split in the middle of an escape sequence. The only situation in which it still incorrectly decodes the input string is when it is split in the middle of multiple escape sequences for a single character.
Public Function decodeURI$(txt$)
'decodes a UTF-8 encoding string
decodeURI = txt
'replace + by spaces first: @URLDecode doesn't conver these back to spaces
If Instr(decodeURI, "+")>0 Then
decodeURI = Replace(decodeURI, "+", " ")
End If
Const MAX_LENGTH = 2048
'NOTE: evaluate has an input limit of 2 Kb (len = 2048)
if Len(decodeURI) <= MAX_LENGTH then
decodeURI = Join( Evaluate( {@URLDecode("utf-8"; "} & decodeURI & {")} ), "" )
Else
Dim s As New NotesSession
Dim stream As NotesSTream
Set stream = s.CreateStream
Dim pStart As Long
Dim part As String, i As Integer, intLen As Integer, char As String
pStart = 1
While pStart <= Len(decodeURI)
part = Mid$( decodeURI, pStart, MAX_LENGTH )
intLen = Len(part)
For i=1 To 2
char = Left(Right(part, i), 1)
If char = "%" Then 'dont split in the middle of an encoded char
part = Left( part, intLen-i )
intLen = Len(part)
Exit For
End If
Next
stream.WriteText Join( Evaluate( {@URLDecode("utf-8"; "} & part & {")} ) )
pStart = pStart + intLen 'new start position
Wend
stream.Position = 0
decodeURI = stream.ReadText
end if
End Function
I also had problems with @URLdecode and long strings (messages in webforms... -> bad)
I submitted the @Urldecode limit to IBM (PMR 62259 SGC 724) and they say it's not a limitation in @URLDecode but in evaluate: @explode will have the same problem. So you could add the value to field in a temp document and call @URLdecode(field) with the tempdoc as context (haven't tried yet).
To split the string correctly, you need to split it at an encoded value which begins NOT with the bytes 10xx xxxx (see http://en.wikipedia.org/wiki/UTF-8 -> table). Also not yet sure how to do that, but I think I will go that way in my websession bugfix...
Lots of fun...
Whenever I do Domino development I always end up coming back to this blog as one of the best sources of information. In this case my thanks to Mark for his insight and code into decoding long strings of data. Many thanks also to Jake for maintaining Codestore.