[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

Bug in base64 decoder of xmlrpc?

Lars Hoss

1/16/2005 9:29:00 PM

Hi,

I am currently implementing support for the MetaWeblog/MovableType API
in my weblog engine. The API is simply a bunch of XML-RPC methods. One
method allows the uploading of files (metaWeblog.newMediaObject). In
order to send the binary file via XML-RPC the data gets base64 encoded.
The xmlrpc library automatically decodes the data. Writing the file to
disk looks something like this:

def newMediaObject(blogid, username, password, data)
fileData = data["bits"] # Decoding is handled by xmlrpc automatically
File.open(data["name"], "wb") { |f| f.write(fileData) }
end

The written data, however, is corrupted. Sometimes even some bytes are
missing. Uploading plain text files seems to be ok. But uploading arger
files such as JPEG files will result in corrupted data. The client I am
using is MarsEdit.

Any suggestions?

Regards,
Lars

--
"Stil ist die Fähigkeit, komplizierte Dinge einfach zu sagen - nicht
umgekehrt." -- Cocteau, Jean
3 Answers

djberg96

1/17/2005 8:57:00 PM

0

Lars Hoss wrote:
> Hi,
>
> I am currently implementing support for the MetaWeblog/MovableType
API
> in my weblog engine. The API is simply a bunch of XML-RPC methods.
One
> method allows the uploading of files (metaWeblog.newMediaObject). In
> order to send the binary file via XML-RPC the data gets base64
encoded.
> The xmlrpc library automatically decodes the data. Writing the file
to
> disk looks something like this:
>
> def newMediaObject(blogid, username, password, data)
> fileData = data["bits"] # Decoding is handled by xmlrpc
automatically
> File.open(data["name"], "wb") { |f| f.write(fileData) }
> end
>
> The written data, however, is corrupted. Sometimes even some bytes
are
> missing. Uploading plain text files seems to be ok. But uploading
arger
> files such as JPEG files will result in corrupted data. The client I
am
> using is MarsEdit.
>
> Any suggestions?
>
> Regards,
> Lars
>
> --
> "Stil ist die Fähigkeit, komplizierte Dinge einfach zu sagen - nicht

> umgekehrt." -- Cocteau, Jean

I remember bringing something like this up with the author last year
after reading Randy Ray's use.perl journal entry at
http://use.perl.org/~rjray/jo... and
http://use.perl.org/~rjray/jo.... Randy Ray is the author of
the Perl module RPC::XML, btw.

This was Michael's response at the time (please correct me if things
have changed Michael):

"Implementing [Base64 streaming support] in xmlrpc4r is possible, but
it would require some parts to be redesigned, especially where strings
are returned (e.g. methodResponse).

One problem is that the XML-RPC spec. requires the size of the
XML-message to
be known in the HTTP header (which is send first). Without this
requirement, the whole
XML-RPC message could be streamed out a socket."

Whether anything has been done since then (about a year ago), I don't
know.
If this is unrelated, then I apologize for the noise.

Regards,

Dan

Michael Neumann

1/17/2005 9:18:00 PM

0

Daniel Berger wrote:
> Lars Hoss wrote:
>
>>Hi,
>>
>>I am currently implementing support for the MetaWeblog/MovableType
>
> API
>
>>in my weblog engine. The API is simply a bunch of XML-RPC methods.
>
> One
>
>>method allows the uploading of files (metaWeblog.newMediaObject). In
>>order to send the binary file via XML-RPC the data gets base64
>
> encoded.
>
>>The xmlrpc library automatically decodes the data. Writing the file
>
> to
>
>>disk looks something like this:
>>
>>def newMediaObject(blogid, username, password, data)
>> fileData = data["bits"] # Decoding is handled by xmlrpc
>
> automatically
>
>> File.open(data["name"], "wb") { |f| f.write(fileData) }
>>end
>>
>>The written data, however, is corrupted. Sometimes even some bytes
>
> are
>
>>missing. Uploading plain text files seems to be ok. But uploading
>
> arger
>
>>files such as JPEG files will result in corrupted data. The client I
>
> am
>
>>using is MarsEdit.
>>
>>Any suggestions?
>>
>>Regards,
>>Lars
>>
>>--
>>"Stil ist die Fähigkeit, komplizierte Dinge einfach zu sagen - nicht
>
>
>>umgekehrt." -- Cocteau, Jean
>
>
> I remember bringing something like this up with the author last year
> after reading Randy Ray's use.perl journal entry at
> http://use.perl.org/~rjray/jo... and
> http://use.perl.org/~rjray/jo.... Randy Ray is the author of
> the Perl module RPC::XML, btw.
>
> This was Michael's response at the time (please correct me if things
> have changed Michael):
>
> "Implementing [Base64 streaming support] in xmlrpc4r is possible, but
> it would require some parts to be redesigned, especially where strings
> are returned (e.g. methodResponse).
>
> One problem is that the XML-RPC spec. requires the size of the
> XML-message to
> be known in the HTTP header (which is send first). Without this
> requirement, the whole
> XML-RPC message could be streamed out a socket."
>
> Whether anything has been done since then (about a year ago), I don't
> know.
> If this is unrelated, then I apologize for the noise.

Hm, it should work if you pass a XMLRPC::Base64 object as argument, and
not a String. Of course streaming does not work, but that's not related
to the corruption. Maybe the best would be to encode every string as
base64, but that would probably break many clients or servers (other
than xmlrpc4r).

Regards,

Michael

Lars Hoss

1/18/2005 12:56:00 PM

0

Hi Michael, hi Daniel,

first thanks for your help. After digging in the sources I think I
discovered the problem. The problem is that somehow all "+" characters
in the base64 encoded string get converted into a " " character. I am
still not sure at which point the conversion happens. Probably Rails
thinks that the XML-RPC data is a request parameter and therefore
automatically converts all "+" characters back into " " characters
which is of course ok for regular request parameters. But as I said
before, I am still searching ...

Yours,
Lars

Am 17.01.2005 um 22:17 schrieb Michael Neumann:

> Daniel Berger wrote:
>> Lars Hoss wrote:
>>> Hi,
>>>
>>> I am currently implementing support for the MetaWeblog/MovableType
>> API
>>> in my weblog engine. The API is simply a bunch of XML-RPC methods.
>> One
>>> method allows the uploading of files (metaWeblog.newMediaObject). In
>>> order to send the binary file via XML-RPC the data gets base64
>> encoded.
>>> The xmlrpc library automatically decodes the data. Writing the file
>> to
>>> disk looks something like this:
>>>
>>> def newMediaObject(blogid, username, password, data)
>>> fileData = data["bits"] # Decoding is handled by xmlrpc
>> automatically
>>> File.open(data["name"], "wb") { |f| f.write(fileData) }
>>> end
>>>
>>> The written data, however, is corrupted. Sometimes even some bytes
>> are
>>> missing. Uploading plain text files seems to be ok. But uploading
>> arger
>>> files such as JPEG files will result in corrupted data. The client I
>> am
>>> using is MarsEdit.
>>>
>>> Any suggestions?
>>>
>>> Regards,
>>> Lars
>>>
>>> --
>>> "Stil ist die Fähigkeit, komplizierte Dinge einfach zu sagen - nicht
>>> umgekehrt." -- Cocteau, Jean
>> I remember bringing something like this up with the author last year
>> after reading Randy Ray's use.perl journal entry at
>> http://use.perl.org/~rjray/jo... and
>> http://use.perl.org/~rjray/jo.... Randy Ray is the author of
>> the Perl module RPC::XML, btw.
>> This was Michael's response at the time (please correct me if things
>> have changed Michael):
>> "Implementing [Base64 streaming support] in xmlrpc4r is possible, but
>> it would require some parts to be redesigned, especially where strings
>> are returned (e.g. methodResponse).
>> One problem is that the XML-RPC spec. requires the size of the
>> XML-message to
>> be known in the HTTP header (which is send first). Without this
>> requirement, the whole
>> XML-RPC message could be streamed out a socket."
>> Whether anything has been done since then (about a year ago), I don't
>> know.
>> If this is unrelated, then I apologize for the noise.
>
> Hm, it should work if you pass a XMLRPC::Base64 object as argument,
> and not a String. Of course streaming does not work, but that's not
> related to the corruption. Maybe the best would be to encode every
> string as base64, but that would probably break many clients or
> servers (other than xmlrpc4r).
>
> Regards,
>
> Michael
>
>
>
--
"Stil ist die Fähigkeit, komplizierte Dinge einfach zu sagen - nicht
umgekehrt." -- Cocteau, Jean