Asp Forum - Image decompression, eruby

Belorion

1/5/2005 11:01:00 PM

I am working on a website with a MySQL backend. The site allows
users to upload files, such as jpg, gif, etc. I'm having issues
loading images from the database and getting them to display in the
webbrowser.

Since the raw image encoding contains characters MySQL doesn't like
when doing an INSERT, I encoded the image using Base64.encode64().

So, when I pulled the data back out, I do a Base64.decode64().

I also do a:

print cgi.header( "type=>image/jpg" ) so I have the correct header,
and then I dump the data by simply using <%= data %> as the only
information that is sent to the browser other than print cgi.header.

However, when I load the image, I get browser errors. In Firefox, it
knows it is image/jpg, but I get "The image (image path) cannot be
displayed, because it contains errors." And IE6 completely barfs,
asking to save the file because it doesn't know what type it is (and
saving the file as a jpg and then opening that up does not work)

What am I doing wrong? Does encoding/decoding in Base64 totally screw
this up? Any other recommended approaches? As I mentioned, Firefox is
aware of the fact it is image/jpg, but IE does not. In either case,
the raw data is not a properly encoded jpeg.

14 Answers

Carlos

1/5/2005 11:25:00 PM

[Belorion <belorion@gmail.com>, 2005-01-06 00.01 CET]
> I am working on a website with a MySQL backend. The site allows
> users to upload files, such as jpg, gif, etc. I'm having issues
> loading images from the database and getting them to display in the
> webbrowser.
>
> Since the raw image encoding contains characters MySQL doesn't like
> when doing an INSERT, I encoded the image using Base64.encode64().
>
> So, when I pulled the data back out, I do a Base64.decode64().
>
> I also do a:
>
> print cgi.header( "type=>image/jpg" ) so I have the correct header,

Shouldn't that be cgi.header("type"=>"image/jpg") ?

> and then I dump the data by simply using <%= data %> as the only
> information that is sent to the browser other than print cgi.header.

Just a guess: check for blanks/new lines around the <%= data %>.

...
> What am I doing wrong? Does encoding/decoding in Base64 totally screw
> this up? Any other recommended approaches? As I mentioned, Firefox is
> aware of the fact it is image/jpg, but IE does not. In either case,
> the raw data is not a properly encoded jpeg.

The other possibility is to add a header "Content-Transfer-Encoding: base64"
and send the base64 encoded photo directly.

Good luck.

Kaspar Schiess

1/6/2005 12:32:00 PM

(In response to news:a48d774d050105150167c9244@mail.gmail.com by Belorion)

> Since the raw image encoding contains characters MySQL doesn't like
> when doing an INSERT, I encoded the image using Base64.encode64().

I recommend storing the binary data in BLOB's or whatever that data type is
called with mysql. Then you can insert data by doing
insert into mytable (blobcol) values (?)

The question mark is a placeholder for the data. You can bind your binary
data to it by calling DBI#execute as follows:
dbh.execute( stmt, data )

Note that there are some RDBMS' out there that disallow BLOB inserts. If
this is the case, you need to first insert all non-blob columns and then go
back and UPDATE your table's blob columns (selecting the id's you have just
inserted).

This data then comes back from a SELECT as normal binary data which is
exactly (and without detour) your image. It can be sent to the browser by
the following code (untested):

# image contained in +data+
require 'cgi'
c = CGI.new
c.print c.header( 'type' => 'image/gif', 'length' => data.length )
STDOUT.binmode if PLATFORM =~ /32/
c.print data

The binmode part is UGLY, but there seems to be no other way to do this.

happy hacking !
kaspar

nb: This is a part of the DBI manual:

execute( stmt, *bindvars ) {|statement_handle| aBlock}

Immediately executes the SQL statement stmt after binding the values
in bindvars to the placeholders in the statement.

kaspar

hand manufactured code - www.tua.ch/ruby

Belorion

1/6/2005 3:37:00 PM

> Shouldn't that be cgi.header("type"=>"image/jpg") ?

Yes, it should, and it is in my code ;)

> Just a guess: check for blanks/new lines around the <%= data %>.

Nope, can't find any blank lines....

> The other possibility is to add a header "Content-Transfer-Encoding: base64"
> and send the base64 encoded photo directly.

Good idea. Gave it a shot, but couldn't get it to work. Same errors.

> I recommend storing the binary data in BLOB's or whatever that data type is
> called with mysql. Then you can insert data by doing
> insert into mytable (blobcol) values (?)
>
> The question mark is a placeholder for the data. You can bind your binary
> data to it by calling DBI#execute as follows:
> dbh.execute( stmt, data )
>

I hadn't seen that before. I gave it a shot. Seems to work fine for
getting the browser to display an image. I take that back, dbh.do(
stmt, data ) works fine, I get an error with dbh.execute. However,
the image is coming back with glaring errors. After further
investigation it appears that the raw data in the MySQL table has
issues, though I can't tell if it's from the way the data is inserted,
or if is corrupted from the actual upload.

So my first question regarding displaying the image via eruby in my
browser has been answered (thanks!), but maybe someone has an idea as
to why my data is being corrupted (the image comes back with a ton of
artifacts, especially with strange bars of color).

The way I am uploading my file is derived from a script I found at
http://www.zytrax.com/tech/lang/ru....

Specifically, my HTML for uploading looks like:

<form name='fileupload' enctype="multipart/form-data"
action='attach.rhtml' method='post'>
<input type=hidden name=index value=<%= index %>>
<input type='file' name="myfile" size="35">
<input type='submit' value="Upload">

And my upload specific ruby in attach.rhtml looks like:

fromfile = cgi.params['myfile'].first
fromfile.binmode # Tried both with and without this line
dbh.do( "INSERT INTO attachments VALUES( ? )", [ index,
fromfile.original_filename, fromfile.read] )

So, somewhere in that process the data gets corrupted, and the image
comes out half garbled. Any ideas? Plain text files, which are
handled using the exact same script, come out just fine.

Carlos

1/6/2005 4:07:00 PM

[Belorion <belorion@gmail.com>, 2005-01-06 16.37 CET]
[...]
> Specifically, my HTML for uploading looks like:
>
> <form name='fileupload' enctype="multipart/form-data"
> action='attach.rhtml' method='post'>
> <input type=hidden name=index value=<%= index %>>
> <input type='file' name="myfile" size="35">
> <input type='submit' value="Upload">
>
> And my upload specific ruby in attach.rhtml looks like:
>
> fromfile = cgi.params['myfile'].first
> fromfile.binmode # Tried both with and without this line
> dbh.do( "INSERT INTO attachments VALUES( ? )", [ index,
> fromfile.original_filename, fromfile.read] )
>
> So, somewhere in that process the data gets corrupted, and the image
> comes out half garbled. Any ideas? Plain text files, which are
> handled using the exact same script, come out just fine.

I don't have any idea about dbi or mysql...

Attacking the problem by the web side :), maybe the upload got interrupted
and you are getting a truncated file.

If you add a parameter like

<input type=hidden name=MARK value=MARK>

at the end of your form, you can check it to know if all the previous fields
arrived whole. At least, if that's not the problem, you'll have a variable
less to ponder (and it is a good general tip, too :).

Belorion

1/6/2005 4:11:00 PM

> I don't have any idea about dbi or mysql...
>
> Attacking the problem by the web side :), maybe the upload got interrupted
> and you are getting a truncated file.
>
> If you add a parameter like
>
> <input type=hidden name=MARK value=MARK>
>
> at the end of your form, you can check it to know if all the previous fields
> arrived whole. At least, if that's not the problem, you'll have a variable
> less to ponder (and it is a good general tip, too :).
>
>

I just tried dumping fromfile.read to a file and then viewing the
file. It turns out that the data at that point in time is *fine*.
So I know the data is arriving whole. So, it has something to do
specifically with inserting the binary data into MySQL, which I
suppose is beyond the normal scope of this list (though I won't
discourage any more suggestions!) I am going to do some digging on
binary data in MySQL, but if anyone has any more suggestions I'd love
to hear them.

Roland Schmitt

1/6/2005 4:25:00 PM

Hi,

Belorion schrieb:
>>I don't have any idea about dbi or mysql...
>>
>>Attacking the problem by the web side :), maybe the upload got interrupted
>>and you are getting a truncated file.
>>
>>If you add a parameter like
>>
>> <input type=hidden name=MARK value=MARK>
>>
>>at the end of your form, you can check it to know if all the previous fields
>>arrived whole. At least, if that's not the problem, you'll have a variable
>>less to ponder (and it is a good general tip, too :).
>>
>>
>
>
> I just tried dumping fromfile.read to a file and then viewing the
> file. It turns out that the data at that point in time is *fine*.
> So I know the data is arriving whole. So, it has something to do
> specifically with inserting the binary data into MySQL, which I
> suppose is beyond the normal scope of this list (though I won't
> discourage any more suggestions!) I am going to do some digging on
> binary data in MySQL, but if anyone has any more suggestions I'd love
> to hear them.
>

you can try to convert the binary data to base64 and then store the
base64-data in the database:

require "base64"
...
b64_data = Base64.encode64(binary_data)
# store to db
...
# read from db
binary_data = Base64.decode64(b64_data)
...

Obviously you must convert back after reading the data from the database
;-)

For me base64-formatted data is easier to handle when looking for
differences between two files.

Regards,
Roland

Belorion

1/6/2005 4:37:00 PM

On Fri, 7 Jan 2005 01:24:36 +0900, Roland Schmitt <Roland.Schmitt@web.de> wrote:
> Hi,
> you can try to convert the binary data to base64 and then store the
> base64-data in the database:
>
> require "base64"
> ...
> b64_data = Base64.encode64(binary_data)
> # store to db
> ...
> # read from db
> binary_data = Base64.decode64(b64_data)
> ...
>
> Obviously you must convert back after reading the data from the database
> ;-)
>
> For me base64-formatted data is easier to handle when looking for
> differences between two files.
>
> Regards,
> Roland

I've tried that once with no success, but maybe I'll give it a shot again.

Although, it occurrs to me... my "test" which told me my MySQL data
was bad may have been incorrect. What, precisely, is the best/correct
way to write out binary data? Basically, what I was doing was:

sth = dbh.prepare( "SELECT content FROM attachment WHERE index=41" )
sth.execute
data = sth.fetch[0]

f = File.new( "test.jpg", "w+" )
f << data
f.close

Is that correct way to write out binary data?

Belorion

1/6/2005 4:47:00 PM

I feel dumb. the Base64 was one of the first approaches I used. I
just tried it again and it worked. I'm not sure what I am doing right
this time that I was doing wrong the last... but it works now! Thanks
everyone for the help!

And, for anyone searching Ruby talk for a binary file upload solution
using MySQL, here ya go:

HTML:

<form name='fileupload' enctype="multipart/form-data"
action='attach.rhtml' method='post'>
<input type='file' name="myfile" size="35">
<input type='submit' value="Upload">

In attach.rhtml, the relavent code for insertion into MySQL is:

require 'cgi'
require 'base64'
require 'dbi'
require 'stringio' # not sure if you need this

myfile = cgi.params['myfile'].first
dbh = DBI.connect( "DBI:Mysql:userdb", "user", "psswd" )
dbh.do( "INSERT INTO table VALUES( ? )", Base64.encode64(myfile.read) ] )
dbh.disconnect

And then, to view the file using eruby:

<%
require 'cgi'
require 'dbi'
require 'base64'

cgi = CGI.new

dbh = DBI.connect( "DBI:Mysql:userdb", "user", "psswd" )
sth = dbh.prepare( " (statement to select out your data) "
sth.execute
data = sth.fetch[0] # assuming your binary data is the only thing returned

print cgi.header("type"=>"image/jpg", "length"=>data.length)
cgi.print Base64.decode64(data)
%>

Osuka Adartse

1/6/2005 5:31:00 PM

Belorion wrote:
>>Shouldn't that be cgi.header("type"=>"image/jpg") ?
>
>
> Yes, it should, and it is in my code ;)
>
>
>>Just a guess: check for blanks/new lines around the <%= data %>.
>
>
> Nope, can't find any blank lines....
>
>
>>The other possibility is to add a header "Content-Transfer-Encoding: base64"
>>and send the base64 encoded photo directly.
>
>
> Good idea. Gave it a shot, but couldn't get it to work. Same errors.
>
>
>>I recommend storing the binary data in BLOB's or whatever that data type is
>>called with mysql. Then you can insert data by doing
>> insert into mytable (blobcol) values (?)
>>
>>The question mark is a placeholder for the data. You can bind your binary
>>data to it by calling DBI#execute as follows:
>> dbh.execute( stmt, data )
>>
>
>
> I hadn't seen that before. I gave it a shot. Seems to work fine for
> getting the browser to display an image. I take that back, dbh.do(
> stmt, data ) works fine, I get an error with dbh.execute. However,
> the image is coming back with glaring errors. After further
> investigation it appears that the raw data in the MySQL table has
> issues, though I can't tell if it's from the way the data is inserted,
> or if is corrupted from the actual upload.
>
> So my first question regarding displaying the image via eruby in my
> browser has been answered (thanks!), but maybe someone has an idea as
> to why my data is being corrupted (the image comes back with a ton of
> artifacts, especially with strange bars of color).
>
> The way I am uploading my file is derived from a script I found at
> http://www.zytrax.com/tech/lang/ru....
>
> Specifically, my HTML for uploading looks like:
>
> <form name='fileupload' enctype="multipart/form-data"
> action='attach.rhtml' method='post'>
> <input type=hidden name=index value=<%= index %>>
> <input type='file' name="myfile" size="35">
> <input type='submit' value="Upload">
>
> And my upload specific ruby in attach.rhtml looks like:
>
> fromfile = cgi.params['myfile'].first
> fromfile.binmode # Tried both with and without this line
> dbh.do( "INSERT INTO attachments VALUES( ? )", [ index,
> fromfile.original_filename, fromfile.read] )
>
> So, somewhere in that process the data gets corrupted, and the image
> comes out half garbled. Any ideas? Plain text files, which are
> handled using the exact same script, come out just fine.

let me guess! The image starts OK but then horizontal lines start to
appear with degenerating colors? if so the data in DB is probably OK as
long as you use MySql::quote(img_data) at insert time.

Ok try this, select the image data and write it to disk,just an idea
since I had once a similar problem.

res=@my.query("select pik from test where idpiks=1;")
res.each{|r| img=r[0]} #well r[0] it's the actual image.
File.open("e:/temp/test.jpg","wb").write(img)

now compare it, if it's OK then no upload problems or in db but at
"browser display". If not make sure the #quote method is being used.

how I displayed the image
imager.rb
...
res=@my.query("select pik from test where idpiks=#{cgi.params['CID']};")
res.each{|r| img=r[0]}
puts "Content-Type: image/jpeg"
puts "Content-Length: #{img.size}"
puts "Content-Title:MyTest.jpg"
puts
$stdout.binmode << img #this is the culprit in windows, notice 'wb'
mode in File#Write so #binmode, it drove me nuts back then, the answer
was obvious...
}

hope it helps
Adartse

Belorion

1/6/2005 5:38:00 PM

> let me guess! The image starts OK but then horizontal lines start to
> appear with degenerating colors? if so the data in DB is probably OK as
> long as you use MySql::quote(img_data) at insert time.

That was precisely the problem!

> Ok try this, select the image data and write it to disk,just an idea
> since I had once a similar problem.
>
> res=@my.query("select pik from test where idpiks=1;")
> res.each{|r| img=r[0]} #well r[0] it's the actual image.
> File.open("e:/temp/test.jpg","wb").write(img)
>
> now compare it, if it's OK then no upload problems or in db but at
> "browser display". If not make sure the #quote method is being used.
>
> how I displayed the image
> imager.rb
> ...
> res=@my.query("select pik from test where idpiks=#{cgi.params['CID']};")
> res.each{|r| img=r[0]}
> puts "Content-Type: image/jpeg"
> puts "Content-Length: #{img.size}"
> puts "Content-Title:MyTest.jpg"
> puts
> $stdout.binmode << img #this is the culprit in windows, notice 'wb'
> mode in File#Write so #binmode, it drove me nuts back then, the answer
> was obvious...
> }
>
> hope it helps
> Adartse
>

Thanks for that solution as well. I currently have it working using
Base64 encoding, so I may not switch it over, but thanks anyway.

And thanks to all for their help.

comp.lang.ruby

Image decompression, eruby

Belorion

Carlos

Kaspar Schiess

Belorion

Carlos

Belorion

Roland Schmitt

Belorion

Belorion

Osuka Adartse

Belorion

x Login to ForumsZone