[lnkForumImage]
TotalShareware - Download Free Software

Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

Proxy Server troubles

Tanner Burson

4/22/2005 12:54:00 AM

I've been working on a proxy server implementation for a project idea
I have. But I've found that it only seems to work on certain pages,
and not on others. I've distilled the problem down to a simple
implementation, but it still occurs. The code below will return a
(mostly) correct website if pointed at 'www.msn.com' but if pointed at
say 'slashdot.org' the browser refuses to display it, in both cases
the page is loaded correctly. If anyone has any ideas I'd really
appreciate it.

===proxy.rb

require 'socket'
require 'net/http'
local_host = '192.168.0.6'
local_port = 8080

host = 'slashdot.org'
server = TCPServer.new(local_host,local_port)
while true
Thread.start(server.accept) do |s|
data = ''
while s.gets && !$_.nil?
data += $_
break if $_ == "\r\n" || $_ == "\n"
end
p "====Received: #{data}"
net = Net::HTTP.new(host,80)
response = net.get('/')
s << "HTTP/#{response.http_version}
#{response.code}/#{response.message} \r\n"
p "====Got back"
response.each_header do |key,value|
tmp = key + ": " + value + "\r\n"
p tmp
s << tmp
end
p response.body
s << "\r\n" << response.body
s.close # We're done with this request, bring on the next.
end
end

====end
--
===Tanner Burson===
tanner.burson@gmail.com
http://tanner... <---Might even work one day...



12 Answers

jm

4/22/2005 1:31:00 AM

0

I gave it a quick run. I had to change
local_host = nil
so that it would bind and fixed the line wrap in the posted code.
Telnetted to port 8080 on localhost and hit enter twice and it came
back with a lot of html from slashdot. So it works on

$ ruby -v
ruby 1.8.1 (2003-12-25) [powerpc-darwin]

Can't see anything obviously wrong. Do you have a transparent proxy or
anything between you and the site as this can cause havoc?

J.

On 22/04/2005, at 10:54 AM, Tanner Burson wrote:

> I've been working on a proxy server implementation for a project idea
> I have. But I've found that it only seems to work on certain pages,
> and not on others. I've distilled the problem down to a simple
> implementation, but it still occurs. The code below will return a
> (mostly) correct website if pointed at 'www.msn.com' but if pointed at
> say 'slashdot.org' the browser refuses to display it, in both cases
> the page is loaded correctly. If anyone has any ideas I'd really
> appreciate it.
>
> ===proxy.rb
>
> require 'socket'
> require 'net/http'
> local_host = '192.168.0.6'
> local_port = 8080



james_b

4/22/2005 1:49:00 AM

0

jm wrote:
> I gave it a quick run. I had to change
> local_host = nil
> so that it would bind and fixed the line wrap in the posted code.
> Telnetted to port 8080 on localhost and hit enter twice and it came back
> with a lot of html from slashdot.

I managed the same, on win32, but could not get the results to render in
Firefox or IE. It fetches the page OK (judging by the debugging p
calls), but whatever is sent back to the browser is off in some way.



James


Tanner Burson

4/22/2005 1:50:00 AM

0

On 4/21/05, jm <jeffm@ghostgun.com> wrote:
> I gave it a quick run. I had to change
> local_host = nil
> so that it would bind and fixed the line wrap in the posted code.
> Telnetted to port 8080 on localhost and hit enter twice and it came
> back with a lot of html from slashdot. So it works on
>
> $ ruby -v
> ruby 1.8.1 (2003-12-25) [powerpc-darwin]
>
> Can't see anything obviously wrong. Do you have a transparent proxy or
> anything between you and the site as this can cause havoc?
>


Not that I'm aware of. I should have been more specific the first
time, try setting it as a proxy in your browser settings, I've yet to
get it to load correctly in a browser.

--
===Tanner Burson===
tanner.burson@gmail.com
http://tanner... <---Might even work one day...



Tanner Burson

4/22/2005 1:51:00 AM

0

On 4/21/05, James Britt <james_b@neurogami.com> wrote:
> jm wrote:
> > I gave it a quick run. I had to change
> > local_host = nil
> > so that it would bind and fixed the line wrap in the posted code.
> > Telnetted to port 8080 on localhost and hit enter twice and it came back
> > with a lot of html from slashdot.
>
> I managed the same, on win32, but could not get the results to render in
> Firefox or IE. It fetches the page OK (judging by the debugging p
> calls), but whatever is sent back to the browser is off in some way.
>
>

This so far has been exactly the problem. Some sites (ie: msn.com )
work just fine, but others won't work at all. I've compared the
output of both requests but haven't been able to determine any
difference other than possibly the amount of content being sent back.
I'm about at my wit's end over this one, any advice would be great.

> James
>
>


--
===Tanner Burson===
tanner.burson@gmail.com
http://tanner... <---Might even work one day...



Jim Freeze

4/22/2005 2:27:00 AM

0

* Tanner Burson <tanner.burson@gmail.com> [2005-04-22 10:49:35 +0900]:

Have you tried setting your HTTP_USER_AGENT to something like:

Mozilla/5.0


--
Jim Freeze
Code Red. Code Ruby


Tanner Burson

4/22/2005 2:33:00 AM

0

On 4/21/05, Jim Freeze <jim@freeze.org> wrote:
> * Tanner Burson <tanner.burson@gmail.com> [2005-04-22 10:49:35 +0900]:
>
> Have you tried setting your HTTP_USER_AGENT to something like:
>
> Mozilla/5.0

When I'm using the full script (not included for brevity), it runs as
a true proxy and relays all the request headers straight on through to
the server. The code I provided is just a simple sample that still
portrays the same problems. It doesn't appear to be anything related
to the request itself, as the response comes back successfully. There
is just something about it the browser doesn't like.

My first thought was that it didn't like receiving the whole page at
once, so I tried reading the body in chunks and sending it on, but it
didn't seem to solve it either.

>
> --
> Jim Freeze
> Code Red. Code Ruby
>
>


--
===Tanner Burson===
tanner.burson@gmail.com
http://tanner... <---Might even work one day...



Ernest Ellingson

4/22/2005 3:35:00 AM

0

Tanner Burson wrote:
> I've been working on a proxy server implementation for a project idea
> I have. But I've found that it only seems to work on certain pages,
> and not on others. I've distilled the problem down to a simple
> implementation, but it still occurs. The code below will return a
> (mostly) correct website if pointed at 'www.msn.com' but if pointed at
> say 'slashdot.org' the browser refuses to display it, in both cases
> the page is loaded correctly. If anyone has any ideas I'd really
> appreciate it.
>
> ===proxy.rb
>
Take a look at socketproxy.rb at

http://dev.ctor.org...

jm

4/22/2005 3:51:00 AM

0


On 22/04/2005, at 11:48 AM, James Britt wrote:

> jm wrote:
>> I gave it a quick run. I had to change
>> local_host = nil
>> so that it would bind and fixed the line wrap in the posted code.
>> Telnetted to port 8080 on localhost and hit enter twice and it came
>> back with a lot of html from slashdot.
>
> I managed the same, on win32, but could not get the results to render
> in Firefox or IE. It fetches the page OK (judging by the debugging p
> calls), but whatever is sent back to the browser is off in some way.
>

Alright. this is wierd I just tried it with firefox on macosx and it
rendered find, except for all the broken images, etc due to the
reference to '/', ie

net = Net::HTTP.new(host,80)
response = net.get('/')

Made a few modification. Really needs a clean up and there's probably
libraries out there which will do this cleaner. I've been staring at
perl code most of the day wishing it wasn't . Anyway, I've included the
full script below.

require 'socket'
require 'net/http'
local_host = nil
local_port = 8080

#
# returns proto,host,path or nil
#
def url_split(h)
reg = Regexp.new('^(\w*)\:\/\/([\w\.]+)(\/.*)')
md = reg.match(h)
return nil unless md
return md[1],md[2],md[3]
end

host = 'slashdot.org'
server = TCPServer.new(local_host,local_port)
while true
Thread.start(server.accept) do |s|
header = ''
data = ''
# this is a little crude
header = s.gets # added
while s.gets && !$_.nil?
data += $_
break if $_ == "\r\n" || $_ == "\n"
end
header = header.split(' ') # added
proto,host,path = url_split(header[1]) # added
p "==== header[1]: #{header[1]}" # added
p "==== #{proto} #{host} #{path}" # added
p "====Received: #{data}"
net = Net::HTTP.new(host,80)
response = net.get(path) # changed
s << "HTTP/#{response.http_version}
#{response.code}/#{response.message} \r\n"
p "====Got back"
response.each_header do |key,value|
tmp = key + ": " + value + "\r\n"
tmp = key + ": " + value + "\r\n"
p tmp
s << tmp
end
p response.body
s << "\r\n" << response.body
s.close # We're done with this request, bring on the next.
end
end



Jim Freeze

4/22/2005 6:49:00 AM

0

* jm <jeffm@ghostgun.com> [2005-04-22 12:50:56 +0900]:

> Alright. this is wierd I just tried it with firefox on macosx and it
> rendered find, except for all the broken images, etc due to the
> reference to '/', ie

Uhh, this may be another dumb suggestion, but are you
by chance sending inspect data with #p instead of #puts
to the browser?

> p response.body
> s << "\r\n" << response.body
> s.close # We're done with this request, bring on the next.

--
Jim Freeze
Code Red. Code Ruby


Craig Moran

4/25/2005 1:06:00 PM

0

I am behind a proxy server at work, yet got this to work. I commented
out your Net::HTTP.new line and added what I believe to be the Proxy
equivalent. Then I set Mozilla Firefox's proxy to be localhost:8080.
NOTE: I do not have the ability to use another machine to test as I
am very restricted with net access here. I launched slashdot.org and
monitored the proxy.rb window while all of the HTML flew by. It came
up and rendered properly in the browser.

I am on Windows 2000
C:\>ruby -v
ruby 1.8.2 (2004-12-25) [i386-mswin32]

Here's the code. Hope it helps:

require 'socket'
require 'net/http'
local_host = 'localhost'
local_port = 8080

host = 'slashdot.org'
server = TCPServer.new(local_host, local_port)
while true
Thread.start(server.accept) do |s|
data = ''
while s.gets && !$_.nil?
data += $_
break if $_ == "\r\n" || $_ == "\n"
end
p "====Received: #{data}"
# net = Net::HTTP.new(host,80)
net = Net::HTTP::Proxy("proxy", "80").start(host)
response = net.get('/')
s <<
"HTTP/#{response.http_version}#{response.code}/#{response.message}
\r\n"
p "====Got back"
response.each_header do |key,value|
tmp = key + ": " + value + "\r\n"
p tmp
s << tmp
end
p response.body
s << "\r\n" << response.body
s.close # We're done with this request, bring on the next.
end
end

On 4/21/05, Tanner Burson <tanner.burson@gmail.com> wrote:
> I've been working on a proxy server implementation for a project idea
> I have. But I've found that it only seems to work on certain pages,
> and not on others. I've distilled the problem down to a simple
> implementation, but it still occurs. The code below will return a
> (mostly) correct website if pointed at 'www.msn.com' but if pointed at
> say 'slashdot.org' the browser refuses to display it, in both cases
> the page is loaded correctly. If anyone has any ideas I'd really
> appreciate it.
>
> ===proxy.rb
>
> require 'socket'
> require 'net/http'
> local_host = '192.168.0.6'
> local_port = 8080
>
> host = 'slashdot.org'
> server = TCPServer.new(local_host,local_port)
> while true
> Thread.start(server.accept) do |s|
> data = ''
> while s.gets && !$_.nil?
> data += $_
> break if $_ == "\r\n" || $_ == "\n"
> end
> p "====Received: #{data}"
> net = Net::HTTP.new(host,80)
> response = net.get('/')
> s << "HTTP/#{response.http_version}
> #{response.code}/#{response.message} \r\n"
> p "====Got back"
> response.each_header do |key,value|
> tmp = key + ": " + value + "\r\n"
> p tmp
> s << tmp
> end
> p response.body
> s << "\r\n" << response.body
> s.close # We're done with this request, bring on the next.
> end
> end
>
> ====end
> --
> ===Tanner Burson===
> tanner.burson@gmail.com
> http://tanner... <---Might even work one day...
>
>