[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

Question about Mechanize and Google API

Jesús Gabriel y Galán

10/8/2007 3:49:00 PM

Hi,

I'm trying to use Mechanize to retrieve spreadsheets from Google Docs
using the Google API. So far I got the ClientLogin to work, but I
can't seem to be able to use the Auth token they give me back. This is
what I have:

require 'logger'
require 'mechanize'

agent = WWW::Mechanize.new {|a| a.log = Logger.new("test.log")}
response = agent.post("https://www.google.com/accounts/ClientL...,
"Email" => "xxx@gmail.com", "Passwd" => "xxxxx", "service" => "wise",
"source" => "personal-test-1.0", "accountType" => "GOOGLE")
h = {}
response.body.each {|line| a = line.split("="); h[a[0]] = a[1]}
auth = h["Auth"]
agent.basic_auth("GoogleLogin", auth)
# this didn't work either
#agent.auth("GoogleLogin", auth)
page = agent.get("http://spreadsheets.google.com/feeds/spreadsheets/private/...)

The problem is, according to google I have to:
"After a successful authentication request, use the Auth value to
create an Authorization header for each request:

Authorization: GoogleLogin auth=yourAuthValue"

I've checked and the ClientLogin is working and returning me an Auth
token, but I can't get Mechanize to use it. In the Mechanize logs I
see this:
I, [2007-10-08T17:01:52.211778 #24929] INFO -- : Net::HTTP::Post:
/accounts/ClientLogin
D, [2007-10-08T17:01:52.394047 #24929] DEBUG -- : request-header:
accept-language => en-us,en;q0.5
D, [2007-10-08T17:01:52.394214 #24929] DEBUG -- : request-header:
connection => keep-alive
D, [2007-10-08T17:01:52.394305 #24929] DEBUG -- : request-header: accept => */*
D, [2007-10-08T17:01:52.394394 #24929] DEBUG -- : request-header:
accept-encoding => gzip,identity
D, [2007-10-08T17:01:52.394482 #24929] DEBUG -- : request-header:
user-agent => WWW-Mechanize/0.6.10
(http://rubyforge.org/projects/...)
D, [2007-10-08T17:01:52.394570 #24929] DEBUG -- : request-header:
content-type => application/x-www-form-urlencoded
D, [2007-10-08T17:01:52.394656 #24929] DEBUG -- : request-header:
accept-charset => ISO-8859-1,utf-8;q=0.7,*;q=0.7
D, [2007-10-08T17:01:52.394742 #24929] DEBUG -- : request-header:
content-length => 104
D, [2007-10-08T17:01:52.394828 #24929] DEBUG -- : request-header:
keep-alive => 300
D, [2007-10-08T17:01:52.613138 #24929] DEBUG -- : Read 563 bytes
D, [2007-10-08T17:01:52.613358 #24929] DEBUG -- : response-header:
cache-control => no-cache
D, [2007-10-08T17:01:52.613483 #24929] DEBUG -- : response-header:
date => Mon, 08 Oct 2007 15:00:17 GMT
D, [2007-10-08T17:01:52.613603 #24929] DEBUG -- : response-header:
content-type => text/plain
D, [2007-10-08T17:01:52.613721 #24929] DEBUG -- : response-header:
server => GFE/1.3
D, [2007-10-08T17:01:52.613837 #24929] DEBUG -- : response-header:
content-length => 563
D, [2007-10-08T17:01:52.613969 #24929] DEBUG -- : response-header:
pragma => no-cache
I, [2007-10-08T17:01:52.614363 #24929] INFO -- : status: 200
I, [2007-10-08T17:13:15.370472 #24929] INFO -- : Net::HTTP::Get:
/feeds/spreadsheets/private/full
D, [2007-10-08T17:13:15.482357 #24929] DEBUG -- : request-header:
accept-language => en-us,en;q0.5
D, [2007-10-08T17:13:15.482481 #24929] DEBUG -- : request-header:
connection => keep-alive
D, [2007-10-08T17:13:15.482534 #24929] DEBUG -- : request-header: accept => */*
D, [2007-10-08T17:13:15.482587 #24929] DEBUG -- : request-header:
accept-encoding => gzip,identity
D, [2007-10-08T17:13:15.482639 #24929] DEBUG -- : request-header:
user-agent => WWW-Mechanize/0.6.10
(http://rubyforge.org/projects/...)
D, [2007-10-08T17:13:15.482692 #24929] DEBUG -- : request-header:
referer => https://www.google.com/accounts/C...
D, [2007-10-08T17:13:15.482745 #24929] DEBUG -- : request-header:
accept-charset => ISO-8859-1,utf-8;q=0.7,*;q=0.7
D, [2007-10-08T17:13:15.482796 #24929] DEBUG -- : request-header:
keep-alive => 300
D, [2007-10-08T17:13:15.652098 #24929] DEBUG -- : Read 141 bytes
D, [2007-10-08T17:13:15.652313 #24929] DEBUG -- : response-header:
cache-control => no-cache, no-store, max-age=0, must-revalidate
D, [2007-10-08T17:13:15.652438 #24929] DEBUG -- : response-header:
expires => Fri, 01 Jan 1990 00:00:00 GMT
D, [2007-10-08T17:13:15.652604 #24929] DEBUG -- : response-header:
date => Mon, 08 Oct 2007 15:11:40 GMT
D, [2007-10-08T17:13:15.652725 #24929] DEBUG -- : response-header:
content-type => text/html; charset=UTF-8
D, [2007-10-08T17:13:15.652859 #24929] DEBUG -- : response-header:
www-authenticate => No credentials were included in your request.
D, [2007-10-08T17:13:15.652977 #24929] DEBUG -- : response-header:
server => GFE/1.3
D, [2007-10-08T17:13:15.653094 #24929] DEBUG -- : response-header:
set-cookie => S=trix=Xc7oom0LVJ8; Path=/
D, [2007-10-08T17:13:15.653211 #24929] DEBUG -- : response-header:
content-length => 141
D, [2007-10-08T17:13:15.653326 #24929] DEBUG -- : response-header:
pragma => no-cache
D, [2007-10-08T17:13:15.659559 #24929] DEBUG -- : saved cookie:
S=trix=Xc7oom0LVJ8
I, [2007-10-08T17:13:15.659765 #24929] INFO -- : status: 404

So, it seems that the credentials are not correctly set. Anybody knows
how to do this?

Thanks,

Jesus.
4 Answers

Richard Conroy

10/8/2007 4:24:00 PM

0

On 10/8/07, Jesús Gabriel y Galán <jgabrielygalan@gmail.com> wrote:
> Hi,
>
> I'm trying to use Mechanize to retrieve spreadsheets from Google Docs
> using the Google API. So far I got the ClientLogin to work, but I
> can't seem to be able to use the Auth token they give me back. This is
> what I have:

From my experience Mechanize is a bit low level, especially in this area.
You are expected to write the code yourself to handle, redirects,
authentication,
SSL etc.

You might want to try open-uri or Hpricot as well. But I think there is no
right-sized solution - you will still need to do a bit of code writing yourself.

BTW - you should check out the new RESTful Web Services book (O'Reilly).
In addition to just being an excellent book, it has a lot of practical advice
on HTTP clients and directly manipulating HTTP headers, which is I
suspect what you will ultimately need to do.

Jesús Gabriel y Galán

10/8/2007 7:21:00 PM

0

On 10/8/07, Matias Pablo Brutti <matiasbrutti@gmail.com> wrote:

> the Gdata gem works with the google Gdata API , I'm currently implementing
> the gcalendar API for that gem , but luckely for you, I think the
> 0.4version has some spreadsheet implementation.

Hi, yes, I had checked the Gdata gem, but saw it was pre alpha. In any
case I installed it and checked the spreadsheet class but I just saw a
couple of methods to demo how to evaluate a cell, and not the complete
functionality of the Google API. Am I right?

I'm thinking of stop using Mechanize for this and use just net/http
for the simple post and get I need to do, so I can customize the
headers myself, as I've seen you do in the gdata classes.

Thanks,

Jesus.

mortee

10/8/2007 10:53:00 PM

0

Jesús Gabriel y Galán

10/9/2007 7:33:00 AM

0

On 10/9/07, mortee <mortee.lists@kavemalna.hu> wrote:
> Richard Conroy wrote:
> > From my experience Mechanize is a bit low level, especially in this area.
> > You are expected to write the code yourself to handle, redirects,
> > authentication,
> > SSL etc.
> >
> > You might want to try open-uri or Hpricot as well. But I think there is no
> > right-sized solution - you will still need to do a bit of code writing yourself.
>
> Just for the record, last time I checked, Mechanize *did* use Hpricot to
> parse pages. AFAIK, you can even provide Mechainze with your own page
> parser class before fetching the stuff.

Yes, Mechanize parses pages with Hpricot, and also allows you to use
an Hpricot-like API on them for search:

agent.get('http://rubyforg...).search("//strong")

The problem I was having with it is the fact that I didn't know how to
specify arbitrary headers for my request, or make the Google
authentication stuff to work.

Working directly with net/https worked for me (code "inspired" by the
Gdata gem :-):

require 'net/http'
require 'net/https'
require 'uri'
require 'hpricot'

module Net
class HTTPS < HTTP
def initialize(address, port = nil)
super(address, port)
self.use_ssl = true
end
end
end


h = {"Email" => "xxxxx@gmail.com",
"Passwd" => "xxxxx",
"service" => "wise",
"source" => "personal-test-1.0",
"accountType" => "GOOGLE"
}

url = URI.parse('https://www.google.com/accounts/Client...)
response = Net::HTTPS.post_form(url, h)
auth = ""
response.body.each do |line|
if line =~ /Auth=/
auth = line.split("=")[1].chomp
end
end

headers = {"Authorization" => "GoogleLogin auth=#{auth}"}
http = Net::HTTP.new("spreadsheets.google.com", 80)
response, data = http.get("/feeds/spreadsheets/private/full", headers)
document = Hpricot(data)
document.search("//entry").each do |entry|
p entry
end

This gives you a document with the list of private spreadsheets of the
user. Then you can use Hpricot or whatever you want to parse the atom
feed.

Jesus.