[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

Hpricot syntax different from Xpath ?

Celine

12/18/2007 10:00:00 PM

Hi all

I'm trying to parse a page with Hpricot in order to retrieve a value.

I use Xpather (a firefox extension) in order to get the path of this
value. But when I use this path with Hpricot, it doesn't work. I have
to change it so that it works.

Here's my path, given by Xpather :

/html/body/div[1]/div[2]/div[1]/div[2]/div[1]/div[2]/div[1]/div/div[1]/
div[1]/table/tbody/tr[1]


And here's what I have to write in order to make it understand by
Hpricot :

/html/body/div/div/div/div/div/div/div/div/div/div/table/tr


Could you explain me why I have to write that ?

Thanks in advance
13 Answers

Chris Shea

12/18/2007 10:21:00 PM

0

On Dec 18, 2007 3:04 PM, Celine <xhanrot@gmail.com> wrote:
> Hi all
>
> I'm trying to parse a page with Hpricot in order to retrieve a value.
>
> I use Xpather (a firefox extension) in order to get the path of this
> value. But when I use this path with Hpricot, it doesn't work. I have
> to change it so that it works.
>
> Here's my path, given by Xpather :
>
> /html/body/div[1]/div[2]/div[1]/div[2]/div[1]/div[2]/div[1]/div/div[1]/
> div[1]/table/tbody/tr[1]
>
>
> And here's what I have to write in order to make it understand by
> Hpricot :
>
> /html/body/div/div/div/div/div/div/div/div/div/div/table/tr
>
>
> Could you explain me why I have to write that ?

Well, it depends. It'd be helpful to see the page you're working with.
You might want to try asking the Hpricot mailing list as well (To
join: Send a message to hpricot@code.whytheluckystiff.net Cc:
why@whytheluckystiff.net).

Chris

Thomas Wieczorek

12/18/2007 10:25:00 PM

0

On Dec 18, 2007 11:04 PM, Celine <xhanrot@gmail.com> wrote:
> Hi all
>
> I'm trying to parse a page with Hpricot in order to retrieve a value.
>
> I use Xpather (a firefox extension) in order to get the path of this
> value. But when I use this path with Hpricot, it doesn't work. I have
> to change it so that it works.
>
> Here's my path, given by Xpather :
>
> /html/body/div[1]/div[2]/div[1]/div[2]/div[1]/div[2]/div[1]/div/div[1]/
> div[1]/table/tbody/tr[1]
>

HPricot doesn't include the whole XPath syntax. You can write a little
function which translates XPath expressions with brackets to HPricot
expressions. I wrote a function for that, but my SVN is right now down
and I can't get it. Drop an answer if you still need it
Firefox includes some missing HTML tags. I ran in it, when I had to
write a little script. <tbody> is added in <table>, probably some more
things, but I didn't find them. You can see the difference when you
download the page with open-uri. Not many pages add <tbody>.

Celine

12/18/2007 10:34:00 PM

0

On Dec 18, 11:21 pm, Chris Shea <ch...@ruby.tie-rack.org> wrote:

>
> Well, it depends. It'd be helpful to see the page you're working with.
> You might want to try asking the Hpricot mailing list as well (To
> join: Send a message to hpri...@code.whytheluckystiff.net Cc:
> w...@whytheluckystiff.net).
>
> Chris

Hi Chris, thanks for your answer
Here is the page I'm working with : http://finance...
I want to retrieve value of Nasdaq (up left of the page).
I sent the same message on Hpricot ML this afternoon, actually no
answer.

Celine

12/18/2007 10:39:00 PM

0

On Dec 18, 11:25 pm, Thomas Wieczorek <wieczo...@googlemail.com>
wrote:
> On Dec 18, 2007 11:04 PM, Celine <xhan...@gmail.com> wrote:
>
> > Hi all
>
> > I'm trying to parse a page with Hpricot in order to retrieve a value.
>
> > I use Xpather (a firefox extension) in order to get the path of this
> > value. But when I use this path with Hpricot, it doesn't work. I have
> > to change it so that it works.
>
> > Here's my path, given by Xpather :
>
> > /html/body/div[1]/div[2]/div[1]/div[2]/div[1]/div[2]/div[1]/div/div[1]/
> > div[1]/table/tbody/tr[1]
>
> HPricot doesn't include the whole XPath syntax. You can write a little
> function which translates XPath expressions with brackets to HPricot
> expressions. I wrote a function for that, but my SVN is right now down
> and I can't get it. Drop an answer if you still need it
> Firefox includes some missing HTML tags. I ran in it, when I had to
> write a little script. <tbody> is added in <table>, probably some more
> things, but I didn't find them. You can see the difference when you
> download the page with open-uri. Not many pages add <tbody>.

Hi Thomas

I'm very interested in your function. Do you know where I can find
differences between XPath syntax and Hpricot syntax ? What reference
did you use to write your function ?

Celine

Chris Shea

12/18/2007 10:44:00 PM

0

On Dec 18, 2007 3:34 PM, Celine <xhanrot@gmail.com> wrote:
> On Dec 18, 11:21 pm, Chris Shea <ch...@ruby.tie-rack.org> wrote:
>
> >
> > Well, it depends. It'd be helpful to see the page you're working with.
> > You might want to try asking the Hpricot mailing list as well (To
> > join: Send a message to hpri...@code.whytheluckystiff.net Cc:
> > w...@whytheluckystiff.net).
> >
> > Chris
>
> Hi Chris, thanks for your answer
> Here is the page I'm working with : http://finance...
> I want to retrieve value of Nasdaq (up left of the page).

I see. It's pretty easy to get using element attributes, which
resilient to page changes. If Yahoo decides to add a div in the
hierarchy, or add a new exchange, this shouldn't suddenly fail:

# assuming doc is the Hpricot object for finance.yahoo.com
doc.at('tr[@title="Nasdaq"]/td[2]')

Looking at the page source, the span element that contains the value
actually has an id (yfs_l10_^ixic), but it doesn't look stable, does
it?

> I sent the same message on Hpricot ML this afternoon, actually no
> answer.

I'm on the Hpricot ML and never saw it. Maybe a hiccup somewhere?

HTH,
Chris

Chris Shea

12/18/2007 10:55:00 PM

0

> > On Dec 18, 11:21 pm, Chris Shea <ch...@ruby.tie-rack.org> wrote:
> Looking at the page source, the span element that contains the value
> actually has an id (yfs_l10_^ixic), but it doesn't look stable, does
> it?

I take that back. That id is almost definitely stable.

doc.at('span[@id="yfs_l10_^ixic"]')

Chris

Thomas Wieczorek

12/18/2007 11:36:00 PM

0

On Dec 18, 2007 11:39 PM, Celine <xhanrot@gmail.com> wrote:
> On Dec 18, 11:25 pm, Thomas Wieczorek <wieczo...@googlemail.com>
> wrote:
>

>
> I'm very interested in your function.
>

I'll post it as soon as the SVN server is up again.

> Do you know where I can find
> differences between XPath syntax and Hpricot syntax ? What reference
> did you use to write your function ?
>

I used http://code.whytheluckystiff.net/hpricot/wiki/SupportedXpathE...
and related pages to get started with it. I found the table/tbody
thing because I didn't get any further with it and thought, that I did
something wrong until I downloaded the page without Firefox using
open-uri.

Vitor Peres

12/19/2007 9:54:00 AM

0

[Note: parts of this message were removed to make it a legal post.]

On Dec 18, 2007 8:34 PM, Celine <xhanrot@gmail.com> wrote:

> Hi Chris, thanks for your answer
> Here is the page I'm working with : http://finance...
> I want to retrieve value of Nasdaq (up left of the page).
> I sent the same message on Hpricot ML this afternoon, actually no
> answer.
>
>
Hi, Celine.

I know it's not nearly as fun as screen-scraping, but you can get the value
for Nasdaq (and many other quotes) on Yahoo! Finance by querying the right
URL for the CSV. The current value can be obtained by fetching:

http://download.finance.yahoo.com/d/quo...[name]&f=sl1d1t1c1ohgv&e=.csv

You just have replace [name] with %5EIXIC for Nasdaq. Historical data is
available (closings only) at:

http://ichart.finance.yahoo.com/table.c...[<http://ichart.finance.yahoo.com/table.c...%5Bquote>name]&a=[start
month]&b=[start_day]&c=[start
_year]&d=[end_month]&e=[end_day]&f=[end_year]&g=d&ignore=.csv

Just replace [name] with the index or stock you wish to query and each
bracketed date info with integers.

I've replied to a topic before that involved Yahoo! Finance, but it was
specifically about searching for a symbol. Since it's not your case, here's
hoping that directly fetching it will suffice.


--
Vitor Peres (dodecaphonic)
------------------------------------
http://twitter.com/do...

Celine

12/19/2007 9:41:00 PM

0

On 18 déc, 23:54, Chris Shea <ch...@ruby.tie-rack.org> wrote:
> > > On Dec 18, 11:21 pm, Chris Shea <ch...@ruby.tie-rack.org> wrote:
> > Looking at the page source, the span element that contains the value
> > actually has an id (yfs_l10_^ixic), but it doesn't look stable, does
> > it?
>
> I take that back. That id is almost definitely stable.
>
> doc.at('span[@id="yfs_l10_^ixic"]')
>
> Chris

Yes, thanks, it runs.
There's something I can't understand : in the Xpath expression I
posted later, when a node has several child DIVs, I access them with
an index (div[2]...), but in Hpricot syntax, DIVs aren't accessed
using an index. So, what trick Hpricot uses to locate "the good" div ?

Celine

12/19/2007 9:42:00 PM

0

On 19 déc, 10:53, Vitor Peres <dodecapho...@gmail.com> wrote:
>
> I know it's not nearly as fun as screen-scraping, but you can get the value
> for Nasdaq (and many other quotes) on Yahoo! Finance by querying the right
> URL for the CSV. The current value can be obtained by fetching:
>
> http://download.finance.yahoo.com/d/quo...[name]&f=sl1d1t1c1ohgv&e=.csv
>
> You just have replace [name] with %5EIXIC for Nasdaq. Historical data is
> available (closings only) at:
>
> http://ichart.finance.yahoo.com/table.c...[<http://ichart.finance.yahoo.com/table.c...%5Bquote>name]&a=[start
> month]&b=[start_day]&c=[start
> _year]&d=[end_month]&e=[end_day]&f=[end_year]&g=d&ignore=.csv
>
> Just replace [name] with the index or stock you wish to query and each
> bracketed date info with integers.
>
> I've replied to a topic before that involved Yahoo! Finance, but it was
> specifically about searching for a symbol. Since it's not your case, here's
> hoping that directly fetching it will suffice.
>
> --
> Vitor Peres (dodecaphonic)
> ------------------------------------http://twitter.com/do...

Hi Victor, thank you very much :)
But, as you said, it isn't very funny, no ? ;)
(but I didn't know that trick, thanks)