[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

Yet another Hpricot question

Rick DeNatale

10/11/2006 5:13:00 PM

I'm trying to scan an html file using Hpricot to produce a table of
links within the file.

Right now I've got something like this.

doc = Hpricot(open(url)).
doc.search('a').each do | element |
puts "#{element.inner_html}
puts " #{element.attributes['href']
end

This works, but in this document some of the a tags use markup on
their contents. Something like
<a href="http://blah.or..."><... blah</b> blah</a>

I'd like to strip out the markup tags so that I'd get

blah blah blah
http://blah.or...

Is there some way to search for or iterate over the leaf elements of
the tree rooted by an element in Hpricot?

--
Rick DeNatale

My blog on Ruby
http://talklikeaduck.denh...
5 Answers

Aaron Patterson

10/11/2006 5:28:00 PM

0

On Thu, Oct 12, 2006 at 02:13:07AM +0900, Rick DeNatale wrote:
> I'm trying to scan an html file using Hpricot to produce a table of
> links within the file.
>
> Right now I've got something like this.
>
> doc = Hpricot(open(url)).
> doc.search('a').each do | element |
> puts "#{element.inner_html}
> puts " #{element.attributes['href']
> end
>
> This works, but in this document some of the a tags use markup on
> their contents. Something like
> <a href="http://blah.or..."><... blah</b> blah</a>
>
> I'd like to strip out the markup tags so that I'd get
>
> blah blah blah
> http://blah.or...
>
> Is there some way to search for or iterate over the leaf elements of
> the tree rooted by an element in Hpricot?

I had to do something similar in Mechanize, and this is what I came up
with:

class Hpricot::Elem
def all_text
text = ''
children.each do |child|
if child.respond_to? :content
text << child.content
end
if child.respond_to? :all_text
text << child.all_text
end
end
text
end
end

doc = Hpricot("<a href=\"http://blah.or...\"><b>blah blah</b> blah</a>")
doc.search('a').each do |e|
puts "#{e.all_text}"
puts " #{e.attributes['href']}"
end

Hope that helps!

--Aaron

--
Aaron Patterson
http://tenderlovem...

Gregory Seidman

10/11/2006 5:50:00 PM

0

On Thu, Oct 12, 2006 at 02:28:13AM +0900, Aaron Patterson wrote:
} On Thu, Oct 12, 2006 at 02:13:07AM +0900, Rick DeNatale wrote:
} > I'm trying to scan an html file using Hpricot to produce a table of
} > links within the file.
} >
} > Right now I've got something like this.
} >
} > doc = Hpricot(open(url)).
} > doc.search('a').each do | element |
} > puts "#{element.inner_html}
} > puts " #{element.attributes['href']
} > end
} >
} > This works, but in this document some of the a tags use markup on
} > their contents. Something like
} > <a href="http://blah.or..."><... blah</b> blah</a>
} >
} > I'd like to strip out the markup tags so that I'd get
} >
} > blah blah blah
} > http://blah.or...
} >
} > Is there some way to search for or iterate over the leaf elements of
} > the tree rooted by an element in Hpricot?
}
} I had to do something similar in Mechanize, and this is what I came up
} with:
}
} class Hpricot::Elem
} def all_text
} text = ''
} children.each do |child|
} if child.respond_to? :content
} text << child.content
} end
} if child.respond_to? :all_text
} text << child.all_text
} end
} end
} text
} end
} end
}
} doc = Hpricot("<a href=\"http://blah.or...\"><b>blah blah</b> blah</a>")
} doc.search('a').each do |e|
} puts "#{e.all_text}"
} puts " #{e.attributes['href']}"
} end

There is a simpler implementation of all_text:

class Hpricot::Elem
def all_text
text = ''
traverse_text {|t| text << t.content }
text
end
end

} Hope that helps!
} --Aaron
--Greg

Rick DeNatale

10/12/2006 1:30:00 PM

0

On 10/11/06, Gregory Seidman <gsslist+ruby@anthropohedron.net> wrote:
> On Thu, Oct 12, 2006 at 02:28:13AM +0900, Aaron Patterson wrote:
> } On Thu, Oct 12, 2006 at 02:13:07AM +0900, Rick DeNatale wrote:
> } > I'm trying to scan an html file using Hpricot to produce a table of
> } > links within the file.
> } >
> } > Right now I've got something like this.
> } >
> } > doc = Hpricot(open(url)).
> } > doc.search('a').each do | element |
> } > puts "#{element.inner_html}
> } > puts " #{element.attributes['href']
> } > end
> } >
> } > This works, but in this document some of the a tags use markup on
> } > their contents. Something like
> } > <a href="http://blah.or..."><... blah</b> blah</a>
> } >
> } > I'd like to strip out the markup tags so that I'd get
> } >
> } > blah blah blah
> } > http://blah.or...
> } >
> } > Is there some way to search for or iterate over the leaf elements of
> } > the tree rooted by an element in Hpricot?
> }
> } I had to do something similar in Mechanize, and this is what I came up
> } with:
> }
> } class Hpricot::Elem
> } def all_text
> } text = ''
> } children.each do |child|
> } if child.respond_to? :content
> } text << child.content
> } end
> } if child.respond_to? :all_text
> } text << child.all_text
> } end
> } end
> } text
> } end
> } end
> }
> } doc = Hpricot("<a href=\"http://blah.or...\"><b>blah blah</b> blah</a>")
> } doc.search('a').each do |e|
> } puts "#{e.all_text}"
> } puts " #{e.attributes['href']}"
> } end
>
> There is a simpler implementation of all_text:
>
> class Hpricot::Elem
> def all_text
> text = ''
> traverse_text {|t| text << t.content }
> text
> end
> end
>
> } Hope that helps!
> } --Aaron
> --Greg
>
>
>

Thanks Aaron and Greg, works a treat!

--
Rick DeNatale

My blog on Ruby
http://talklikeaduck.denh...

why the lucky stiff

10/12/2006 5:38:00 PM

0

On Thu, Oct 12, 2006 at 10:29:54PM +0900, Rick DeNatale wrote:
> >There is a simpler implementation of all_text:
> >
> >class Hpricot::Elem
> > def all_text
> > text = ''
> > traverse_text {|t| text << t.content }
> > text
> > end
> >end
>
> Thanks Aaron and Greg, works a treat!

If three of you have independantly used this, let's check it in. Elements#text
which parallels Elements#html (I think jQuery also has this) and Elem#inner_text
as well.

_why

Gregory Seidman

10/12/2006 10:52:00 PM

0

On Fri, Oct 13, 2006 at 02:38:10AM +0900, why the lucky stiff wrote:
} On Thu, Oct 12, 2006 at 10:29:54PM +0900, Rick DeNatale wrote:
} > >There is a simpler implementation of all_text:
} > >
} > >class Hpricot::Elem
} > > def all_text
} > > text = ''
} > > traverse_text {|t| text << t.content }
} > > text
} > > end
} > >end
} >
} > Thanks Aaron and Greg, works a treat!
}
} If three of you have independantly used this, let's check it in.
} Elements#text which parallels Elements#html (I think jQuery also has
} this) and Elem#inner_text as well.

Actually, I haven't used it. I just knew how to go about it. What I would
find much more useful is an #inject_text (and a corresponding
#inject_elements, though that isn't as important since the / notation
retrieves an Enumerable).

} _why
--Greg