Asp Forum - [ANN] FasterCSV 0.1.6 -- With Header Support!

James Gray

2/26/2006 1:18:00 AM

FasterCSV 0.1.6 Released
========================

The first couple of releases brought raw speed and plenty of it.
This release puts all those spare cycles to work with a ton of new
data-centric features.

Want to access you CSV files by the names of the header rows or get
back real data objects instead of just Strings? Then this release
has what you need.

This release also fixes the number one complaint with automatic line
ending detection (now the default).

What is FasterCSV?
------------------

(from the README)

FasterCSV is intended as a replacement to Ruby's standard CSV
library. It was designed to address concerns users of that library
had and it has three primary goals:

1. Be significantly faster than CSV while remaining a pure Ruby
library.
2. Use a smaller and easier to maintain code base. (We're about
even now,
but not if you compare the features!)
3. Improve on the CSV interface.

What's New?
-----------

(highlights from the CHANGELOG)

* Added built-in and custom data converters. Built-in handle numbers
and dates.
* Added Array#to_csv and String#parse_csv. Both accept normal options.
* Added auto-discovery for <tt>:row_sep</tt> (now the default).
* Added FasterCSV::filter() for easy Unix-like CSV filters.
* Added support for accessing fields by headers.
* Headers can have their own converters.
* Headers can be skipped or returned as needed.
* FasterCSV::Row allows index or header access while retaining
order and
allowing for duplicate headers.

Migrating from CSV to FasterCSV?
--------------------------------

The README includes a section on the differences and you can read
that here:

http://fastercsv.ruby...

You call also see general usage in the documentation of the
interface, right here:

http://fastercsv.ruby...classes/FasterCSV.html

If FasterCSV isn't meeting your needs, I want to here about it:

james@grayproductions.net

Where can I learn more?
-----------------------

FasterCSV is hosted on RubyForge.

Project page: http://rubyforge.org/projects/...
Documentation: http://fastercsv.ruby...
Downloads: http://rubyforge.org/frs/?gro...

How do I get FasterCSV?
-----------------------

FasterCSV is a gem, so as long as you have RubyGems installed it's as
simple as:

$ sudo gem install fastercsv

If you need to install RubyGems, you can download it from:

http://rubyforge.org/frs/?group_id=126&relea...

FasterCSV can also be installed manually. Just download the latest
release and follow the instructions in INSTALL:

http://rubyforge.org/frs/?gro...&release_id=4438

James Edward Gray II

25 Answers

Gregory Brown

2/26/2006 1:22:00 AM

On 2/25/06, James Edward Gray II <james@grayproductions.net> wrote:
> FasterCSV 0.1.6 Released
> ========================
>
> The first couple of releases brought raw speed and plenty of it.
> This release puts all those spare cycles to work with a ton of new
> data-centric features.
>
> Want to access you CSV files by the names of the header rows or get
> back real data objects instead of just Strings? Then this release
> has what you need.
>
> This release also fixes the number one complaint with automatic line
> ending detection (now the default).

Cool James! I just ran the units on Ruport, for which you are an
(optional) dependency. Your new release didn't break anything :)

James Gray

2/26/2006 5:26:00 AM

On Feb 25, 2006, at 7:17 PM, James Edward Gray II wrote:

> FasterCSV 0.1.6 Released
> ========================

I was in such a hurry to get this out (was almost late to the
symphony!), I almost forgot the most important part! The features in
this release were partially funded by B-Tree Technology and Stone
Code Productions. How cool is that to get paid to right free
software? Thanks guys!

James Edward Gray II

Petite Abeille

2/26/2006 1:41:00 PM

On Feb 26, 2006, at 02:17, James Edward Gray II wrote:

> 1. Be significantly faster than CSV while remaining a pure Ruby
> library.

Couldn't resist the temptation to check how much faster 8^)

Here are some numbers parsing ip-to-country.csv (3MB, 63726 lines)[1].

[ruby csv]
% ruby -v
ruby 1.8.4 (2005-12-24) [powerpc-darwin7.9.0]
% /usr/bin/time ruby TestCSV.rb
79.06 real 74.42 user 0.30 sys

[ruby faster_csv]
% ruby -v
ruby 1.8.4 (2005-12-24) [powerpc-darwin7.9.0]
% /usr/bin/time ruby TestFasterCSV.rb
7.56 real 7.16 user 0.19 sys

[lua LUCSV][2][3]
% lua -v
Lua 5.1 Copyright (C) 1994-2006 Lua.org, PUC-Rio
% /usr/bin/time lua TestCSV.lua
2.79 real 2.45 user 0.09 sys

[python csv][4][5]
% python -V
Python 2.4.2
% /usr/bin/time python TestCSV.py
0.85 real 0.79 user 0.04 sys

Cheers

--
PA, Onnay Equitursay
http://alt.text...

[1] http://ip-to-country.webhosting.info/n...
[2] http://www.lua.org/...
[3] http://dev.alt.textdrive.com/browser/lu...
[4] http://www.python.org/doc/2.4.2/lib/modul...
[5] Python provides a C implementation of the CSV parser.

James Gray

2/27/2006 4:20:00 PM

On Feb 25, 2006, at 7:22 PM, Gregory Brown wrote:

> On 2/25/06, James Edward Gray II <james@grayproductions.net> wrote:
>> FasterCSV 0.1.6 Released
>> ========================
>>
>> The first couple of releases brought raw speed and plenty of it.
>> This release puts all those spare cycles to work with a ton of new
>> data-centric features.
>>
>> Want to access you CSV files by the names of the header rows or get
>> back real data objects instead of just Strings? Then this release
>> has what you need.
>>
>> This release also fixes the number one complaint with automatic line
>> ending detection (now the default).
>
> Cool James! I just ran the units on Ruport, for which you are an
> (optional) dependency. Your new release didn't break anything :)

That's either a miracle or a sign of poor test coverage, because I
broke quite a bit. :)

Following my annoying new habit of release-then-make-it-work,
FasterCSV 0.1.8 is now out, and resolves all of the issues found so
far...

Sorry about the hassle.

James Edward Gray II

Gregory Brown

2/27/2006 6:22:00 PM

On 2/27/06, James Edward Gray II <james@grayproductions.net> wrote:
> On Feb 25, 2006, at 7:22 PM, Gregory Brown wrote:

> > Cool James! I just ran the units on Ruport, for which you are an
> > (optional) dependency. Your new release didn't break anything :)
>
> That's either a miracle or a sign of poor test coverage, because I
> broke quite a bit. :)

It's a result of simple needs. I only call two FasterCSV functions.

> Following my annoying new habit of release-then-make-it-work,
> FasterCSV 0.1.8 is now out, and resolves all of the issues found so
> far...

Still passing tests on 0.1.8, FYI

Is this release-then-make-it-work an extension of your svn/cvs double
commit habit?
;)

Brian Moelk

2/27/2006 6:35:00 PM

I've found some discussions about support for Windows CE devices
(http://tinyurl...), I was wondering if there was any new information
regarding Windows Mobile 5.0 support?

Regards,
Brian Moelk

James Gray

2/27/2006 7:23:00 PM

On Feb 27, 2006, at 12:21 PM, Gregory Brown wrote:

> Is this release-then-make-it-work an extension of your svn/cvs double
> commit habit?
> ;)

Hey, that move is my unique signature. We all need at least one. ;)

Sadly, FasterCSV required one more release to get it working
everywhere: 0.1.9 is out now. :(

A big thanks to Michael Schoen for help in resolving all of these new
issues!

James Edward Gray II

Wilson Bilkovich

2/27/2006 8:08:00 PM

On 2/27/06, James Edward Gray II <james@grayproductions.net> wrote:
> On Feb 25, 2006, at 7:22 PM, Gregory Brown wrote:
>
> > On 2/25/06, James Edward Gray II <james@grayproductions.net> wrote:
> >> FasterCSV 0.1.6 Released
> >> ========================
> >>
> >> The first couple of releases brought raw speed and plenty of it.
> >> This release puts all those spare cycles to work with a ton of new
> >> data-centric features.
> >>
> >> Want to access you CSV files by the names of the header rows or get
> >> back real data objects instead of just Strings? Then this release
> >> has what you need.
> >>
> >> This release also fixes the number one complaint with automatic line
> >> ending detection (now the default).
> >
> > Cool James! I just ran the units on Ruport, for which you are an
> > (optional) dependency. Your new release didn't break anything :)
>
> That's either a miracle or a sign of poor test coverage, because I
> broke quite a bit. :)
>
> Following my annoying new habit of release-then-make-it-work,
> FasterCSV 0.1.8 is now out, and resolves all of the issues found so
> far...
>

Ruport has something like 3400 unit tests, if I recall, so I'm going
to cast my vote for 'miracle'. :)

Gregory Brown

2/28/2006 12:13:00 AM

On 2/27/06, Wilson Bilkovich <wilsonb@gmail.com> wrote:

> Ruport has something like 3400 unit tests, if I recall, so I'm going
> to cast my vote for 'miracle'. :)
>
>

34000+ assertions, almost all of them for Ruport::Parser. (Which are
the units from Parse::Input that James wrote! ;) )

Lest people say 'wow', there are only 60ish tests.

Test coverage went from near 1:1 to about 60% in the last release. :-/
However, the tests covering CSV/FasterCSV are so simple I'd be amazed
if they were faulty.

I'm releasing again either tonight or tomorrow with a large chunk of
code cleanup and a whole lot more tests...

I am making a promise that if anyone finds a problem in Ruport,from
now on I'll at least write a failing test cornering it, so um... let's
try to avoid miracles AND poor test coverage? ;)

Still passing here, despite the triple shot :)

Sascha Ebach

2/28/2006 12:43:00 AM

Hi James,

first, thanks for FasterCSV. It is very useful. I have been wildly using it
the last couple of days.

> Sadly, FasterCSV required one more release to get it working
> everywhere: 0.1.9 is out now. :(

Could you please make a couple of small examples of how each of those new
features is supposed to be used?

Another idea. I am no C expert. But maybe it is worth to do an optional
parser as a C module. Maybe with the help of RubyInline? Especially the
parse method would be a great target.

Saša Ebach

comp.lang.ruby

[ANN] FasterCSV 0.1.6 -- With Header Support!

James Gray

Gregory Brown

James Gray

Petite Abeille

James Gray

Gregory Brown

Brian Moelk

James Gray

Wilson Bilkovich

Gregory Brown

Sascha Ebach

x Login to ForumsZone