Asp Forum - Package, a future replacement for setup.rb and mkmf.rb

Christian Neukirchen

6/4/2005 10:59:00 PM

Hello,

During a discussion on IRC, I started to wonder if Ruby's install
scripts are state of the art, what could be done better and how.

Ruby's mkmf.rb and Aoki's setup.rb probably have their roots in the
oldest pieces of Ruby source still in use. While setup.rb had some
changes in the latter time, mkmf.rb more or less stayed the same.

I have looked into how other languages install source and compile
extensions, and the library I liked best so far is Python's distutils.
I'm not very familiar with Python, but I like the general approach and
the essence of API. Basically, you create a file, setup.py, like
this:

from distutils.core import setup

setup (name = "Distutils",
version = "0.1.1",
description = "Python Module Distribution Utilities",
author = "Greg Ward",
author_email = "gward@python.net",
url = "http://www.python.org/sigs/distutils-...,

packages = ['distutils', 'distutils.command'])

In Ruby, this would maybe look like that:

require 'package'

setup {
name "Distutils"
version "0.1.1"
description "Python Module Distribution Utilities"
author "Greg Ward"
author_email "gward@python.net"
url "http://www.python.org/sigs/distutils-...

packages ['distutils', 'distutils/command']
}

Given this file, we can simply run:

python setup.py install

and the files will get installed where they belong to. distutils can
also handle different prefixes, installing into home directories, and
complex cases like putting scripts to /usr/bin, but libraries to
/opt/local and whatever.

Python's distutils also handles compiling extensions:

name = 'DateTime.mxDateTime.mxDateTime'
src = 'mxDateTime/mxDateTime.c'
setup (
...
ext_modules =
[(name,
{ 'sources': [src]
'include_dirs': ['mxDateTime'] }
)]
)

Here, something like this would be possible in Ruby (I'm not yet sure
about exact semantics of the Python version):

setup {
# ...
extension("DateTime/mxDateTime/mxDateTime") {
sources "mxDateTime/mxDateTime.c"
include_dirs "mxDateTime"
}
}

Of course, more complex build descriptions can be represented too:

extension("foolib") {
sources "foo.c", "bar.c"
if have_library("foo", "fooble")
define "HAVE_FOO_H"
cflags << `foo-config --cflags`
ldflags << `foo-config --libs`
else
fail "foolib is needed"
end
}

Whether this will generate a Makefile (like mkmf.rb), a Rakefile
or compile directly (like distutils) is still an open question.

To allow for an easy conversion of setup.rb usage, Package will
provide convenience methods that will make it behave like setup.rb
with respect to the directory structure.

Package doesn't try to conquer the world, however, it aims to be just
a tool that would be useful if it was standard and everyone could
build on due to it's policy-neutrality

What advantages will Package have over setup.rb and mkmf.rb, as
they are now?

* simple, clean and consistent working
* unified library to handle both extensions and libraries
* lightweight approach (if included in the standard library)
* easy adaption
* more flexible directory layout: especially small projects
profit from this, as setup.rb's directory layout is quite
bulky by default and not very customizable
* easier packaging by third-party packagers due to simple
but flexible and standardized invocation

What do we need to get a wide adoption of Package?

* inclusion in the standard library so it doesn't need to be
shipped with every package (as setup.rb unfortunately is).
* backing from the community to make use of Package.
* acceptance from packaging projects like RPA, RubyGems and
distributions like Debian, FreeBSD and PLD.

Coding of Package has not started yet (the name is also not set into
stone yet, so if you have better ideas, please tell me) because it
would be pointless without a strong feedback from the community. I
expect to get a first version done rather quickly, possibly borrowing
code from setup.rb and mkmf.rb, but Package will not depend on these
both. If anyone is interested in helping development, please mail me;
helpful hands are always of use. Also, there will be need for testers
on all and even the most weird platforms.

But now, I'll ask you: Are you satisfied with the way installing Ruby
extensions and libraries works? Do you think there is a place for
Package? Do you have further improvements or can provide alternative
ideas?

--
Christian Neukirchen <chneukirchen@gmail.com> http://chneuk...

27 Answers

Austin Ziegler

6/4/2005 11:15:00 PM

On 6/4/05, Christian Neukirchen <chneukirchen@gmail.com> wrote:
> But now, I'll ask you: Are you satisfied with the way installing Ruby
> extensions and libraries works? Do you think there is a place for
> Package? Do you have further improvements or can provide alternative
> ideas?

I think that there is room for improvement, definitely. However, if
you're going to do extension support with this, it absolutely *must*
work perfectly well on Windows. It has to work better than setup.rb,
and setup.rb works mostly well for that. Gems, much less so.

-austin
--
Austin Ziegler * halostatue@gmail.com
* Alternate: austin@halostatue.ca

Jim Freeze

6/4/2005 11:43:00 PM

* Christian Neukirchen <chneukirchen@gmail.com> [2005-06-05 07:59:15 +0900]:

> In Ruby, this would maybe look like that:
>
> require 'package'
>
> setup {
> name "Distutils"
> version "0.1.1"
> description "Python Module Distribution Utilities"
> author "Greg Ward"
> author_email "gward@python.net"
> url "http://www.python.org/sigs/distutils-...
>
> packages ['distutils', 'distutils/command']
> }

If you have #name(arg) do the same thing as #name=(arg), how
does one get access to the instance variable?

> Package doesn't try to conquer the world, however, it aims to be just
> a tool that would be useful if it was standard and everyone could
> build on due to it's policy-neutrality

> What advantages will Package have over setup.rb and mkmf.rb, as
> they are now?

I find mkmf.rb very useful to create a Makefile when I am building a C wrapper
to some library. Will Package do the same thing, or will mkmf still
be needed?

--
Jim Freeze
Theory and practice are the same, in theory. -- Ryan Davis

ES

6/5/2005 12:23:00 AM

Le 4/6/2005, "Christian Neukirchen" <chneukirchen@gmail.com> a écrit:
>
>Hello,
>
>During a discussion on IRC, I started to wonder if Ruby's install
>scripts are state of the art, what could be done better and how.
>
>Ruby's mkmf.rb and Aoki's setup.rb probably have their roots in the
>oldest pieces of Ruby source still in use. While setup.rb had some
>changes in the latter time, mkmf.rb more or less stayed the same.
>
>I have looked into how other languages install source and compile
>extensions, and the library I liked best so far is Python's distutils.
>I'm not very familiar with Python, but I like the general approach and
>the essence of API. Basically, you create a file, setup.py, like
>this:
>
> from distutils.core import setup
>
> setup (name = "Distutils",
> version = "0.1.1",
> description = "Python Module Distribution Utilities",
> author = "Greg Ward",
> author_email = "gward@python.net",
> url = "http://www.python.org/sigs/distutils-...,
>
> packages = ['distutils', 'distutils.command'])
>
>In Ruby, this would maybe look like that:
>
> require 'package'
>
> setup {
> name "Distutils"
> version "0.1.1"
> description "Python Module Distribution Utilities"
> author "Greg Ward"
> author_email "gward@python.net"
> url "http://www.python.org/sigs/distutils-...
>
> packages ['distutils', 'distutils/command']
> }
>
>Given this file, we can simply run:
>
> python setup.py install
>
>and the files will get installed where they belong to. distutils can
>also handle different prefixes, installing into home directories, and
>complex cases like putting scripts to /usr/bin, but libraries to
>/opt/local and whatever.
>
>Python's distutils also handles compiling extensions:
>
> name = 'DateTime.mxDateTime.mxDateTime'
> src = 'mxDateTime/mxDateTime.c'
> setup (
> ...
> ext_modules =
> [(name,
> { 'sources': [src]
> 'include_dirs': ['mxDateTime'] }
> )]
> )
>
>Here, something like this would be possible in Ruby (I'm not yet sure
>about exact semantics of the Python version):
>
> setup {
> # ...
> extension("DateTime/mxDateTime/mxDateTime") {
> sources "mxDateTime/mxDateTime.c"
> include_dirs "mxDateTime"
> }
> }
>
>Of course, more complex build descriptions can be represented too:
>
> extension("foolib") {
> sources "foo.c", "bar.c"
> if have_library("foo", "fooble")
> define "HAVE_FOO_H"
> cflags << `foo-config --cflags`
> ldflags << `foo-config --libs`
> else
> fail "foolib is needed"
> end
> }
>
>Whether this will generate a Makefile (like mkmf.rb), a Rakefile
>or compile directly (like distutils) is still an open question.
>
>To allow for an easy conversion of setup.rb usage, Package will
>provide convenience methods that will make it behave like setup.rb
>with respect to the directory structure.
>
>Package doesn't try to conquer the world, however, it aims to be just
>a tool that would be useful if it was standard and everyone could
>build on due to it's policy-neutrality
>
>What advantages will Package have over setup.rb and mkmf.rb, as
>they are now?
>
>* simple, clean and consistent working
>* unified library to handle both extensions and libraries
>* lightweight approach (if included in the standard library)
>* easy adaption
>* more flexible directory layout: especially small projects
> profit from this, as setup.rb's directory layout is quite
> bulky by default and not very customizable
>* easier packaging by third-party packagers due to simple
> but flexible and standardized invocation
>
>What do we need to get a wide adoption of Package?
>
>* inclusion in the standard library so it doesn't need to be
> shipped with every package (as setup.rb unfortunately is).
>* backing from the community to make use of Package.
>* acceptance from packaging projects like RPA, RubyGems and
> distributions like Debian, FreeBSD and PLD.
>
>Coding of Package has not started yet (the name is also not set into
>stone yet, so if you have better ideas, please tell me) because it
>would be pointless without a strong feedback from the community. I
>expect to get a first version done rather quickly, possibly borrowing
>code from setup.rb and mkmf.rb, but Package will not depend on these
>both. If anyone is interested in helping development, please mail me;
>helpful hands are always of use. Also, there will be need for testers
>on all and even the most weird platforms.
>
>But now, I'll ask you: Are you satisfied with the way installing Ruby
>extensions and libraries works? Do you think there is a place for
>Package? Do you have further improvements or can provide alternative
>ideas?

No. setup.rb is OK but inflexible. Package management (gems etc.)
nor build management (Rant, Rake) should not be in any way incorporated
in 'Package', either.

Might be a good idea to have the extension builder as a completely
separate tool, too.

Just a simple, easy packager/setup utility, please!
(I will help, too, if I can!)

>Christian Neukirchen

E

--
template<typename duck>
void quack(duck& d) { d.quack(); }

dave

6/5/2005 5:49:00 AM

> I started to wonder if Ruby's install
> scripts are state of the art, what could be done better and how.

clap, clap.
this is wonderfull: we need it.

> Basically, you create a file, setup.py

I can suggest you to look at scons (http://www...)

A great thing would be to replace:
- configure make and make install

As said on this mailing list to rake author you could:

work on a configure script template like configure.in.

and provide macro as ruby functions like:

- Ac.init_automake("package", 1.0.1)

and Makefile.am that generate a Makefile.in and and a Makefile.rb

- Am.subdirs(src)
- Am.sources("package", [files])

This approach will led you to a great success, imo.

--
>here are more things in heaven and earth,
horatio, than are dreamt of in your philosophy.

Christian Neukirchen

6/5/2005 8:47:00 AM

Austin Ziegler <halostatue@gmail.com> writes:

> On 6/4/05, Christian Neukirchen <chneukirchen@gmail.com> wrote:
>> But now, I'll ask you: Are you satisfied with the way installing Ruby
>> extensions and libraries works? Do you think there is a place for
>> Package? Do you have further improvements or can provide alternative
>> ideas?
>
> I think that there is room for improvement, definitely. However, if
> you're going to do extension support with this, it absolutely *must*
> work perfectly well on Windows. It has to work better than setup.rb,
> and setup.rb works mostly well for that. Gems, much less so.

I don't think making the installer work on windows "perfectly" would
be a hard job. For building extensions, things look a bit
different... I'd certainly need a win32 expert for that.

How well does extconf.rb work on win32?

> -austin
--
Christian Neukirchen <chneukirchen@gmail.com> http://chneuk...

Christian Neukirchen

6/5/2005 8:49:00 AM

Jim Freeze <jim@freeze.org> writes:

> * Christian Neukirchen <chneukirchen@gmail.com> [2005-06-05 07:59:15 +0900]:
>
>> Package doesn't try to conquer the world, however, it aims to be just
>> a tool that would be useful if it was standard and everyone could
>> build on due to it's policy-neutrality
>
>> What advantages will Package have over setup.rb and mkmf.rb, as
>> they are now?
>
> I find mkmf.rb very useful to create a Makefile when I am building a C wrapper
> to some library. Will Package do the same thing, or will mkmf still
> be needed?

Package will try to provide a more clean (no icky globals, for
example) API for the things mkmf.rb does. I think I'll start with a
recent mkmf.rb and refactor it heavily.

> Jim Freeze
--
Christian Neukirchen <chneukirchen@gmail.com> http://chneuk...

Stefan Lang

6/5/2005 8:49:00 AM

On Sunday 05 June 2005 07:48, dave wrote:
> > I started to wonder if Ruby's install
> > scripts are state of the art, what could be done better and how.
>
> clap, clap.
> this is wonderfull: we need it.
>
> > Basically, you create a file, setup.py
>
> I can suggest you to look at scons (http://www...)

SCons is a full blown build tool. AFAIK they use the distutils
(setup.py) package for SCons installation.

Stefan

Christian Neukirchen

6/5/2005 8:56:00 AM

ES <ruby-ml@magical-cat.org> writes:

> Le 4/6/2005, "Christian Neukirchen" <chneukirchen@gmail.com> a écrit:
>>
>>But now, I'll ask you: Are you satisfied with the way installing Ruby
>>extensions and libraries works? Do you think there is a place for
>>Package? Do you have further improvements or can provide alternative
>>ideas?
>
> No. setup.rb is OK but inflexible. Package management (gems etc.)
> nor build management (Rant, Rake) should not be in any way incorporated
> in 'Package', either.

Package will not handle dependencies on it's own (You are strongly
recommended to check for libraries in the build part, though).

Package will not provide general "build management", but does generate
instructions how to build extensions. That is, if you before used an
extconf.rb, you'll use Package, if you used a custom Rakefile, you'll
continue to do so.

It would be imaginable that Package could generate Rake tasks
dynamically, however, Rake is not (yet?) included in the Ruby standard
library.

> Might be a good idea to have the extension builder as a completely
> separate tool, too.

I'll try to make it easy to use outside of Package.

> Just a simple, easy packager/setup utility, please!
> (I will help, too, if I can!)

Thank you, I'll come back on that.

> E
--
Christian Neukirchen <chneukirchen@gmail.com> http://chneuk...

Christian Neukirchen

6/5/2005 8:59:00 AM

dave <dave.m@email.it> writes:

>> I started to wonder if Ruby's install
>> scripts are state of the art, what could be done better and how.
>
> clap, clap.
> this is wonderfull: we need it.
>
>
>> Basically, you create a file, setup.py
>
> I can suggest you to look at scons (http://www...)
>
>
> A great thing would be to replace:
> - configure make and make install

Please note that Package aims to specifically help Ruby developers
installing *Ruby* libraries and extensions. Package is not trying to
be a general purpose tool for other languages, especially not autoconf
or automake.

> This approach will led you to a great success, imo.

How about you write the autotools in Ruby and have your own great
success? :-)

--
Christian Neukirchen <chneukirchen@gmail.com> http://chneuk...

Stefan Lang

6/5/2005 8:59:00 AM

On Sunday 05 June 2005 10:48, Christian Neukirchen wrote:
> Jim Freeze <jim@freeze.org> writes:
> > * Christian Neukirchen <chneukirchen@gmail.com> [2005-06-05 07:59:15
+0900]:
> >> Package doesn't try to conquer the world, however, it aims to be just
> >> a tool that would be useful if it was standard and everyone could
> >> build on due to it's policy-neutrality
> >>
> >> What advantages will Package have over setup.rb and mkmf.rb, as
> >> they are now?
> >
> > I find mkmf.rb very useful to create a Makefile when I am building a C
> > wrapper to some library. Will Package do the same thing, or will mkmf
> > still be needed?
>
> Package will try to provide a more clean (no icky globals, for
> example) API for the things mkmf.rb does. I think I'll start with a
> recent mkmf.rb and refactor it heavily.

A clean API sounds good, it would be nice if it could be used as a
library to allow easy integration into other tools, e.g. Rant.
OTOH if Package uses Rant or Rake it is easier to make it portable
and there would be less dependencies on external (non-Ruby) tools.

Stefan

comp.lang.ruby

Package, a future replacement for setup.rb and mkmf.rb

Christian Neukirchen

Austin Ziegler

Jim Freeze

ES

dave

Christian Neukirchen

Christian Neukirchen

Stefan Lang

Christian Neukirchen

Christian Neukirchen

Stefan Lang

x Login to ForumsZone