[lnkForumImage]
TotalShareware - Download Free Software

Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

wrapping a C struct[] constant

Martin DeMello

6/12/2009 10:50:00 AM

I have some C code with a struct definition

struct flag_str {
unsigned int val;
const char *str;
};

and an inline array

struct flag_str extent_flags[] = {
{ FIEMAP_EXTENT_LAST, "last" },
{ FIEMAP_EXTENT_UNKNOWN, "unkown" },
{ FIEMAP_EXTENT_DELALLOC, "delalloc" },
{ FIEMAP_EXTENT_NO_BYPASS, "no_bypass" },
{ FIEMAP_EXTENT_SECONDARY, "secondary" },
{ FIEMAP_EXTENT_NET, "net" },
{ FIEMAP_EXTENT_DATA_COMPRESSED, "data_compressed" },
{ FIEMAP_EXTENT_DATA_ENCRYPTED, "data_encrypted" },
{ FIEMAP_EXTENT_NOT_ALIGNED, "not_aligned" },
{ FIEMAP_EXTENT_DATA_INLINE, "data_inline" },
{ FIEMAP_EXTENT_DATA_TAIL, "data_tail" },
{ FIEMAP_EXTENT_UNWRITTEN, "unwritten" },
{ FIEMAP_EXTENT_MERGED, "merged" },
{ 0, NULL },
};

What's the simplest way to expose that array as a ruby constant (other
than just copy/pasting the data, of course :))?

martin

14 Answers

Eero Saynatkari

6/12/2009 11:43:00 AM

0

Martin DeMello wrote:
> I have some C code with a struct definition
>
> struct flag_str {
> unsigned int val;
> const char *str;
> };
>
> and an inline array
>
> struct flag_str extent_flags[] = {
> { FIEMAP_EXTENT_LAST, "last" },
> { FIEMAP_EXTENT_UNKNOWN, "unkown" },
> { FIEMAP_EXTENT_DELALLOC, "delalloc" },
> { FIEMAP_EXTENT_NO_BYPASS, "no_bypass" },
> { FIEMAP_EXTENT_SECONDARY, "secondary" },
> { FIEMAP_EXTENT_NET, "net" },
> { FIEMAP_EXTENT_DATA_COMPRESSED, "data_compressed" },
> { FIEMAP_EXTENT_DATA_ENCRYPTED, "data_encrypted" },
> { FIEMAP_EXTENT_NOT_ALIGNED, "not_aligned" },
> { FIEMAP_EXTENT_DATA_INLINE, "data_inline" },
> { FIEMAP_EXTENT_DATA_TAIL, "data_tail" },
> { FIEMAP_EXTENT_UNWRITTEN, "unwritten" },
> { FIEMAP_EXTENT_MERGED, "merged" },
> { 0, NULL },
> };
>
> What's the simplest way to expose that array as a ruby constant (other
> than just copy/pasting the data, of course :))?

How/where are you *using* it?

FFI is generally a good solution, although it
would probably be best to actually define the
Array on the Ruby side and pass it in as needed.


Eero
--
Magic is insufficiently advanced technology.
--
Posted via http://www.ruby-....

Jason Roelofs

6/12/2009 1:32:00 PM

0

[Note: parts of this message were removed to make it a legal post.]

On Fri, Jun 12, 2009 at 7:42 AM, Eero Saynatkari <ruby-ml@kittensoft.org>wrote:

> Martin DeMello wrote:
> > I have some C code with a struct definition
> >
> > struct flag_str {
> > unsigned int val;
> > const char *str;
> > };
> >
> > and an inline array
> >
> > struct flag_str extent_flags[] = {
> > { FIEMAP_EXTENT_LAST, "last" },
> > { FIEMAP_EXTENT_UNKNOWN, "unkown" },
> > { FIEMAP_EXTENT_DELALLOC, "delalloc" },
> > { FIEMAP_EXTENT_NO_BYPASS, "no_bypass" },
> > { FIEMAP_EXTENT_SECONDARY, "secondary" },
> > { FIEMAP_EXTENT_NET, "net" },
> > { FIEMAP_EXTENT_DATA_COMPRESSED, "data_compressed" },
> > { FIEMAP_EXTENT_DATA_ENCRYPTED, "data_encrypted" },
> > { FIEMAP_EXTENT_NOT_ALIGNED, "not_aligned" },
> > { FIEMAP_EXTENT_DATA_INLINE, "data_inline" },
> > { FIEMAP_EXTENT_DATA_TAIL, "data_tail" },
> > { FIEMAP_EXTENT_UNWRITTEN, "unwritten" },
> > { FIEMAP_EXTENT_MERGED, "merged" },
> > { 0, NULL },
> > };
> >
> > What's the simplest way to expose that array as a ruby constant (other
> > than just copy/pasting the data, of course :))?
>
> How/where are you *using* it?
>
> FFI is generally a good solution, although it
> would probably be best to actually define the
> Array on the Ruby side and pass it in as needed.
>
>
> Eero
> --
> Magic is insufficiently advanced technology.
> --
> Posted via http://www.ruby-....
>
>
Even using FFI, you'll need to define the wrapping between C constant to
Ruby constant, and using Ruby's API, you'll be define_const-ing every one of
those, then putting them into an array. Just define them in the Ruby
yourself, there isn't a faster way to do it.

Jason

Martin DeMello

6/12/2009 1:55:00 PM

0

On Fri, Jun 12, 2009 at 5:12 PM, Eero Saynatkari<ruby-ml@kittensoft.org> wr=
ote:
> Martin DeMello wrote:
>> I have some C code with a struct definition
>>
>> struct flag_str {
>> =A0 unsigned int val;
>> =A0 const char *str;
>> };
>>
>> and an inline array
>>
>> struct flag_str extent_flags[] =3D {
>> =A0 { FIEMAP_EXTENT_LAST, =A0 =A0"last" },
>> =A0 { FIEMAP_EXTENT_UNKNOWN, =A0"unkown" },
>> =A0 { FIEMAP_EXTENT_DELALLOC, =A0"delalloc" },
>> =A0 { FIEMAP_EXTENT_NO_BYPASS, =A0"no_bypass" },
>> =A0 { FIEMAP_EXTENT_SECONDARY, =A0"secondary" },
>> =A0 { FIEMAP_EXTENT_NET, =A0 =A0"net" },
>> =A0 { FIEMAP_EXTENT_DATA_COMPRESSED, "data_compressed" },
>> =A0 { FIEMAP_EXTENT_DATA_ENCRYPTED, "data_encrypted" },
>> =A0 { FIEMAP_EXTENT_NOT_ALIGNED, =A0"not_aligned" },
>> =A0 { FIEMAP_EXTENT_DATA_INLINE, =A0"data_inline" },
>> =A0 { FIEMAP_EXTENT_DATA_TAIL, =A0"data_tail" },
>> =A0 { FIEMAP_EXTENT_UNWRITTEN, =A0"unwritten" },
>> =A0 { FIEMAP_EXTENT_MERGED, =A0 =A0"merged" },
>> =A0 { 0, =A0 =A0 =A0 =A0NULL },
>> };
>>
>> What's the simplest way to expose that array as a ruby constant (other
>> than just copy/pasting the data, of course :))?
>
> How/where are you *using* it?

Using it (or want to use it!) from the ruby side to generate a hash of
constant =3D> name and use that to unpack and display flag settings.

m.

Martin DeMello

6/12/2009 1:57:00 PM

0

On Fri, Jun 12, 2009 at 7:01 PM, Jason Roelofs<jameskilton@gmail.com> wrote:
>
> Even using FFI, you'll need to define the wrapping between C constant to
> Ruby constant, and using Ruby's API, you'll be define_const-ing every one of
> those, then putting them into an array. Just define them in the Ruby
> yourself, there isn't a faster way to do it.

I was afraid of that :( The docs are a bit on the sketchy side too. I
guess some sort of code generation would be the way to go then (swig
seems like overkill). Will give cgenerator a look.

martin

Joel VanderWerf

6/12/2009 8:18:00 PM

0

Martin DeMello wrote:
> Using it (or want to use it!) from the ruby side to generate a hash of
> constant => name and use that to unpack and display flag settings.

If all you want is this data structure in ruby, you could parse the
source (gccxml?), and then you have a pure ruby library that defines
this hash. Define the parse->xml>rb part as a rake task if you need to
keep up to date with the source files.

--
vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

Joel VanderWerf

6/12/2009 9:23:00 PM

0

Martin DeMello wrote:
> On Fri, Jun 12, 2009 at 7:01 PM, Jason Roelofs<jameskilton@gmail.com> wrote:
>> Even using FFI, you'll need to define the wrapping between C constant to
>> Ruby constant, and using Ruby's API, you'll be define_const-ing every one of
>> those, then putting them into an array. Just define them in the Ruby
>> yourself, there isn't a faster way to do it.
>
> I was afraid of that :( The docs are a bit on the sketchy side too. I
> guess some sort of code generation would be the way to go then (swig
> seems like overkill). Will give cgenerator a look.
>
> martin

Cgenerator will work, and might be a good idea if the source (the list
of values and strings) changes from time to time, and you would rather
have your program regenerate the ruby extension (and data structure)
automatically.

Here's how it works, assuming your source files are flag.c and flag.h:

$ ls
flag.c flag.h flag.rb
$ cat flag.h
struct flag_str {
unsigned int val;
const char *str;
};

extern struct flag_str extent_flags[];

typedef enum {
FIEMAP_EXTENT_LAST = 1,
FIEMAP_EXTENT_UNKNOWN,
FIEMAP_EXTENT_DELALLOC,
FIEMAP_EXTENT_NO_BYPASS,
FIEMAP_EXTENT_SECONDARY,
FIEMAP_EXTENT_NET,
FIEMAP_EXTENT_DATA_COMPRESSED,
FIEMAP_EXTENT_DATA_ENCRYPTED,
FIEMAP_EXTENT_NOT_ALIGNED,
FIEMAP_EXTENT_DATA_INLINE,
FIEMAP_EXTENT_DATA_TAIL,
FIEMAP_EXTENT_UNWRITTEN,
FIEMAP_EXTENT_MERGED
} FIEMAP_EXTENT;

$ cat flag.c
#include "flag.h"

#ifndef NULL
#define NULL 0
#endif

struct flag_str extent_flags[] = {
{ FIEMAP_EXTENT_LAST, "last" },
{ FIEMAP_EXTENT_UNKNOWN, "unkown" },
{ FIEMAP_EXTENT_DELALLOC, "delalloc" },
{ FIEMAP_EXTENT_NO_BYPASS, "no_bypass" },
{ FIEMAP_EXTENT_SECONDARY, "secondary" },
{ FIEMAP_EXTENT_NET, "net" },
{ FIEMAP_EXTENT_DATA_COMPRESSED, "data_compressed" },
{ FIEMAP_EXTENT_DATA_ENCRYPTED, "data_encrypted" },
{ FIEMAP_EXTENT_NOT_ALIGNED, "not_aligned" },
{ FIEMAP_EXTENT_DATA_INLINE, "data_inline" },
{ FIEMAP_EXTENT_DATA_TAIL, "data_tail" },
{ FIEMAP_EXTENT_UNWRITTEN, "unwritten" },
{ FIEMAP_EXTENT_MERGED, "merged" },
{ 0, NULL },
};

$ cat flag.rb
#!/usr/bin/env ruby

require 'cgen/cgen'
require 'fileutils'

module Flag; end

# Generate the extension source code.
lib = CGenerator::Library.new "flag_lib"
lib.include "flag.h"

lib.define_c_singleton_method(Flag, :extent_flags).instance_eval {
# no args
body %{ VALUE a;
struct flag_str *pfs;

a = rb_ary_new();
for (pfs = extent_flags; pfs->val; pfs++) {
rb_ary_push(a,
rb_ary_new3(2,
INT2NUM(pfs->val),
rb_str_new2(pfs->str)
));
}
}
returns "a"
}

# Normally, this isn't needed, but we need to set up the external files
# as symlinks in this dir before calling commit.
FileUtils.mkdir_p("flag_lib")

# Put links to sources where they will be found.
Dir.chdir "flag_lib" do
%w{ flag.h flag.c }.each do |file|
FileUtils.ln_s("../#{file}", file) rescue nil
end
end

# Write and build
lib.commit

# Use the library
Flag::EXTENT_FLAGS = Hash[*Flag.extent_flags.flatten]
p Flag::EXTENT_FLAGS

$ ruby flag.rb
{5=>"secondary", 11=>"data_tail", 6=>"net", 12=>"unwritten", 1=>"last",
7=>"data_compressed", 13=>"merged", 2=>"unkown", 8=>"data_encrypted",
3=>"delalloc", 9=>"not_aligned", 4=>"no_bypass", 10=>"data_inline"}


--
vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

Martin DeMello

6/15/2009 9:14:00 AM

0

On Sat, Jun 13, 2009 at 2:52 AM, Joel VanderWerf<vjoel@path.berkeley.edu> wrote:
>
> Cgenerator will work, and might be a good idea if the source (the list of
> values and strings) changes from time to time, and you would rather have
> your program regenerate the ruby extension (and data structure)
> automatically.
>
> Here's how it works, assuming your source files are flag.c and flag.h:

Thanks, that looks great. I have to say, I'm a bit disappointed in the
state of ruby/C integration - another basic thing I missed is the
ability to define a typemap and then have a ruby class
serialised/deserialised to a C struct. (Is that what Cgenerator's
CShadow is all about?). Are these not common use cases?

martin

Jason Roelofs

6/15/2009 1:43:00 PM

0

[Note: parts of this message were removed to make it a legal post.]

On Mon, Jun 15, 2009 at 5:13 AM, Martin DeMello <martindemello@gmail.com>wrote:

> On Sat, Jun 13, 2009 at 2:52 AM, Joel VanderWerf<vjoel@path.berkeley.edu>
> wrote:
> >
> > Cgenerator will work, and might be a good idea if the source (the list of
> > values and strings) changes from time to time, and you would rather have
> > your program regenerate the ruby extension (and data structure)
> > automatically.
> >
> > Here's how it works, assuming your source files are flag.c and flag.h:
>
> Thanks, that looks great. I have to say, I'm a bit disappointed in the
> state of ruby/C integration - another basic thing I missed is the
> ability to define a typemap and then have a ruby class
> serialised/deserialised to a C struct. (Is that what Cgenerator's
> CShadow is all about?). Are these not common use cases?
>
> martin
>
>
Could you give an example of how this would work? I'm having a hard time
picturing how such a thing would be possible to have done automagically.
There has to be something somewhere that defines how a Ruby class would map
to said C struct.

Jason

Joel VanderWerf

6/15/2009 5:30:00 PM

0

Martin DeMello wrote:
> another basic thing I missed is the
> ability to define a typemap and then have a ruby class
> serialised/deserialised to a C struct. (Is that what Cgenerator's
> CShadow is all about?). Are these not common use cases?

Would this be like swig in reverse? Input a ruby class with type
annotations, output C header, accessors, etc? I'd like to do that for
the special case of BitStruct classes (fixed-length, packed fields,
usually numeric or character, stored in a ruby String using
pack/unpack), so that once you define a network protocol or other binary
format in ruby, you can share your code with C programmers (and via swig
with other languages). That seems feasible, and I did actually do this
with a predecessor of BitStruct, using cgen. But how would you handle a
more general case, like a class with a hash or array attribute?

CShadow is similar, but instead of inheriting from String like
BitStruct, its instances are T_DATA (ruby objects with a blob that can
only be accessed from C). Unlike a BitStruct, this blob is not intended
for sending outside of the ruby+C world. For example, a "self" pointer
is automatically added at the beginning of the blob.

Including CShadow in your class lets you define the structure of the
T_DATA object in ruby, in terms of accessors of various types. When your
code runs, it uses cgenerator to build an extension that takes care of
all the boiler plate functions: mark, free, accessors with type
checking/conversion, alloc, init, inspect, marshal, and yaml. Plus it
handles inheritance and sets up introspection methods. [See example below.]

CShadow also gives you tools to distribute source code in separate .c
and .h files, so that when you make a minor change in the definitions
embedded in ruby, you minimize how much needs to be recompiled. This
works well enough that even if the generated code is a few Mb, you can
let cgenerator do its full generate/make cycle every time you run your
program, without noticing much delay.

You can add C functions to a CShadow class inline in ruby source just as
with cgenerator. But CShadow only supports a few kinds of attributes
"out of the box": ruby object references, native scalar data of various
sizes, and native pointers to char, double, array of double, etc. You
can add new attr types with a little work, but it's far from automatic.

These are just two special case solutions. I'm not seeing what a general
solution would look like. What kinds of classes/attributes are there in
your use cases?

--------------------------------
require 'cgen/cshadow'

class Parent
include CShadow

shadow_attr_accessor :ruby_str => String # type-checked VALUE type
shadow_attr_accessor :c_int => "int c_int"
end

class Child < Parent
shadow_attr_accessor :c_str => "char *c_str"
shadow_attr_accessor :obj => Object # VALUE type
end

Parent.commit
# we're done adding attrs and methods, so make.

x = Child.new
x.ruby_str = "foo"
x.c_str = "bar"
x.obj = [1,2,3]
x.c_int = 3

p x

CShadow.allow_yaml
y x

__END__

Output:

#<Child:0xb79716cc ruby_str="foo", c_int=3, c_str="bar", obj=[1, 2, 3]>
--- !path.berkeley.edu,2006/cshadow:Child
ruby_str: foo
c_int: 3
c_str: bar
obj:
- 1
- 2
- 3

--
vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

Martin DeMello

6/15/2009 6:01:00 PM

0

On Mon, Jun 15, 2009 at 11:00 PM, Joel
VanderWerf<vjoel@path.berkeley.edu> wrote:
> Martin DeMello wrote:
>>
>> another basic thing I missed is the
>> ability to define a typemap and then have a ruby class
>> serialised/deserialised to a C struct. (Is that what Cgenerator's
>> CShadow is all about?). Are these not common use cases?
>
> Would this be like swig in reverse? Input a ruby class with type
> annotations, output C header, accessors, etc?

I actually meant like a simplified swig - you have a C struct
(arbitrary nesting of structs and arrays, actually), and you have a
ruby class that you define a typemap for and then have it
serialise/deserialise. Basically you know that you can mirror C
structures in ruby with a bit of gruntwork, all you need is a tool to
do the gruntwork for you

> I'd like to do that for the
> special case of BitStruct classes (fixed-length, packed fields, usually
> numeric or character, stored in a ruby String using pack/unpack), so that
> once you define a network protocol or other binary format in ruby, you can
> share your code with C programmers (and via swig with other languages). That
> seems feasible, and I did actually do this with a predecessor of BitStruct,
> using cgen. But how would you handle a more general case, like a class with
> a hash or array attribute?

Network protocols were one of the use cases I was thinking of, except
that you usually see them defined in C. I wanted the ability to use
the structs from ruby, without going through C the way
data_wrap_struct makes you do. Not asking for magic, just the ability
to make the common case trivial.

> Including CShadow in your class lets you define the structure of the T_DATA
> object in ruby, in terms of accessors of various types. When your code runs,
> it uses cgenerator to build an extension that takes care of all the boiler
> plate functions: mark, free, accessors with type checking/conversion, alloc,
> init, inspect, marshal, and yaml. Plus it handles inheritance and sets up
> introspection methods. [See example below.]

Okay, that's most of what I was looking for :) If it can convert an
array of structs into a ruby array, that's *all* of what I'm looking
for. Need to look through the CGenerator docs some more - great piece
of work!

martin