Chris
1/4/2008 2:07:00 PM
On Jan 4, 3:57 pm, wanzathe <wanza...@gmail.com> wrote:
> hi everyone
> i'm a newbie to python :)
> i have a binary file named test.dat including 9600000 records.
> the record format is int a + int b + int c + int d
> i want to build a dict like this: key=int a,int b values=int c,int d
> i choose using bsddb and it takes about 140 seconds to build the dict.
> what can i do if i want to make my program run faster?
> or is there another way i can choose?
> Thanks in advance.
>
> My Code:
> -----------------------------------------------------------------------------------
> my_file = file('test.dat','rb')
> content = my_file.read()
> record_number = len(content) / 16
>
> db = bsddb.btopen('test.dat.db','n',cachesize=500000000)
> for i in range(0,record_number):
> a = struct.unpack("IIII",content[i*16:i*16+16])
> db['%d_%d' % (a[0],a[1])] = '%d_%d' % (a[2],a[3])
>
> db.close()
> my_file.close()
my_file = file('test.dat','rb')
db = bsddb.btopen('test.dat.db','n',cachesize=500000000)
content = myfile.read(16)
while content:
a = struct.unpack('IIII',content)
db['%d_%d' % (a[0],a[1])] = '%d_%d' % (a[2],a[3])
content = myfile.read(16)
db.close()
my_file.close()
That would be more memory efficient, as for speed you would need to
time it on your side.