Asp Forum - A more scalable Parallel Varfiler and a more scalable Parallel Hashlist

Ramine

11/16/2014 11:30:00 PM

Hello,

I have implemented a more scalable Parallel varfiler that uses my
scalable RWLock in each bucket of the Parallel hashtable, and it can be
used also as a parallel hashtable and that uses ParallelHashList
(Parallel Hashtable) with O(1) best case and when there is collisions in
the hashtable, the average time complexity will be O(log(n)) in each
bucket were the collisions happened. The parallel Hahstable uses lock
striping and my scalable RWLock in each bucket , this allows multiple
threads to write and read concurently. also ParallelHashList maintains
an independant counter , that counts the number of entries , for each
segment of the hashtable and uses a lock for each counter, this is also
for better scalability.

ParallelVarFiler is a Parallel HashTable that can be saved automatically
or manually to a file or to a stream or to a string and it can be
restored from a file or from a stream or from a string back to the
Hashtable in memory, and it's fault tolerant to power failures etc.,
When you use a file name in the constructor, many readers and one writer
can proceed concurrently. But when you pass an empty file name to the
constructor, many writers and many readers can proceed concurrently in
memory. and when you read a data item it only use the parallel hashtable
in memory, hardisk is not used.

You can collect different types of variables and save them into one file
or stream. TParallelVarFiler reads and writes on files, streams and
Strings. You can collect Strings, Integers and Dates, in fact everything
that can be stored in a variant. In addition you can collect Streams.

You can use also ParallelVarFiler to send parameters over network. It
can be used for example to share data by any IPC mechanism.

An important note:

I have used my scalable RWLock in each bucket so that it scales better
on multicores, but to scale better on multicores please use a number
of rwlocks equal to 2000 in the constructor.

My Parallel Varfiler is now compatible with Delphi 7 to 2007 and
with FreePascal and Lazarus and with all the Delphi XE versions,
please download the zipfile called "parallelvarfiler_srw_xe.zip" for the
Delphi XE versions , and please download the zipfile called
"parallelvarfiler_srw.zip" for the Delphi7 to 2007 and for the
FreePascal and Lazarus.

You can download my more scalable Parallel Varfiler from:

https://sites.google.com/site/aminer68/more-scalable-paralle...

I have also implemented a more scalable Parallel Hashlist that scales
better , please download it from:

https://sites.google.com/site/aminer68/more-scalable-paralle...

Thank you,
Amine Moulay Ramdane.

3 Answers

Ramine

11/16/2014 11:47:00 PM

Hello,

I have done some benchmarks on multicores with an x86 Quadcore processor
and Parallel Hashlist (a parallel hashtable) and my Parallel Varfiler
scale in-memory to 3.9x (please look at the test2.pas example inside the
zipfile and test them yourself) , so my Parallel Varfiler and Parallel
Hashlist are great tools ! so i hope you will enjoy my inventions !

You can download them from:

https://sites.google.com/site/aminer68/more-scalable-paralle...

and from:

https://sites.google.com/site/aminer68/more-scalable-paralle...

Thank you,
Amine Moulay Ramdane.

Tom

11/24/2014 2:21:00 PM

the only one who responds to or comments on your posts is yourself. Does this tell you anything about the usefulness of your posts? or work? or life?

Tom

11/24/2014 2:22:00 PM

the only one who responds to or comments on your posts is yourself. Does this tell you anything about the usefulness of your posts? or work? or life?

comp.programming.threads

A more scalable Parallel Varfiler and a more scalable Parallel Hashlist

Ramine

Ramine

Tom

Tom

x Login to ForumsZone