[lnkForumImage]
TotalShareware - Download Free Software
Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

multithreaded file access

Matias Surdi

12/23/2005 4:38:00 PM

14 Answers

Jellen

12/23/2005 5:08:00 PM

0

Well, I think it's OK to do that.
Seeing is believing:
# first one
a = Thread.new do
5.times do
f = File.new("qq.txt", "a").puts "I am a..."
f.close if f
end
end
b = Thread.new do
5.times do
f = File.new("qq.txt", "a").puts "I am b..."
f.close if f
end
end
a.join
b.join

# and this one
f = File.new("test.txt", "a")
a = Thread.new do
5.times do
f.puts "I am a..."
sleep 1
end
end
b = Thread.new do
5.times do
f.puts "I am b..."
sleep 1
end
end
a.join
b.join
f.close


Both program are ok. (But I am not sure myself:)

J. Ryan Sobol

12/23/2005 5:54:00 PM

0

On Dec 23, 2005, at 12:12 PM, Jellen wrote:

> Well, I think it's OK to do that.
> Seeing is believing:
> # first one
> a = Thread.new do
> 5.times do
> f = File.new("qq.txt", "a").puts "I am a..."
> f.close if f
> end
> end
> b = Thread.new do
> 5.times do
> f = File.new("qq.txt", "a").puts "I am b..."
> f.close if f
> end
> end
> a.join
> b.join
>
> # and this one
> f = File.new("test.txt", "a")
> a = Thread.new do
> 5.times do
> f.puts "I am a..."
> sleep 1
> end
> end
> b = Thread.new do
> 5.times do
> f.puts "I am b..."
> sleep 1
> end
> end
> a.join
> b.join
> f.close
>
>
> Both program are ok. (But I am not sure myself:)

Interesting examples, Jellen, but I don't think it answers Matias'
question, which was

> File.new('filename','a').puts("this is the string")
>
>
> Is this already thread safe??? how can I make it so?????

Correct me if I'm wrong, but your examples only prove that the thread
on the CPU will be able to append the file. I *think* Matias wants
to know if the statement ( File.new('filename','a').puts("this is the
string") ) is atomic. Or in other words, do you need to enforce
mutual exclusive access to the file with a mutex? Unfortunately, I
don't have an answer to that question.

Ilmari Heikkinen

12/23/2005 9:14:00 PM

0

On 12/23/05, J. Ryan Sobol <ryansobol@gmail.com> wrote:> Correct me if I'm wrong, but your examples only prove that the thread> on the CPU will be able to append the file. I *think* Matias wants> to know if the statement ( File.new('filename','a').puts("this is the> string") ) is atomic. Or in other words, do you need to enforce> mutual exclusive access to the file with a mutex? Unfortunately, I> don't have an answer to that question.[kig@jugend:~] cat fw_test.rbdef writer(i, fn, ok) Thread.new{ t_str = "#{i}" * 65536 while ok.first File.open(fn, 'a'){|f| f.puts t_str } end }endok = [true]fn = 'fw_test.dat'ts = (1..3).map{|i| writer i, fn, ok}sleep 10ok[0] = falseif File.readlines(fn).uniq.size == ts.size puts "puts in different threads seems to be atomic"else puts "puts in different threads isn't atomic"endFile.unlink fn[kig@jugend:~] ruby fw_test.rbputs in different threads seems to be atomic

J. Ryan Sobol

12/23/2005 10:00:00 PM

0

On Dec 23, 2005, at 4:13 PM, Ilmari Heikkinen wrote:

> [kig@jugend:~] cat fw_test.rb
> def writer(i, fn, ok)
> Thread.new{
> t_str = "#{i}" * 65536
> while ok.first
> File.open(fn, 'a'){|f|
> f.puts t_str
> }
> end
> }
> end
>
> ok = [true]
> fn = 'fw_test.dat'
> ts = (1..3).map{|i| writer i, fn, ok}
>
> sleep 10
>
> ok[0] = false
>
> if File.readlines(fn).uniq.size == ts.size
> puts "puts in different threads seems to be atomic"
> else
> puts "puts in different threads isn't atomic"
> end
>
> File.unlink fn
>
> [kig@jugend:~] ruby fw_test.rb
> puts in different threads seems to be atomic

Crafty test program. Coincidentally, my results differ from yours.

$ ruby fw_test.rb
puts in different threads isn't atomic
$ ruby -v
ruby 1.8.2 (2004-12-25) [powerpc-darwin8.3.0]

I'm getting 8 unique lines in File.readlines(fn).uniq, as opposed to
the 3 thread objects in ts. Assuming I understand the program, that
means threads are not waiting their turn like they should. FYI, I've
compiled my own ruby executable from DarwinPorts instead of using the
'broken' one shipped in OS X Tiger.

However, if I change the multiplication factor in line 3 to 655, then

$ ruby fw_test.rb
puts in different threads seems to be atomic

~ ryan ~

PS - I love the 205+ MB of text this app generates! :-D


Ilmari Heikkinen

12/23/2005 10:23:00 PM

0

On 12/23/05, J. Ryan Sobol <ryansobol@gmail.com> wrote:> On Dec 23, 2005, at 4:13 PM, Ilmari Heikkinen wrote:> > [kig@jugend:~] ruby fw_test.rb> > puts in different threads seems to be atomic>> Crafty test program. Coincidentally, my results differ from yours.>> $ ruby fw_test.rb> puts in different threads isn't atomic> $ ruby -v> ruby 1.8.2 (2004-12-25) [powerpc-darwin8.3.0]Good! I thought it was odd that they seemed to be atomic..And now we know to mutex disk writes (or use flock.)> I'm getting 8 unique lines in File.readlines(fn).uniq, as opposed to> the 3 thread objects in ts. Assuming I understand the program, that> means threads are not waiting their turn like they should. FYI, I've> compiled my own ruby executable from DarwinPorts instead of using the> 'broken' one shipped in OS X Tiger.>> However, if I change the multiplication factor in line 3 to 655, then>> $ ruby fw_test.rb> puts in different threads seems to be atomicI don't know the reason (can guess, but too unsure)Explanation anyone?> ~ ryan ~>> PS - I love the 205+ MB of text this app generates! :-DIlmari

J. Ryan Sobol

12/23/2005 10:46:00 PM

0

On Dec 23, 2005, at 5:23 PM, Ilmari Heikkinen wrote:

> On 12/23/05, J. Ryan Sobol <ryansobol@gmail.com> wrote:
>> On Dec 23, 2005, at 4:13 PM, Ilmari Heikkinen wrote:
>>> [kig@jugend:~] ruby fw_test.rb
>>> puts in different threads seems to be atomic
>>
>> Crafty test program. Coincidentally, my results differ from yours.
>>
>> $ ruby fw_test.rb
>> puts in different threads isn't atomic
>> $ ruby -v
>> ruby 1.8.2 (2004-12-25) [powerpc-darwin8.3.0]
>
> Good! I thought it was odd that they seemed to be atomic..
>
> And now we know to mutex disk writes (or use flock.)
>
>> I'm getting 8 unique lines in File.readlines(fn).uniq, as opposed to
>> the 3 thread objects in ts. Assuming I understand the program, that
>> means threads are not waiting their turn like they should. FYI, I've
>> compiled my own ruby executable from DarwinPorts instead of using the
>> 'broken' one shipped in OS X Tiger.
>>
>> However, if I change the multiplication factor in line 3 to 655, then
>>
>> $ ruby fw_test.rb
>> puts in different threads seems to be atomic
>
> I don't know the reason (can guess, but too unsure)
> Explanation anyone?
>
>> ~ ryan ~
>>
>> PS - I love the 205+ MB of text this app generates! :-D
>
> Ilmari

Regardless, the important thing is to know that it seems file IO via
puts is not thread safe, which I kind of assumed from the beginning.
This begs the question: which methods, especially concerning file IO,
are thread safe? (if any)

~ ryan ~

Matias Surdi

12/23/2005 11:43:00 PM

0

Eero Saynatkari

12/24/2005 12:16:00 AM

0

Matias Surdi wrote:
> J. Ryan Sobol
> escribi󺦧t;>> Crafty test program. Coincidentally, my results differ from yours.
>>>
>>>
>>
>>
>> Regardless, the important thing is to know that it seems file IO via
>> puts is not thread safe, which I kind of assumed from the beginning.
>> This begs the question: which methods, especially concerning file IO,
>> are thread safe? (if any)
>>
>> ~ ryan ~
>>
>>
>
>
> Hey!!.... I'm now more confused than before posting :-D

You should guard your critical section :) The simplest way
is using Mutex from the core. Consult your ri or make a visit
to http://www.ru....


E

--
Posted via http://www.ruby-....

Matias Surdi

12/24/2005 8:09:00 AM

0

J. Ryan Sobol

12/24/2005 3:27:00 PM

0


On Dec 23, 2005, at 6:42 PM, Matias Surdi wrote:
> Hey!!.... I'm now more confused than before posting :-D

http://www.rubycentral.com/book/tut_th...

Pay special attention to the section on mutual exclusion.

~ ryan ~