Announcement
iPhone DZone: Fresh Links On Your iPhone by matt at Sat Jul 05 14:09:51 EDT 2008
Reading DZone on your iPhone just got a little bit easier. Visit http://dzone.com/iphone from your iPhone to try out our beta iPhone support. You can view stories, filter by tags, and login. We'd appreciate your feedback at feedback@dzone.com.
If you were logged in, then you could vote. Don't wait. Login and vote now.
By bloid
via peach.rubyforge.org
Published: Apr 04 2008 / 21:36
It is pretty common to have iterations over Arrays that can be safely run in parallel. With multicore chips becoming pretty common, single threaded processing is about as cool as Pog. Unfortunately, standard Ruby hates real threads pretty hardcore at the present time; however, for some ruby projects alternate VMs like JRuby do give multicores some lovin'. Peach exists to make this power simple to use with minimal code changes.
Comments
aliz replied ago:
Interesting concept, but this could quickly get you into trouble. Spinning off threads whenever an iteration is requested is not the best implementation. I would suggest creating a pool of workers and have your 'peach' routine pull workers from the pool. Doing this would control the number of threads are running at any point in time. You also need to be careful of stepping on data that may be in use in another thread. Keep working on it; you have a good idea that could be great with some additional work.
schleyfox replied ago:
Jruby has automatic threadpooling which should minimize most of the trouble. I also have peach take an optional number of threads argument per call. There are weaknesses in the implementation, but I think the spinning off of threads is relatively safe given the VM built in features. The more worrisome part is having to convert the block into a proc, but I don't think I can win that one.
murphee replied ago:
The defaults on the current implementation aren't going to help... iterating over a 10000 element array will fire off 10000 threads, unless you specify a lower number of threads.
Also: I'm not sure about opening the Array class... some kind of Adapter would be nicer and less intrusive.
Eg. parallel([1,2,3]).each {|x| foo(x)}
This could inspect the object passed in, and return and adapter that acts as an Enumerable, with specialized algorithms for some class types... eg. classes that support slice() could be easily sliced up into pieces for individual worker threads.
schleyfox replied ago:
all the peach functions take an optional number_of_threads argument which divides up the elements among that number of worker threads (using slice() as you suggested). An adapter class would be a decent alternative to monkey-patching Array, but monkey patching is much simpler and I would hope there wouldn't be too many conflicts with peach, pmap, or pdelete_if (they are rather silly names).
Thanks for the feedback.
djberg96 replied ago:
I would look at the implementation in Fortress to see how they do it.
research.sun.com/projects/plrg/fortress.pdf
Voters For This Link (10)
Voters Against This Link (2)