News for hackers. Stuff that matters.
Re:PiCloud : Cloud Computing. Simplified. by liuliu on Saturday February 27, 2010 @02:51PM (#1155712)
How they scale automatically on python (a non-functional language)? I am puzzled.
Re:PiCloud : Cloud Computing. Simplified. by m0th87 on Saturday February 27, 2010 @03:02PM (#1155731)
They probably spawn several python processes
Re:PiCloud : Cloud Computing. Simplified. by miloshh on Saturday February 27, 2010 @03:10PM (#1155745)
Indeed, seems like a single cloud.call will not give you any parallelism, you need many calls. So the whole parallelization challenge is left completely up to you. Still, this is a nice approach, and I'd like to see more companies in this space.
Re:PiCloud : Cloud Computing. Simplified. by willbmoss on Saturday February 27, 2010 @03:48PM (#1155814)
Agreed, a very cool idea. I believe you are right on the parallelism front, cloud.call just offloads the processing on their servers, but you'd have to call it multiple times to get any parallelism.
Re:PiCloud : Cloud Computing. Simplified. by usaar333 on Saturday February 27, 2010 @05:05PM (#1155923)
PiCloud dev here.

That is correct. We also offer a mapping function, cloud.map(func, arg_list), where every func(arg_list[i]) will be evaluated in parallel.

Re:PiCloud : Cloud Computing. Simplified. by sgk284 on Saturday February 27, 2010 @03:52PM (#1155822)
This is just a way to call a function and run it on a different server(s). You still have to design your program to handle parallelism.
Re:PiCloud : Cloud Computing. Simplified. by frisco on Saturday February 27, 2010 @04:11PM (#1155849)
Python is a functional language in the sense that it has first class functions and you can pass them around.
Re:PiCloud : Cloud Computing. Simplified. by m0th87 on Saturday February 27, 2010 @04:50PM (#1155898)
Yes but it's not purely functional in that you cannot automagically make a script scale to n threads, as is the case with Haskell. This is what the OP is referring to.
Re:PiCloud : Cloud Computing. Simplified. by _delirium on Saturday February 27, 2010 @08:07PM (#1156161)
You can't really do that either in Haskell. Well, in theory you can, but in practice nobody has yet written a way to autoparallelize Haskell that actually gets you reasonable speedups, because it requires predicting which parts are worth parallelizing.

Instead, the Haskell devs are mostly taking the same approach, of providing constructs for the programmer to explicitly write parallel code: http://www.haskell.org/ghc/docs/6.6/html/libraries/base/Cont...

Re:PiCloud : Cloud Computing. Simplified. by usaar333 on Saturday February 27, 2010 @05:10PM (#1155928)
Correct. cloud.call, etc. are all higher-level functions.

PiCloud does treat cloud.call(func) dispatches as completely functional. The function evaluated cannot have side-effects in terms of the program space; within the program, you access its output via cloud.result(). (Note: You can within the func have external side-effects - e.g. opening a connection to a database and modifying entries).

Re:PiCloud : Cloud Computing. Simplified. by nico_h on Saturday February 27, 2010 @08:17PM (#1156178)
- edit : nevermind, found the answer in the 'pitfalls' doc : yes you can create picloud call from within picloud, but they of course go against your maximum count of parallel calls -

Hi

I have read (quickly) your webscraper example and i still have one question : Is it possible to launch piclouds jobs from within a picloud job ? i.e. from picloud download some list of url as csvs and then start webcrapers then have the original function return the results of these scrapers ? Thanks for your answer.