Why is parallel computing harder than using a web service?
Why isn't there a library that lets me call a map function and split the workload across 2, 10, or more computers?
Hmm, how to properly say "in soviet Russia, web service calls you?"
With a web service, the client calls a function on the server, and gets some data in return.
Now, have a pool of clients connect to the server. Then, when the server needs a lot of data processed it sends a subset to the data to each client. It then sends the processing function and dependencies to the clients. The clients process the data and return it to the server.
I'm aiming for something like Google's
MapReduce, just more accessible for programmers without Google-scale resources.
Now, for a twist, what if the server could send the function in bytecode form to the clients along with a wrapper specifying how to run the function (what interpreter, parameters, etc.)?
I know that this has about as much to do with web services as a cow does with the moon, but if you invert the web service model, I think you get close. There is a better way to explain this, but I'm stuck for now. I've started writing a test implementation in Python. Hopefully I'll have something to show this weekend.