PyMW: Week one

Today is the official end of my 7th day of working on the PyMW interface for BOINC for Google Summer of Code.

It took me three days to get the my first PyMW app to run (monte_pi.py), which ran on 4 virtual nodes (4 tasks). More than four tasks was causing problems, it turned out that there was a bug in the PyMW BOINC interface. Now that that’s fixed, I ran with 200 nodes yesterday and 800 today.

This morning, I tried running with 2 physical nodes: my laptop and my Ubuntu VM, which failed at the end of computation. Somehow the canonical results are not being recognized which causes the BOINC interface to get lost in limbo and hang forever.

This week, I created a pure-Python assimilator for PyMW, which works pretty well, but is perhaps causing the error above.

I also rewrote a big swath of the BOINC interface to stop it from using a new thread for each task during task reclamation (getting data back from BOINC). Since it was using one thread per task, it was reaching the maximum number of scheduler threads. This in turn caused the execution thread to hang until some of the tasks completed. Now it reclaims tasks in a single thread and is able to queue all tasks in a single shot, greatly improving the throughput from PyMW -> BOINC.

Overall, it’s been really fun so far. The first few days were trying, since there was little documentation of how to get PyMW to play nice with BOINC. But seeing the first application run was great

Tagged Tags: BOINC, GSoC, Parallel, PyMW, Python on June 5, 2009 at 12:01 pm