aboutsummaryrefslogtreecommitdiff
path: root/test/git/async/test_pool.py
AgeCommit message (Collapse)Author
2010-06-08Its getting better already - intermediate commit before further chaning the ↵Sebastian Thiel
task class
2010-06-08The new channeldesign actually works, but it also shows that its located at ↵Sebastian Thiel
the wrong spot. The channel is nothing more than an adapter allowing to read multiple items from a thread-safe queue, the queue itself though must be 'closable' for writing, or needs something like a writable flag.
2010-06-08both versions of the async queue still have trouble in certain situations, ↵Sebastian Thiel
at least with my totally overwritten version of the condition - the previous one was somewhat more stable it seems. Nonetheless, this is the fastest version so far
2010-06-08test implementation of async-queue with everything stripped from it that ↵Sebastian Thiel
didn't seem necessary - its a failure, something is wrong - performance not much better than the original one, its depending on the condition performance actually, which I don't get faster
2010-06-07introduced a new counter keeping track of the scheduled tasks - this prevent ↵Sebastian Thiel
unnecessary tasks to be scheduled as we keep track of how many items will be produced for the task at hand. This introduces additional locking, but performns well in multithreaded mode. Performance of the master queue is still a huge issue, its currently the limiting factor, as bypassing the master queue in serial moode gives 15x performance, wich is what I would need
2010-06-07improved testing to test the actual async handling of the pool. there are ↵Sebastian Thiel
still inconsistencies that need to be fixed, but it already improved, especially the 4-thread performance which now is as fast as the dual-threaded performance
2010-06-07task: Fixed incorrect handling of channel closure. Performance is alright ↵Sebastian Thiel
for up to 2 threads, but 4 are killing the queue
2010-06-07added high-speed locking facilities, allowing our Queue to be faster, at ↵Sebastian Thiel
least in tests, and with multiple threads. There is still an sync bug in regard to closed channels to be fixed, as the Task.set_done handling is incorrecft
2010-06-07Added task order cache, and a lock to prevent us walking the graph while ↵Sebastian Thiel
changing tasks Now processing more items to test performance, in dual-threaded mode as well, and its rather bad, have to figure out the reason for this, probably gil, but queues could help
2010-06-07pool: First version which works as expected in async mode. Its just using a ↵Sebastian Thiel
single task for now, but next up are dependent tasks
2010-06-06Plenty of fixes in the chunking routine, made possible by a serialized ↵Sebastian Thiel
chunking test. Next up, actual async processing
2010-06-06First step of testing the pool - tasks have been separated into a new module ↵Sebastian Thiel
including own tests, their design improved to prepare them for some specifics that would be needed for multiprocessing support
2010-06-05Renamed mp to async, as this is a much better name for what is actually ↵Sebastian Thiel
going on. The default implementation uses threads, which ends up being nothing more than async, as they are all locked down by internal and the global interpreter lock