aboutsummaryrefslogtreecommitdiff
path: root/lib/git/async/util.py
AgeCommit message (Collapse)Author
2010-06-09thread: fixed initialization problem if an empty iterable was handed inSebastian Thiel
queue: Queue now derives from deque directly, which safes one dict lookup as the queue does not need to be accessed through self anymore pool test improved to better verify threads are started correctly
2010-06-09queue: fixed critical bug in the notify method, as it was not at all ↵Sebastian Thiel
thread-safe, causing locks to be released multiple times. Now it runs very fast, and very stable apparently. Now its about putting previous features back in, and studying their results, before more complex task graphs can be examined
2010-06-08workerthread: adjusted to use a blocking queue, it will receive termination ↵Sebastian Thiel
events only with its queue, with boosts performance into brigt green levels
2010-06-08Its getting better already - intermediate commit before further chaning the ↵Sebastian Thiel
task class
2010-06-08queue: adjusted queue to be closable ( without own testing yet, except for ↵Sebastian Thiel
the pool which runs it ) - its not yet stable, but should be solvable.
2010-06-08The new channeldesign actually works, but it also shows that its located at ↵Sebastian Thiel
the wrong spot. The channel is nothing more than an adapter allowing to read multiple items from a thread-safe queue, the queue itself though must be 'closable' for writing, or needs something like a writable flag.
2010-06-08both versions of the async queue still have trouble in certain situations, ↵Sebastian Thiel
at least with my totally overwritten version of the condition - the previous one was somewhat more stable it seems. Nonetheless, this is the fastest version so far
2010-06-08test implementation of async-queue with everything stripped from it that ↵Sebastian Thiel
didn't seem necessary - its a failure, something is wrong - performance not much better than the original one, its depending on the condition performance actually, which I don't get faster
2010-06-07Task scheduled items lock now uses a dummy lock in serial mode, improving ↵Sebastian Thiel
its performance considerably. Channels now use the AsyncQueue, boosting their throughput to about 5k items / s - this is something one can work with, considering the runtime of each item should be large enough to keep the threads busy. This could be a basis, further testing needed
2010-06-07introduced a new counter keeping track of the scheduled tasks - this prevent ↵Sebastian Thiel
unnecessary tasks to be scheduled as we keep track of how many items will be produced for the task at hand. This introduces additional locking, but performns well in multithreaded mode. Performance of the master queue is still a huge issue, its currently the limiting factor, as bypassing the master queue in serial moode gives 15x performance, wich is what I would need
2010-06-07improved testing to test the actual async handling of the pool. there are ↵Sebastian Thiel
still inconsistencies that need to be fixed, but it already improved, especially the 4-thread performance which now is as fast as the dual-threaded performance
2010-06-07Moved pool utilities into util module, fixed critical issue that caused ↵Sebastian Thiel
havok - lets call this a safe-state
2010-06-06First step of testing the pool - tasks have been separated into a new module ↵Sebastian Thiel
including own tests, their design improved to prepare them for some specifics that would be needed for multiprocessing support