B ~d@sJdZddlZddlmZmZddlmZddlmZddZ d d d Z dS) zUtilities for multi-processing N)QueueProcess) itemgetter) progress_barc Cshxb|\}}|dkrPy||||fWqtk r^}z|||fWdd}~XYqXqWdS)aIterate through a Queue, call, ``func`, and Queue the result Parameters ---------- func : `callable` any function that can take an element of the input `Queue` as the only argument q_in : `multiprocessing.queue.Queue` the input `Queue` q_out : `multiprocessing.queue.Queue` the output `Queue` Notes ----- To close the input `Queue`, add ``(None, None)`` as the last item N)getput Exception)funcq_inq_outidxargexcrZ/work/yifan.wang/ringdown/master-ringdown-env/lib/python3.7/site-packages/gwpy/utils/mp.py_process_in_out_queuess rFc  s|dddk rtdtt|rdt|ts:t||d<t|ttfrX| dt |t f|nd|dkrfdd}tt ||St t fd d t|D}x|D]}d |_|qWfd d t|D}xt|D]} d qWg} x6tt |D]&} } r,| | qWx|D]}|qBWrbdd t| tddD} x| D]} t| tr| qW| S)aMap a function over a list of inputs using multiprocess This essentially duplicates `multiprocess.map` but allows for arbitrary functions (that aren't necessarily importable) Parameters ---------- nproc : `int` number of processes to use, if ``1`` is given, the current process is used, and no child processes are forked func : `callable` the function to call in each iteration, should take a single argument that is the next element from ``inputs`` inputs : `iterable` iterable (e.g. `list`) of inputs, each element of which is passed to ``func`` in one of the child processes verbose : `bool`, `str`, optional if `True`, print progress to the console as a bar, pass a `str` to customise the heading for the progress bar, default: `False`, (default heading ``'Processing:'`` if ``verbose=True`) Returns ------- outputs : `list` the `list` of results from calling ``func(x)`` for each element of ``inputs`` Zraise_exceptionsNzthe `raise_exceptions` keyword to multiprocess_with_queues is deprecated, and will be removed in a future release, all exceptions will be raised if they occurdesctotalrc sz|SrdXdS)Nr)update)x)r pbarrr_innertsz(multiprocess_with_queues.._innercsg|]}ttfdqS))targetargs)rr).0_)r r r rr sz,multiprocess_with_queues..Tcsg|]}j|ddqS)F)block)r)rr)r rrrs)NNcSsg|] \}}|qSrr)rroutrrrrsr)key)popwarningswarnDeprecationWarningbool isinstancestrlisttuple setdefaultlenrmaprrangedaemonstart enumeraterrrappendjoinclosesortedrr ) Znprocr inputsverboseZ progress_kwrZproclistprocsentrresrresultsr)r rr r rmultiprocess_with_queues@sL           r;)F) __doc__r"multiprocessingrroperatorrprogressrrr;rrrrs   #