B }d@sTdZddlmZddlmZddlmZddlm Z ddlm Z d d Z d d Z d S)zMulti-processing utilities for input/output This module provides the `read_multi` method, which enables spreading reading multiple files across multiple cores, returning a flattened result. ) SAXException)read)get_read_format) file_list)mpcs$dd}y t|}Wntk r6|g}d}YnX|rD|dnd}|s^tdjddddkrt||fd<tdd t||d krd t d }fd d|D}t j t ||dd} x2| D]*\} } t | trd| d| f| _| qWt| \} } || S)aRead sources into a `cls` with multiprocessing This method should be called by `cls.read` and uses the `nproc` keyword to enable and handle pool-based multiprocessing of multiple source files, using `flatten` to combine the chunked data into a single object of the correct type. Parameters ---------- flatten : `callable` a method to take a list of ``cls`` instances, and combine them into a single ``cls`` instance cls : `type` the object type to read source : `str`, `list` of `str`, ... the input data source, can be of in many different forms *args positional arguments to pass to the reader **kwargs keyword arguments to pass to the reader verboseFNrz cannot read z from empty source listformatnprocrTz Reading ()csg|]}|fqSr ).0f)argsclskwargsr r W/work/yifan.wang/ringdown/master-ringdown-env/lib/python3.7/site-packages/gwpy/io/mp.py Wszread_multi..files)r unitzFailed to read z: )popr ValueError IndexError__name__getrminlenr mp_utilsZmultiprocess_with_queues_read_single_file isinstance Exceptionrzip)flattenrsourcerrr rpathinputsoutputfobjexc_outr )rrrr r read_multi"s.      r,c Csp|\}}}}}y|t||f||fStk rj}z(|dkr@t|trV||fS||fSd}~XYnXdS)aOReads a single file and returns the output This is designed only to be passed to :func:`gwpy.utils.mp.multiprocess_with_queues` Returns ------- f, output : the input file (object) that was (attempted to be) read, and the result of the read operation, which may be an exception object. rN)io_readr!r rZ getException)Zbundler(rr rrr)r r rrhs   rN)__doc__Zxml.saxrZastropy.io.registryrr-registryrutilsrrrr,rr r r rs     F