B }dF9 @sdZddlZddlZddlmZmZddlmZddlm Z m Z m Z yddl m Z Wnek rld ZYnXd Zydd lmZWnek rd ZYnXd Zd Zd dZddZefddZddZefddZGdddedddddgZefddZe eddd fdd Zefd!d"Zd#d$Ze d%dd&d=d'd(Zd)d*Zd+d,Z d-d.Z!d/d0Z"d1d2Z#d3d4Z$d5d6Z%d>d7d8Z&d9d:Z'd;d<Z(dS)?zInput/Output utilities for GW Cache files. A cache file is a specially-formatted ASCII file that contains file paths and associated metadata for those files, designed to make identifying relevant data, and sieving large file lists, easier for the user. N) namedtuple OrderedDict) LIGOTimeGPS) FILE_LIKE file_path with_open) CacheEntryFT)Cachez(Duncan Macleod cCsTt|}t|\}}}|\}}|r.t|}|r>t|}|||t|||fS)N)rfilename_metadata is_integerinttype)entrypathobstagsegstartendrZ/work/yifan.wang/ringdown/master-ringdown-env/lib/python3.7/site-packages/gwpy/io/cache.py_preformat_entry6src Cs8t|\}}}}|d|d|ddt|d| S)N r)rabs)rrrrrrrr_format_entry_lalAsrc CsBddlm}|\}}}}}||}|t|}t||||||S)Nr)Segment)segmentsrfloat _CacheEntry) linegpstyperrdescrdurrrrrr_parse_entry_lalFs   r%cCs.t|\}}}}|d|ddt|dS)Nrrz 0 0)rr)rrrrrrrr_format_entry_fflNsr&c Csddlm}|\}}}}}||}|t|}y$tj|dddd\}} Wn$tk rvtdd||||SXt|| ||||S)Nr)r-) rrrosrbasenamesplit ValueErrorr ) r!r"rrrr$_r observatory descriptionrrr_parse_entry_fflSs  $r/c@s(eZdZdZddZeefddZdS)r zQuick version of lal.utils.CacheEntry for internal purposes only Just to allow metadata handling for files that don't follow LIGO-T050017. cCs|jS)N)r)selfrrr__str__esz_CacheEntry.__str__c Cst|tr|d}|}t|dkrF|d}|t||fSy t||dStt t fk r}z:y t ||dSt k rYnXd|f|_ Wdd}~XYnXdS)Nzutf-8rr)r"z'Cannot identify format for cache entry ) isinstancebytesdecodestripr*lenr r/ RuntimeError TypeErrorr+r%args)clsr!r"partsrexcrrrparsehs      z_CacheEntry.parseN)__name__ __module__ __qualname____doc__r1 classmethodrr=rrrrr _sr r-r.segmentrc csytj|j}Wntk r*d}YnXx|D]}ytj||dVWq2tk r|}t |dkrtj|d|krt |dd"}xt ||dD] }|VqWWdQRXnYq2Xq2WdS)zInternal method that yields a `_CacheEntry` for each line in the file This method supports reading LAL- and (nested) FFL-format cache files. N)r"rr) r(rabspathnameAttributeErrorr r=r+r*r6open _iter_cache) cachefiler"rr!r;Zcache2rrrrrJs   rJcCs<ddt||dD}|r(t|||d}|r8|j|d|S)aBRead a LAL- or FFL-format cache file as a list of file paths Parameters ---------- cachefile : `str`, `pathlib.Path`, `file` Input file or file path to read. coltype : `LIGOTimeGPS`, `int`, optional Type for GPS times. sort : `callable`, optional A callable key function by which to sort the output list of file paths segment : `gwpy.segments.Segment`, optional A GPS `[start, stop)` interval, if given only files overlapping this interval will be returned. strict : `bool`, optional If `False` warn about entries that don't follow the LIGO-T050017 standard, then skip them; if `True` all errors are raised as exceptions. Returns ------- paths : `list` of `str` A list of file paths as read from the cache file. cSsg|] }|jqSr)r).0xrrr szread_cache..)r")rCstrict)key)rJsievesort)rKcoltyperRrCrOcacherrr read_caches $ rUcCstj||djS)akRead a file path from a line in a cache file. Parameters ---------- line : `str`, `bytes` Line of text to parse gpstype : `LIGOTimeGPS`, `int`, optional Type for GPS times. Returns ------- path : `str` The file path. Raises ------ ValueError if the line cannot be parsed successfully )r")r r=r)r!r"rrrread_cache_entrysrVcOstdtt||S)Nz/gwpy.io.cache.open_cache was renamed read_cache)warningswarnDeprecationWarningrU)r9kwargsrrr open_cachesr[w)modeposc Cs|dkrt}n2|dkr t}n |dkr2t}ntd|xNt||D]@}yt||dWqLtk r||d dYqLXqLWdS)aWrite a `list` of cache entries to a file Parameters ---------- cache : `list` of `str` The list of file paths to write fobj : `file`, `str`, `pathlib.Path` The open file object, or file path to write to. format : `str`, optional The format to write to, one of - `None` : format each entry using `str` - ``'lal'`` : write a LAL-format cache - ``'ffl'`` : write an FFL-format cache NZlalfflzUnrecognised cache format )file zutf-8) strlowerrr&r+mapprintr8writeencode)rTZfobjformat formatterr!rrr write_caches  rjcCstt|ttjftr>yttt|tdSt k r<dSXt rPt|t rPdStt|t t fop|opttt|S)a<Returns `True` if ``cache`` is a readable cache file or object Parameters ---------- cache : `str`, `file`, `list` Object to detect as cache Returns ------- iscache : `bool` `True` if the input object is a cache, or a file in LAL cache format, otherwise `False` )rSFT)r2rbr(PathLikerboolr6rUr Exception HAS_CACHEr listtupleallrdis_cache_entry)rTrrris_cachesrsc Cs>trt|trdSy t|Wntttfk r8dSXdS)aReturns `True` if ``path`` can be represented as a cache entry In practice this just tests whether the input is |LIGO-T050017|_ compliant. Parameters ---------- path : `str`, :class:`lal.utils.CacheEntry` The input to test Returns ------- isentry : `bool` `True` if ``path`` is an instance of `CacheEntry`, or can be parsed using |LIGO-T050017|_. TF)HAS_CACHEENTRYr2r file_segmentr+r8rH)rrrrrr,s rrc Csddlm}tj|}y|d\}}}}Wn4tk rb}zd|df|_Wdd}~XYnXt|}| ddd }xBy t|}Wn.tk rd|kr| ddd }Yq~XPq~W||||||fS) aReturn metadata parsed from a filename following LIGO-T050017 This method is lenient with regards to integers in the GPS start time of the file, as opposed to `gwdatafind.utils.filename_metadata`, which is strict. Parameters ---------- filename : `str` the path name of a file Returns ------- obs : `str` the observatory metadata tag : `str` the file tag segment : `gwpy.segments.Segment` the GPS ``[float, float)`` interval for this file Notes ----- `LIGO-T050017 `__ declares a file naming convention that includes documenting the GPS start integer and integer duration of a file, see that document for more details. Examples -------- >>> from gwpy.io.cache import filename_metadata >>> filename_metadata("A-B-0-1.txt") ('A', 'B', Segment(0, 1)) >>> filename_metadata("A-B-0.456-1.345.txt") ("A", "B", Segment(0.456, 1.801)) r)rr'zFailed to parse z& as a LIGO-T050017-compatible filenameN.rr) rrr(rr)r*r+r9rrsplit)filenamerrGrr#rr$r<rrrr Gs$%   r cCs8ddlm}y ||jStk r2t|dSXdS)aReturn the data segment for a filename following T050017 Parameters --------- filename : `str`, :class:`~lal.utils.CacheEntry` the path name of a file Returns ------- segment : `~gwpy.segments.Segment` the ``[start, stop)`` GPS segment covered by the given file Notes ----- |LIGO-T050017|_ declares a filenaming convention that includes documenting the GPS start integer and integer duration of a file, see that document for more details. r)rN)rrrCrHr )rxrrrrrus   rucGs<ddlm}|}x |D]}|dd|DqW|S)aVReturns the segments of data covered by entries in the cache(s). Parameters ---------- *caches : `list` One or more lists of file paths (`str` or :class:`~lal.utils.CacheEntry`). Returns ------- segments : `~gwpy.segments.SegmentList` A list of segments for when data should be available r) SegmentListcss|]}t|VqdS)N)ru)rLerrr sz!cache_segments..)rryextendZcoalesce)cachesryoutrTrrrcache_segmentss   rcGsttdd|DS)a;Flatten a nested list of cache entries Parameters ---------- *caches : `list` One or more lists of file paths (`str` or :class:`~lal.utils.CacheEntry`). Returns ------- flat : `list` A flat `list` containing the unique set of entries across each input. css|]}|D] }|Vq qdS)Nr)rLcrzrrrr{szflatten..)rorfromkeys)r}rrrflattensrcgs,t|}xt|D]}t||dVqWdS)aDSeparate one or more cache entry lists into time-contiguous sub-lists Parameters ---------- *caches : `list` One or more lists of file paths (`str` or :class:`~lal.utils.CacheEntry`). Returns ------- caches : `iter` of `list` an interable yielding each contiguous cache )rCN)rrrQ)r}ZflatrCrrrfind_contiguoussrc Csvt|}xf|D]^}y t|}Wn8tk rX}z|r8tt|wWdd}~XYnX||r||qW|S)aVFilter the cache to find those entries that overlap ``segment`` Parameters ---------- cache : `list` Input list of file paths segment : `~gwpy.segments.Segment` The ``[start, stop)`` interval to match against. strict : `bool`, optional If `True` raise all exceptions, if `False` emit warnings when the file segment cannot be determined. Warns ----- UserWarning For any files for which the file segment cannot be determined, if ``strict=False`` is given; these files are excluded from the sieved cache. N)rrur+rWrXrbZ intersectsappend)rTrCrOr~rzrr<rrrrQs    rQcOs"ddlm}tdt|||S)Nr)rz7this function has been moved to gwpy.io.utils.file_path)utilsrrWrXrY)r9rZrrrr file_names rcOs"ddlm}tdt|||S)Nr) file_listz7this function has been moved to gwpy.io.utils.file_list)rrrWrXrY)r9rZrrrrrs r)N)NT))rAr(rW collectionsrrtimerrrrr Z lal.utilsr ImportErrorrtZglue.lalr rn __author__rrr%r&r/r rJrUrVr[rjrsrrr rurrrrQrrrrrrsV      $ *   #< (