B ~df@sdZddlZddlmZddlmZddlmZddlZddl m Z ddl m Z mZddlmZd d lmZd d lmZd d lmZmZdZdddgZddZddZeGddde ZdS)z2Extend :mod:`astropy.table` with the `EventTable` N)wraps) attrgetter)ceil) DEFAULT_URL)Tablevstack)registry) read_multi) gps_types) filter_tableparse_operatorz(Duncan Macleod timeZgpsZpeakGPScCs|jd}xtj|dD]}|d}|ddkrNtj||t||dd|ddkrxtj||t||dd|d dkrtj||tj ||fddqW|S) Nr )Z data_classZFormatZReadyesF)forceZWritez Auto-identify) __mro__rZ get_formatslowerZregister_readerZ get_readerZregister_writerZ get_writerZregister_identifierZ _identifiers)clsparentrownamer]/work/yifan.wang/ringdown/master-ringdown-env/lib/python3.7/site-packages/gwpy/table/table.pyinherit_io_registrations2s,     rcstfdd}|S)Nc s|d}|d}|d}|dkr6|dks<|dks<|jsy||d<Wn8tk r}zd|jdf|_Wdd}~XYnX|d|jd||d}|dkr||d<|dkr||d<|f||S)N timecolumnstartendz%{0}, please give `timecolumn` keywordr) getZcolnames_get_time_column ValueErrorformatargs setdefaultminmax)selfr"kwargsrrrexctimes)funcrr wrapped_funcRs$      z'_rates_preprocess..wrapped_func)r)r*r+r)r*r_rates_preprocessQsr,c@seZdZdZddZddZeddZdd Zed d Z ed d e fd dZ ddZ e d$ddZe d%ddZddZddZddZddZddZd d!Zd"d#Zd S)& EventTableaKA container for a table of events. This object expands the default :class:`~astropy.table.Table` with extra read/write formats, and methods to perform filtering, rate calculations, and visualisation. See also -------- astropy.table.Table for details on parameters for creating an `EventTable` cCsN|tkrdS||jtkr"dSyt||dtStk rHdSXdS)aKReturn `True` if a column in this table represents 'time' This method checks the name of the column against a hardcoded list of time-like names, then checks the `dtype` of the column (or the first element in the column) against a hardcoded list of time-like dtypes (`gwpy.time.gps_types`). TrFN)rTIME_LIKE_COLUMN_NAMESdtyper isinstance IndexError)r&rrrr_is_time_columns zEventTable._is_time_columnc Cs~tt|j|j}y |\}Wn\tk rxddttt dd}d |}t |dkrl| dd}t|YnX|S)a^Return the name of the 'time' column in this table. This method tries the following: - look for a single column named 'time', 'gps', or 'peakGPS' - look for a single column with a GPS type (e.g. `LIGOTimeGPS`) So, its not foolproof. Raises a `ValueError` if either 0 or multiple matches are found. z, or z, r zOcannot identify time column for table, no columns named {}, or with a GPS dtypez no columnszmultiple columns) listfilterr2columnsr joinmapreprr.rsplitr!lenreplace)r&matchesrZ tcolnamesmsgrrrrs    zEventTable._get_time_columncOstt||f||S)aRead data into an `EventTable` Parameters ---------- source : `str`, `list` Source of data, any of the following: - `str` path of single data file, - `str` path of LAL-format cache file, - `list` of paths. *args other positional arguments will be passed directly to the underlying reader method for the given format format : `str`, optional the format of the given source files; if not given, an attempt will be made to automatically identify the format columns : `list` of `str`, optional the list of column names to read selection : `str`, or `list` of `str`, optional one or more column filters with which to downselect the returned table rows as they as read, e.g. ``'snr > 5'``; multiple selections should be connected by ' && ', or given as a `list`, e.g. ``'snr > 5 && frequency < 1000'`` or ``['snr > 5', 'frequency < 1000']`` nproc : `int`, optional, default: 1 number of CPUs to use for parallel reading of multiple files verbose : `bool`, optional print a progress bar showing read status, default: `False` .. note:: Keyword arguments other than those listed here may be required depending on the `format` Returns ------- table : `EventTable` Raises ------ astropy.io.registry.IORegistryError if the `format` cannot be automatically identified IndexError if ``source`` is an empty list Notes -----) io_read_multir)rsourcer"r'rrrreads7zEventTable.readcOstj||f||S)aWrite this table to a file Parameters ---------- target: `str` filename for output data file *args other positional arguments will be passed directly to the underlying writer method for the given format format : `str`, optional format for output data; if not given, an attempt will be made to automatically identify the format based on the `target` filename **kwargs other keyword arguments will be passed directly to the underlying writer method for the given format Raises ------ astropy.io.registry.IORegistryError if the `format` cannot be automatically identified Notes -----)rwrite)r&targetr"r'rrrrAszEventTable.writec Osyddlm}Wntk r$Yn,Xt||rPddlm}|||f||Sddlm}|||}|||}t||st|t |ry||St k r} zd |j t | f| _Wdd} ~ XYnXtd |j |S) aFetch a table of events from a database Parameters ---------- format : `str`, `~sqlalchemy.engine.Engine` the format of the remote data, see _Notes_ for a list of registered formats, OR an SQL database `Engine` object *args all other positional arguments are specific to the data format, see below for basic usage columns : `list` of `str`, optional the columns to fetch from the database table, defaults to all selection : `str`, or `list` of `str`, optional one or more column filters with which to downselect the returned table rows as they as read, e.g. ``'snr > 5'``; multiple selections should be connected by ' && ', or given as a `list`, e.g. ``'snr > 5 && frequency < 1000'`` or ``['snr > 5', 'frequency < 1000']`` **kwargs all other positional arguments are specific to the data format, see the online documentation for more details Returns ------- table : `EventTable` a table of events recovered from the remote database Examples -------- >>> from gwpy.table import EventTable To download a table of all blip glitches from the Gravity Spy database: >>> EventTable.fetch( ... 'gravityspy', ... 'glitches', ... selection=['ml_label=Blip', 'ml_confidence>0.9'], ... ) To download a table from any SQL-type server >>> from sqlalchemy.engine import create_engine >>> engine = create_engine(...) >>> EventTable.fetch(engine, 'mytable') Notes -----r)Enginer )fetch) get_fetcherz,could not convert fetch() output to {0}: {1}Nz$fetch() should return a {0} instance)Zsqlalchemy.enginerC ImportErrorr0Zio.sqlrDZio.fetchrE issubclasstype Exceptionr!__name__strr" TypeError) rZformat_r"r'rCrDrEZfetcheroutr(rrrrD s*7      zEventTable.fetchNcKs<ddlm}||f|||d|}t||kr4|S||S)aFetch events from an open-data catalogue hosted by GWOSC. Parameters ---------- catalog : `str` the name of the catalog to fetch, e.g. ``'GWTC-1-confident'`` columns : `list` of `str`, optional the list of column names to read selection : `str`, or `list` of `str`, optional one or more column filters with which to downselect the returned events as they as read, e.g. ``'mass1 < 30'``; multiple selections should be connected by ' && ', or given as a `list`, e.g. ``'mchirp < 3 && distance < 500'`` or ``['mchirp < 3', 'distance < 500']`` host : `str`, optional the open-data host to use r ) fetch_catalog)r5 selectionhost)Zio.loscrNrH)rcatalogr5rOrPr'rNtabrrrfetch_open_data_s    zEventTable.fetch_open_datacCs||S)aReturn the `Column` with the given name This method is provided only for compatibility with the :class:`ligo.lw.table.Table`. Parameters ---------- name : `str` the name of the column to return Returns ------- column : `astropy.table.Column` Raises ------ KeyError if no column is found with the given name r)r&rrrr get_columnszEventTable.get_columnc Csddlm}||}|jjdkr.|jddd}tt|||}t|d||}|tj ||ddt |||d d d S) aCalculate the rate `~gwpy.timeseries.TimeSeries` for this `Table`. Parameters ---------- stride : `float` size (seconds) of each time bin start : `float`, `~gwpy.time.LIGOTimeGPS`, optional GPS start epoch of rate `~gwpy.timeseries.TimeSeries` end : `float`, `~gwpy.time.LIGOTimeGPS`, optional GPS end time of rate `~gwpy.timeseries.TimeSeries`. This value will be rounded up to the nearest sample if needed. timecolumn : `str`, optional name of time-column to use when binning events, attempts are made to guess this Returns ------- rate : `~gwpy.timeseries.TimeSeries` a `TimeSeries` of events per second (Hz) Raises ------ ValueError if the ``timecolumn`` cannot be guessed from the table contents r) TimeSeriesobjectZ longdoubleF)copyr )binsZHzz Event rate)t0dtunitr) gwpy.timeseriesrUr/rZastypeintrnumpyZarangeZ histogramfloat) r&striderrrrUr)ZnsampZtimebinsrrr event_rates  zEventTable.event_rate>=csddlm}s tj tjfg|dkrVtdtsVfddtddDnt|trjt|} n|} ||} |} xrD]j} t| tr| | dk| | dk@} n | | | } || j ||||d | | <d |t|t| f| | _ qW| S) aCalculate an event rate `~gwpy.timeseries.TimeSeriesDict` over a number of bins. Parameters ---------- stride : `float` size (seconds) of each time bin column : `str` name of column by which to bin. bins : `list` a list of `tuples ` marking containing bins, or a list of `floats ` defining bin edges against which an math operation is performed for each event. operator : `str`, `callable` one of: - ``'<'``, ``'<='``, ``'>'``, ``'>='``, ``'=='``, ``'!='``, for a standard mathematical operation, - ``'in'`` to use the list of bins as containing bin edges, or - a callable function that takes compares an event value against the bin value and returns a boolean. .. note:: If ``bins`` is given as a list of tuples, this argument is ignored. start : `float`, `~gwpy.time.LIGOTimeGPS`, optional GPS start epoch of rate `~gwpy.timeseries.TimeSeries`. end : `float`, `~gwpy.time.LIGOTimeGPS`, optional GPS end time of rate `~gwpy.timeseries.TimeSeries`. This value will be rounded up to the nearest sample if needed. timecolumn : `str`, optional, default: ``time`` name of time-column to use when binning events Returns ------- rates : ~gwpy.timeseries.TimeSeriesDict` a dict of (bin, `~gwpy.timeseries.TimeSeries`) pairs describing a rate of events per second (Hz) for each of the bins. r)TimeSeriesDictincs g|]\}}||dfqS)r r).0ibin_)rXrr sz1EventTable.binned_event_rates..Nr )rrr ) r\rcr^infr0tuple enumeraterKrrar6r)r&r`columnrXoperatorrrrrcZop_funcZcoldatarMrgZkeepr)rXrbinned_event_ratess$3        "zEventTable.binned_event_ratescOs$tdt|jt|j||S)z-DEPRECATED, use `EventTable.scatter` zK{0}.plot was renamed {0}.scatter and will be removed in an upcoming release)warningswarnr!rHrJDeprecationWarningscatter)r&r"r'rrrplots zEventTable.plotcKs:|dd}|dk r |||d<|jd||||f|S)aMake a scatter plot of column ``x`` vs column ``y``. Parameters ---------- x : `str` name of column defining centre point on the X-axis y : `str` name of column defining centre point on the Y-axis color : `str`, optional, default:`None` name of column by which to color markers **kwargs any other keyword arguments, see below Returns ------- plot : `~gwpy.plot.Plot` the newly created figure See also -------- matplotlib.pyplot.figure for documentation of keyword arguments used to create the figure matplotlib.figure.Figure.add_subplot for documentation of keyword arguments used to create the axes gwpy.plot.Axes.scatter for documentation of keyword arguments used to display the table colorNcrt)pop_plot)r&xyr'rvrrrrts!  zEventTable.scattercKsF|dd}|dk r |||d<|jd||||||||f|S)a7Make a tile plot of this table. Parameters ---------- x : `str` name of column defining anchor point on the X-axis y : `str` name of column defining anchor point on the Y-axis w : `str` name of column defining extent on the X-axis (width) h : `str` name of column defining extent on the Y-axis (height) color : `str`, optional, default:`None` name of column by which to color markers **kwargs any other keyword arguments, see below Returns ------- plot : `~gwpy.plot.Plot` the newly created figure See also -------- matplotlib.pyplot.figure for documentation of keyword arguments used to create the figure matplotlib.figure.Figure.add_subplot for documentation of keyword arguments used to create the axes gwpy.plot.Axes.tile for documentation of keyword arguments used to display the table rvNtile)rxry)r&rzr{whr'rvrrrr|=s'  zEventTable.tilec Osddlm}ddlm}ddlm}||djrL|dd|dd ||d <|||}| }x~t t t d |j |jf|ddD]V\} } | j} |d rd || j} | jdk r| d| jd7} | | d| _qW|S)Nr)rcParamsr )Plot)label_to_latexZfigsize) Zxscalezauto-gpsmethodisDefault_labelz text.usetexz\texttt{{{0}}}z [{0}]Z latex_inlineT)Z matplotlibrrurZplot.texrr2rr#Zgcazipr4rZxaxisZyaxisr!r[Z to_stringZset_label_textr) r&rr"r'rrrruZaxZaxiscolrrrrryis(         zEventTable._plotcKs$ddlm}|||fddi|S)aGenerate a `HistogramPlot` of this `Table`. Parameters ---------- column : `str` Name of the column over which to histogram data method : `str`, optional Name of `~matplotlib.axes.Axes` method to use to plot the histogram, default: ``'hist'``. **kwargs Any other keyword arguments, see below. Returns ------- plot : `~gwpy.plot.Plot` The newly created figure. See also -------- matplotlib.pyplot.figure for documentation of keyword arguments used to create the figure. matplotlib.figure.Figure.add_subplot for documentation of keyword arguments used to create the axes. gwpy.plot.Axes.hist for documentation of keyword arguments used to display the histogram, if the ``method`` keyword is given, this method might not actually be the one used. r )rrhist)rur)r&rnr'rrrrrs! zEventTable.histcGst|f|S)aApply one or more column slice filters to this `EventTable` Multiple column filters can be given, and will be applied concurrently Parameters ---------- column_filter : `str`, `tuple` a column slice filter definition, e.g. ``'snr > 10``, or a filter tuple definition, e.g. ``('snr', , )`` Notes ----- See :ref:`gwpy-table-filter` for more details on using filter tuples Returns ------- table : `EventTable` a new table with only those rows matching the filters Examples -------- To filter an existing `EventTable` (``table``) to include only rows with ``snr`` greater than `10`, and ``frequency`` less than `1000`: >>> table.filter('snr>10', 'frequency<1000') Custom operations can be defined using filter tuple definitions: >>> from gwpy.table.filters import in_segmentlist >>> table.filter(('time', in_segmentlist, segs)) )r )r&Zcolumn_filtersrrrr4s"zEventTable.filterc s|dkrtdt|dkr$|St||}|||}|||tt||kd}t|dkrv|St|tt|dkdd}dd|D}fdd|D} tj|t d} d | t |<d | | <||| S) a~Cluster this `EventTable` over a given column, `index`, maximizing over a specified column in the table, `rank`. The clustering algorithm uses a pooling method to identify groups of points that are all separated in `index` by less than `window`. Each cluster of nearby points is replaced by the point in that cluster with the maximum value of `rank`. Parameters ---------- index : `str` name of the column which is used to search for clusters rank : `str` name of the column to maximize over in each cluster window : `float` window to use when clustering data points, will raise ValueError if `window > 0` is not satisfied Returns ------- table : `EventTable` a new table that has had the clustering algorithm applied via slicing of the original Examples -------- To cluster an `EventTable` (``table``) whose `index` is `end_time`, `window` is `0.1`, and maximize over `snr`: >>> table.cluster('end_time', 'snr', 0.1) gzWindow must be a positive valuerr c Ss(g|] }t|t|ddgqS)rir )r^appendarray)resrrrrh sz&EventTable.cluster..csg|]}|t|qSr)r^Zargmax)rer)paramrrrh s)r/FT) r r:rWr^ZargsortwherediffsplitZ ones_likeboolZ concatenate) r&indexZrankZwindowZorderidxrZ clusterpointsZsublistsZpadded_sublistsZmaxidxmaskr)rrclusters&#    zEventTable.cluster)NNN)rbNNN)rJ __module__ __qualname____doc__r2r classmethodr@rArDDEFAULT_GWOSC_URLrSrTr,rarprurtr|ryrr4rrrrrr-ss,  9 T ) L&,$$r-)rrq functoolsrrormathrr^Z gwosc.apirrZ astropy.tablerrZ astropy.iorZio.mpr r>rr r4r r __author__r.rr,r-rrrrs&       "