B dL@sdZddlmZddlmZddlmZddlm Z ddl Z ddl m Z ddlmZd d lmZd d lmZmZd Zd dddddddgZd)dd Zee ddZ ddZddZd*ddZd+ddZd,ddZd-ddZd.d!d"Zd/d%dZd0d&dZ d1d(dZ!dS)2zUser interface functions for GWDataFind. These are all imported to the top-level namespace, so should be referred to in usage as ``gwdatafind.`` and not ``gwdatafind.ui.`` )wraps)compile)urlparse)warnN)get)segments)api) file_segmentget_default_hostz(Duncan Macleod connectpingfind_observatories find_types find_times find_latestfind_url find_urlsc Csddl}ddlm}m}ddlm}tdt|dkr>> find_observatories(host="datafind.gw-openscience.org") ['L', 'V', 'H'] >>> find_observatories(match="H", host="datafind.gw-openscience.org") ['H'] rB) rAr Zfind_observatories_pathsetr: compile_regexsearchr0filter)matchrrBrCrDsitesr'r'r(rs /  cKsLt|tj|d}tt|fd|i|}|rDt|j}tt||St|S)aJQuery a GWDataFind host for dataset types. Parameters ---------- site : `str`, optional single-character name of site to match; if not given types for all sites will be returned. match : `str`, `re.Pattern`, optional regular expression to match against known types host : `str`, optional the URL/name of the GWDataFind host to query; if not given :func:`~gwdatafind.utils.get_default_host` will be used to discover the default host. session : `requests.Session`, optional the connection session to use; if not given, a :class:`~igwn_auth_utils.Session` will be created on-the-fly token, token_audience, token_scope passed directly to :class:`~igwn_auth_utils.Session`, see :ref:`scitokens` for more details. cert passed directly to :class:`~igwn_auth_utils.Session`, see :ref:`x509` for more details. request_kw other keywords are passed to :func:`igwn_auth_utils.get` Returns ------- types : `list` of `str` list of dataset types Raises ------ requests.RequestsException if the request fails for any reason Examples -------- >>> find_types(host="datafind.gw-openscience.org") ['H2_LOSC_4_V1', 'V1_GWOSC_O3a_16KHZ_R1', 'H1_LOSC_16_V1', 'L1_LOSC_4_V1', 'V1_GWOSC_O2_16KHZ_R1', 'V1_GWOSC_O3a_4KHZ_R1', 'L1_GWOSC_O3a_4KHZ_R1', 'L1_GWOSC_O2_16KHZ_R1', 'L1_GWOSC_O2_4KHZ_R1', 'V1_GWOSC_O2_4KHZ_R1', 'H1_LOSC_4_V1', 'H1_GWOSC_O3a_16KHZ_R1', 'H1_GWOSC_O2_16KHZ_R1', 'H1_GWOSC_O3a_4KHZ_R1', 'L1_GWOSC_O3a_16KHZ_R1', 'H1_GWOSC_O2_4KHZ_R1', 'L1_LOSC_16_V1'] >>> find_types(site='V', host="datafind.gw-openscience.org") ['V1_GWOSC_O3a_4KHZ_R1', 'V1_GWOSC_O3a_16KHZ_R1', 'V1_GWOSC_O2_4KHZ_R1', 'V1_GWOSC_O2_16KHZ_R1'] (accurate as of Nov 18 2021) )siterB) rAr Zfind_types_pathrEr:rFrGr0rH)rKrIrrBrCrDtypesr'r'r(rs 4 c Ks:t|tj||||}t|fd|i|}tttj|S)aQuery a GWDataFind host for times in which data are available. Parameters ---------- site : `str` single-character name of site to match frametype : `str` name of dataset to match gpsstart : `int`, optional GPS start time of query gpsend : `int`, optional GPS end time of query match : `str`, `re.Pattern`, optional regular expression to match against known types host : `str`, optional the URL/name of the GWDataFind host to query; if not given :func:`~gwdatafind.utils.get_default_host` will be used to discover the default host. session : `requests.Session`, optional the connection session to use; if not given, a :class:`igwn_auth_utils.requests.Session` will be created on-the-fly token, token_audience, token_scope passed directly to :class:`~igwn_auth_utils.Session`, see :ref:`scitokens` for more details. cert passed directly to :class:`~igwn_auth_utils.Session`, see :ref:`x509` for more details. request_kw other keywords are passed to :func:`igwn_auth_utils.get` Returns ------- segments : `segments.segmentlist` the list of ``[start, end)`` intervals during which data are available for the relevant dataset. Examples -------- >>> find_times( ... "V", ... "V1_GWOSC_O3a_4KHZ_R1", ... gpsstart=1238249472, ... gpsend=1239429120, ... host="datafind.gw-openscience.org", ... ) [segment(1238249472, 1238417408), segment(1238421504, 1238605824), segment(1238609920, 1238827008), segment(1238839296, 1239429120)] Raises ------ requests.RequestsException if the request fails for any reason rB)rAr Zfind_times_pathr:r segmentlistmapsegment) rK frametypegpsstartgpsendrrBrCrDtimesr'r'r(rTsGignorec sZt|f|}r&ttfdd|}|s2|dkr6|Sd}|dkrNt||St|dS)Ncst|jkS)N)rr5)e)r5r'r(z_get_urls..rTzno files foundr)r:r0rHr RuntimeError)rDr5 on_missingr4urlserrr')r5r( _get_urlss  r\fileerrorcKs&t|tj|}t|f|||d|S)a*Query a GWDataFind host for the URL of a single filename. Parameters ---------- framefile : `str` the name of the file to match; note that only the basename of the file is relevant. urltype : `str`, optional URL scheme to search for on_missing : `str`, optional what to do when the requested file isn't found, one of: - ``'error'``: raise a `RuntimeError` - ``'warn'``: print a warning but return an empty `list` - ``'ignore'``: return an empty `list` with no warnings host : `str`, optional the URL/name of the GWDataFind host to query; if not given :func:`~gwdatafind.utils.get_default_host` will be used to discover the default host. session : `requests.Session`, optional the connection session to use; if not given, a :class:`igwn_auth_utils.requests.Session` will be created on-the-fly token, token_audience, token_scope passed directly to :class:`~igwn_auth_utils.Session`, see :ref:`scitokens` for more details. cert passed directly to :class:`~igwn_auth_utils.Session`, see :ref:`x509` for more details. request_kw other keywords are passed to :func:`igwn_auth_utils.get` Returns ------- urls : `list` of `str` a list of URLs for all instances of ``filename`` Raises ------ requests.RequestsException if the request fails for any reason RuntimeError if no matching URLs are found and ``on_missing="error"`` was given )r5rYrB)rAr Z find_url_pathr\)Z framefileurltyperYrrBrCrDr'r'r(rs<cKs(t|tj|||d}t|fd|i|S)a,Query a GWDataFind host for the latest file in a given dataset. Parameters ---------- site : `str` single-character name of site to match frametype : `str` name of dataset to match urltype : `str`, optional URL scheme to search for on_missing : `str`, optional what to do when the requested file isn't found, one of: - ``'error'``: raise a `RuntimeError` - ``'warn'``: print a warning but return an empty `list` - ``'ignore'``: return an empty `list` with no warnings host : `str`, optional the URL/name of the GWDataFind host to query; if not given :func:`~gwdatafind.utils.get_default_host` will be used to discover the default host. session : `requests.Session`, optional the connection session to use; if not given, a :class:`igwn_auth_utils.requests.Session` will be created on-the-fly token, token_audience, token_scope passed directly to :class:`~igwn_auth_utils.Session`, see :ref:`scitokens` for more details. cert passed directly to :class:`~igwn_auth_utils.Session`, see :ref:`x509` for more details. request_kw other keywords are passed to :func:`igwn_auth_utils.get` Returns ------- urls : `list` of `str` a list of URLs for the latest file found Raises ------ requests.RequestsException if the request fails for any reason RuntimeError if no latest file is found and ``on_missing="error"`` was given Examples -------- >>> find_latest('H', 'H1_GWOSC_O2_4KHZ_R1', urltype='file', host='datafind.gw-openscience.org')) ['file://localhost/cvmfs/gwosc.osgstorage.org/gwdata/O2/strain.4k/frame.v1/H1/1186988032/H-H1_GWOSC_O2_4KHZ_R1-1187733504-4096.gwf'] )r_rY)rAr Zfind_latest_pathr\)rKrPr_rYrrBrCrDr'r'r(rsDrc  Kst|tj||||||d} t| fd|i| } |dkr:| St||} ttt|  } t| g|  }|sv| Sdd tt |}|dkrt || St |dS)aQuery a GWDataFind host for all URLs for a dataset in an interval. Parameters ---------- site : `str` single-character name of site to match frametype : `str` name of dataset to match gpsstart : `int`, optional GPS start time of interval gpsend : `int`, optional GPS end time of interval match : `str`, `re.Pattern`, optional regular expression pattern to match URLs against urltype : `str`, optional URL scheme to search for on_gaps : `str`, optional what to do when the requested all or some of the GPS interval is not covereed by the dataset, one of: - ``'error'``: raise a `RuntimeError` - ``'warn'``: print a warning but return all available URLs - ``'ignore'``: return the list of URLs with no warnings host : `str`, optional the URL/name of the GWDataFind host to query; if not given :func:`~gwdatafind.utils.get_default_host` will be used to discover the default host. session : `requests.Session`, optional the connection session to use; if not given, a :class:`igwn_auth_utils.requests.Session` will be created on-the-fly token, token_audience, token_scope passed directly to :class:`~igwn_auth_utils.Session`, see :ref:`scitokens` for more details. cert passed directly to :class:`~igwn_auth_utils.Session`, see :ref:`x509` for more details. request_kw other keywords are passed to :func:`igwn_auth_utils.get` Raises ------ requests.RequestsException if the request fails for any reason RuntimeError if gaps in the dataset are found and ``on_gaps="error"`` was given )r_rIrBrTzMissing segments: %s rN)rAr Zfind_urls_pathr\rrOrMrNr ZcoalescejoinstrrrX)rKrPrQrRrIr_Zon_gapsrrBrCrDrZspanZseglistmissingmsgr'r'r(r?s,G )NN)NN)NNN)NNNN)NNNN)NrT)r]r^NN)r]r^NN)Nr]rNN)"__doc__ functoolsrrerrF urllib.parserwarningsrr,Zigwn_auth_utils.requestsrr1Zligorr rr r __author____all__r r:rAr rrrr\rrrr'r'r'r(sV        5% $ 7 ? F  D G