pycbc_make_inference_workflow: A parameter estimation workflow generator

Introduction

The executable pycbc_make_inference_workflow is a workflow generator to setup a parameter estimation analysis.

Workflow configuration file

A sample workflow configuration file:

[workflow]
; basic information used by the workflow generator
file-retention-level = all_triggers
h1-channel-name = H1:DCS-CALIB_STRAIN_C02
l1-channel-name = L1:DCS-CALIB_STRAIN_C02
start-time = 1126259452
end-time = 1126259468

[workflow-ifos]
; the IFOs to analyze
h1 =
l1 =

[workflow-datafind]
; how the workflow generator should get frame data
datafind-h1-frame-type = H1_HOFT_C02
datafind-l1-frame-type = L1_HOFT_C02
datafind-method = AT_RUNTIME_SINGLE_FRAMES
datafind-check-segment-gaps = raise_error
datafind-check-frames-exist = raise_error
datafind-check-segment-summary = no_test

[workflow-inference]
; how the workflow generator should setup inference nodes
num-events = 1
plot-1d-mass = mass1 mass2 mchirp q
plot-1d-orientation = ra dec tc polarization inclination coa_phase
plot-1d-distance = distance redshift

[executables]
; paths to executables to use in workflow
inference = ${which:pycbc_inference}
inference_posterior = ${which:pycbc_inference_plot_posterior}
inference_prior = ${which:pycbc_inference_plot_prior}
inference_rate = ${which:pycbc_inference_plot_acceptance_rate}
inference_samples = ${which:pycbc_inference_plot_samples}
inference_table = ${which:pycbc_inference_table_summary}
plot_spectrum = ${which:pycbc_plot_psd_file}
results_page = ${which:pycbc_make_html_page}

[datafind]
; datafind options
urltype = file

[inference]
; command line options use --help for more information
verbose =
sample-rate = 2048
strain-high-pass = 20
pad-data = 8
psd-estimation = median
psd-segment-length = 16
psd-segment-stride = 8
psd-inverse-length = 16
processing-scheme = mkl
data-conditioning-low-freq = 25

[pegasus_profile-inference]
; pegasus profile for inference nodes
condor|request_memory = 20G
condor|request_cpus = 12 

[inference_posterior]
; command line options use --help for more information
plot-density =
plot-contours =
plot-marginal =
z-arg = snr

[inference_prior]
; command line options use --help for more information

[inference_rate]
; command line options use --help for more information

[inference_samples]
; command line options use --help for more information

[inference_table]
; command line options use --help for more information

[plot_spectrum]
; command line options use --help for more information

[results_page]
; command line options use --help for more information
analysis-title = "PyCBC Inference Test"

Download

Inference configuration file

You will also need a configuration file with sections that tells pycbc_inference how to construct the priors. A sample inference configuration file is:

[model]
name = marginalized_phase
h1-low-frequency-cutoff = 20
l1-low-frequency-cutoff = 20

[sampler]
name = emcee_pt
nwalkers = 1000
ntemps = 4
effective-nsamples = 2000
checkpoint-interval = 2000
max-samples-per-chain = 1000

[sampler-burn_in]
burn-in-test = nacl & max_posterior

[variable_params]
; waveform parameters that will vary in MCMC
tc =
mass1 =
mass2 =
spin1_a =
spin1_azimuthal =
spin1_polar =
spin2_a =
spin2_azimuthal =
spin2_polar =
distance =
inclination =
polarization =
ra =
dec =

[static_params]
; waveform parameters that will not change in MCMC
approximant = IMRPhenomPv2
f_lower = 18
f_ref = 20

[prior-tc]
; coalescence time prior
name = uniform
min-tc = 1126259462.32
max-tc = 1126259462.52

[prior-mass1]
name = uniform
min-mass1 = 10.
max-mass1 = 80.

[prior-mass2]
name = uniform
min-mass2 = 10.
max-mass2 = 80.

[prior-spin1_a]
name = uniform
min-spin1_a = 0.0
max-spin1_a = 0.99

[prior-spin1_polar+spin1_azimuthal]
name = uniform_solidangle
polar-angle = spin1_polar
azimuthal-angle = spin1_azimuthal

[prior-spin2_a]
name = uniform
min-spin2_a = 0.0
max-spin2_a = 0.99

[prior-spin2_polar+spin2_azimuthal]
name = uniform_solidangle
polar-angle = spin2_polar
azimuthal-angle = spin2_azimuthal

[prior-distance]
; following gives a uniform volume prior
name = uniform_radius
min-distance = 10
max-distance = 1000

[prior-inclination]
; inclination prior
name = sin_angle

[prior-ra+dec]
; sky position prior
name = uniform_sky

[prior-polarization]
; polarization prior
name = uniform_angle

[waveform_transforms-spin1x+spin1y+spin1z]
name = spherical_spin_1_to_cartesian_spin_1

[waveform_transforms-spin2x+spin2y+spin2z]
name = spherical_spin_2_to_cartesian_spin_2

;
;   Sampling transforms
;
[sampling_params]
; parameters on the left will be sampled in
; parametes on the right
mass1, mass2 : mchirp, q

[sampling_transforms-mchirp+q]
; inputs mass1, mass2
; outputs mchirp, q
name = mass1_mass2_to_mchirp_q

Download

A sample configuration file for parameter estimation on the ringdown is:

[model]
name = gaussian_noise
h1-low-frequency-cutoff = 20
l1-low-frequency-cutoff = 20

[sampler]
name = emcee_pt
nwalkers = 1000
ntemps = 4
effective-nsamples = 2000
checkpoint-interval = 1000

[sampler-burn_in]
burn-in-test = nacl & max_posterior

[variable_args]
; waveform parameters that will vary in MCMC
final_mass =
final_spin =
amp220 =
phi220 =
inclination =

[static_args]
; waveform parameters that will not change in MCMC
approximant = TdQNMfromFinalMassSpin
lmns = ['221']
tc = 1126259462
ra = 2.2
dec = -1.24
polarization = 0.
f_lower = 18.
f_ref = 20.

[prior-final_mass]
name = uniform
min-final_mass = 20.
max-final_mass = 100.

[prior-final_spin]
name = uniform
min-final_spin = -0.99
max-final_spin = 0.99

[prior-amp220]
name = uniform_log10
min-amp220 = 1e-30
max-amp220 = 1e-10

[prior-phi220]
name = uniform_angle

[prior-inclination]
; inclination prior
name = sin_angle

Download

If you want to use another variable parameter in the inference sampler then add its name to [variable_args] and add a prior section like shown above.

Generate the workflow

To generate a workflow you will need your configuration files. If you want to run on the loudest triggers from a PyCBC coincident search workflow then run:

#! /bin/bash

set -e

# name of the workflow
WORKFLOW_NAME="search_triggers"

# path to output dir
OUTPUT_DIR=output

# input configuration files
CONFIG_PATH=workflow.ini
INFERENCE_CONFIG_PATH=inference.ini

# directory that will be populated with HTML pages
HTML_DIR=${HOME}/public_html/inference_test

# run workflow generator on triggers from workflow
pycbc_make_inference_workflow --workflow-name ${WORKFLOW_NAME} \
    --config-files ${CONFIG_PATH} \
    --inference-config-file ${INFERENCE_CONFIG_PATH} \
    --output-dir ${OUTPUT_DIR} \
    --output-file ${WORKFLOW_NAME}.dax \
    --output-map ${OUTPUT_MAP_PATH} \
    --bank-file ${BANK_PATH} \
    --statmap-file ${STATMAP_PATH} \
    --single-detector-triggers ${SNGL_H1_PATHS} ${SNGL_L1_PATHS}
    --config-overrides workflow:start-time:${WORKFLOW_START_TIME} \
                       workflow:end-time:${WORKFLOW_END_TIME} \
                       workflow-inference:data-seconds-before-trigger:8 \
                       workflow-inference:data-seconds-after-trigger:8 \
                       results_page:output-path:${HTML_DIR} \
                       results_page:analysis-subtitle:${WORKFLOW_NAME}

Download

Where ${BANK_FILE} is the path to the template bank HDF file, ${STATMAP_FILE} is the path to the combined statmap HDF file, ${SNGL_H1_PATHS} and ${SNGL_L1_PATHS} are the paths to the merged single-detector HDF files, and ${WORKFLOW_START_TIME} and ${WORKFLOW_END_TIME} are the start and end time of the coincidence workflow.

Else you can run from a specific GPS end time with the --gps-end-time option like:

#! /bin/bash

set -e

# name of the workflow
WORKFLOW_NAME="event"

# path to output dir
OUTPUT_DIR=output

# input configuration files
CONFIG_PATH=workflow.ini
INFERENCE_CONFIG_PATH=inference.ini

# directory that will be populated with HTML pages
HTML_DIR=${HOME}/public_html/inference_test

# run workflow generator on specific GPS end time
pycbc_make_inference_workflow --workflow-name ${WORKFLOW_NAME} \
    --config-files ${CONFIG_PATH} \
    --inference-config-file ${INFERENCE_CONFIG_PATH} \
    --output-dir ${OUTPUT_DIR} \
    --output-file ${WORKFLOW_NAME}.dax \
    --output-map ${OUTPUT_MAP_PATH} \
    --gps-end-time ${GPS_END_TIME} \
    --config-overrides workflow:start-time:$((${GPS_END_TIME}-2)) \
                       workflow:end-time:$((${GPS_END_TIME}+2)) \
                       workflow-inference:data-seconds-before-trigger:2 \
                       workflow-inference:data-seconds-after-trigger:2 \
                       inference:psd-start-time:$((${GPS_END_TIME}-300)) \
                       inference:psd-end-time:$((${GPS_END_TIME}+1748)) \
                       results_page:output-path:${HTML_DIR} \
                       results_page:analysis-subtitle:${WORKFLOW_NAME}

Download

Where ${GPS_END_TIME} is the GPS end time of the trigger.

For the CBC example above define the environment variables GPS_END_TIME=1126259462 and OUTPUT_MAP_PATH=output.map.

Plan and execute the workflow

If you are on LDG, you need to define an accounting group. Plan and submit the workflow with:

# submit workflow
cd ${OUTPUT_DIR}
pycbc_submit_dax --dax ${WORKFLOW_NAME}.dax \
    --no-grid \
    --enable-shared-filesystem \
    --accounting-group ${ACCOUNTING_GROUP}

Where ${ACCOUNTING_GROUP} is the appropriate tag for your workflow.