Installation

This page provides a guide to the installation of the WAT and external libraries needed to run an analysis with CWB. The packages and directories required for the installation are listed below.

Packages repository	list of all packages needed by CWB
Create directories	directories where to install libraries

After installing the libraries, run My first CWB pipeline test to check that the installed has been done correctly.

How to install the WAT libraries

Two scenarios are considered: 1) the user wants to perform an analysis with the libraries already installed at the LSC computing clusters, and 2) the user wants to perform an analysis with a local copy of the libraries (recommended for users willing to costumise the libraries).

• Using Pre-installed Librares

  The CWB WAT libraries are available at the following computing clusters: ATLAS, CIT and CNAF. The pre-installed libraries can be used by running the WAT configuration file. The environment is set up as follows (assuming tcsh shell):

  source $HOME_LIBS/GIT/cWB/library/[atlas/cit/cnaf]_watenv.csh
  cp $HOME_LIBS/GIT/cWB/library/tools/cwb/cwb.rootrc ~/.rootrc
  

  Here $HOME_LIBS depends on the considered cluster:

  ATLAS: /home/waveburst/SOFT
  CIT: /home/waveburst/SOFT
  CNAF: /opt/exp_software/virgo/virgoDev/waveburst/SOFT
  

  If using a bash shell, replace *_watenv.csh with *_watenv.sh.

  Note

  The Pre-installed Librares are aligned with the last updated master version in the local git repository. If you need a specific version you must install a local copy of the WAT library

• Installing a local copy of the WAT library (Setup Environment)

  The WAT libraries can be downloaded from the original sites or can be found in the pakages repository (see, e.g., the CIT Repository). The libraries have been tested on the Linux distributions installed at the LIGO-Virgo clusters and on Mac OS X 10.7. Instructions on the local installation of the libraries are available at the following links:

  WAT installation

  CWB config installation

  WAT filters installation

  WAT WWW installation

  CWB setting

  CWB documentation

  User WWW installation

  Test CWB pipeline

• A quick list of commands to be used for installation.

How to install the external libraries

The CWB libraries require external libraries, see below. In case the user makes use of the WAT libraries on the LIGO-Virgo clusters, the pre-installed external libraries can be considered. In case the user wants to install the WAT libraries on his/her personal computer, the external libraries must be present on the same machine.

Instructions on the installation of the external libraries are reported below.

ROOT installation	mandatory
FRAMELIB installation	mandatory
CFITSIO installation	optional (needed by HEALPix)
HEALPix installation	optional
Baudline installation	optional
LAL installation	optional
CVODE installation	optional (needed by eBBH)
SKYMAP installation	optional (produce skymap-statistics)

Packages repository

A directory containing the software packages used by CWB is stored on each cluster:

CIT : /home/waveburst/SOFT/PACKAGES

ATLAS : /home/waveburst/SOFT/PACKAGES

CNAF : /opt/exp_software/virgo/virgoDev/waveburst/SOFT/PACKAGES

1. root_v6.14.00_icc.tgz

   An object oriented framework for large scale data analysis

2. Healpix_3.40_2018Jun22.tar - optional

   HEALPix is an acronym for Hierarchical Equal Area isoLatitude Pixelization of a sphere

3. cfitsio3450.tar - optional

   CFITSIO is a library of C and Fortran subroutines for reading and writing data files in FITS (Flexible Image Transport System) data format

4. libframe-8.30_root-6.14.00_icc.tgz

   The Frame Library is a software dedicated to the frame manipulation including file input/output

5. baudline_1.08_linux_x86_64.tar - optional

   Baudline is a time-frequency browser designed for scientific visualization of the spectral domain.

6. lal - optional

   LALSuite is a package including various gravitational wave data analysis routines

7. wat

   gravitational wave burst analysis software

8. data64_wat-4.8.2.tgz

   wat filters for 1G analysis

9. www-ced-1.0.tar

   set of javascripts used by ced

10. cvode-2.7.0.tar.gz - optional

    used by eBBH : solves initial value problems for ordinary differential equation (ODE) systems

Create directories

Note

all instructions are based on tcsh shell (.csh), but can be easily converted to bash shell (.sh)

set environment :

- PCKS_REPO is the directory which contains the pakages

  setenv PCKS_REPO "/home/waveburst/SOFT/PACKAGES"                             # CIT
  setenv PCKS_REPO "/home/waveburst/SOFT/PACKAGES"                             # ATLAS
  setenv PCKS_REPO "/opt/exp_software/virgo/virgoDev/waveburst/SOFT/PACKAGES"  # CNAF

- HOME_LIBS is the directory where to install the libraries

  setenv HOME_LIBS "user libraries path"

install :

mkdir $HOME_LIBS
cd $HOME_LIBS
mkdir HEALPix CFITSIO ROOT FRAMELIB BAUDLINE LAL WAT

ROOT installation

whatis : An object oriented framework for large scale data analysis

git repository : http://github.com/root-project/root.git

home site : https://root.cern.ch/

install :

#  clone root from git repository
git clone http://github.com/root-project/root.git
# checkout a specific version
git checkout -b v6-14-00 v6-14-00
# setup cmake (root uses cmake)
export PATH=/home/waveburst/SOFT/SYSLIBS/cmake-3.7.2-Linux-x86_64/bin:$PATH
# setup intel compiler
source /ldcg/intel/2018u2/compilers_and_libraries_2018.2.199/linux/bin/iccvars.sh -arc intel64
# compile root
mkdir root_v6.14.00_icc; cd root_v6.14.00_icc; cmake -DCMAKE_C_COMPILER=icc -DCMAKE_CXX_COMPILER=icpc ..; make -j 20
# move to SOFT/ROOT
mv root_v6.14.00_icc /home/waveburst/SOFT/ROOT

FRAMELIB installation

whatis : The Frame Library is a software dedicated to the frame manipulation including file input/output

source : http://lappweb.in2p3.fr/virgo/FrameL/

infos : http://lappweb.in2p3.fr/virgo/FrameL/FrDoc.html

Installation :

wget http://lappweb.in2p3.fr/virgo/FrameL/libframe-8.30.tar.gz
tar zxvf libframe-8.30.tar.gz
mv libframe-8.30 libframe-8.30_root-6.14.00_icc

# edit ibframe-8.30_root-6.14.00_icc/root/build
# set ROOTSYS = "/home/waveburst/SOFT/ROOT/root_v6.14.00_icc/"

source $ICC_INIT_SCRIPT intel64

cd mgr
./makeicc

cd ../root
./build_icc

cd ..
ln -s Linux-x86_64 Linux

CFITSIO installation

whatis : CFITSIO is a library for reading and writing data files in FITS (Flexible Image Transport System) data format

source : NASA site: http://heasarc.gsfc.nasa.gov/docs/software/fitsio/fitsio.html

Download cfitsio-3.45

wget ftp://heasarc.gsfc.nasa.gov/software/fitsio/c/cfitsio3450.tar.gz

Installation :

gunzip cfitsio3450.tar.gz
tar xvf cfitsio3450.tar
mv cfitsio cfitsio-3.45
cd cfitsio-3.45
./configure CFLAGS=-O
make shared

HEALPix installation

whatis : HEALPix is an acronym for Hierarchical Equal Area isoLatitude Pixelization of a sphere

source : https://sourceforge.net/projects/healpix/

versions : https://sourceforge.net/projects/healpix/files/

infos : see What is HEALPix

Download HEALPix_3.40

wget https://sourceforge.net/projects/healpix/files/Healpix_3.40/Healpix_3.40_2018Jun22.tar.gz
gunzip Healpix_3.40_2018Jun22.tar.gz
tar xvf Healpix_3.40_2018Jun22.tar

Installation :

cd Healpix_3.40

./configure
  -> Do you want to:
select option 4 : [(4): configure Healpix C++ package, and edit Makefile]
  -> enter location of cfitsio library (/usr/local/lib):
type path - [Ex : cfitsio_path/cfitsio-3.45]
  -> enter location of cfitsio header fitsio.h (cfitsio_path/cfitsio-3.34):
type return
  -> Available configurations for C++ compilation are:
type 2 : [ 2: generic_gcc]
  -> Enter your choice (configuration of packages can be done in any order):
type 0

Edit : Healpix_3.40/src/cxx/config/config.generic_gcc
Replace :
ARCREATE= ar cr
With :
ARCREATE= gcc -shared -fno-common -dynamiclib -O2 -g -o

make cpp-all

libraries are created under src/cxx/generic_gcc/lib
cd src/cxx
ln -s generic_gcc shared
cd shared/lib

ln -s libc_utils.a       libc_utils.so
ln -s libcxxsupport.a    libcxxsupport.so
ln -s libfftpack.a       libfftpack.so
ln -s libhealpix_cxx.a   libhealpix_cxx.so
ln -s libsharp.a         libsharp.so

Baudline installation

whatis : Baudline is a time-frequency browser designed for scientific visualization of the spectral domain.

source : http://www.baudline.com/download.html

infos : FrDisplay

Installation :

cd $HOME_LIBS/BAUDLINE
tar xvf $PCKS_REPO/baudline_1.08_linux_x86_64.tar

LAL installation

whatis : LALSuite is a package including various gravitational wave data analysis routines

git repository : http://github.com/root-project/root.git

Note

LAL is used by cwb to generate on the fly the inspiral mdc

Installation :

# get lalsuite from git repository

git clone https://git.ligo.org/lscsoft/lalsuite.git

# get the short hash

cd lalsuite
git rev-parse --short HEAD

#!/bin/bash -e

export LSCSOFT_ROOTDIR="${HOME}/SOFT/LAL/lalsuite_lalinference_o2_995dfede" # Place where the built lal goes, Make sure folder exists before
export LSCSOFT_SRCDIR="/home/waveburst/git"

OLD_WD=$PWD
cd ${LSCSOFT_SRCDIR}/lalsuite

# reset repo to clean state
git clean -f -d -x

# wipe old build
rm -rf ${LSCSOFT_ROOTDIR}/*

# do build

./00boot

CONF_OPTS="--prefix=${LSCSOFT_ROOTDIR} \
    --enable-swig-python \
    --enable-lalstochastic \
    --enable-lalpulsar \
    --enable-lalinference \
    --disable-gcc-flags "


./configure ${CONF_OPTS} > >(tee /tmp/stdout.log) 2> >(tee /tmp/stderr.log >&2)


make && make install

# this is necessary to get pylal libs linked properly
source ${LSCSOFT_ROOTDIR}/etc/lal-user-env.sh
source ${LSCSOFT_ROOTDIR}/etc/lalapps-user-env.sh

cd ${OLD_WD}

CVODE installation

whatis : CVODE solves initial value problems for ordinary differential equation (ODE) systems.

source : https://computation.llnl.gov/casc/sundials/download/download.html

Installation :

cd $HOME_LIBS/CVODE
1. Download the CVODE package version 2.7.0 from

   http://computation.llnl.gov/casc/sundials/download/download.html
   or use the package :
   tar xvf $PCKS_REPO/cvode-2.7.0.tar.gz

Currently (May 08 2013) this is the latest version, but in the future you might need to scroll down
and look under "Previous releases".

2. Uncompress it

   tar -xzvf cvode-2.7.0.tar.gz

3. Configure it to produce shared libraries and provide the location
for the installed code ("dist" in this example)

   cd cvode-2.7.0
   ./configure --enable-shared --prefix=/home/waveburst/SOFT/CVODE/cvode-2.7.0/dist

4. Make and install it

   make
   make install

5. You should find in dist/lib the two shared libraries sundials_cvode and sundials_nvecserial.

SKYMAP installation

whatis : simple module and exectutables to quantify skymaps and compare them. Also contains plotting and visualization routines, as well as a script that builds friendly html documents summarizing this information (intended to be used with GraceDb).

source : https://github.com/reedessick/skymap_statistics

Installation :

cd $HOME_LIBS/SKYMAP
1. clone package from git repository :

   git clone https://github.com/reedessick/skymap_statistics.git

2. Define environment variables

   - HOME_SKYMAP_LIB : path of skymap-statistics library
   - CWB_USER_URL    : defines the url site where the user reports are pubblished

   Examples :

   - setenv HOME_SKYMAP_LIB     "${HOME_LIBS}/SKYMAP/skymap_statistics"
   - setenv CWB_USER_URL        "https://ldas-jobs.ligo.caltech.edu/~waveburst/LSC/reports"

3. For html web pages productions it is necessary to add
       skymap_statistics/opt/lib/python2.7/site-packages directory.
   Such directory must be setup according to the python site-package definition
   See the example in the ATLAS cluster : /home/waveburst/SOFT/SKYMAP/skymap_statistics/opt

WAT installation

whatis : gravitational wave burst analysis software

git repository : https://git.ligo.org/cWB/library

Installation :

- define the env $HOME_LIBS to point to the pre-installed cluster libraries path, see CWB quick startup

  setenv HOME_LIBS  "cluster libraries path"

- download WAT library from WAT GIT Repository

  The suggested local directory where to download the repository is $HOME/git/cWB

  Download the git master version (developing version):

  cd $HOME/git/cWB
  git clone git@git.ligo.org:cWB/library.git

  Check hash version:

  git rev-parse HEAD
  git rev-parse --short HEAD

  if a specific hash version is needed then first list log:

  git log

  then select a suitable hash (selected_hash) and do:

  git checkout selected_hash (Ex: git checkout 4e0d80fd8da5b751ba62ce6402aa270354560145)

- define the environment variables :

  - make a copy the default cluster shell script defined here CWB quick startup
  - source your_watenv.csh
  - cp $HOME/git/cWB/library/tools/cwb/cwb.rootrc ~/.rootrc

- compile wat with the default settings (up to 4 detectors)
  make

- compile wat with up to 8 detectors capability
  make XIFO=8

- to compile wat with lal (default : enabled) set

  setenv _USE_LAL      1     # enable LAL in CWB
  setenv HOME_LAL            "${HOME_LIBS}/LAL/lalsuite/master"
  setenv LAL_INC             "${HOME_LAL}/include"


- to compile wat with HEALPix (default : enabled) set

  setenv _USE_HEALPIX 1      # enable HEALPix in CWB


- to compile wat with eBBH (default : enabled) set

  setenv _USE_EBBH 1          # enable EBBH in CWB
  setenv HOME_CVODE            "${HOME_LIBS}/CVODE/cvode-2.7.0/dist"

- setup the cWB config
  setenv CWB_CONFIG          "/home/vedovato/git/cWB/config"   # Location of CWB Configuration files, DQ files, FRAMES files

- copy WAT libraries to the default install directory ($HOME/git/cWB/library/tools/install)
  make install
  or
  make winstall

- It is possible to copy the compiled cWB libraries into a specific directory:
  make install ODIR=your_specific_dir

  - define the environment variables :
    - source your_specific_dir/your_watenv.csh

CWB config installation

whatis : configuration files used by the cWB pipeline to perform the analysis

git repository : https://git.ligo.org/cWB/config

Installation :

- define the env $HOME_LIBS to point to the pre-installed cluster libraries path, see CWB quick startup

  setenv HOME_LIBS  "cluster libraries path"

- download WAT config from WAT GIT Repository

  The suggested local directory where to download the repository is $HOME/git/cWB

  Download the git master version (developing version):

  cd $HOME/git/cWB
  git clone git@git.ligo.org:cWB/config.git

  Check hash version:

  git rev-parse HEAD
  git rev-parse --short HEAD

  if a specific hash version is needed then first list log:

  git log

  then select a suitable hash (selected_hash) and do:

  git checkout selected_hash (Ex: git checkout 4e0d80fd8da5b751ba62ce6402aa270354560145)

WAT filters installation

Note

filters are used by CWB to perform data time shifts (used in the likelihood stage)

mkdir $HOME_LIBS/WAT/filters
cd $HOME_LIBS/WAT/filters
- 1G filters
tar zxvf $PCKS_REPO/data64_wat-4.8.2.tgz
- 2G filters
tar xvf $PCKS_REPO/wdmXTalk.tar

WAT WWW installation

Note

contains index.html & javascripts used to display ceds

mkdir $HOME_LIBS/WAT/www
cd $HOME_LIBS/WAT/www
tar xvf $PCKS_REPO/www-ced-1.0.tar
cd $HOME_LIBS/WAT/trunk/tools/cwb/www
make
cd $HOME_LIBS/WAT/www/ced-1.0
ln -s $HOME_LIBS/WAT/trunk/tools/cwb/www/index

- create public dir for ced

  mkdir ~/public_html/waveburst
  cd ~/public_html/waveburst
  ln -s $HOME_LIBS/WAT/www/ced-1.0

CWB setting

For each cluster there is a tcsh/bash script that can be used to set the cWB environment.

These scripts are configured for the pre-installed libraries : see Using Pre Installed Librares

Here we show how to customize the CWB environment

Note

all instructions are based on tcsh shell (.csh), but can be easily translate in bash shell (.sh)

1) define the environment variables

- set HOME_LIBS (is the PATH where are located the libraries used by cWB)

  setenv HOME_LIBS "/home/waveburst/SOFT"     # ATLAS (for other clusters see CWB quick startup)

- set HOME_WAT (is the PATH where is located the WAT library)

  for a full installation
  setenv HOME_WAT "${HOME_LIBS}/WAT/trunk"
  for a user custom installation
  setenv HOME_WAT "the PATH where is located the WAT library"

2) make a copy of watenv configuration file to a custom watenv file

cp $HOME_LIBS/WAT/trunk/atlas_watenv.csh $HOME_WAT/local_watenv.csh   # ATLAS

cp $HOME_LIBS/WAT/trunk/cit_watenv.csh   $HOME_WAT/local_watenv.csh   # CIT

cp $HOME_LIBS/WAT/trunk/cnaf_watenv.csh  $HOME_WAT/local_watenv.csh   # CNAF

3) customize the local_watenv.csh configuration file

- set HOME_LIBS (is the PATH where are located the libraries used by cWB)

  setenv HOME_LIBS "/home/waveburst/SOFT"     # ATLAS (for other clusters see CWB quick startup)

- set HOME_WAT (is the PATH where is located the WAT library)

  for a full installation
  setenv HOME_WAT "${HOME_LIBS}/WAT/trunk"
  for a user custom installation
  setenv HOME_WAT "the PATH where is located the WAT library"

- enable/disable LAL  (optional)

  unsetenv _USE_LAL                            # disable LAL in CWB
  setenv _USE_LAL 1                            # enable LAL in CWB

- enable/disable HEALPix (optional)

  unsetenv _USE_HEALPIX                        # disable LAL in CWB
  setenv _USE_HEALPIX 1                        # enable LAL in CWB

4) add script to login

- add local_watenv.csh to ~/.tcshrc

  source $HOME_LIBS/WAT/trunk/tools/local_watenv.csh

- make "$HOME_WAT/tools/cwb/macros/cwb_rootlogon.C" the default rootlogon.C :

  cp $HOME_WAT/tools/cwb/cwb.rootrc ~/.rootrc

Note

cwb.rootrc contains the ROOT defaults environment setup used by CWB to make further customizations see the ROOT default setup in : $ROOTSYS/etc/system.rootrc

User WWW installation

The user must create a public www directory where to store the reports

produced by the CWB analysis.

For ATLAS do :

  mkdir ~/WWW/LSC/reports

  reports can be browser at (user must be substituted with the user name) :
    https://ldas-jobs.ligo.caltech.edu/~user/LSC/reports

For CIT do :

  mkdir ~/public_html/reports

  reports can be browser at (user must be substituted with the user name) :
    https://ldas-jobs.ligo.caltech.edu/~user/reports/

Test CWB pipeline

Note

this is a test to verify if cwb is working

see also CWB quick startup and My first CWB pipeline test)

source $HOME_LIBS/WAT/trunk/tools/local_watenv.csh
- cd to a working a user working directory
  cp -r $HOME_LIBS/WAT/trunk/tools/cwb/examples/ADV_SIM_SGQ9_L1H1V1_TST  ADV_SIM_SGQ9_L1H1V1_TST
  cd ADV_SIM_SGQ9_L1H1V1_TST
- read the README file
  make setup
- simulation
- the output root file is produced under data directory
  make cwb
- simulation with ced
- output ced files are produced under data directory
  make ced
- to see ced from browser
  mv data/* report/ced/
- to viwew ced bowser this link (substitute user with the user account) :
- https://ldas-jobs.ligo.caltech.edu/~user/reports/