The atlas is provided as files in Mapped Vector Library (MVL) format:
the main data set O3a_atlas.mvl
the summary data set O3a_atlas_summary.mvl
the spatial index O3a_spatial_index.mvl
Instead of downloading, the user can generate the spatial index O3a_spatial_index.mvl using the script create_spatial_index.R.
For the O3a all-sky search the Atlas contains nearly 2 billion entries. The MVL format was chosen for this data because it allows it to be memory mapped, allowing analysis on personal computers with small amounts of random access memory. We strongly recommend that the user has a solid state drive allowing random access to the data.
The MVL format specification and description of data layout is in the file data_format.txt.
The main data set O3a_atlas.mvl contains upper limits (for circular and linear polarization) and signal-to-noise ratios (SNR), parameterized by sky position and frequency.
The sky resolution increases with frequency. The sky coordinates are in radians, J2000.
The summary data set O3a_atlas_summary.mvl also has upper limits and signal-to-noise ratios, but it has been mapped onto a coarser uniform sky grid that is the same for all frequencies. The summary data set is used by the script visualize.R.
Once in the R environment, a script runs with the following command:
source("SCRIPT_NAME.R")
Before running any of the scripts in R the RMVL package has to be installed using the following R command:
install.packages("RMVL")
A number of example R scripts that make use of the atlas data are provided. The user is encouraged to open the scripts and modify them as needed.
visualize.R - generates a skymap and a frequency plot from any sub-band of the atlas, plus some summary information. There are examples of how to make plots and extract data at the end of the script.
visualize_parallel.R - a parallel version of visualize.R script. Because MVL data is memory mapped the data is shared between parallel R processes.
full_scan_example1.R - example of finding all points above a fixed SNR cutoff
full_scan_example2.R - example of finding all points that were analyzed by the second stage of the pipeline
full_scan_example3.R - example of finding all points that were not analyzed by the second stage of the pipeline
create_spatial_index.R - creates spatial index allowing fast queries by sky location. The size of the generated index is 56GB. Instead of running this script you can download the spatial index.
spatial_index_example1.R - example of finding points closest to points in provided file points_of_interest.csv with help of spatial index
spatial_index_example2.R - example of using spatial index to retrieve data for a single sky location across all frequencies
The convenience of MVL files depends on the speed of the underlying storage. At the time of writing this README file we recommend using a solid-state drive with 2TB capacity, attached either using USB 3.0/USB C or NVME/PCI express. SATA or USB 2.0 connections are much slower.
The 2 TB capacity is recommended because larger solid state drives usually have larger endurance and faster access speeds. While it is possible to use drives of 1 TB and smaller, keep in mind that it is desirable to have spare capacity for the creation of temporary files, such as used by full scan examples.
MD5 checksums of MVL files:
69f04d5752acdb4f63700ebef25d22c0 O3a_atlas.mvl
1c7e17ebc5c6b1e8951bc0f530de81f5 O3a_atlas_summary.mvl
8629475d1117d8820fdf27a517660f95 O3a_spatial_index.mvl