Gaia DR3 data in MVL format

The MVL stands for Mappable Vector Library and is a file format designed for memory mapping. With a solid state drive, you can map the entire Gaia data into memory and access it at will - even on a small notebook. You can also run parallel computations because the data will be shared between processes running on the same computer.

There are two libraries that access MVL files: C library libMVL and an R package RMVL.

RMVL is great for exploring the data and making plots. The files below were built with RMVL scripts.

Instead of running the scripts that take a long time to read in all the compressed CSV files that are part of Gaia DR3 release, you can simply download the MVL files here. We recommend using bittorrent client with http support - this way you can dowload using all the mirrors and the webserver. The torrent file is hosted by Academic Torrents.

• torrent file with all MVL files below - fastest way to download
• gaia_dr3.mvl - main data (1.3 TB - very large)
• parallax_index.mvl - an index that makes it easy to find nearby stars (16 GB)
• coordinates_index.mvl - a sky position index, allowing to search for stars near a particular sky position (64 GB)
• source_id_index.mvl - an index that speeds up lookup by Gaia source id (57 GB)
• examples - example R scripts showing to how use the data
• Analyzing large data with Mappable Vector Library - Vladimir Dergachev

MD5 sums:

• gaia_dr3.mvl : f3660c78c0f9fd4915674ab9722b5170
• parallax_index.mvl : a1c82feac9548ae82db9f897334cbc6f
• coordinates_index.mvl : d6283ef55c35ac76a6da08a5f015a1e8
• source_id_index.mvl : 6a46b1f576345b9a0bb254067435151e

We also have MVL files for the expanded sky atlas of continuous gravitational waves.

Questions, comments or general discussion: vladimir.dergachevDaei.mpg.de