.. _catalog-formats: Catalog formats *************** Catalogs refer to datasets which are essentially particle-based (stars, galaxies, etc.). There are several off-the-shelf options to get catalog data in various formats into Gaia Sky. The most important are ``VOTable``, ``FITS`` and ``CSV``. They are all handled by the :ref:`STIL data provider `. They are defined in the same way as any other object is, using :ref:`JSON descriptor files `. Let's see an example of the definition of one such catalog (the Oort cloud) using JSON: .. code:: json { "name" : "Oort cloud", "position" : [0.0, 0.0, 0.0], "color" : [0.9, 0.9, 0.9, 0.8], "size" : 2.0, "labelColor" : [0.3, 0.6, 1.0, 1.0], "labelPosition" : [0.0484814, 0.0, 0.0484814], "componentType" : "Others", "fadeIn" : [0.0004, 0.004], "fadeOut" : [0.1, 15.0], "profileDecay" : 1.0, "parent" : "Universe", "archetype" : "ParticleGroup", "provider" : "gaiasky.data.PointDataProvider", "factor" : 149.597871, "dataFile" : "$data/oort-cloud/oortcloud/oort_10000particles.dat" } This is based on the ``ParticleSet`` component, which is fully documented :ref:`here `. Let's go over the attributes that appear in this example: - ``name`` -- The name of the particle group. - ``position`` -- The mean cartesian position (see :ref:`internal reference system `) in parsecs, used for sorting purposes and also for positioning the label. If this is not provided, the mean position of all the particles is used. - ``color`` -- The color of the particles as an ``rgba`` array. - ``size`` -- The size of the particles. In a non HiDPI screen, this is in pixel units. In HiDPI screens, the size will be scaled up to maintain the proportions. - ``labelColor`` -- The color of the label as an ``rgba`` array. - ``labelPosition`` -- The cartesian position (see :ref:`internal reference system `) of the label, in parsecs. - ``componentType`` (alias: ``ct``) -- The ``ComponentType`` (see `ComponentTypes `__). This is basically a ``string`` that will be matched to the entity type in ``ComponentType`` enum. Valid component types are ``Stars``, ``Planets``, ``Moons``, ``Satellites``, ``Atmospheres``, ``Constellations``, etc. - ``fadeIn`` -- The fade in interpolation distances, in parsecs. If this property is defined, there will be a fade-in effect applied to the particle group between the distance ``fadeIn[0]`` and the distance ``fadeIn[1]``. - ``fadeOut`` -- The fade out interpolation distances, in parsecs. If this property is defined, there will be a fade-out effect applied to the particle group between the distance ``fadeOut[0]`` and the distance ``fadeOut[1]``. - ``profileDecay`` -- This attribute controls how particles are rendered. This is basically the opacity profile decay of each particle, as in ``(1.0 - dist)^profileDecay``, where dist is the distance from the center (center dist is 0, edge dist is 1). - ``parent`` -- The name of the parent object in the scene graph. - ``archetype`` (alias: ``impl``) -- The :ref:`archetype ` name (or legacy class name, but this should be avoided). - ``provider`` -- The full name of the data provider class. This must extend ``gaiasky.data.api.IParticleGroupDataProvider`` (see `IParticleGroupDataProvider `__). - ``factor`` -- A factor to be applied to each coordinate of each data point. If not specified, defaults to 1. - ``dataFile`` -- The actual file with the data. It must be in a format that the data provider specified in ``provider`` knows how to load. - ``texture`` -- Optional attribute that points to a texture or directory with textures to render the particles of this set with. If this is available, ``profileDecay`` is ignored. - ``textures`` -- Same as ``texture``, but with an array of textures and/or directories. Star catalogs ============= Star catalogs are special because, in addition to positional information, they contain extra properties such as proper motions, magnitudes, colors and more. All of these are important to be able to render stars faithfully. The easiest way to load star catalogs is by loading them from VOTable files. Let's see how these catalogs can be defined in Gaia Sky. For example, the new Hipparcos reduction uses this ``dataset.json`` file that contains some catalog metadata and pointers to the actual data files: .. code:: json { "key": "catalog-hipparcos", "name" : "Hipparcos (new reduction)", "version" : 4, "mingsversion" : 30301, "type" : "catalog-star", "description" : "Hipparcos new reduction (van Leeuwen, 2007) with curated star names.", "releasenotes" : "- Add type in catalog descriptor file.\n- Update to new data format.", "link" : "http://adsabs.harvard.edu/abs/2007ASSL..350.....V", "size" : 5433174, "nobjects" : 177955, "check": "$data/catalog-hipparcos/dataset.json", "files" : [ "$data/catalog-hipparcos" ], "data" : [ { "loader": "gaiasky.data.JsonLoader", "files": [ "$data/catalog-hipparcos/particles-hip.json" ] }] } The file ``particles-hip.json`` contains a single object with the actual pointer to the VOTable data file, and some additional metadata such as the color of labels, a description of the catalog or the data provider: .. code:: json { "objects" : [ { "name" : "Hipparcos (new red.)", "position" : [0.0, 0.0, 0.0], "color" : [1.0, 1.0, 1.0, 0.25], "size" : 6.0, "labelColor" : [1.0, 1.0, 1.0, 1.0], "labelPosition" : [0.0, -5.0e7, -4.0e8], "componentType" : "Stars", "fadeout" : [21.0e2, 0.5e5], "profiledecay" : 1.0, "parent" : "Universe", "archetype" : "StarGroup", "catalogInfo" : { "name" : "Hipparcos", "description" : "Hipparcos new reduction (van Leeuwen, 2007). 117995 stars.", "type" : "INTERNAL" }, "provider" : "gaiasky.data.group.STILDataProvider", "dataFile" : "$data/catalog-hipparcos/catalog/hipparcos/hipparcos.vot" } ]} Regular star catalogs --------------------- Gaia Sky supports all formats supported by the ``STIL`` `library `__. Since the data held by the formats supported by ``STIL`` is not of a unique nature, this catalog loader makes a series of assumptions. More information can be found in :ref:`STIL data provider `. In particular, it is possible to load a ``VOTable``, ``CSV``, ``FITS`` or ``ASCII`` file directly into Gaia Sky using the ``Open file`` icon at the bottom of the control panel. Level-of-detail star catalogs ----------------------------- Gaia Sky uses level-of-detail structures to represent catalogs with hundreds of millions of stars. This broad and deep topic is covered in its own section: - :ref:`Level-of-detail: Octree `. Particle catalogs ================= Particle catalogs (point cloud data) can be loaded from CSV, VOTable or FITS files (see :ref:`STIL data loader `), or also from a fast and compact binary format. More information can be found in the :ref:`particle catalogs section `.