Data format *********** Gaia Sky needs to first load datasets in order to display data. Dataset files contain objects, which are organized by Gaia Sky into a ``scenegraph``. A scenegraph is a tree that contains objects and orders them hierarchically depending on their geometrical and spatial relations. .. contents:: :backlinks: none .. raw:: html
Since 3.3.1
All datasets are partially or totally described in a JSON format. Each dataset lives in its own directory in the data location (referred to as ``$data/``, see :ref:`folders `), and must contain a description in the file ``dataset.json``. If a dataset does not have this file in its directory, it won't be recognized by Gaia Sky. Below is an example of the contents of the data location. .. code:: $data/ ├─ default-data/ │ ├─ dataset.json │ └─ ... ├─ catalog-hipparcos/ │ ├─ dataset.json │ └─ ... ├─ catalog-whitedwarfs-dr2/ │ ├─ dataset.json │ └─ ... └─ .../ .. _general-data-loading: Where are the data files defined? ================================= Gaia Sky implements a very flexible and open data loading mechanism. The data files to be loaded are defined in a couple of keys in the ``config.yaml`` :ref:`configuration file `, which is usually located in the ``$GS_CONFIG`` folder (see :ref:`folders `). The keys are the following (double-colon indicates nesting): - ``data::dataFiles`` -- an array containing the list of enabled JSON data files. Each file should be a relative path from the data directory with the prefix ``$data/``. For instance, the default dataset, containing the Solar System and some necessary objects for Gaia Sky to run, is specified in the array as ``$data/default-data/dataset.json``. $data/[dataset-name]/dataset.json example ----------------------------------------- Dataset descriptor files contain the metadata of a catalog (name, description, version, etc.) and a pointer to the actual data. Below is a made-up file file ``dataset.json`` which describes my super-awesome dataset. .. code:: json { "key" : "dataset-key-without-spaces", "name": "Dataset name", "version": 1, "mingsversion" : 30106, "type": "catalog-gaia", "description": "The description here.\nCan contain line breaks.", "releasenotes" : "- What changed since the last version?\n- List here." "link": "https://arxiv.org/abs/1805.00425", "check" : "dataset-descriptor.json" "size": 368633, "nobjects": 1365, "check": "$data/my-dataset/dataset.json", "files": [ "$data/my-dataset" ], "data": [{ "loader": "gaiasky.data.JsonLoader", "files": [ "$data/my-dataset/particles-particles.json" ] }] } Notice that the data (``data::files``) points to another JSON file, which contains some additional info about how to load the data, and a pointer to the actual data file ``particles-particles.json``. Here it is: .. code:: json { "objects": [{ "name": "My dataset", "position": [0.0, 0.0, 0.0], "componentType": "Stars", "fadeout": [1.0e5, 0.5e8], "parent": "Universe", "archetype": "StarGroup", "provider": "gaiasky.data.group.STILDataProvider", "datafile": "$data/my-dataset/catalog/dataset.vot" }] } As you can see, the ``STILDataProvider`` is the one in charge of loading the data form this dataset, which resides in a VOTable file, ``dataset.vot``. default-data/dataset.json example file -------------------------------------- This is an example of what the default data pack contains. The dataset descriptor file loads different files using different loaders. .. code:: json { "data" : [ { "loader": "gaiasky.data.JsonLoader", "files": [ "$data/default-data/planets-normal.json", "$data/default-data/moons-normal.json", "$data/default-data/satellites.json", "$data/default-data/asteroids.json", "$data/default-data/orbits_planet.json", "$data/default-data/orbits_moon.json", "$data/default-data/orbits_asteroid.json", "$data/default-data/orbits_satellite.json", "$data/default-data/extra-low.json", "$data/default-data/locations.json", "$data/default-data/locations_earth.json", "$data/default-data/locations_moon.json"] }, { "loader": "gaiasky.data.GeoJsonLoader", "files": [ "$data/default-data/countries/countries.geo.json" ] }] } The ``dataset.json`` file contains an array, ``"data"``, which is a list of pairs containing ``[loader: files]`` correspondences. Each ``"loader"`` contains the classes that will load the list of files under the corresponding ``"files"`` property. The main loader, the ``JsonLoader``, expects JSON files as inputs. Each of these files must have an attribute called ``"objects"``, which is an array containing the metadata on the objects to load. Data loaders ============ The files are sent to the Scene Graph JSON Loader, which iterates on each loader-files pair in each file, instantiates the loader and uses it to load the files. All loaders need to adhere to a contract, defined in the interface ``ISceneLoader`` (`here `__). The ``loadData()`` method of each loader must return a list of objects, which is then added to a global list containing all the previously loaded files. At the end, we have a list with all the objects in the scene. This list is passed on to the Scene Graph instance, which constructs the scene graph tree structure which will contains the object model. As we said, each loader will load a different kind of data; the ``JSONLoader`` (`here `__) loads JSON files containing planets, satellites, orbits, star catalogs, etc. The ``STILDataProvider`` (`here `__) loads ``VOTables``, ``FITS``, ``CSV`` and other files through the `STIL `__ library, ``GeoJsonLoader`` (`here `__) loads geographic data, and so on. .. _catalog-formats: Catalog formats =============== Catalogs refer to datasets which are essentially particle-based (stars, galaxies, etc.). There are several off-the-shelf options to get catalog data in various formats into Gaia Sky. The most important are ``VOTable``, ``FITS`` and ``CSV``. They are all handled by the :ref:`STIL data provider `. The way they are defined in Gaia Sky is the same any other object is defined, that is, using :ref:`JSON descriptor files `. Let's see an example of the definition of one such catalog (the Oort cloud) using JSON: .. code:: json { "name" : "Oort cloud", "position" : [0.0, 0.0, 0.0], "color" : [0.9, 0.9, 0.9, 0.8], "size" : 2.0, "labelColor" : [0.3, 0.6, 1.0, 1.0], "labelPosition" : [0.0484814, 0.0, 0.0484814], "componentType" : "Others", "fadeIn" : [0.0004, 0.004], "fadeOut" : [0.1, 15.0], "profileDecay" : 1.0, "parent" : "Universe", "archetype" : "ParticleGroup", "provider" : "gaiasky.data.PointDataProvider", "factor" : 149.597871, "dataFile" : "$data/oort-cloud/oortcloud/oort_10000particles.dat" } This is based on the ``ParticleSet`` component, which is fully documented :ref:`here `. Let's go over the attributes that appear in this example: - ``name`` -- The name of the particle group. - ``position`` -- The mean cartesian position (see :ref:`internal reference system `) in parsecs, used for sorting purposes and also for positioning the label. If this is not provided, the mean position of all the particles is used. - ``color`` -- The color of the particles as an ``rgba`` array. - ``size`` -- The size of the particles. In a non HiDPI screen, this is in pixel units. In HiDPI screens, the size will be scaled up to maintain the proportions. - ``labelColor`` -- The color of the label as an ``rgba`` array. - ``labelPosition`` -- The cartesian position (see :ref:`internal reference system `) of the label, in parsecs. - ``componentType`` (alias: ``ct``) -- The ``ComponentType`` (see `here `__). This is basically a ``string`` that will be matched to the entity type in ``ComponentType`` enum. Valid component types are ``Stars``, ``Planets``, ``Moons``, ``Satellites``, ``Atmospheres``, ``Constellations``, etc. - ``fadeIn`` -- The fade in inetrpolation distances, in parsecs. If this property is defined, there will be a fade-in effect applied to the particle group between the distance ``fadeIn[0]`` and the distance ``fadeIn[1]``. - ``fadeOut`` -- The fade out inetrpolation distances, in parsecs. If this property is defined, there will be a fade-in effect applied to the particle group between the distance ``fadeIn[0]`` and the distance ``fadeIn[1]``. - ``profileDecay`` -- This attribute controls how particles are rendered. This is basically the opacity profile decay of each particle, as in ``(1.0 - dist)^profileDecay``, where dist is the distance from the center (center dist is 0, edge dist is 1). - ``parent`` -- The name of the parent object in the scene graph. - ``archetype`` (alias: ``impl``) -- The :ref:`archetype ` name (or legacy class name, but this should be avoided). - ``provider`` -- The full name of the data provider class. This must extend ``gaiasky.data.api.IParticleGroupDataProvider`` (see `here `__). - ``factor`` -- A factor to be applied to each coordinate of each data point. If not specified, defaults to 1. - ``dataFile`` -- The actual file with the data. It must be in a format that the data provider specified in ``provider`` knows how to load. - ``texture`` -- Optional attribute that points to a texture or directory with textures to render the particles of this set with. If this is available, ``profileDecay`` is ignored. - ``textures`` -- Same as ``texture``, but with an array of textures and/or directories. Star catalogs ------------- Star catalogs are special because, additionally to positional information, they contain extra properties such as proper motions, magnitudes, colors and more. All of these are important to be able to render stars faithfully. The easiest way to load star catalogs is by loading them from VOTable files. Let's see how these catalogs can be defined in Gaia Sky. For example, the new Hipparcos reduction uses this ``dataset.json`` file that contains some catalog metadata and pointers to the actual data files: .. code:: json { "key": "catalog-hipparcos", "name" : "Hipparcos (new reduction)", "version" : 4, "mingsversion" : 30301, "type" : "catalog-star", "description" : "Hipparcos new reduction (van Leeuwen, 2007) with curated star names.", "releasenotes" : "- Add type in catalog descriptor file.\n- Update to new data format.", "link" : "http://adsabs.harvard.edu/abs/2007ASSL..350.....V", "size" : 5433174, "nobjects" : 177955, "check": "$data/catalog-hipparcos/dataset.json", "files" : [ "$data/catalog-hipparcos" ], "data" : [ { "loader": "gaiasky.data.JsonLoader", "files": [ "$data/catalog-hipparcos/particles-hip.json" ] }] } The file ``particles-hip.json`` contains a single object with the actual pointer to the VOTable data file, and some additional metadata such as the color of labels, a description of the catalog or the data provider: .. code:: json { "objects" : [ { "name" : "Hipparcos (new red.)", "position" : [0.0, 0.0, 0.0], "color" : [1.0, 1.0, 1.0, 0.25], "size" : 6.0, "labelColor" : [1.0, 1.0, 1.0, 1.0], "labelPosition" : [0.0, -5.0e7, -4.0e8], "componentType" : "Stars", "fadeout" : [21.0e2, 0.5e5], "profiledecay" : 1.0, "parent" : "Universe", "archetype" : "StarGroup", "catalogInfo" : { "name" : "Hipparcos", "description" : "Hipparcos new reduction (van Leeuwen, 2007). 117995 stars.", "type" : "INTERNAL" }, "provider" : "gaiasky.data.group.STILDataProvider", "dataFile" : "$data/catalog-hipparcos/catalog/hipparcos/hipparcos.vot" } ]} Regular star catalogs ~~~~~~~~~~~~~~~~~~~~~ Gaia Sky supports all formats supported by the ``STIL`` `library `__. Since the data held by the formats supported by ``STIL`` is not of a unique nature, this catalog loader makes a series of assumptions. More information can be found in :ref:`STIL data provider `. Particularly, it is possible to directly load a ``VOTable``, ``CSV``, ``FITS`` or ``ASCII`` file into Gaia Sky using the ``Open file`` icon at the bottom of the control panel. Level-of-detail star catalogs ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Gaia Sky uses level-of-detail structures to represent catalogs with hundreds of millions of stars. This broad and deep topic is covered in its own section: - :ref:`Level-of-detail: Octree `. Particle catalogs ----------------- Particle catalogs (point cloud data) can be loaded from CSV, VOTable or FITS files (see :ref:`STIL data loader `), or also from a fast and compact binary format. More information can be found in the :ref:`particle catalogs section `. .. _json-format: JSON data format ================ Most of the entities and celestial bodies that are not stars in the Gaia Sky scene are defined in a series of JSON files and are loaded using the ``JsonLoader`` (`here `__). The format is very flexible and loosely matches the underneath object model, which is a scene graph tree. An example about defining an extrasolar system with a couple of stars orbiting each other and a couple of planets can be found :ref:`here `. .. _data-morphology: Data morphology --------------- Before starting, we need to do a little detour to cover the data morphology. In Gaia Sky, objects organize and store their data into :ref:`components `, conforming to the Entity Component System (ECS) paradigm. Components are simple bags of data (attributes). Objects in Gaia Sky are assigned an :ref:`archetype `, which are simple component groups. - **Archetype** -- an :ref:`archetype ` is a definition of a group of components to create objects of a certain type. Ever object has **one and only one** archetype. The archetype is specified with the ``"archetype"`` or the ``"impl"`` attributes, and take in the name of the archetype (case sensitive!). For instance, the *Planet* archetype contains, amongst others, the components *Base*, *Model*, *Coordinates* and *Atmosphere*. - **Component** -- a :ref:`component ` is a simple bag of data. For example, the *Base* component contains a color, the object type, and the object name or names. Components are mostly hidden from users, as they are not defined directly in the JSON data files. Instead, components define what attributes are accepted by each object. An exhaustive list of all the attributes per component, together with descriptions and data types, is provided in :ref:`components`. Depending on the archetype of an object (i.e. depending on the components it has), objects are processed differently by different systems. Additionally, archetypes can extend other archetypes, so that the extending archetype gets all the components defined in the parent. You can find a full description of all the archetypes and components in the data format, together with their attributes, here: - :ref:`archetypes`. - :ref:`components`. .. _objects-updates: Objects vs Updates ------------------ Every JSON file that contains objects must have a named array as the only top-level object in the file. Depending on the name of this array, two things can happen: - ``objects`` -- when the array is named *objects*, it contains new objects to load. - ``updates`` -- when the array is named *updates*, it contains updates to pre-existing objects. Objects in ``updates`` arrays are kept and applied at the end of the loading stage, when all objects in ``objects`` array have been loaded. They are matched by name. So far, only the following objects and attributes can be updated: - ``material`` -- material and all its sub-attributes. In particular all, regular textures, cubemaps and virtual textures: - ``diffuse``, ``diffuseCubemap``, ``diffuseSVT``. - ``specular``, ``specularCubemap``, ``specularSVT``. - ``normal``, ``normalCubemap``, ``normalSVT``. - ``height``, ``heightCubemap``, ``heightSVT``. - ``emissive``, ``emissiveCubemap``, ``emissiveSVT``. - ``metallic``, ``metallicCubemap``, ``metallicSVT``. - ``roughness``, ``roughnessCubemap``, ``roughnessSVT``. - ``cloud`` -- describes the cloud layer. Can also have a virtual texture. - ``diffuse``, ``diffuseCubemap``, ``diffuseSVT``. - ``atmosphere`` -- all its direct attributes. - ``rotation`` -- all its direct attributes. See the :ref:`virtual textures section ` for some examples. Basic attributes ---------------- All :ref:`archetypes ` have the :ref:`comp-base`, the :ref:`comp-body` and the :ref:`comp-graphnode` components. These components hold basic attributes, which can be specified with these (usually required) keys: - ``name`` -- The name of the object. You can specify multiple names in a string array by using the ``names`` key. - ``color`` -- The color of the object. This will translate to the line color in orbits, to the color of the point for planets when they are far away and to the color of the grid in grids. - ``componentType`` (alias: ``ct``) -- The ``ComponentType`` (see `here `__). This is basically a ``string`` that will be matched to the entity type in ``ComponentType`` enum. Valid component types are ``Stars``, ``Planets``, ``Moons``, ``Satellites``, ``Atmospheres``, ``Constellations``, etc. - ``archetype`` (alias: ``impl``) -- The :ref:`archetype ` name (or legacy package and class name of the implementing class). - ``parent`` -- The name of the parent entity. Additionally, different types of entities accept different additional parameters which are matched to the model using reflection. Here are some examples of these parameters: - ``sizeKm`` (with variants ``sizePc``, ``sizeM``, ``sizeAU``, ``size``) -- The diameter of the entity. The unitless version uses internal units. - ``pos`` (with variants ``position``, ``positionKm``, ``positionPc``, ``posKm``, ``posPc``) -- The position of the object. This is the position at epoch (if the object has proper motion), or just a static position. Given in cartesian coordinates in the internal reference system. - ``labelColor`` -- Color of the label of this object. Below is an example of a simple entity, the equatorial grid: .. code:: json { "name" : "Equatorial grid", "color" : [1.0, 0.0, 0.0, 0.5], "size" : 1.2e12, "componentType" : "Equatorial", "parent" : "Universe", "archetype" : "SphericalGrid" } Proper motions -------------- Objects of an :ref:`archetype ` with a :ref:`ProperMotion component ` can define proper motion attributes: - ``muAlpha`` (``muAlphaMasYr``) -- The :math:`\mu_{\alpha\star}`, in mas/yr. - ``muDelta`` (``muDeltaMasYr``) -- The :math:`\mu_{\delta}`, in mas/yr. - ``radialVelocity`` (``radialVelocityKms``) -- The radial velocity in km/s. - ``epochJd`` (``epochYear``) -- The proper motion epoch as a Julian date, or as a year fraction (2015.5). Magnitudes ---------- Objects of an :ref:`archetype ` with a :ref:`Magnitude component ` can define an apparent and absolute magnitudes. - ``appMag`` -- The apparent magnitude. - ``absMag`` -- The absolute magnitude. The apparent and absolute magnitudes are only used in celestial bodies. In stars, if only one is set, the other is computed automatically. If both are set, **consistency is not checked together with the distance**. Also in stars, the absolute magnitude is used to compute a pseudo-size which is used for rendering purposes only. See the :ref:`star rendering section ` for more information. .. _labels: Labels ------ All labels in Gaia Sky are applied the component type of the object they are attached to, plus the "Labels" component type. Here are some of the attributes related to labels. Attributes marked with a star (\*) can only be applied to objects whose :ref:`archetype ` has a :ref:`Label component `. - ``label``\* -- Whether to render the label at all. Takes in a boolean. - ``labelColor`` -- The color of the label, as a RGBA array. - ``forceLabel`` -- Whether to force-display the label for this entity, regardless of distance and size. - ``labelPositionPc``\* (``labelPositionKm``, ``labelPosition``) -- Override the position of the label. - ``labelFactor``\* -- Factor to apply to the size of the label for this object. - ``labelMax``\* -- Internal rendering factor, should not be set externally unless you know what you are doing. - ``textScale``\* -- Internal rendering factor, should not be set externally unless you know what you are doing. .. _coordinate-providers: Coordinates and ephemerides --------------------------- Within the ``coordinates`` object (see the :ref:`Coordinates component `) one specifies how to get the positional data of the entity given a time. This object contains a reference to the implementation class (which must implement ``IBodyCoordinates`` `here `__) and the necessary parameters to initialize it. There are currently a few implementations that can be used with the ``"impl"`` attribute. They are described in the following sub-sections. Orbit coordinates ~~~~~~~~~~~~~~~~~ Using ``OrbitLintCoordinates``, the coordinates of the object are linearly interpolated using its orbit, which is defined in a separated entity. See the :ref:`orbits section ` for more info. The ``name`` of the orbit entity must be given. For instance, the Hygieia moon uses orbit coordinates. .. code:: json { "coordinates" : { "impl" : "gaiasky.util.coord.OrbitLintCoordinates", "orbitName" : "Hygieia orbit" } } Static coordinates ~~~~~~~~~~~~~~~~~~ Use ``StaticCoordinates`` to specify a static position (or a position at epoch for entities with proper motion). This is equivalent to using the top-level ``pos`` attribute, which also specifies the position at epoch. Static coordinates can also be applied a transformation using the ``transformMatrix`` and ``transformName`` attributes. The position may be given in cartesian or spherical coordinates. - ``position`` (with variants: ``positionKm``, ``positionPc``) -- position in cartesian coordinates in the internal reference system. - ``positionEquatorial`` -- equatorial coordinates (:math:`\alpha` [deg], :math:`\delta` [deg], and distance [parsecs]). - ``positionEcliptic`` -- ecliptic coordinates (:math:`\lambda` [deg], :math:`\beta` [deg], and distance [parsecs]). - ``positionGalactic`` -- ecliptic coordinates (l [deg], b [deg], and distance [parsecs]). .. code:: json { "coordinates" : { "impl" : "gaiasky.util.coord.StaticCoordinates", "position" : [-2.169e17, -1.257e17, -1.898e16] } } VSOP87 ~~~~~~ All classes that extend ``AbstractVSOP87`` provide ephemerides for the major planets. These implement the ``VSOP87`` analytical solution. Our implementation of VSOP87 contains a class for each body, with all the terms hard-coded. For instance, to set up VSOP87 ephemeris for the Earth use the following: .. code:: json { "coordinates" : { "impl" : "gaiasky.util.coord.vsop87.EarthVSOP87", "orbitName" : "Earth orbit" } } .. raw:: html
Since 3.5.0
VSOP2000 ~~~~~~~~ Implementation of the `analytical planetary solution VSOP2000 `__. You just need to point to the file for the particular bodies. Data files are available for Mercury, Venus, Earth, Moon, the Earth-Moon barycentre, Mars, Jupiter, Saturn, Uranus, Neptune and Pluto in ftp://syrte.obspm.fr/francou/vsop2000/. In order to use those, use the VSOP2000 class with the desired data file. .. code:: json { "coordinates" : { "impl" : "gaiasky.util.coord.vsop2000.VSOP2000", "dataFile" : "/path/to/data/vsop2000-p03.dat", "orbitName" : "Earth orbit" } } .. raw:: html
Since 3.5.0
Chebyshev polynomials ~~~~~~~~~~~~~~~~~~~~~ Implementation of Chebyshev polynomials using the coefficients to compute the Ephemeris. Just like VSOP2000, each body needs a different data file, containing the coefficients for the body. .. code:: json { "coordinates" : { "impl" : "gaiasky.util.coord.chebyshev.ChebyshevEphemeris", "dataFile" : "/path/to/data/EARTH.position.data", "orbitName" : "Earth orbit" } } Heliotropic orbits ~~~~~~~~~~~~~~~~~~ Provides coordinates of objects in heliotropic orbits using those orbits' data, like Gaia or JWST. The implementation is in ``HeliotropicOrbitCoordinates``. Moon AA coordinates ~~~~~~~~~~~~~~~~~~~ ``MoonAACoordinates`` contains a special implementation of the algorithm described in the book Astronomical Algorithms by Jean Meeus that provides the position of the Moon. Pluto coordinates ~~~~~~~~~~~~~~~~~ ``PlutoCoordinates`` is a special implementation, described `here `__, which provides very fast but not very accurate positions for Pluto. Python scripting coordinates ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``PythonBodyCoordinates``, reserved for coordinate providers implemented in Python via scripting. This object uses ``IPythonCoordinatesProvider`` instances implemented in a Python script to source coordinates. For more information, see :ref:`this section `. .. _json-desc-format: Model objects ------------- Planets, moons, asteroids, etc. all use the model object ``Planet`` (`here `__). This provides a series of utilities that make their JSON specifications look similar. .. _rotation: .. _orientation: Orientation ~~~~~~~~~~~ Orientations in Gaia Sky may be given in two different formats: - :ref:`Rigid rotation ` -- the orientation is described with basic rotation parameters such as the period, inclination, axial tilt, etc. - :ref:`Quaternion orientation ` -- the orientation of the object is sourced from an entity that provides quaternions. .. _rigid-rotation: **Rigid rotation** The ``rigidRotation`` map (aliased to ``rotation``) describes, as you may imagine, the rigid rotation of the body in question by means of a series of parameters. Rotations are stored in the :ref:`Orientation component `, and use the ``rigidRotation`` map. A rigid rotation is described by the following parameters: - ``period`` -- The rotation period in hours. - ``axialtilt`` -- The axial tilt is the angle between the equatorial plane of the body and its orbital plane. In degrees. - ``inclination`` -- The inclination is the angle between the orbital plane and the ecliptic. In degrees. - ``ascendingnode`` -- The ascending node in degrees. - ``meridianangle`` -- The meridian angle in degrees. For instance, the rotation of Mars: .. code:: json { "rigidRotation": { "period" : 24.622962156, "axialtilt" : 25.19, "inclination" : 1.850, "ascendingnode" : 47.68143, "meridianangle" : 176.630 } } .. _quaternion-orientation: **Quaternion orientation** Some objects like satellites are typically oriented using quaternions. Since every satellite may have its own attitude analytical implementation, we support a generic way of providing quaternions, based on the orientation provider and source. The orientation provider contains the name of a class that extends `OrientationServer `__. We provide two general implementations, based on the spherical linear interpolation of quaternions (`QuaternionSlerpOrientationServer `__) and on normalized linear interpolation (`QuaternionNlerpOrientationServer `__. These implementations read the quaternion data from a CSV file in the following format: .. code:: text time-iso-8601,x,y,z,w For example, a valid quaternion slerp file would be ``quaternions.csv``: .. code:: text #time,x,y,z,w 2020-01-01T12:00:00Z,0.0,0.0,0.0,1.0 2020-02-01T12:00:00Z,1.0,0.0,0.0,0.0 2020-03-01T12:00:00Z,0.0,1.0,0.0,0.0 2020-04-01T12:00:00Z,0.0,0.0,1.0,0.0 2020-05-01T12:00:00Z,0.0,0.0,0.0,1.0 Each line contains: - **Time [ISO-8601]** -- the time of the quaternion. - **x** -- x component. - **y** -- y component. - **z** -- z component. - **w** -- w component. Once we have the file, we can use it in our object by using the :ref:`comp-orientation` properties ``orientationProvider`` and ``orientationSource``. .. code:: json { "orientationProvider": "gaiasky.data.orientation.QuaternionSlerpOrientationServer", "orientationSource": "path/to/quaternions.csv" } We also offer a specific implementation of OrientationServer for Gaia in the form of the `GaiaAttitudeServer `__. .. _model: Model ~~~~~ This section describes the format to specify models, but omits the procedural generation attributes. These are documented in the :ref:`procedural generation section `. The model object describes the model which must be used to represent the entity. Models are described in the :ref:`Model component `, and can have two origins: - They may come from a **3D model file**. In this case, you just need to specify the file. .. code:: json { "model": { "args" : [true], "model" : "$data/default-data/models/gaia/gaia.g3db" } } - They may be **generated on the fly**. In this case, you need to specify the type of model, a series of parameters and the material. .. code:: json { "model": { "args" : [true], "type" : "sphere", "staticLight" : true, "useColor" : "false", "ambientLevel" : 0.5, "ambientColor" : [0.1, 0.3, 0.6, 1.0], "params" : { "quality" : 180, "diameter" : 1.0, "flip" : false }, "material" : { "diffuse" : "$data/default-data/tex/base/earth-day*.jpg", "diffuseCubemap" : "$data/default-data/tex/cubemap/earth-day*", "diffuseSVT" : { "location" : "$data/virtualtex-earth-diffuse/tex", "tileSize" : 1024 }, "specular" : "$data/default-data/tex/base/earth-specular*.jpg", "normal" : "$data/default-data/tex/base/earth-normal*.jpg", "emissive" : "$data/default-data/tex/base/earth-night*.jpg", "hieght" : "$data/default-data/tex/base/earth-height*.jpg", "heightScale" : 8.12, "reflection" : [ 1.0, 1.0, 0.0 ] } } - ``type`` -- the type of model. Possible values are ``sphere``, ``disc``, ``cylinder`` and ``ring``. - ``staticLight`` -- this attribute takes in a boolean (``true`` or ``false``) or a floating point number. If present, this disconnects the ambient light of this model from the global ambient level, **and** does not apply directional lighting to the model. If set to ``true``, the ambient level of this model is set to the default 0.6. Otherwise, it is set to the given floating-point value in [0,1]. - ``useColor`` -- a boolean that indicates that the object color (in the ``color`` attribute) is to be used as the model color. If this is true, the object color is set as a model diffuse color attribute. - ``ambientLevel`` -- a single floating-point value with the ambient light level (in [0,1]) to apply to this model. If present, the model is disconnected from the global ambient light setting. - ``ambientColor`` -- a 3- or 4-component color (RGB or RGBA) for the ambient light of this model. If present, the model is disconnected from the global ambient light setting. - ``params`` -- parameters of the model. This depends on the type. The ``quality`` is the number of both horizontal and vertical divisions. The ``diameter`` is the diameter of the model and ``flip`` indicates whether the normals should be flipped to face outwards. The ``ring`` type also accepts ``innerradius`` and ``outerradius``. - ``material`` -- properties of the material, such as textures, reflections, elevation, etc. - ``diffuse`` -- the diffuse texture to use. - ``diffuseCubemap`` -- the location of the 6 sides of the diffuse cubemap to use. Takes precedence over ``diffuse``. The sides must be images with the ``_bk.jpg``, ``_ft.jpg``, ``_up.jpg``, ``_dn.jpg``, ``_rt.jpg``, ``_lf.jpg`` suffixes. The file formats can be JPG or PNG. Can be applied to all channels (specular, normal, emissive, height, metallic, roughness, etc.) More information on this can be found in the :ref:`cubemaps section `. - ``diffuseSVT`` -- an object with a ``location`` (path) and a ``tileSize`` (integer). Defines a diffuse virtual texture for this model. Can be applied to more channels. More information on this cam be found in the :ref:`virtual texture section `. - ``specular`` -- the specular map to produce specular reflections. This attribute also accepts a specular index or a specular color (RGB). More than one can be specified. - ``normal`` -- normal map to produce extra detail in the lighting. - ``emissive`` -- emissive texture, color or value. For planets, this acts as the night texture, which is applied to the part of the model in the shade. This attribute also accepts an emissive color (RGB) and an emissive index. - ``height`` -- height map which will be represented with tessellation or parallax mapping (see :ref:`elevation (height) `) and whose scale is defined in ``heightScale`` (in Km). - ``heightScale`` -- indicates the extent, in Km, of the top mapping value in the height map (corresponding to full white, or RGB [1,1,1]). When the elevation multiplier slider is set to 1, the highest point in the height map is displaced by this amount of kilometers. - ``roughness`` -- roughness texture, color or value for the PBR shader. - ``metallic`` -- metallic texture or value for the PBR shader. - ``ao`` -- ambient occlusion texture for the PBR shader. - ``diffuseScattering`` -- a color (vec[3]) or a single floating point number with the diffuse scattering value for this model. Diffuse scattering weighs the diffuse color and re-emits it if there are no shadows. - ``occlusionMetallicRoughness`` -- occlusion/metallic/roughness texture (in R, G and B channels respectively) texture for the PBR shader. Follows the glTF specification. - ``reflection`` -- specifies an index or a color. If this is present, the default skymap will be used to generate reflections on the surface of the material. Hint: look up the ``Reflections`` object in Gaia Sky. It is defined in ``satellites.json``. Additionally, we may use the following attributes for **ringed** models, also in the ``material`` group: - ``ringDiffuse`` -- diffuse texture for the ring. - ``ringNormal`` -- diffuse texture for the ring. - ``ringDiffuseScattering`` -- a color (vec[3]) or a single floating point number with the diffuse scattering value for this model. Diffuse scattering weighs the diffuse color and re-emits it if there are no shadows. .. _clouds: Clouds ~~~~~~ This defines the clouds layer (see :ref:`Cloud component `). It can be :ref:`procedurally generated ` or described with textures. Here we deal only with the textures mode. Let's see: .. code:: json "cloud" : { "size" : 6395.0, "cloud" : "$data/default-data/tex/base/earth-cloud*.jpg", "params" : { "quality" : 200, "diameter" : 2.0, "flip" : false } } Contains the ``size`` of the cloud model, its ``parameters`` (``quality``, ``diameter``, etc.) and the clouds texture. The clouds are combined with the planet using the equation: .. math:: \bar{C}_{result} = \bar{C}_{source} \times \bar{F}_{source} + \bar{C}_{destination} \times \bar{F}_{destination} where :math:`\bar{F}_{source}` is 1, and :math:`\bar{F}_{destination}` is :math:`1 - \bar{C}_{source}`. .. _atmospheric-parameters-description: .. _atmospheric-scattering: Atmospheric scattering parameters ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Planet atmospheres can also be defined using this object (see :ref:`Atmosphere component `). The ``atmosphere`` object gets a number of physical quantities that are fed in the atmospheric scattering algorithm (`Sean O'Neil, GPU Gems `__). .. code:: json { "atmosphere" : { "size" : 6600.0, "wavelengths" : [0.650, 0.570, 0.475], "m_Kr" : 0.0025, "m_Km" : 0.0015, "m_eSun" : 1.0, "fogdensity" : 2.5, "fogcolor" : [1.0, 0.7, 0.6], "params" : { "quality" : 180, "diameter" : 2.0, "flip" : true } } } The parameters are the following: - ``size`` -- radius of the sphere model used for the atmosphere, in km. - ``wavelengths``-- the values of :math:`1\over{\lambda^4}` for the red, green and blue channels. These are the Rayleigh scattering rates of different light wavelengths. - ``Kr`` -- Rayleigh scattering constant. - ``Km`` -- Mie scattering constant. - ``eSun`` -- the brightness of the illuminating star. - ``fog density`` -- density of the simulated fog when inside the atmosphere. - ``fog color`` -- the color of the fog. .. _mesh-objects: Mesh objects ------------ Gaia Sky supports Galaxy-size arbitrary meshes. These are usually used to represent iso-density surfaces for stars, dust or HII regions, among others. Mesh objects have all the regular attributes of model bodies (``name``, ``description``, ``color``, ``size``, etc.). Additionally, we offer three shading modes for meshes: - ``additive`` -- renders the mesh with transparency via additive blending. The DR2 hot star and HII density meshes use this shading mode. - ``dust`` -- renders an opaque mesh. An opacity value is computed for the edges (:math:`V \cdot N`), where :math:`V` is the pixel view vector from the camera and :math:`N` is the normal vector at that pixel. Opacity is rendered using dithering to avoid sorting issues. - ``regular`` -- renders the mesh with the regular, general-purpose per-pixel lighting shader. In order to specify the shading mode, a new top-level attribute ``"shading" : "additive|dust|regular"`` must be used: .. code:: json { "name" : "DR3 star density", "description" : "Star density iso-surface based on DR3 data", "color" : [0.95, 0.2, 0.2, 0.75], "size" : 3.0856775814913705E7, "labelcolor" : [0.95, 0.1, 0.1, 1.0], "shading" : "regular", "labelposition" : [1000.0, 0.0, 0.0], "componentType" : "Meshes", "fadeout" : [60000.0, 90000.0], "parent" : "Universe", "archetype" : "MeshObject", "transformName" : "galacticToEquatorialF", "model" : { "args" : [true], "staticLight" : true, "model" : "$data/mesh-dr3-stardenstiy/meshes/dr3/star_density.obj" } } .. _orbit-format: .. _orbits: Orbits ------ When we talk about orbits in this context we talk about orbit lines. In the Gaia Sky orbit lines may be created from two different sources. The sources are used by a class implementing the ``IOrbitDataProvider`` (`here `__) interface, which is also specified in their ``orbit`` object. Orbits are stored in the :ref:`Trajectory component `. - An **orbit data file**. In this case, the orbit data provider is ``OrbitFileDataProvider``. - The **orbital elements**, where the orbit data provider is ``OrbitalParametersProvider``. If the orbit is sampled it comes from an **orbit data file**. In the Gaia Sky the orbits of all major planets are sampled, as well as the orbit of Gaia. For instance, the orbit of **Venus**. .. code:: json { "name" : "Venus orbit", "color" : [1.0, 1.0, 1.0, 0.55], "componentType" : "Orbits", "parent" : "Sol", "archetype" : "Orbit", "provider" : "gaiasky.data.orbit.OrbitFileDataProvider", "orbit" : { "source" : "$data/default-data/orb.VENUS.dat", } } If the orbit is defined with its `orbital elements `__, the elements need to be specified in the ``orbit`` object. For example, the orbit of **Phobos**. .. code:: json { "name" : "Phobos orbit", "color" : [0.7, 0.7, 1.0, 0.4], "componentType" : "Orbits", "parent" : "Mars", "archetype" : "Orbit", "provider" : "gaiasky.data.orbit.OrbitalParametersProvider", "orbit" : { "period" : 0.31891023, "epoch" : 2455198, "semiMajorAxis" : 9377.2, "eccentricity" : 0.0151, "inclination" : 1.082, "ascendingNode" : 16.946, "argOfPericenter" : 157.116, "meanAnomaly" : 241.138, "mu" : 9.9e18 } } The orbit object is represented with the orbital elements: - ``period`` -- in days. - ``epoch`` -- in Julian days. - ``semiMajorAxis`` -- in km. - ``eccentricity`` -- no units. - ``inclination`` -- in degrees. - ``ascendingNode`` -- in degrees. - ``argOfPericenter`` -- in degrees. - ``meanAnomaly`` -- in degrees. - ``mu`` -- :math:`G*M` of central body (gravitational constant). Defaults to the Sun's. This will be anyway automatically recomputed from the period (:math:`T`) and the semi-major axis (:math:`a`), if set, as :math:`\mu=4 * \pi^{2} a^{3} / T^{2}`. Note that if the ``epoch`` is fully defined, the ``argOfPericenter`` is not needed. The orbital elements of extrasolar systems are typically given in a special reference system. In this reference system, the **reference plane** is the plane whose normal is the line of sight vector from the Sun to the planet or star for whom the orbit is defined. The **reference direction** is the direction from the object to the north celestial pole projected on the reference plane. In order to apply such a transformation automatically, Gaia Sky provides an additional attribute ``"model"`` to objects of the :ref:`archetype ` Orbit. When this attribute has the value ``"extasolar_system"``, the abovementioned transformation is applied automatically. The default value of ``"model"`` is ``"default"``. Below is an example: .. code:: json { "name": "J0805+4812 star orbit", "color": [ 1.0, 0.0, 1.0, 1.0 ], "componentType": [ "Orbits", "Stars" ], "parent": "J0805+4812 Center", "archetype": "Orbit", "provider": "gaiasky.data.orbit.OrbitalParametersProvider", "model": "extrasolar_system", "newMethod": true, "trail" : true, "trailMap" : 0.5, "fadeDistanceUp" : 50.0, "fadeDistanceDown" : 100.0, "orbit": { "period": 735.9078666506588, "epoch": 2457397.4170103897, "semiMajorAxis": 48464416.969729796, "eccentricity": 0.4203524474493615, "inclination": 107.89617887219426, "ascendingNode": 175.09066812003118, "argOfPericenter": 326.7130552945523, "meanAnomaly": 0.0, "mu": 9.9998e+21 } } .. raw:: html
Since 3.5.0
At object level, we can set the following attributes to control some visual properties: - ``trail`` -- fades the orbit as it gets further away from the object in the direction opposite to travel. By default, the end closest to the object position is mapped to 1, and the furthermost position from the object is mapped to 0. - ``trailMap`` -- the bottom mapping position for the trail. The orbit trail assigns an opacity value to each point of the orbit, where 1 is the location of the object and 0 is the other end. This mapping parameter defines the location in the orbit (in [0,1]) where we map the opacity value of 0. Set to 0 to have a full trail. Set to 0.5 to have a trail that spans half the orbit. Set to 1 to have no orbit at all. - ``fadeDistanceUp`` -- orbits with a body fade out as the camera get closer to the body. This is the far distance, in body radius units, where the orbit starts the fade (mapped to 1). This attribute only has effect if this trajectory has a body attached to it. - ``fadeDistanceDown`` -- orbits with a body fade out as the camera get closer to the body. This is the near distance, in body radius units, where the orbit finishes the fade (mapped to 0). This attribute only has effect if this trajectory has a body attached to it. Grids and other special objects ------------------------------- There are a last family of objects which do not fall in any of the previous categories. These are grids and other objects such as the Milky Way (inner and outer parts). These objects usually have a special implementation and specific parameters, so they are a good example of how to implement new objects. .. code:: json { "name" : "Galactic grid", "color" : [0.3, 0.5, 1.0, 0.5], "size" : 1.4e12, "componentType" : "Galactic", "transformName" : "equatorialToGalactic", "parent" : "Universe", "archetype" : "Grid" } For example, the grids accept a parameter ``transformName``, which specifies the geometric transform to use. In the case of the galactic grid, we need to use the ``equatorialToGalactic`` transform to have the grid correctly positioned in the celestial sphere. .. _affine-transformations: Affine transformations ---------------------- Model objects (meshes, shapes, models, etc.) of an :ref:`archetype ` that contains an :ref:`AffineTransformations component ` can define arbitrary affine transformations (rotation, translation, scale) in any order, as top-level attributes. These transformations will be applied to the local transformation matrix of the model in the same order they are defined in the JSON file. The supported attributes, and their names, are: - ``translate`` -- contains a 3-vector with a translation in :ref:`internal units `. *Example:* ``"translate" : [2.0, 0.0, 5.0]``. - ``translatePc`` -- contains a 3-vector with a translation in parsecs. *Example:* ``"translatePc" : [2.0, 0.0, 5.0]``. - ``translateKm`` -- contains a 3-vector with a translation in parsecs. *Example:* ``"translateKm" : [25000.0, 0.0, 0.0]``. - ``scale`` -- contains either a single floating-point value or a 3-vector with the scaling factor, in local model coordinates. *Example:* ``"scale" : [1.0, 1.0, 2.0]``, scales the model :math:`\times 2` in the Z component. - ``rotate`` -- contains a 4-vector where the first three components are the rotation axis, and the last component is the rotation angle in degrees. *Example:* ``"rotate": [1.0, 0.0, 0.0, 45.0]``, rotates the model :math:`45^\circ` around the :math:`\begin{pmatrix} 1 & 0 & 0 \end{pmatrix}` vector. For instance, the following JSON object, .. code:: json { "name": "Object name", "color": [ 0.4, 0.4, 1.0, 0.4 ], "labelColor": [ 0.4, 0.4, 1.0, 1.0 ], "labelFactor": 0.0004, "sizePc": 1.0, "componentType": "Others", "parent": "Parent name", "archetype": "ShapeObject", "renderGroup": "MODEL_VERT_ADDITIVE", "rotate": [ 0.0, 1.0, 0.0, 60.0 ], "scale": [ 2.0, 0.5, 0.5 ], "coordinates": { "impl": "gaiasky.util.coord.StaticCoordinates", "position": [ 0.0, 0.0, 0.0 ] }, "model": { "args": [ true ], "staticLight": true, "type": "sphere", "primitiveType": "lines", "blendMode": "additive", "params": { "quality": 30, "diameter": 1.0, "flip": true } } } defines a sphere which is scaled :math:`\times 2` in X and :math:`\times 0.5` in Y and Z, and rotated around the Y axis :math:`60^\circ`. .. _ref-sys-transform: Reference system transformations -------------------------------- Gaia Sky uses an equatorial :ref:`internal reference system `. Objects of an :ref:`archetype ` that contains a :ref:`RefSysTransform component ` can specify reference system transformations directly in the JSON file as top-level attributes using the ``"transformName"`` or ``"transformFunction"`` names. These attributes take in a string with the name of the transformation. The possible values are: - "transformFunction": **"equatorialToEcliptic"** - "transformFunction": **"equatorialToGalactic"** - "transformFunction": **"eclipticToEquatorial"** - "transformFunction": **"eclipticToGalactic"** - "transformFunction": **"galacticToEquatorial"** - "transformFunction": **"galacticToEcliptic"** These essentially get transform to a 4x4 matrix with the necessary reference system rotation. Additionally, it is possible to specify the 16 values of the matrix themselves, in **column-major** order, using the top-level attributes ``"transformMatrix"`` or the alias ``"transformValues"``. *Example:* .. code:: json { "name": "Object name", "color": [ 0.4, 0.4, 1.0, 0.4 ], "labelColor": [ 0.4, 0.4, 1.0, 1.0 ], "labelFactor": 0.0004, "sizePc": 1.0, "componentType": "Others", "parent": "Parent name", "archetype": "ShapeObject", "renderGroup": "MODEL_VERT_ADDITIVE", "transformMatrix": [0.7660444431189781, 0.0, -0.12855752193730788, 0.0, 0.0, 0.5, 0.0, 0.0, 0.6427876096865394, 0.0, 0.15320888862379564, 0.0, 0.0, 0.0, 0.0, 1.0], "coordinates": { "impl": "gaiasky.util.coord.StaticCoordinates", "position": [ 0.0, 0.0, 0.0 ] }, "model": { "args": [ true ], "staticLight": true, "type": "sphere", "primitiveType": "lines", "blendMode": "additive", "params": { "quality": 30, "diameter": 1.0, "flip": true } } } Creating your own catalog loaders ================================= If you want to load your data files into Gaia Sky, chances are that the :ref:`STIL data provider ` can already do it. If you still need to create your own loader, keep reading. In order to create a loader for your catalog, one only needs to provide an implementation to the ``ISceneLoader`` (`here `__) interface. .. code:: java public interface ISceneLoader { public List loadData() throws FileNotFoundException; public void initialize(String[] files, Scene scene) throws RuntimeException; } The main method to implement is ``List loadData()`` (`here `__), which must return a list of ``Entity`` objects. But how do we know which file to load? You need to create a ``dataset.json`` descriptor file, add your loader there and create the properties you desire. Usually, there is a property called ``files`` which contains a list of files to load. Once you've done that, implement the ``initialize(String[], Scene)`` (`here `__) method knowing that all the properties defined in the ``dataset.json`` file with your catalog loader as a prefix will be passed in the ``Properties p`` object without prefix. Also, you will need to connect this new catalog file with the Gaia Sky configuration so that it is loaded at startup. To do so, locate your ``config.yaml`` file (usually under ``$GS_CONFIG``, see :ref:`folders `) and add your new file to the property ``data::catalogFiles``. You can also drop your new catalog into a subdirectory in the data directory and enable it using the dataset manager in Gaia Sky. Add your implementing ``jar`` file to the ``classpath`` (usually putting it in the lib/ folder should do the trick) and you are good to go. You can use existing loaders as examples, such as the ``OctreeLoader`` (`here `__) to see how it works. Loading data using scripts ========================== Data can also be loaded at any time from a ``Python`` script. See :ref:`the scripting section ` for more info.