.. _qs-datasets:

External datasets
-----------------

We can also load datasets into Gaia Sky at run time. Right now, the VOTable, CSV and FITS formats are supported. Gaia Sky needs some metadata in the form of *UCDs* or column names in order to parse the dataset columns correctly. 

.. admonition:: Docs

   See to the :ref:`stil-data-provider` section of the Gaia Sky user manual for more information on how to prepare your datasets for Gaia Sky.

The datasets loaded in Gaia Sky at a certain moment can be found in the :ref:`datasets pane <datasets-pane>` of the control panel. Open it by clicking on the |datasets| button or by pressing :guilabel:`d`.

.. figure:: ../img/quickstart/qs-datasets-pane.jpg
    :alt: Datasets pane
    :align: center
    :target: _images/gs-datasets-pane.jpg

    Datasets pane of Gaia Sky.


There are four main ways to load new datasets into Gaia Sky:

* Directly from the UI, using the |dsload| button (anchored to the bottom-left) or pressing :guilabel:`ctrl` + :guilabel:`o`.
* Through :ref:`SAMP <samp>`, via a connection to another astronomy software package such as Topcat or Aladin.
* Via a script, using one of the `dataset loading API calls <https://gaia.ari.uni-heidelberg.de/gaiasky/docs/javadoc/latest/gaiasky/script/IScriptingInterface.html#loadDataset(java.lang.String,java.lang.String)>`__.
* Creating a dataset in the Gaia Sky format so that it appears in the dataset manager (see :ref:`here <catalog-formats>`.

.. admonition:: Docs

   See the :ref:`data format section <catalog-formats>` to know how to create a Gaia Sky dataset (advanced!).

**Loading a dataset from the UI** -- Go ahead and remove the current star catalog by clicking on the |binicon| icon in the datasets pane. Now, download a raw `Hipparcos dataset VOTable <https://gaia.ari.uni-heidelberg.de/gaiasky/files/catalogs/hip/hipparcos.vot>`__, click on the |dsload| icon (or press :guilabel:`ctrl` + :guilabel:`o`) and select the file. In the next dialog just click :guilabel:`Ok` to start loading the catalog. In a few moments the Hipparcos new reduction dataset should be loaded into Gaia Sky.

**Loading a dataset via SAMP** -- This section presupposes that Topcat is installed on the machine and that the user knows how to use it to connect to the VO to get some data. The following video demonstrates how to do this (`Odysee mirror <https://odysee.com/@GaiaSky:8/gaia-sky-loading-data-with-topcat:9>`__, `YouTube mirror <https://youtu.be/sc0q-VbeoPE>`__):

.. figure:: ../img/quickstart/qs-gaiasky-topcat.jpg
	:width: 500px
	:align: center
	:target: https://gaia.ari.uni-heidelberg.de/gaiasky/files/temp/scripts/quickstartguide/tap-load-topcat.mkv

	Loading a dataset from Topcat through SAMP (click for video)

**Loading a dataset via scripting** -- Wait for the scripting section of this course.

**Preparing a descriptor file** -- Not addressed in this tutorial. See :ref:`the catalog formats section <catalog-formats>` for more information.

Working with datasets
^^^^^^^^^^^^^^^^^^^^^

.. tip:: Expand and collapse the datasets pane by clicking on the hard disk |datasets| button or with :guilabel:`d`.


All datasets loaded are displayed in the datasets pane in the control panel.
A few useful tips for working with datasets:

- The visibility of individual datasets can be switched on and off by clicking on the |eye-s-on| button
- Remove datasets with the |binicon| button
- We can **highlight a dataset** by clicking on the |highlight-s-off| button. The highlight color is defined by the color selector right on top of it. Additionally, we can map an attribute to the highlight color using a color map. Let's try it out:
  
  1. Click on the color box in the Hipparcos dataset we have just loaded from Topcat via SAMP
  2. Select the radio button "Color map"
  3. Select the *rainbow* color map
  4. Choose the attribute. In this case, we will use the number of transits, *ntr*
  5. Click :guilabel:`Ok`
  6. Click on the highlight dataset |highlight-s-off| icon to apply the color map

- We can **define basic filters** on the objects of the dataset using their attributes from the dataset preferences window |prefs-s-icon|. For example, we can filter out all stars with :math:`\delta > 50^{\circ}`:

  1. Click on the dataset preferences button |prefs-s-icon|
  2. Click on :guilabel:`Add filter`
  3. Select your attribute (declination :math:`\delta`)
  4. Select your comparator (:math:`<`)
  5. Enter your value, in this case 50
  6. Click :guilabel:`Ok`
  7. The stars with a declination greater than 50 degrees should be filtered out

Multiple filters can be combined with the **AND** and **OR** operators

.. |docs| image:: ../img/ui/iconic-book.png
   :class: filter-invert
.. |eye| image:: ../img/ui/eye-icon.png
   :class: filter-invert
.. |dsload| image:: ../img/ui/open-icon.png
   :class: filter-invert
.. |datasets| image:: ../img/ui/iconic-hard-drive.png
   :class: filter-invert
.. |binicon| image:: ../img/ui/bin-icon.png
   :class: filter-invert
.. |eye-s-on| image:: ../img/ui/eye-s-on.png
   :class: filter-invert
.. |prefs-s-icon| image:: ../img/ui/prefs-s-icon.png
   :class: filter-invert
.. |highlight-s-off| image:: ../img/ui/highlight-s-off.png
   :class: filter-invert