Package gaiasky.data.group

Class AbstractStarGroupDataProvider

java.lang.Object
gaiasky.data.group.AbstractStarGroupDataProvider
All Implemented Interfaces:
IParticleGroupDataProvider, IStarGroupDataProvider
Direct Known Subclasses:
BinaryDataProvider, CsvCatalogDataProvider, SerializedDataProvider, STILDataProvider
public abstract class AbstractStarGroupDataProvider
extends java.lang.Object
implements IStarGroupDataProvider

  • Field Details

    • logger

      protected static Logger.Log logger

    • NEGATIVE_DIST

      public static double NEGATIVE_DIST

    • indexMap

      protected java.util.Map<AbstractStarGroupDataProvider.ColId,​java.lang.Integer> indexMap

    • list

      protected java.util.List<IParticleRecord> list

    • sphericalPositions

      protected com.badlogic.gdx.utils.LongMap<double[]> sphericalPositions

    • colors

      protected com.badlogic.gdx.utils.LongMap<float[]> colors

    • countsPerMag

      protected long[] countsPerMag

    • mustLoadIds

      protected java.util.Set<java.lang.Long> mustLoadIds

    • additional

      protected java.util.List<AbstractStarGroupDataProvider.AdditionalCols> additional

    • additionalFiles

      protected java.lang.String[] additionalFiles
      Files or folders with optionally gzipped CSVs containing additional columns to be matched by sourceid with the main catalog. Column names must comport to AbstractStarGroupDataProvider.ColId

    • ruwe

      protected java.lang.Double ruwe
      RUWE cap value. Will accept all stars with star_ruwe <= ruwe

    • distCap

      protected double distCap
      Distance cap in parsecs

    • parallaxErrorFactorFaint

      protected double parallaxErrorFactorFaint

      The loader will only load stars for which the parallax error is at most the percentage given here, in [0..1]. Faint stars (gmag >= 13.1) More specifically, the following must be met:

      pllx_err < pllx * pllxErrFactor

    • parallaxErrorFactorBright

      protected double parallaxErrorFactorBright

      The loader will only load stars for which the parallax error is at most the percentage given here, in [0..1]. Bright stars (gmag < 13.1) More specifically, the following must be met:

      pllx_err < pllx * pllxErrFactor

    • adaptiveParallax

      protected boolean adaptiveParallax
      Whether to use an adaptive threshold which lets more bright stars in to avoid artifacts.

    • parallaxZeroPoint

      protected double parallaxZeroPoint
      The zero point for the parallaxes in mas. Gets added to all loaded parallax values

    • magCorrections

      protected boolean magCorrections
      Apply magnitude/color corrections for extinction/reddening

    • fileNumberCap

      protected int fileNumberCap
      Maximum number of files to load. Negative for unlimited

    • starNumberCap

      protected int starNumberCap
      Maximum number of files to load per file

    • parallelism

      protected final int parallelism
      Parallelism value

  • Constructor Details

    • AbstractStarGroupDataProvider

      public AbstractStarGroupDataProvider()

  • Method Details

    • colIdFromStr

      public AbstractStarGroupDataProvider.ColId colIdFromStr​(java.lang.String name)

    • idx

      protected int idx​(AbstractStarGroupDataProvider.ColId colId)

    • hasCol

      protected boolean hasCol​(AbstractStarGroupDataProvider.ColId colId)

    • hasAdditional

      protected boolean hasAdditional​(AbstractStarGroupDataProvider.ColId col, java.lang.Long sourceId)

    • hasAdditionalColumn

      protected boolean hasAdditionalColumn​(AbstractStarGroupDataProvider.ColId col)

    • getAdditionalValue

      protected java.lang.Double getAdditionalValue​(AbstractStarGroupDataProvider.ColId col, java.lang.Long sourceId)

    • initLists

      protected void initLists​(com.badlogic.gdx.files.FileHandle f)
      Initialises the lists and structures given a file by counting the number of lines
      Parameters:
      f - The file handle to count the lines

    • initLists

      protected void initLists​(int elems)
      Initialises the lists and structures given number of elements

    • initLists

      protected void initLists()

    • loadData

      public java.util.List<IParticleRecord> loadData​(java.lang.String file)
      Description copied from interface: IParticleGroupDataProvider
      Loads the data as it is.
      Specified by:
      loadData in interface IParticleGroupDataProvider
      Parameters:
      file - The file to load
      Returns:
      Array of particle beans

    • mustLoad

      protected boolean mustLoad​(long id)
      Returns whether the star must be loaded or not
      Parameters:
      id - The star ID
      Returns:
      Whether the star with the given ID must be loaded

    • acceptParallax

      protected boolean acceptParallax​(double appmag, double pllx, double pllxerr)
      Checks whether the parallax is accepted or not.

      If adaptive is not enabled: 

       accepted = pllx > 0 && pllx_err < pllx * pllx_err_factor && pllx_err <= 1

      If adaptive is enabled: 

       accepted = pllx > 0 && pllx_err < pllx * max(0.5, pllx_err_factor) && pllx_err <= 1, if apparent_magnitude < 13.2
 accepted = pllx > 0 && pllx_err < pllx * pllx_err_factor && pllx_err <= 1, otherwise

      Parameters:
      appmag - Apparent magnitude of star
      pllx - Parallax of star
      pllxerr - Parallax error of star
      Returns:
      True if parallax is accepted, false otherwise

    • getRuweValue

      protected float getRuweValue​(long sourceId, java.lang.String[] tokens)

    • getGeoDistance

      protected double getGeoDistance​(long sourceId)
      Gets the distance in parsecs to the star from the additional columns map, if it exists. Otherwise, it returns a negative value.
      Parameters:
      sourceId - The source id of the source
      Returns:
      The geometric distance in parsecs if it exists, -1 otherwise.

    • acceptDistance

      protected boolean acceptDistance​(double distance)
      Checks whether to accept the distance
      Parameters:
      distance - Distance in parsecs
      Returns:
      Whether to accept the distance or not

    • countLines

      protected int countLines​(com.badlogic.gdx.files.FileHandle f) throws java.io.IOException
      Throws:
      java.io.IOException

    • setColumns

      public void setColumns​(java.lang.String columns)
      Description copied from interface: IStarGroupDataProvider
      List of column names, separated by commas, indicating the position of each field to load
      Specified by:
      setColumns in interface IStarGroupDataProvider
      Parameters:
      columns - The column name list

    • setMustLoadIds

      public void setMustLoadIds​(java.util.Set<java.lang.Long> ids)
      Description copied from interface: IStarGroupDataProvider
      Adds a set with all the ids which will be loaded regardless of any other conditions (i.e. parallax error thresholds)
      Specified by:
      setMustLoadIds in interface IStarGroupDataProvider
      Parameters:
      ids - The ids that must be loaded

    • countLines

      protected int countLines​(java.io.InputStream is) throws java.io.IOException
      Counts the lines on this input stream
      Parameters:
      is - The input stream
      Returns:
      The number of lines
      Throws:
      java.io.IOException

    • dumpToDisk

      protected void dumpToDisk​(java.util.List<IParticleRecord> data, java.lang.String filename, java.lang.String format)

    • dumpToDiskBin

      protected void dumpToDiskBin​(java.util.List<IParticleRecord> data, java.lang.String filename, boolean serialized)

    • dumpToDiskCsv

      protected void dumpToDiskCsv​(java.util.List<IParticleRecord> data, java.lang.String filename)

    • getColors

      public com.badlogic.gdx.utils.LongMap<float[]> getColors()
      Specified by:
      getColors in interface IStarGroupDataProvider

    • setParallaxErrorFactorFaint

      public void setParallaxErrorFactorFaint​(double parallaxErrorFactor)
      Description copied from interface: IStarGroupDataProvider

      The loader will only load stars for which the parallax error is at most the percentage given here, in [0..1]. This applies to faint stars (gmag >= 13.1) More specifically, the following must be met:

      pllx_err < pllx * pllxErrFactor
      Specified by:
      setParallaxErrorFactorFaint in interface IStarGroupDataProvider
      Parameters:
      parallaxErrorFactor - The percentage value of parallax errors with respect to parallax

    • setParallaxErrorFactorBright

      public void setParallaxErrorFactorBright​(double parallaxErrorFactor)
      Description copied from interface: IStarGroupDataProvider

      The loader will only load stars for which the parallax error is at most the percentage given here, in [0..1]. This applies to bright stars (gmag < 13.1) More specifically, the following must be met:

      pllx_err < pllx * pllxErrFactor
      Specified by:
      setParallaxErrorFactorBright in interface IStarGroupDataProvider
      Parameters:
      parallaxErrorFactor - The percentage value of parallax errors with respect to parallax

    • setAdaptiveParallax

      public void setAdaptiveParallax​(boolean adaptive)
      Description copied from interface: IStarGroupDataProvider
      Whether to use an adaptive threshold, relaxing it for bright (appmag >= 13) stars to let more bright stars in.
      Specified by:
      setAdaptiveParallax in interface IStarGroupDataProvider

    • setParallaxZeroPoint

      public void setParallaxZeroPoint​(double parallaxZeroPoint)
      Description copied from interface: IStarGroupDataProvider
      Sets the zero point of the parallax as an addition to the parallax values, in [mas]
      Specified by:
      setParallaxZeroPoint in interface IStarGroupDataProvider
      Parameters:
      parallaxZeroPoint - The parallax zero point

    • setMagCorrections

      public void setMagCorrections​(boolean magCorrections)
      Description copied from interface: IStarGroupDataProvider
      Sets the flag to apply magnitude and color corrections for extinction and reddening
      Specified by:
      setMagCorrections in interface IStarGroupDataProvider
      Parameters:
      magCorrections - Whether to apply the corrections

    • getCountsPerMag

      public long[] getCountsPerMag()
      Description copied from interface: IStarGroupDataProvider
      Gets the star counts per magnitude
      Specified by:
      getCountsPerMag in interface IStarGroupDataProvider

    • setAdditionalFiles

      public void setAdditionalFiles​(java.lang.String additionalFiles)
      Description copied from interface: IStarGroupDataProvider
      Set location of additional columns file or directory
      Specified by:
      setAdditionalFiles in interface IStarGroupDataProvider
      Parameters:
      additionalFiles - File or directory with additional columns per sourceId

    • setDistanceCap

      public void setDistanceCap​(double distCap)
      Description copied from interface: IStarGroupDataProvider
      Sets a distance cap. Stars beyond this distance will not be loaded
      Specified by:
      setDistanceCap in interface IStarGroupDataProvider
      Parameters:
      distCap - The distance cap, in parsecs

    • setRUWECap

      public void setRUWECap​(double RUWE)
      Description copied from interface: IStarGroupDataProvider
      Sets the RUWE criteria. RUWE file must have been set
      Specified by:
      setRUWECap in interface IStarGroupDataProvider
      Parameters:
      RUWE - The criteria (usually 1.4)

    • setFileNumberCap

      public void setFileNumberCap​(int cap)
      Description copied from interface: IParticleGroupDataProvider
      Sets a cap on the number of files to load. Set to negative for unlimited
      Specified by:
      setFileNumberCap in interface IParticleGroupDataProvider
      Parameters:
      cap - The file cap number

    • setStarNumberCap

      public void setStarNumberCap​(int starNumberCap)
      Description copied from interface: IParticleGroupDataProvider
      Sets the maximum number of stars to be processed per file. Set to negative for unlimited
      Specified by:
      setStarNumberCap in interface IParticleGroupDataProvider
      Parameters:
      starNumberCap - The star cap number

    • setOutputFormatVersion

      public void setOutputFormatVersion​(int version)
      Description copied from interface: IStarGroupDataProvider
      Set the preferred output format version, if applicable
      Specified by:
      setOutputFormatVersion in interface IStarGroupDataProvider
      Parameters:
      version - The version number