Class AbstractStarGroupDataProvider

java.lang.Object
gaiasky.data.group.AbstractStarGroupDataProvider
All Implemented Interfaces:
IParticleGroupDataProvider, IStarGroupDataProvider
Direct Known Subclasses:
BinaryDataProvider, SerializedDataProvider, STILDataProvider
public abstract class AbstractStarGroupDataProvider extends Object implements IStarGroupDataProvider
Base abstract class for star group data providers.

This class implements basic loading and processing of star catalog data. It provides common parameters like distance caps, RUWE caps, and parallax error factors, which can be configured to filter out stars during loading. Providers can also handle additional data files and perform coordinate transformations.

  • Field Details

    • logger

      protected static Logger.Log logger

    • parallelism

      protected final int parallelism
      Parallelism value

    • params

      protected Map<String,Object> params

    • indexMap

      protected Map<AbstractStarGroupDataProvider.ColId, Integer> indexMap

    • list

      protected List<IParticleRecord> list

    • sphericalPositions

      protected com.badlogic.gdx.utils.LongMap<double[]> sphericalPositions

    • colors

      protected com.badlogic.gdx.utils.LongMap<float[]> colors

    • countsPerMag

      protected long[] countsPerMag

    • transform

      protected Matrix4D transform

    • mustLoadIds

      protected Set<Long> mustLoadIds

    • additional

      protected List<AbstractStarGroupDataProvider.AdditionalCols> additional

    • additionalFiles

      protected String[] additionalFiles
      Files or folders with optionally gzipped CSVs containing additional columns to be matched by sourceId with the main catalog. Column names must comport to AbstractStarGroupDataProvider.ColId.

    • ruwe

      protected Double ruwe
      RUWE cap value. Will accept all stars with star_ruwe ≤ ruwe

    • distCap

      protected double distCap
      Distance cap in parsecs

    • parallaxErrorFactorFaint

      protected double parallaxErrorFactorFaint

      The loader will only load stars for which the parallax error is at most the percentage given here, in [0..1]. Faint stars (gmag ≥ 13.1) More specifically, the following must be met:

      pllx_err < pllx * pllxErrFactor

    • parallaxErrorFactorBright

      protected double parallaxErrorFactorBright

      The loader will only load stars for which the parallax error is at most the percentage given here, in [0..1]. Bright stars (gmag < 13.1) More specifically, the following must be met:

      pllx_err < pllx * pllxErrFactor

    • adaptiveParallax

      protected boolean adaptiveParallax
      Whether to use an adaptive threshold which lets more bright stars in to avoid artifacts.

    • parallaxZeroPoint

      protected double parallaxZeroPoint
      The zero point for the parallaxes in mas. Gets added to all loaded parallax values

    • magCorrections

      protected boolean magCorrections
      Apply magnitude/color corrections for extinction/reddening

    • fileNumberCap

      protected int fileNumberCap
      Maximum number of files to load. Negative for unlimited

    • starNumberCap

      protected int starNumberCap
      Maximum number of files to load per file

  • Constructor Details

    • AbstractStarGroupDataProvider

      public AbstractStarGroupDataProvider()

  • Method Details

    • colIdFromStr

      public AbstractStarGroupDataProvider.ColId colIdFromStr(String name)

    • idx

      protected int idx(AbstractStarGroupDataProvider.ColId colId)

    • hasCol

      protected boolean hasCol(AbstractStarGroupDataProvider.ColId colId)

    • hasAdditional

      protected boolean hasAdditional(AbstractStarGroupDataProvider.ColId col, Long sourceId)

    • hasAdditionalColumn

      protected boolean hasAdditionalColumn(AbstractStarGroupDataProvider.ColId col)

    • getAdditionalValue

      protected Double getAdditionalValue(AbstractStarGroupDataProvider.ColId col, Long sourceId)

    • initLists

      protected void initLists(com.badlogic.gdx.files.FileHandle f)
      Initialises the lists and structures given a file by counting the number of lines
      Parameters:
      f - The file handle to count the lines

    • initLists

      protected void initLists(int elems)
      Initialises the lists and structures given number of elements

    • initLists

      protected void initLists()

    • loadData

      public List<IParticleRecord> loadData(String file)
      Description copied from interface: IParticleGroupDataProvider
      Loads the data as it is.
      Specified by:
      loadData in interface IParticleGroupDataProvider
      Parameters:
      file - The file to load.
      Returns:
      The array of particle records.

    • mustLoad

      protected boolean mustLoad(long id)
      Returns whether the star must be loaded or not
      Parameters:
      id - The star ID
      Returns:
      Whether the star with the given ID must be loaded

    • acceptParallax

      protected boolean acceptParallax(double appMag, double parallax, double parallaxError)
      Checks whether the parallax is accepted or not. If adaptive is not enabled: 
      accepted = pllx > 0 && pllx_err < pllx * pllx_err_factor && pllx_err ≤ 1

      If adaptive is enabled: 

      accepted = pllx > 0 && pllx_err < pllx * max(0.5, pllx_err_factor) && pllx_err ≤ 1, if apparent_magnitude < 13.2
accepted = pllx > 0 && pllx_err < pllx * pllx_err_factor && pllx_err ≤ 1, otherwise
      Parameters:
      appMag - Apparent magnitude of star.
      parallax - Parallax of star.
      parallaxError - Parallax error of star.
      Returns:
      True if parallax is accepted, false otherwise

    • getRuweValue

      protected float getRuweValue(long sourceId, String[] tokens)

    • getGeoDistance

      protected double getGeoDistance(long sourceId)
      Gets the distance in parsecs to the star from the additional columns map, if it exists. Otherwise, it returns a negative value.
      Parameters:
      sourceId - The source id of the source
      Returns:
      The geometric distance in parsecs if it exists, -1 otherwise.

    • acceptDistance

      protected boolean acceptDistance(double distance)
      Checks whether to accept the distance
      Parameters:
      distance - Distance in parsecs
      Returns:
      Whether to accept the distance or not

    • countLines

      protected int countLines(com.badlogic.gdx.files.FileHandle f) throws IOException
      Throws:
      IOException

    • setColumns

      public void setColumns(String columns)
      Description copied from interface: IStarGroupDataProvider
      List of column names, separated by commas, indicating the position of each field to load.
      Specified by:
      setColumns in interface IStarGroupDataProvider
      Parameters:
      columns - The column name list.

    • setMustLoadIds

      public void setMustLoadIds(Set<Long> ids)
      Description copied from interface: IStarGroupDataProvider
      Adds a set with all the ids which will be loaded regardless of any other conditions (i.e. parallax error thresholds).
      Specified by:
      setMustLoadIds in interface IStarGroupDataProvider
      Parameters:
      ids - The ids that must be loaded.

    • countLines

      protected int countLines(InputStream is) throws IOException
      Counts the lines on this input stream
      Parameters:
      is - The input stream
      Returns:
      The number of lines
      Throws:
      IOException

    • dumpToDisk

      protected void dumpToDisk(List<IParticleRecord> data, String filename, String format)

    • dumpToDiskBin

      protected void dumpToDiskBin(List<IParticleRecord> data, String filename)

    • dumpToDiskCsv

      protected void dumpToDiskCsv(List<IParticleRecord> data, String filename)

    • getColors

      public com.badlogic.gdx.utils.LongMap<float[]> getColors()
      Specified by:
      getColors in interface IStarGroupDataProvider

    • setParallaxErrorFactorFaint

      public void setParallaxErrorFactorFaint(double parallaxErrorFactor)
      Description copied from interface: IStarGroupDataProvider

      The loader will only load stars for which the parallax error is at most the percentage given here, in [0..1]. This applies to faint stars (gmag ≥ 13.1) More specifically, the following must be met:

      pllx_err < pllx * pllxErrFactor
      Specified by:
      setParallaxErrorFactorFaint in interface IStarGroupDataProvider
      Parameters:
      parallaxErrorFactor - The percentage value of parallax errors with respect to parallax.

    • setParallaxErrorFactorBright

      public void setParallaxErrorFactorBright(double parallaxErrorFactor)
      Description copied from interface: IStarGroupDataProvider

      The loader will only load stars for which the parallax error is at most the percentage given here, in [0..1]. This applies to bright stars (gmag < 13.1) More specifically, the following must be met:

      pllx_err < pllx * pllxErrFactor
      Specified by:
      setParallaxErrorFactorBright in interface IStarGroupDataProvider
      Parameters:
      parallaxErrorFactor - The percentage value of parallax errors with respect to parallax.

    • setAdaptiveParallax

      public void setAdaptiveParallax(boolean adaptive)
      Description copied from interface: IStarGroupDataProvider
      Whether to use an adaptive threshold, relaxing it for bright (appmag ≥ 13) stars to let more bright stars in.
      Specified by:
      setAdaptiveParallax in interface IStarGroupDataProvider

    • setParallaxZeroPoint

      public void setParallaxZeroPoint(double parallaxZeroPoint)
      Description copied from interface: IStarGroupDataProvider
      Sets the zero point of the parallax as an addition to the parallax values, in [mas].
      Specified by:
      setParallaxZeroPoint in interface IStarGroupDataProvider
      Parameters:
      parallaxZeroPoint - The parallax zero point.

    • setMagCorrections

      public void setMagCorrections(boolean magCorrections)
      Description copied from interface: IStarGroupDataProvider
      Sets the flag to apply magnitude and color corrections for extinction and reddening.
      Specified by:
      setMagCorrections in interface IStarGroupDataProvider
      Parameters:
      magCorrections - Whether to apply the corrections.

    • getCountsPerMag

      public long[] getCountsPerMag()
      Description copied from interface: IStarGroupDataProvider
      Gets the star counts per magnitude.
      Specified by:
      getCountsPerMag in interface IStarGroupDataProvider

    • setAdditionalFiles

      public void setAdditionalFiles(String additionalFiles)
      Description copied from interface: IStarGroupDataProvider
      Set location of additional columns file or directory.
      Specified by:
      setAdditionalFiles in interface IStarGroupDataProvider
      Parameters:
      additionalFiles - File or directory with additional columns per sourceId.

    • setDistanceCap

      public void setDistanceCap(double distCap)
      Description copied from interface: IStarGroupDataProvider
      Sets a distance cap. Stars beyond this distance will not be loaded.
      Specified by:
      setDistanceCap in interface IStarGroupDataProvider
      Parameters:
      distCap - The distance cap, in parsecs.

    • setRUWECap

      public void setRUWECap(double RUWE)
      Description copied from interface: IStarGroupDataProvider
      Sets the RUWE criteria. RUWE file must have been set.
      Specified by:
      setRUWECap in interface IStarGroupDataProvider
      Parameters:
      RUWE - The criteria (usually 1.4).

    • setFileNumberCap

      public void setFileNumberCap(int cap)
      Description copied from interface: IParticleGroupDataProvider
      Sets a cap on the number of files to load. Set to negative for unlimited.
      Specified by:
      setFileNumberCap in interface IParticleGroupDataProvider
      Parameters:
      cap - The file cap number.

    • setStarNumberCap

      public void setStarNumberCap(int starNumberCap)
      Description copied from interface: IParticleGroupDataProvider
      Sets the maximum number of stars to be processed per file. Set to negative for unlimited.
      Specified by:
      setStarNumberCap in interface IParticleGroupDataProvider
      Parameters:
      starNumberCap - The star cap number.

    • setOutputFormatVersion

      public void setOutputFormatVersion(int version)
      Description copied from interface: IStarGroupDataProvider
      Set the preferred output format version, if applicable.
      Specified by:
      setOutputFormatVersion in interface IStarGroupDataProvider
      Parameters:
      version - The version number.

    • setProviderParams

      public void setProviderParams(Map<String,Object> params)
      Description copied from interface: IParticleGroupDataProvider
      Set provider parameters as a map.
      Specified by:
      setProviderParams in interface IParticleGroupDataProvider
      Parameters:
      params - The parameters map.

    • setTransformMatrix

      public void setTransformMatrix(Matrix4D transform)
      Description copied from interface: IParticleGroupDataProvider
      Sets a transform matrix to apply to all data points.
      Specified by:
      setTransformMatrix in interface IParticleGroupDataProvider
      Parameters:
      transform - The transform matrix.