Package gaiasky.data.group

Class AbstractStarGroupDataProvider

java.lang.Object
gaiasky.data.group.AbstractStarGroupDataProvider
All Implemented Interfaces:
IParticleGroupDataProvider, IStarGroupDataProvider
Direct Known Subclasses:
BinaryDataProvider, SerializedDataProvider, STILDataProvider
public abstract class AbstractStarGroupDataProvider extends Object implements IStarGroupDataProvider

  • Field Details

    • NEGATIVE_DIST

      public static double NEGATIVE_DIST

    • logger

      protected static Logger.Log logger

    • parallelism

      protected final int parallelism
      Parallelism value

    • params

      protected Map<String,Object> params

    • indexMap

      protected Map<AbstractStarGroupDataProvider.ColId,Integer> indexMap

    • list

      protected List<IParticleRecord> list

    • sphericalPositions

      protected com.badlogic.gdx.utils.LongMap<double[]> sphericalPositions

    • colors

      protected com.badlogic.gdx.utils.LongMap<float[]> colors

    • countsPerMag

      protected long[] countsPerMag

    • transform

      protected Matrix4d transform

    • mustLoadIds

      protected Set<Long> mustLoadIds

    • additional

      protected List<AbstractStarGroupDataProvider.AdditionalCols> additional

    • additionalFiles

      protected String[] additionalFiles
      Files or folders with optionally gzipped CSVs containing additional columns to be matched by sourceid with the main catalog. Column names must comport to AbstractStarGroupDataProvider.ColId.

    • ruwe

      protected Double ruwe
      RUWE cap value. Will accept all stars with star_ruwe invalid input: '<'= ruwe

    • distCap

      protected double distCap
      Distance cap in parsecs

    • parallaxErrorFactorFaint

      protected double parallaxErrorFactorFaint

      The loader will only load stars for which the parallax error is at most the percentage given here, in [0..1]. Faint stars (gmag >= 13.1) More specifically, the following must be met:

      pllx_err < pllx * pllxErrFactor

    • parallaxErrorFactorBright

      protected double parallaxErrorFactorBright

      The loader will only load stars for which the parallax error is at most the percentage given here, in [0..1]. Bright stars (gmag invalid input: '<' 13.1) More specifically, the following must be met:

      pllx_err < pllx * pllxErrFactor

    • adaptiveParallax

      protected boolean adaptiveParallax
      Whether to use an adaptive threshold which lets more bright stars in to avoid artifacts.

    • parallaxZeroPoint

      protected double parallaxZeroPoint
      The zero point for the parallaxes in mas. Gets added to all loaded parallax values

    • magCorrections

      protected boolean magCorrections
      Apply magnitude/color corrections for extinction/reddening

    • fileNumberCap

      protected int fileNumberCap
      Maximum number of files to load. Negative for unlimited

    • starNumberCap

      protected int starNumberCap
      Maximum number of files to load per file

  • Constructor Details

    • AbstractStarGroupDataProvider

      public AbstractStarGroupDataProvider()

  • Method Details

    • colIdFromStr

      public AbstractStarGroupDataProvider.ColId colIdFromStr(String name)

    • idx

      protected int idx(AbstractStarGroupDataProvider.ColId colId)

    • hasCol

      protected boolean hasCol(AbstractStarGroupDataProvider.ColId colId)

    • hasAdditional

      protected boolean hasAdditional(AbstractStarGroupDataProvider.ColId col, Long sourceId)

    • hasAdditionalColumn

      protected boolean hasAdditionalColumn(AbstractStarGroupDataProvider.ColId col)

    • getAdditionalValue

      protected Double getAdditionalValue(AbstractStarGroupDataProvider.ColId col, Long sourceId)

    • initLists

      protected void initLists(com.badlogic.gdx.files.FileHandle f)
      Initialises the lists and structures given a file by counting the number of lines
      Parameters:
      f - The file handle to count the lines

    • initLists

      protected void initLists(int elems)
      Initialises the lists and structures given number of elements

    • initLists

      protected void initLists()

    • loadData

      public List<IParticleRecord> loadData(String file)
      Description copied from interface: IParticleGroupDataProvider
      Loads the data as it is.
      Specified by:
      loadData in interface IParticleGroupDataProvider
      Parameters:
      file - The file to load.
      Returns:
      The array of particle records.

    • mustLoad

      protected boolean mustLoad(long id)
      Returns whether the star must be loaded or not
      Parameters:
      id - The star ID
      Returns:
      Whether the star with the given ID must be loaded

    • acceptParallax

      protected boolean acceptParallax(double appmag, double pllx, double pllxerr)
      Checks whether the parallax is accepted or not.

      If adaptive is not enabled: 

       accepted = pllx > 0 invalid input: '&'invalid input: '&' pllx_err invalid input: '<' pllx * pllx_err_factor invalid input: '&'invalid input: '&' pllx_err invalid input: '<'= 1

      If adaptive is enabled: 

       accepted = pllx > 0 invalid input: '&'invalid input: '&' pllx_err invalid input: '<' pllx * max(0.5, pllx_err_factor) invalid input: '&'invalid input: '&' pllx_err invalid input: '<'= 1, if apparent_magnitude invalid input: '<' 13.2
 accepted = pllx > 0 invalid input: '&'invalid input: '&' pllx_err invalid input: '<' pllx * pllx_err_factor invalid input: '&'invalid input: '&' pllx_err invalid input: '<'= 1, otherwise

      Parameters:
      appmag - Apparent magnitude of star
      pllx - Parallax of star
      pllxerr - Parallax error of star
      Returns:
      True if parallax is accepted, false otherwise

    • getRuweValue

      protected float getRuweValue(long sourceId, String[] tokens)

    • getGeoDistance

      protected double getGeoDistance(long sourceId)
      Gets the distance in parsecs to the star from the additional columns map, if it exists. Otherwise, it returns a negative value.
      Parameters:
      sourceId - The source id of the source
      Returns:
      The geometric distance in parsecs if it exists, -1 otherwise.

    • acceptDistance

      protected boolean acceptDistance(double distance)
      Checks whether to accept the distance
      Parameters:
      distance - Distance in parsecs
      Returns:
      Whether to accept the distance or not

    • countLines

      protected int countLines(com.badlogic.gdx.files.FileHandle f) throws IOException
      Throws:
      IOException

    • setColumns

      public void setColumns(String columns)
      Description copied from interface: IStarGroupDataProvider
      List of column names, separated by commas, indicating the position of each field to load.
      Specified by:
      setColumns in interface IStarGroupDataProvider
      Parameters:
      columns - The column name list.

    • setMustLoadIds

      public void setMustLoadIds(Set<Long> ids)
      Description copied from interface: IStarGroupDataProvider
      Adds a set with all the ids which will be loaded regardless of any other conditions (i.e. parallax error thresholds).
      Specified by:
      setMustLoadIds in interface IStarGroupDataProvider
      Parameters:
      ids - The ids that must be loaded.

    • countLines

      protected int countLines(InputStream is) throws IOException
      Counts the lines on this input stream
      Parameters:
      is - The input stream
      Returns:
      The number of lines
      Throws:
      IOException

    • dumpToDisk

      protected void dumpToDisk(List<IParticleRecord> data, String filename, String format)

    • dumpToDiskBin

      protected void dumpToDiskBin(List<IParticleRecord> data, String filename, boolean serialized)

    • dumpToDiskCsv

      protected void dumpToDiskCsv(List<IParticleRecord> data, String filename)

    • getColors

      public com.badlogic.gdx.utils.LongMap<float[]> getColors()
      Specified by:
      getColors in interface IStarGroupDataProvider

    • setParallaxErrorFactorFaint

      public void setParallaxErrorFactorFaint(double parallaxErrorFactor)
      Description copied from interface: IStarGroupDataProvider

      The loader will only load stars for which the parallax error is at most the percentage given here, in [0..1]. This applies to faint stars (gmag >= 13.1) More specifically, the following must be met:

      pllx_err < pllx * pllxErrFactor
      Specified by:
      setParallaxErrorFactorFaint in interface IStarGroupDataProvider
      Parameters:
      parallaxErrorFactor - The percentage value of parallax errors with respect to parallax.

    • setParallaxErrorFactorBright

      public void setParallaxErrorFactorBright(double parallaxErrorFactor)
      Description copied from interface: IStarGroupDataProvider

      The loader will only load stars for which the parallax error is at most the percentage given here, in [0..1]. This applies to bright stars (gmag invalid input: '<' 13.1) More specifically, the following must be met:

      pllx_err < pllx * pllxErrFactor
      Specified by:
      setParallaxErrorFactorBright in interface IStarGroupDataProvider
      Parameters:
      parallaxErrorFactor - The percentage value of parallax errors with respect to parallax.

    • setAdaptiveParallax

      public void setAdaptiveParallax(boolean adaptive)
      Description copied from interface: IStarGroupDataProvider
      Whether to use an adaptive threshold, relaxing it for bright (appmag >= 13) stars to let more bright stars in.
      Specified by:
      setAdaptiveParallax in interface IStarGroupDataProvider

    • setParallaxZeroPoint

      public void setParallaxZeroPoint(double parallaxZeroPoint)
      Description copied from interface: IStarGroupDataProvider
      Sets the zero point of the parallax as an addition to the parallax values, in [mas].
      Specified by:
      setParallaxZeroPoint in interface IStarGroupDataProvider
      Parameters:
      parallaxZeroPoint - The parallax zero point.

    • setMagCorrections

      public void setMagCorrections(boolean magCorrections)
      Description copied from interface: IStarGroupDataProvider
      Sets the flag to apply magnitude and color corrections for extinction and reddening.
      Specified by:
      setMagCorrections in interface IStarGroupDataProvider
      Parameters:
      magCorrections - Whether to apply the corrections.

    • getCountsPerMag

      public long[] getCountsPerMag()
      Description copied from interface: IStarGroupDataProvider
      Gets the star counts per magnitude.
      Specified by:
      getCountsPerMag in interface IStarGroupDataProvider

    • setAdditionalFiles

      public void setAdditionalFiles(String additionalFiles)
      Description copied from interface: IStarGroupDataProvider
      Set location of additional columns file or directory.
      Specified by:
      setAdditionalFiles in interface IStarGroupDataProvider
      Parameters:
      additionalFiles - File or directory with additional columns per sourceId.

    • setDistanceCap

      public void setDistanceCap(double distCap)
      Description copied from interface: IStarGroupDataProvider
      Sets a distance cap. Stars beyond this distance will not be loaded.
      Specified by:
      setDistanceCap in interface IStarGroupDataProvider
      Parameters:
      distCap - The distance cap, in parsecs.

    • setRUWECap

      public void setRUWECap(double RUWE)
      Description copied from interface: IStarGroupDataProvider
      Sets the RUWE criteria. RUWE file must have been set.
      Specified by:
      setRUWECap in interface IStarGroupDataProvider
      Parameters:
      RUWE - The criteria (usually 1.4).

    • setFileNumberCap

      public void setFileNumberCap(int cap)
      Description copied from interface: IParticleGroupDataProvider
      Sets a cap on the number of files to load. Set to negative for unlimited.
      Specified by:
      setFileNumberCap in interface IParticleGroupDataProvider
      Parameters:
      cap - The file cap number.

    • setStarNumberCap

      public void setStarNumberCap(int starNumberCap)
      Description copied from interface: IParticleGroupDataProvider
      Sets the maximum number of stars to be processed per file. Set to negative for unlimited.
      Specified by:
      setStarNumberCap in interface IParticleGroupDataProvider
      Parameters:
      starNumberCap - The star cap number.

    • setOutputFormatVersion

      public void setOutputFormatVersion(int version)
      Description copied from interface: IStarGroupDataProvider
      Set the preferred output format version, if applicable.
      Specified by:
      setOutputFormatVersion in interface IStarGroupDataProvider
      Parameters:
      version - The version number.

    • setProviderParams

      public void setProviderParams(Map<String,Object> params)
      Description copied from interface: IParticleGroupDataProvider
      Set provider parameters as a map.
      Specified by:
      setProviderParams in interface IParticleGroupDataProvider
      Parameters:
      params - The parameters map.

    • setTransformMatrix

      public void setTransformMatrix(Matrix4d transform)
      Description copied from interface: IParticleGroupDataProvider
      Sets a transform matrix to apply to all data points.
      Specified by:
      setTransformMatrix in interface IParticleGroupDataProvider
      Parameters:
      transform - The transform matrix.