Class AbstractStarGroupDataProvider

java.lang.Object
gaiasky.data.group.AbstractStarGroupDataProvider
All Implemented Interfaces:
IParticleGroupDataProvider, IStarGroupDataProvider
Direct Known Subclasses:
BinaryDataProvider, SerializedDataProvider, STILDataProvider

public abstract class AbstractStarGroupDataProvider extends Object implements IStarGroupDataProvider
  • Field Details

    • NEGATIVE_DIST

      public static double NEGATIVE_DIST
    • logger

      protected static Logger.Log logger
    • parallelism

      protected final int parallelism
      Parallelism value
    • params

      protected Map<String,Object> params
    • indexMap

    • list

      protected List<IParticleRecord> list
    • sphericalPositions

      protected com.badlogic.gdx.utils.LongMap<double[]> sphericalPositions
    • colors

      protected com.badlogic.gdx.utils.LongMap<float[]> colors
    • countsPerMag

      protected long[] countsPerMag
    • transform

      protected Matrix4d transform
    • mustLoadIds

      protected Set<Long> mustLoadIds
    • additional

    • additionalFiles

      protected String[] additionalFiles
      Files or folders with optionally gzipped CSVs containing additional columns to be matched by sourceid with the main catalog. Column names must comport to AbstractStarGroupDataProvider.ColId.
    • ruwe

      protected Double ruwe
      RUWE cap value. Will accept all stars with star_ruwe invalid input: '<'= ruwe
    • distCap

      protected double distCap
      Distance cap in parsecs
    • parallaxErrorFactorFaint

      protected double parallaxErrorFactorFaint

      The loader will only load stars for which the parallax error is at most the percentage given here, in [0..1]. Faint stars (gmag >= 13.1) More specifically, the following must be met:

      pllx_err < pllx * pllxErrFactor
    • parallaxErrorFactorBright

      protected double parallaxErrorFactorBright

      The loader will only load stars for which the parallax error is at most the percentage given here, in [0..1]. Bright stars (gmag invalid input: '<' 13.1) More specifically, the following must be met:

      pllx_err < pllx * pllxErrFactor
    • adaptiveParallax

      protected boolean adaptiveParallax
      Whether to use an adaptive threshold which lets more bright stars in to avoid artifacts.
    • parallaxZeroPoint

      protected double parallaxZeroPoint
      The zero point for the parallaxes in mas. Gets added to all loaded parallax values
    • magCorrections

      protected boolean magCorrections
      Apply magnitude/color corrections for extinction/reddening
    • fileNumberCap

      protected int fileNumberCap
      Maximum number of files to load. Negative for unlimited
    • starNumberCap

      protected int starNumberCap
      Maximum number of files to load per file
  • Constructor Details

    • AbstractStarGroupDataProvider

      public AbstractStarGroupDataProvider()
  • Method Details

    • colIdFromStr

      public AbstractStarGroupDataProvider.ColId colIdFromStr(String name)
    • idx

      protected int idx(AbstractStarGroupDataProvider.ColId colId)
    • hasCol

      protected boolean hasCol(AbstractStarGroupDataProvider.ColId colId)
    • hasAdditional

      protected boolean hasAdditional(AbstractStarGroupDataProvider.ColId col, Long sourceId)
    • hasAdditionalColumn

      protected boolean hasAdditionalColumn(AbstractStarGroupDataProvider.ColId col)
    • getAdditionalValue

      protected Double getAdditionalValue(AbstractStarGroupDataProvider.ColId col, Long sourceId)
    • initLists

      protected void initLists(com.badlogic.gdx.files.FileHandle f)
      Initialises the lists and structures given a file by counting the number of lines
      Parameters:
      f - The file handle to count the lines
    • initLists

      protected void initLists(int elems)
      Initialises the lists and structures given number of elements
    • initLists

      protected void initLists()
    • loadData

      public List<IParticleRecord> loadData(String file)
      Description copied from interface: IParticleGroupDataProvider
      Loads the data as it is.
      Specified by:
      loadData in interface IParticleGroupDataProvider
      Parameters:
      file - The file to load.
      Returns:
      The array of particle records.
    • mustLoad

      protected boolean mustLoad(long id)
      Returns whether the star must be loaded or not
      Parameters:
      id - The star ID
      Returns:
      Whether the star with the given ID must be loaded
    • acceptParallax

      protected boolean acceptParallax(double appmag, double pllx, double pllxerr)
      Checks whether the parallax is accepted or not.

      If adaptive is not enabled:

       accepted = pllx > 0 invalid input: '&'invalid input: '&' pllx_err invalid input: '<' pllx * pllx_err_factor invalid input: '&'invalid input: '&' pllx_err invalid input: '<'= 1
       

      If adaptive is enabled:

       accepted = pllx > 0 invalid input: '&'invalid input: '&' pllx_err invalid input: '<' pllx * max(0.5, pllx_err_factor) invalid input: '&'invalid input: '&' pllx_err invalid input: '<'= 1, if apparent_magnitude invalid input: '<' 13.2
       accepted = pllx > 0 invalid input: '&'invalid input: '&' pllx_err invalid input: '<' pllx * pllx_err_factor invalid input: '&'invalid input: '&' pllx_err invalid input: '<'= 1, otherwise
       

      Parameters:
      appmag - Apparent magnitude of star
      pllx - Parallax of star
      pllxerr - Parallax error of star
      Returns:
      True if parallax is accepted, false otherwise
    • getRuweValue

      protected float getRuweValue(long sourceId, String[] tokens)
    • getGeoDistance

      protected double getGeoDistance(long sourceId)
      Gets the distance in parsecs to the star from the additional columns map, if it exists. Otherwise, it returns a negative value.
      Parameters:
      sourceId - The source id of the source
      Returns:
      The geometric distance in parsecs if it exists, -1 otherwise.
    • acceptDistance

      protected boolean acceptDistance(double distance)
      Checks whether to accept the distance
      Parameters:
      distance - Distance in parsecs
      Returns:
      Whether to accept the distance or not
    • countLines

      protected int countLines(com.badlogic.gdx.files.FileHandle f) throws IOException
      Throws:
      IOException
    • setColumns

      public void setColumns(String columns)
      Description copied from interface: IStarGroupDataProvider
      List of column names, separated by commas, indicating the position of each field to load.
      Specified by:
      setColumns in interface IStarGroupDataProvider
      Parameters:
      columns - The column name list.
    • setMustLoadIds

      public void setMustLoadIds(Set<Long> ids)
      Description copied from interface: IStarGroupDataProvider
      Adds a set with all the ids which will be loaded regardless of any other conditions (i.e. parallax error thresholds).
      Specified by:
      setMustLoadIds in interface IStarGroupDataProvider
      Parameters:
      ids - The ids that must be loaded.
    • countLines

      protected int countLines(InputStream is) throws IOException
      Counts the lines on this input stream
      Parameters:
      is - The input stream
      Returns:
      The number of lines
      Throws:
      IOException
    • dumpToDisk

      protected void dumpToDisk(List<IParticleRecord> data, String filename, String format)
    • dumpToDiskBin

      protected void dumpToDiskBin(List<IParticleRecord> data, String filename, boolean serialized)
    • dumpToDiskCsv

      protected void dumpToDiskCsv(List<IParticleRecord> data, String filename)
    • getColors

      public com.badlogic.gdx.utils.LongMap<float[]> getColors()
      Specified by:
      getColors in interface IStarGroupDataProvider
    • setParallaxErrorFactorFaint

      public void setParallaxErrorFactorFaint(double parallaxErrorFactor)
      Description copied from interface: IStarGroupDataProvider

      The loader will only load stars for which the parallax error is at most the percentage given here, in [0..1]. This applies to faint stars (gmag >= 13.1) More specifically, the following must be met:

      pllx_err < pllx * pllxErrFactor
      Specified by:
      setParallaxErrorFactorFaint in interface IStarGroupDataProvider
      Parameters:
      parallaxErrorFactor - The percentage value of parallax errors with respect to parallax.
    • setParallaxErrorFactorBright

      public void setParallaxErrorFactorBright(double parallaxErrorFactor)
      Description copied from interface: IStarGroupDataProvider

      The loader will only load stars for which the parallax error is at most the percentage given here, in [0..1]. This applies to bright stars (gmag invalid input: '<' 13.1) More specifically, the following must be met:

      pllx_err < pllx * pllxErrFactor
      Specified by:
      setParallaxErrorFactorBright in interface IStarGroupDataProvider
      Parameters:
      parallaxErrorFactor - The percentage value of parallax errors with respect to parallax.
    • setAdaptiveParallax

      public void setAdaptiveParallax(boolean adaptive)
      Description copied from interface: IStarGroupDataProvider
      Whether to use an adaptive threshold, relaxing it for bright (appmag >= 13) stars to let more bright stars in.
      Specified by:
      setAdaptiveParallax in interface IStarGroupDataProvider
    • setParallaxZeroPoint

      public void setParallaxZeroPoint(double parallaxZeroPoint)
      Description copied from interface: IStarGroupDataProvider
      Sets the zero point of the parallax as an addition to the parallax values, in [mas].
      Specified by:
      setParallaxZeroPoint in interface IStarGroupDataProvider
      Parameters:
      parallaxZeroPoint - The parallax zero point.
    • setMagCorrections

      public void setMagCorrections(boolean magCorrections)
      Description copied from interface: IStarGroupDataProvider
      Sets the flag to apply magnitude and color corrections for extinction and reddening.
      Specified by:
      setMagCorrections in interface IStarGroupDataProvider
      Parameters:
      magCorrections - Whether to apply the corrections.
    • getCountsPerMag

      public long[] getCountsPerMag()
      Description copied from interface: IStarGroupDataProvider
      Gets the star counts per magnitude.
      Specified by:
      getCountsPerMag in interface IStarGroupDataProvider
    • setAdditionalFiles

      public void setAdditionalFiles(String additionalFiles)
      Description copied from interface: IStarGroupDataProvider
      Set location of additional columns file or directory.
      Specified by:
      setAdditionalFiles in interface IStarGroupDataProvider
      Parameters:
      additionalFiles - File or directory with additional columns per sourceId.
    • setDistanceCap

      public void setDistanceCap(double distCap)
      Description copied from interface: IStarGroupDataProvider
      Sets a distance cap. Stars beyond this distance will not be loaded.
      Specified by:
      setDistanceCap in interface IStarGroupDataProvider
      Parameters:
      distCap - The distance cap, in parsecs.
    • setRUWECap

      public void setRUWECap(double RUWE)
      Description copied from interface: IStarGroupDataProvider
      Sets the RUWE criteria. RUWE file must have been set.
      Specified by:
      setRUWECap in interface IStarGroupDataProvider
      Parameters:
      RUWE - The criteria (usually 1.4).
    • setFileNumberCap

      public void setFileNumberCap(int cap)
      Description copied from interface: IParticleGroupDataProvider
      Sets a cap on the number of files to load. Set to negative for unlimited.
      Specified by:
      setFileNumberCap in interface IParticleGroupDataProvider
      Parameters:
      cap - The file cap number.
    • setStarNumberCap

      public void setStarNumberCap(int starNumberCap)
      Description copied from interface: IParticleGroupDataProvider
      Sets the maximum number of stars to be processed per file. Set to negative for unlimited.
      Specified by:
      setStarNumberCap in interface IParticleGroupDataProvider
      Parameters:
      starNumberCap - The star cap number.
    • setOutputFormatVersion

      public void setOutputFormatVersion(int version)
      Description copied from interface: IStarGroupDataProvider
      Set the preferred output format version, if applicable.
      Specified by:
      setOutputFormatVersion in interface IStarGroupDataProvider
      Parameters:
      version - The version number.
    • setProviderParams

      public void setProviderParams(Map<String,Object> params)
      Description copied from interface: IParticleGroupDataProvider
      Set provider parameters as a map.
      Specified by:
      setProviderParams in interface IParticleGroupDataProvider
      Parameters:
      params - The parameters map.
    • setTransformMatrix

      public void setTransformMatrix(Matrix4d transform)
      Description copied from interface: IParticleGroupDataProvider
      Sets a transform matrix to apply to all data points.
      Specified by:
      setTransformMatrix in interface IParticleGroupDataProvider
      Parameters:
      transform - The transform matrix.