Class BinaryDataProvider

java.lang.Object

gaiasky.data.group.AbstractStarGroupDataProvider

gaiasky.data.group.BinaryDataProvider

All Implemented Interfaces:

IParticleGroupDataProvider, IStarGroupDataProvider

public class BinaryDataProvider
extends AbstractStarGroupDataProvider

Reads binary files using the infrastructure under BinaryIO. The format includes a header token and a version number. Depending on the version number a different loader is activated.

Nested Class Summary

Nested classes/interfaces inherited from class AbstractStarGroupDataProvider

AbstractStarGroupDataProvider.AdditionalCols, AbstractStarGroupDataProvider.ColId

Modifier and Type	Class	Description
static class	AbstractStarGroupDataProvider.AdditionalCols
static enum	AbstractStarGroupDataProvider.ColId	Represents a column type.

Field Summary

Fields

Modifier and Type	Field	Description
static int	DEFAULT_OUTPUT_VERSION	The default output format version to use for writing
static int	MAX_OUTPUT_VERSION
static int	MIN_OUTPUT_VERSION

Fields inherited from class AbstractStarGroupDataProvider

adaptiveParallax, additional, additionalFiles, colors, countsPerMag, distCap, fileNumberCap, indexMap, list, logger, magCorrections, mustLoadIds, parallaxErrorFactorBright, parallaxErrorFactorFaint, parallaxZeroPoint, parallelism, params, ruwe, sphericalPositions, starNumberCap, transform

Modifier and Type	Field	Description
protected boolean	adaptiveParallax	Whether to use an adaptive threshold which lets more bright stars in to avoid artifacts.
protected List<AbstractStarGroupDataProvider.AdditionalCols>	additional
protected String[]	additionalFiles	Files or folders with optionally gzipped CSVs containing additional columns to be matched by sourceId with the main catalog.
protected com.badlogic.gdx.utils.LongMap<float[]>	colors
protected long[]	countsPerMag
protected double	distCap	Distance cap in parsecs
protected int	fileNumberCap	Maximum number of files to load.
protected Map<AbstractStarGroupDataProvider.ColId, Integer>	indexMap
protected List<IParticleRecord>	list
protected static Logger.Log	logger
protected boolean	magCorrections	Apply magnitude/color corrections for extinction/reddening
protected Set<Long>	mustLoadIds
protected double	parallaxErrorFactorBright	The loader will only load stars for which the parallax error is at most the percentage given here, in [0..1].
protected double	parallaxErrorFactorFaint	The loader will only load stars for which the parallax error is at most the percentage given here, in [0..1].
protected double	parallaxZeroPoint	The zero point for the parallaxes in mas.
protected final int	parallelism	Parallelism value
protected Map<String,Object>	params
protected Double	ruwe	RUWE cap value.
protected com.badlogic.gdx.utils.LongMap<double[]>	sphericalPositions
protected int	starNumberCap	Maximum number of files to load per file
protected Matrix4D	transform

Constructor Summary

Constructors

Constructor	Description
BinaryDataProvider()

Method Summary

Modifier and Type	Method	Description
List<IParticleRecord>	loadData(InputStream is, double factor)	Loads the data applying a factor.
List<IParticleRecord>	loadData(String file, double factor)	Loads the data applying a factor.
List<IParticleRecord>	loadDataMapped(String file, double factor)	Loads the data applying a factor using a memory mapped file for improved speed.
List<IParticleRecord>	loadDataMapped(String file, double factor, int versionHint)	Loads data mapped with a version hint.
List<IParticleRecord>	readData(InputStream in, double factor)
void	setOutputFormatVersion(int version)	Set the preferred output format version, if applicable.
void	writeData(List<IParticleRecord> data, OutputStream out)
void	writeData(List<IParticleRecord> data, OutputStream out, int version)

Methods inherited from class AbstractStarGroupDataProvider

acceptDistance, acceptParallax, colIdFromStr, countLines, countLines, dumpToDisk, dumpToDiskBin, dumpToDiskCsv, getAdditionalValue, getColors, getCountsPerMag, getGeoDistance, getRuweValue, hasAdditional, hasAdditionalColumn, hasCol, idx, initLists, initLists, initLists, loadData, mustLoad, setAdaptiveParallax, setAdditionalFiles, setColumns, setDistanceCap, setFileNumberCap, setMagCorrections, setMustLoadIds, setParallaxErrorFactorBright, setParallaxErrorFactorFaint, setParallaxZeroPoint, setProviderParams, setRUWECap, setStarNumberCap, setTransformMatrix

Modifier and Type	Method	Description
protected boolean	acceptDistance(double distance)	Checks whether to accept the distance
protected boolean	acceptParallax(double appMag, double parallax, double parallaxError)	Checks whether the parallax is accepted or not.
AbstractStarGroupDataProvider.ColId	colIdFromStr(String name)
protected int	countLines(com.badlogic.gdx.files.FileHandle f)
protected int	countLines(InputStream is)	Counts the lines on this input stream
protected void	dumpToDisk(List<IParticleRecord> data, String filename, String format)
protected void	dumpToDiskBin(List<IParticleRecord> data, String filename)
protected void	dumpToDiskCsv(List<IParticleRecord> data, String filename)
protected Double	getAdditionalValue(AbstractStarGroupDataProvider.ColId col, Long sourceId)
com.badlogic.gdx.utils.LongMap<float[]>	getColors()
long[]	getCountsPerMag()	Gets the star counts per magnitude.
protected double	getGeoDistance(long sourceId)	Gets the distance in parsecs to the star from the additional columns map, if it exists.
protected float	getRuweValue(long sourceId, String[] tokens)
protected boolean	hasAdditional(AbstractStarGroupDataProvider.ColId col, Long sourceId)
protected boolean	hasAdditionalColumn(AbstractStarGroupDataProvider.ColId col)
protected boolean	hasCol(AbstractStarGroupDataProvider.ColId colId)
protected int	idx(AbstractStarGroupDataProvider.ColId colId)
protected void	initLists()
protected void	initLists(int elems)	Initialises the lists and structures given number of elements
protected void	initLists(com.badlogic.gdx.files.FileHandle f)	Initialises the lists and structures given a file by counting the number of lines
List<IParticleRecord>	loadData(String file)	Loads the data as it is.
protected boolean	mustLoad(long id)	Returns whether the star must be loaded or not
void	setAdaptiveParallax(boolean adaptive)	Whether to use an adaptive threshold, relaxing it for bright (appmag ≥ 13) stars to let more bright stars in.
void	setAdditionalFiles(String additionalFiles)	Set location of additional columns file or directory.
void	setColumns(String columns)	List of column names, separated by commas, indicating the position of each field to load.
void	setDistanceCap(double distCap)	Sets a distance cap.
void	setFileNumberCap(int cap)	Sets a cap on the number of files to load.
void	setMagCorrections(boolean magCorrections)	Sets the flag to apply magnitude and color corrections for extinction and reddening.
void	setMustLoadIds(Set<Long> ids)	Adds a set with all the ids which will be loaded regardless of any other conditions (i.e.
void	setParallaxErrorFactorBright(double parallaxErrorFactor)	The loader will only load stars for which the parallax error is at most the percentage given here, in [0..1].
void	setParallaxErrorFactorFaint(double parallaxErrorFactor)	The loader will only load stars for which the parallax error is at most the percentage given here, in [0..1].
void	setParallaxZeroPoint(double parallaxZeroPoint)	Sets the zero point of the parallax as an addition to the parallax values, in [mas].
void	setProviderParams(Map<String,Object> params)	Set provider parameters as a map.
void	setRUWECap(double RUWE)	Sets the RUWE criteria.
void	setStarNumberCap(int starNumberCap)	Sets the maximum number of stars to be processed per file.
void	setTransformMatrix(Matrix4D transform)	Sets a transform matrix to apply to all data points.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface IParticleGroupDataProvider

isUniformEpoch

Modifier and Type	Method	Description
default boolean	isUniformEpoch()	Do all particles in the group have the same epoch?

Field Details

DEFAULT_OUTPUT_VERSION

public static int DEFAULT_OUTPUT_VERSION

The default output format version to use for writing

MIN_OUTPUT_VERSION

public static int MIN_OUTPUT_VERSION

MAX_OUTPUT_VERSION

public static int MAX_OUTPUT_VERSION

Constructor Details

BinaryDataProvider

public BinaryDataProvider()

Method Details

loadData

public List<IParticleRecord> loadData(String file,
 double factor)

Description copied from interface: IStarGroupDataProvider

Loads the data applying a factor.

Parameters:

file - The file to load.

factor - Factor to apply to the positions.

Returns:

The array of particle records.

loadData

public List<IParticleRecord> loadData(InputStream is,
 double factor)

Description copied from interface: IStarGroupDataProvider

Loads the data applying a factor.

Parameters:

is - Input stream to load the data from.

factor - Factor to apply to the positions.

Returns:

The array of particle records.

writeData

public void writeData(List<IParticleRecord> data,
 OutputStream out)

writeData

public void writeData(List<IParticleRecord> data,
 OutputStream out,
 int version)

readData

public List<IParticleRecord> readData(InputStream in,
 double factor)

loadDataMapped

public List<IParticleRecord> loadDataMapped(String file,
 double factor)

Description copied from interface: IStarGroupDataProvider

Loads the data applying a factor using a memory mapped file for improved speed.

Parameters:

file - The file to load.

factor - Factor to apply to the positions.

Returns:

The array of particle records.

loadDataMapped

public List<IParticleRecord> loadDataMapped(String file,
 double factor,
 int versionHint)

Loads data mapped with a version hint.

Parameters:

file - The file to load

factor - Distance factor, if any

versionHint - Data version number, in case of version 0 or 1, since these formats were not annotated. If version >=2, the version number is read from the file header

Returns:

The list of particle records.

setOutputFormatVersion

public void setOutputFormatVersion(int version)

Description copied from interface: IStarGroupDataProvider

Set the preferred output format version, if applicable.

Specified by:

setOutputFormatVersion in interface IStarGroupDataProvider

Overrides:

setOutputFormatVersion in class AbstractStarGroupDataProvider

Parameters:

version - The version number.