Virtual Textures
Gaia Sky supports Sparse Virtual Textures (SVT), which enable ultra-high resolution partially resident textures to be used to map planets and other objects. From the user’s perspective, virtual textures are transparent, meaning that the user does not even need to be aware they are being used.
Contents
Hint
The implementation of Sparse Virtual Textures in Gaia Sky is thoroughly explained in this external article.
Overview
Virtual Textures (VT), also known as Sparse Virtual Textures (SVT), MegaTextures, and Partially Resident Textures (PRT), have at their core the idea of splitting large textures into several tiles and only streaming the necessary ones (i.e. the ones required to render the current view) to graphics memory in order to optimize memory usage and enable the display of textures so large that they can’t be handled effectively by the graphics hardware. In this article we use VT and SVT interchangeably to refer to virtual textures.
This technique aims at drastically increasing the size of usable textures in real time rendering applications by splitting them up in tiles and streaming only the necessary ones to graphics memory. It was initially described in a primitive form by Chris Hall in 1993 and has subsequently been improved upon. My understanding is that most modern implementations are based on Sean Barret’s GDC 2008 talk on the topic.
Committed texture pages are kept in a texture, called cache, which is unique for all virtual textures. The size of the cache (in tiles) can be adjusted in the Graphics Settings, virtual textures section.
Creating Virtual Texture Datasets
An SVT is essentially a quadtree which contains a downsized version of the whole texture in the root node. Each level contains 4 times the amount of tiles as the level above, and each tile covers 4 times less area. The pixel count and resolution of all tiles in all levels is always the same.
An example of a virtual texture with 3 levels (0 to 2) for the Earth laid out as a quadtree. Note that the root (level 0, top), covers the whole area, while successive levels have equally-sized tiles that cover less and less area each. This VT has an aspect ratio of 2:1, so it has two root nodes at the top.
In Gaia Sky, SVTs can be packed into a dataset. To do so, we create a new directory for the dataset, preferably using the naming convention vt-[object]-[channel]-[source]. For example, vt-earth-diffuse-nasa is a good name for a VT for the Earth’s surface generated from a NASA dataset. Virtual Textures, like regular textures and cubemaps, can be applied to several material properties:
Diffuse – the color of the surface of a planet or moon, for shading.
Specular – the specular map, for shading.
Normal – the normal map, for shading.
Height – the elevation map, to be used by the tessellation shader or by the parallax mapping process, depending on the height representation chosen.
Metallic – the metallic map, for PBR shading.
Roughness – the roughness map, for PBR shading.
Clouds – the cloud layer.
Typically, we create a virtual texture dataset for a pre-existing object, like the Earth, the Moon or Mars. The Gaia Sky JSON format incorporates some syntax to update already loaded objects. For instance, we can add a diffuse virtual texture to the Earth with the following JSON descriptor in the file vt-earth-diffuse-nasa.json:
{"updates" : [
{
"name" : "Earth",
"model": {
"material" : {
"diffuseSVT" : {
"location" : "$data/virtualtex-earth-diffuse/tex",
"tileSize" : 1024
}
}
}
}
]}
The "updates" object name at the top marks the objects in the list as updates. Then, we define the name of the object and the properties we need to update, with the same structure as in the original description file. For instance, if diffuseSVT is a property of material, which is inside model, the same structure must be maintained in the update file.
The following are the objects and attributes that can be updated:
material– material and all its sub-attributes. In particular all, regular textures, cubemaps and virtual textures: -diffuse,diffuseCubemap,diffuseSVT. -specular,specularCubemap,specularSVT. -normal,normalCubemap,normalSVT. -height,heightCubemap,heightSVT. -emissive,emissiveCubemap,emissiveSVT. -metallic,metallicCubemap,metallicSVT. -roughness,roughnessCubemap,roughnessSVT.cloud– describes the cloud layer. Can also have a virtual texture. -diffuse,diffuseCubemap,diffuseSVT.atmosphere– all its direct attributes.rotation– all its direct attributes.
Any SVT needs to specify a location and a tileSize. The location is the directory where the tiles for the different levels are located. The tile size is just the resolution of the tiles of this SVT.
Gaia Sky can work with multiple SVTs, but they all need to have the same tile size. Additionally, the tile size needs to be a power of two in [4, 1024].
Preparing the tiles
The dataset directory must contain a dataset descriptor file named dataset.json, and the actual data descriptor seen in the previous section (vt-earth-diffuse-nasa.json).
A VT dataset directory looks like this:
tex/
dataset.json
vt-earth-diffuse-nasa.json
The tiles are located in the tex directory within the dataset directory. Tile files are separated by levels using directories. Every level has the name level[level]. For example, the tiles for level 3 are all inside the tex/level3 directory.
The tex/ directory looks like the following, for a dataset with 7 levels, from 0 to 6:
level0/
level1/
level2/
level3/
level4/
level5/
level6/
Each level directory contains the tiles for that level. The first level contains either or two tiles (depending on the aspect ratio of the virtual texture), the second level contains 4 times that number, and so on (each tile is subdivided into 4 sub-tiles in the next level). Tile files are named tx_[col]_[row].ext, where col is the column, and row is the row. Supported formats are JPG and PNG.
The level1/ directory looks like this:
tx_0_0.jpg
tx_0_1.jpg
tx_1_0.jpg
tx_1_1.jpg
tx_2_0.jpg
tx_2_1.jpg
tx_3_0.jpg
tx_3_1.jpg
When in doubt, look at the existing VT datasets.
It is important to know that levels (except level 0) do not need to be complete. Missing tiles will be queried at higher levels automatically.
Tools
We did not find any open-source tools to our liking to create virtual texture tiles from high-resolution texture data, so we created our own. You can find them in the virtual texture tools repository. This repository contains two scripts:
split-tiles— can split a texture into square tiles of a given resolution, and names the tiles in the format expected by Gaia Sky (and also Celestia), which istx_[col]_[row].ext. The output format, quality and starting column and row are configurable via arguments.generate-lod— given a bunch of tiles and a level number, this script generates all the upper levels by stitching and resizing tiles. It lays them out in directories with the formatlevelN, where N is the zero-based level. The input tiles are also expected in a directory. The output format and quality are configurable.
Limitations
The limitations of our implementation are the following:
Due to the fact that all SVTs in the scene share the same cache, right now we can’t have SVTs with different tile sizes in the same scene.
Similarly, only square tiles are supported. Actually, I can’t think of a single good use case for non-square tiles.
Supported virtual texture aspect ratios are n:1, with \(n\geq1\). This is due to the fact that VT quadtrees are square by definition (1:1), and we have an array of root quadtree nodes that stack horizontally in the tree object. It is currently not possible to have a VT with a greater height than width.
Performance is not very good, especially with many SVTs running at once. This may be due to the shader mimpmap level lookups. This produces \(depth\) texture lookups (mip levels) in the worst-case scenario when only the root node is available in the cache. A workaround would be to fill lower levels, additionally to the tile level, in the indirection buffer whenever a tile enters the cache. This would also have a (CPU) overhead. Might be faster.
All SVTs in the scene share the same tile detection pass. This means that there is only one render operation in that pass. This might be good or bad, I’m not quite sure yet.