.. _camera-paths: .. role:: red .. role:: green .. role:: yellow .. role:: orange .. role:: blue .. role:: magenta .. role:: mouse Camera paths ************ Gaia Sky offers the possibility to record camera paths out of the box and later play them back. These camera paths are saved in a ``.gsc`` (for Gaia Sky Camera) file in ``$GS_DATA/camera`` (see :ref:`folders `). .. contents:: :backlinks: none .. _camera-path-file: Camera path file format ======================= The format of the ``.gsc`` files is pretty straightforward. It is a comma- or white space-separated text file (both are supported), each row containing the **state** of the camera and the **time** for a given frame. The state of the camera consists of 9 double-precision floating point numbers, 3 for the **position** and 3 for the **direction** vector and 3 for the **up** vector. Lines starting with the character '#' are ignored (with the exception of the frame rate annotation). .. raw:: html
Since 3.6.1
The **first line** in a camera path file may contain the target frame rate. If so, it should look like this: .. code:: text #fps 60.0 #time,pos_x,pos_y,pos_z,dir_x,dir_y,dir_z,up_x,up_y,up_z 2021-12-03T10:15:30Z,17.663479293630523,7.669932047249439,-147.39800210363168,0.8908469033686479,0.15523508799939112,-0.42695885306703063,-0.15546522466942284,0.9872364428222099,0.03456544346939234 2021-12-03T12:15:30Z,17.66336523250385,7.669924111844091,-147.39824473603434,0.8930812247239476,0.15539061859390402,-0.4222081023601746,-0.15543885339145308,0.9872410377214266,0.03455280444582746 [...] In that case, when the camera file is played back, the frame rate is automatically detected and used. As for the content, the reference system used for positions and directions is explained in the :ref:`reference-system` section. The units are :math:`1*10^{-9} m`. The format of each row is as follows: - time, which may be in one of these formats: - ``long integer``, defined as the number of milliseconds since epoch, 1970-01-01T00:00:00Z (UTC). - ``ISO-8601 date string`` that contains an instant in UTC, like '2011-12-03T10:15:30Z'. - 3x ``double-precision float`` -- [px, py, pz] position of the camera. - 3x ``double-precision float`` -- [dx, dy, dz] direction vector of the camera. - 3x ``double-precision float`` -- [ux, uy, yz] up vector of the camera. Gaia Sky provides two ways to **record camera paths**: real time recording and keyframes. :ref:`Keyframes ` are dealt with in the next sub-section. Here we look at the real time recording of camera paths using the integrated **camcorder**. .. _camcorder: Camcorder ========= The camcorder is located at the top of the :ref:`camera pane `. It offers buttons to do the following actions: - |rec-icon-gray|/|rec-icon-red| -- start and stop recording a camera path. When the |rec-icon-gray| :guilabel:`REC button` is gray, the camcorder is not recording. Press it to start recording a new camera path. When the camcorder is recording, the |rec-icon-red| :guilabel:`REC button` is red. Press it to stop the current recording. When a recording is stopped, it is automatically saved to a file in the ``$GS_DATA/camera`` :ref:`directory `. The file name is auto-generated. - |rec-key-icon-gray| -- open the :ref:`keyframes editor `. - |play-icon| -- :ref:`play a camera path file `. .. |rec-icon-gray| image:: img/ui/rec-icon-gray.png :class: filter-invert .. |rec-icon-red| image:: img/ui/rec-icon-red.png .. _camera-path-recording: Recording camera paths ====================== In order to **start recording** the camera path, click on the |rec-icon-gray| :guilabel:`REC button` next to the Camera section title in the GUI Controls window. The :guilabel:`REC button` turns red, |rec-icon-red|, which indicates the camera is being recorded. .. hint:: You can't record camera paths while the camcorder is in *playing* mode. In order to **stop the recording** and write the result to a file, click again on the |rec-icon-red| :guilabel:`REC button`. The button turns grey and a notification pops up indicating the location of the camera file. Camera files are by default saved in the ``$GS_DATA/camera`` directory (see :ref:`folders `). Frame rate ---------- **Mind the FPS!** The camcorder stores the time, position and orientation of the camera every frame. It is important that recording and playing back are done with the same (stable) frame rate. To set the target recording frame rate, edit the :guilabel:`Target FPS` field in the :ref:`camcorder settings ` of the preferences window. This makes sure the camera path is recorded using the target frame rate. .. raw:: html
Since 3.6.1
:ref:`Camera path files are annotated with the target frame rate ` In order to play back the camera file at the right frame rate. When the frame rate is in the camera file, the playback system automatically uses it. .. _keyframe-system: Keyframes editor ================ The keyframe editor offers the possibility to create keyframes at specific positions from which the camera file will be generated. In order start creating a keyframed camera path, click on the |rec-key-icon-gray| ``REC`` button in the camera pane of the control panel (marked with a red arrow in the screenshot below). A new window will pop up from which you'll be able to create and manage the keyframes. .. |rec-key-icon-gray| image:: img/ui/rec-key-icon-gray.png :class: filter-invert .. |export| image:: img/ui/export.png :class: filter-invert .. |open| image:: img/ui/open.png :class: filter-invert .. |save| image:: img/ui/save.png :class: filter-invert .. |seam| image:: img/ui/seam.png :class: filter-invert .. |go| image:: img/ui/go-to.png :class: filter-invert .. |plus| image:: img/ui/tree-plus.png :class: filter-invert .. |bin| image:: img/ui/bin-icon.png :class: filter-invert .. |others| image:: img/ui/icon-elem-others.png :class: filter-invert .. |keyframes| image:: img/ui/ct/icon-elem-keyframes.png :class: filter-invert .. |prefs| image:: img/ui/prefs-icon.png :class: filter-invert .. |clock| image:: img/ui/iconic-clock.png :class: filter-invert .. |target| image:: img/ui/iconic-target.png :class: filter-invert .. |project| image:: img/ui/iconic-project.png :class: filter-invert .. |caret-up| image:: img/ui/iconic-caret-top.png :class: filter-invert .. |caret-down| image:: img/ui/iconic-caret-bottom.png :class: filter-invert .. |slr| image:: img/ui/iconic-camera-slr.png :class: filter-invert Keyframes file format --------------------- Keyframes can be saved and loaded to and from ``.gkf`` files using the **keyframes file format**. These files only contain the information on the keyframes themselves. Once the keyframes have been created, they can be exported |export| to a ``.gsc`` camera path file. Both keyframe files and camera path files are stored by default in the ``$GS_DATA/camera`` folder (:ref:`see folders `). The keyframes file format is a text format with comma-separated values. Lines starting with '#' are ignored.. It contains a line for each keyframe. Each line has the following columns: - ``double-precision float`` -- keyframe duration since the last keyframe, in seconds. The duration of the first keyframe must be 0.0. - simulation time of the keyframe, which may be in one of these formats: - ``integer``, defined as the number of milliseconds since epoch, 1970-01-01T00:00:00Z (UTC). - ``ISO-8601 date string`` that contains an instant in UTC, like '2011-12-03T10:15:30Z'. - 3x ``double-precision float`` -- [px, py, pz] position of the camera in this keyframe. - 3x ``double-precision float`` -- [vx, vy, vz] direction of the camera in this keyframe. - 3x ``double-precision float`` -- [ux, uy, uz] up vector of the camera in this keyframe. - :red:`OPTIONAL`: 3x ``double-precision float`` -- [tx, ty, tz] position of the camera target, or point of interest, for this keyframe. This vector is only present if the camera was in focus mode when the keyframe was created or modified. - ``integer`` -- seam mark. This is 1 if the keyframe is a seam, and 0 otherwise. Creating and editing keyframes ------------------------------ .. figure:: img/screenshots/keyframes_screenshot.jpg :alt: Creating a keyframed camera path around Gaia :align: center Creating a keyframed camera path around Gaia. To the bottom-left, we see the keyframes editor window with some keyframes. The red arrow points to the button used to launch the keyframes editor. A visual representation of keyframes is displayed in the 3D world (see screenshot above) as lines (splines, orientations, etc.) and points (keyframe locations). The colors are as following: - :yellow:`Yellow lines` -- linear interpolation paths between keyframes. - :green:`Green lines` -- spline paths (either B-Splines or Catmull-Rom splines) between keyframes. They represent the true camera position. If the position interpolation is set to *linear* in the keyframes settings, the green lines coincide with the yellow lines. If the position interpolation is set to *Catmull-Rom splines*, the green line should hit every keyframe position perfectly. If the position interpolation is set to *B-splines*, the green line should only hit perfectly the path start and end points. - :red:`Red lines` -- for each keyframe, the red line represents the camera direction vector. - :blue:`Blue lines` -- for each keyframe, the blue line represents the camera up vector. - :green:`Green points` -- represent keyframe locations for keyframes which are not currently selected. - :magenta:`Magenta points` -- represent the keyframe that is currently selected and acts as a camera focus, if any. The keyframe visual representation is only displayed when the visibility of keyframes in enabled, so make sure that the visibility of ``Keyframes`` |keyframes| is on (you can use the :guilabel:`Keframe visuals` button in the keyframes editor directly). Keyframes can be **selected** and **dragged** with the right mouse button. The currently selected keyframe is highlighted in the keyframes list and also in the scene, using a magenta color. Here are the basic controls: - :mouse:`Right mouse` -- select keyframes (click) and move them around (drag). - :guilabel:`Shift` + :mouse:`Right mouse` -- drag to rotate the keyframe orientation around the up vector (in blue). - :guilabel:`Ctrl` + :mouse:`Right mouse` -- drag to rotate the keyframe orientation around the direction vector (in red). - :guilabel:`Alt` + :mouse:`Right mouse` -- drag to rotate the keyframe orientation around the perpendicular to the up and the direction vector (not represented in the scene). Adding keyframes ---------------- In order to add a new keyframe, click on the |plus| :guilabel:`Add keyframe at the end` button. The new keyframe will be created after the current one (if any), and with the current camera state (position, orientation). If the camera is in focus mode, the keyframe name will appear in :yellow:`yellow`, and the keyframe will have a target, or point of interest. See the :ref:`OptFlowCam export section ` for more information. The keyframe is also created with as many seconds after the previous keyframe as stated in the *Seconds after prev.* text field, and with the name given in the *Name (optional)* text field. If no name is entered, the default name of "Keyframe x" is used, where x is a monotonically increasing integer. Keyframes list -------------- To the right of the keyframes editor is the keyframes list. It is a list of keyframes with their properties, along with some buttons to perform some actions. The list is sorted top-to-bottom in the same order as they are in the path. Each entry in the list has the following elements, in order, left to right: - |caret-up| -- move the keyframe up in the list. This moves the keyframe one position to the left in the path. - |caret-down| -- move the keyframe down in the list. This moves the keyframe one position to the right in the path. - :green:`Keyframe time` -- the time of the keyframe relative to the first one, in seconds. The first keyframe in the list always has the time 000.00. By default, keyframes are created 1.0 seconds after the previous one. **Left click** on the :green:`green keyframe time label` to **edit** it on the fly. Once the edition is done, press :guilabel:`Enter` to persist. - (Frame number) -- the frame number relative to the first keyframe. This is just the keyframe time times the target FPS (defined in the :ref:`keyframe preferences `). - |clock| -- hover over this icon to see the simulation time attached to this keyframe. - |target| -- when this icon and the keyframe name are :yellow:`in yellow color`, the keyframe has a **target** position (also referred to as 'point of interest'). A keyframe has a target only when it was created when the camera was in focus mode. In that case, the keyframe target is the position of the camera focus object at the time. Targets are used only by the :ref:`OptFlowCam export function `. Refer to that section for more information. - :yellow:`Keyframe name` -- the name of the keyframe. By default, this will be "keyframe x", where x is a monotonically increasing integer number in the keyframes list, starting at 1. **Left click** on the keyframe name label to **edit** it on the fly. As with the target icon, the keyframe name appears in :yellow:`yellow` if the keyframe has a target position (point of interest). Targets are used only by the :ref:`OptFlowCam export function `. Refer to that section for more information. - |go| -- go to the keyframe. Puts the camera in the state specified by the keyframe. - |slr| -- set keyframe to the current camera state. This allows to modify the given keyframe by setting it to the current state of the camera (including position, orientation and target, if any). - |seam| -- mark the keyframe as seam. In case the spline interpolation method is chosen, this will break the spline path. - |plus| -- add a new keyframe after this one, interpolating position, orientation and time with the next one. If the two keyframes have a target, the target position is also interpolated. - |bin| -- remove the keyframe. .. raw:: html
Since 3.5.3
Playback controls ----------------- Below the keyframes list is a series of playback controls and a timeline slider. The slider is annotated with the current frame, and can be used to position the camera at any location in the path in real time. No need to export the camera path. - |skipb| **Beginning** -- Move to the beginning of the timeline. - |stepb| **Step back** -- Step one frame backward. - |play| **Play/pause** -- Play or pause the camera path. - |stepf| **Step forward** -- Step one frame forward. - |skipf| **End** -- Move to the end of the timeline. .. |play| image:: img/ui/media/iconic-media-play.png :class: filter-invert .. |stepf| image:: img/ui/media/iconic-media-step-forward.png :class: filter-invert .. |skipf| image:: img/ui/media/iconic-media-skip-forward.png :class: filter-invert .. |stepb| image:: img/ui/media/iconic-media-step-backward.png :class: filter-invert .. |skipb| image:: img/ui/media/iconic-media-skip-backward.png :class: filter-invert Export keyframes to camera paths -------------------------------- To the top of the keyframes window there are a few buttons to load, export and save keyframes projects. - |open| **Open...** -- load a new ``.gkf`` keyframes file. The button opens a file picker in the default camera directory (``$GS_DATA/camera``, see :ref:`system directories `). - |save| **Save...** -- save the current keyframes to a keyframes file ``.gkf`` in ``$GS_DATA/camera``. You can choose the file name, but not the directory. If another file with the same name exists, a unique file name is generated from the given one. - |export| **Export...** -- export the current project to a camera path file ``.gsc``. Optionally, the :ref:`OptFlowCam method ` can be used to export. The export process uses the settings defined in the :ref:`kefyrame preferences `. You can choose the file name, but not the directory. If another file with the same name exists, a unique file name is generated from the given one. - |project| **Normalize** -- recompute all keyframe times so that speed is constant. The total length and distance are unaltered. - |prefs| **Preferences** -- see next section, Keyframes preferences. .. _keyframe-preferences: Keyframes preferences --------------------- The ``Preferences`` button (lower right in the Keyframes window) opens a window which contains some settings related to the keyframes system: - **Target FPS** -- the target frame rate to use when generating the camera file from the keyframes. - **Interpolation type** -- method used to interpolate between positions (orientations are always interpolated using quaternion interpolation). The time is always interpolated linearly to prevent unwanted speed-ups and slow-downs. Two types of interpolation are available: - **Catmull-Rom splines** -- produce smoothed paths which hit every keyframe. In this mode, keyframes can be seams |seam|. Seam keyframes break up the path into two sections, so that two splines will be used ending and beginning at the keyframe. - **B-splines** -- produce smoothed paths which **do not** hit every keyframe. In this mode, keyframes can be seams |seam|. Seam keyframes break up the path into two sections, so that two splines will be used ending and beginning at the keyframe. - **Linear interpolation** -- keyframe positions are joined by straight lines. In this mode, the yellow and green lines above are the same. .. _optflowcam: Export keyframes with OptFlowCam -------------------------------- Gaia Sky includes the option to use the `OptFlowCam method `__ [#]_ to export the keyframes to camera path files. This method works very well in smoothing paths which span over long distance ranges and extremely varying scales. .. [#] *Piotrowski, Motejat, Roessl, Theisel. "OptFlowCam: A 3D-Image-Flow-Based Metric in Camera Space for Camera Paths in Scenes with Extreme Scale Variations", Computer Graphics Forum, 2024 (10.1111/cgf.15056)*, `link `__. .. figure:: img/screenshots/keyframes-export.jpg :alt: Keyframes export window. :width: 50% :align: center The keyframes export window contains a check box to activate the OptFlowCam post-processing. A few caveats need to be taken into account to use this functionality: - The **OptFlowCam** processing is implemented as an external **Python script**, so a local installation of `Python 3.x `__ must be in place and accessible via the operating system. NumPy is also required. Typically, once Python is installed, you can install NumPy by running ``pip3 install numpy`` in a terminal. In some cases, if the Python environment is externally managed, you may need to install it via your package manager (for instance, in Arch Linux you would do ``pacman -S python-numpy``). The Flatpak version of Gaia Sky already includes Python and NumPy. - `Install Python on Windows `__. - `Install Python on macOS `__. - Install Python on Linux: use the package manager provided by your distribution. - The technique works best if **every keyframe has a target** or point of interest (see sections above for more information), so make sure that all your keyframes are created when the camera is in focus mode. Otherwise, a default distance-to-target of 500.000 Km is assumed. - Note that the :green:`green line` visually indicating the camera path **is not respected** in this mode. Only the keyframes themselves are guaranteed to be hit, but not so the interpolated positions between them. - Following up on the previous point, the method is **not interactive**, and only kicks in at export time. Do not use the visual path lines for guidance, as they are not updated with this method in real time. .. _camera-path-playback: Playing camera paths ==================== In order to play a camera file, click on the |play-icon| :guilabel:`PLAY button` at the top of the :ref:`camera pane `. A file picker dialog opens, where you can select the camera path file to play. The file picker opens by default in the ``$GS_DATA/camera`` folder (see :ref:`folders `). .. |play-icon| image:: img/ui/play-icon.png :class: filter-invert You may also combine the camera file playback with the frame output system to save each frame as an image file to disk during playback. To do so, enable the :guilabel:`Activate frame output automatically` checkbox in the preferences dialog as described in the :ref:`camcorder-settings` section. .. hint:: You can't play camera paths while the camcorder is in *recording* mode. Camera paths are **recorded at a fixed frame rate**. Starting in version 3.6.1, these files are annotated with the target frame rate, so that the player automatically uses it. In previous versions of Gaia Sky, however it was necessary to manually cap the frame rate to the target frame rate. To do so, see the :ref:`graphics configuration section `.