Camera#

Description#

The Camera component simulates a camera attached to a link on the drone. The camera itself can be gimballed to achieve a horizon lock effect. In addition, the field-of-view (FOV), tilt angle, resolution, and offset distance from the main link can be adjusted. On image capture, the camera returns an RGBA image, a depth map, and a segmentation map with pixel values representing the IDs of objects in the environment.

Camera FPS#

Camera capture consumes a significant amount of overhead, see this github issue. It is not recommended to perform camera capture on every physics step.

Class Description#

class PyFlyt.core.abstractions.Camera(p: BulletClient, uav_id: int, camera_id: int, use_gimbal: bool, camera_FOV_degrees: float, camera_angle_degrees: float, camera_resolution: tuple[int, int], camera_position_offset: ndarray = array([0., 0., 0.]), is_tracking_camera: bool = False, cinematic: bool = False)#

A camera component.

The Camera component simulates a camera attached to a link on the drone. The camera itself can be gimballed to achieve a horizon lock effect. In addition, the field-of-view (FOV), tilt angle, resolution, and offset distance from the main link can be adjusted. On image capture, the camera returns an RGBA image, a depth map, and a segmentation map with pixel values representing the IDs of objects in the environment.

Parameters:

  • p (bullet_client.BulletClient) – PyBullet physics client ID.

  • uav_id (int) – ID of the drone.

  • camera_id (int) – integer representing the ID of the link that the camera is attached to.

  • use_gimbal (bool) – whether to lock the horizon of the camera.

  • camera_FOV_degrees (float) – the field-of-view of the camera in degrees.

  • camera_angle_degrees (float) – when gimballed, this is the angle of downtilt from horizon; when not gimballed, this is theh angle of uptile from horizon.

  • camera_resolution (tuple[int, int]) – the resolution of the camera in terms of [height, width].

  • camera_position_offset (np.ndarray = np.array([0.0, 0.0, 0.0])) – an (3,) array representing an offset of where the camera is from the center of the link in camera_id.

  • is_tracking_camera (bool = False) – if the camera is a tracking camera, the focus point of the camera is adjusted to focus on the center body of the aircraft instead of at infinity.

  • cinematic (bool = False) – it’s not a bug, it’s a feature.

capture_image() → tuple[ndarray, ndarray, ndarray]#

Captures the 3 relevant images from the camera.

Returns:

rgbaImg, depthImg, segImg

Return type:

tuple[np.ndarray, np.ndarray, np.ndarray]

get_states()#

This does not need to be called for camera. Call capture_image() instead within update_last().

physics_update()#

This does not need to be called for camera, call capture_image instead.

state_update()#

This does not need to be called for camera.

property view_mat: ndarray#

Generates the view matrix for the camera depending on the current orientation and implicit parameters.

Returns:

view matrix.

Return type:

np.ndarray