l5kit.sampling.agent_sampling module

l5kit.sampling.agent_sampling.compute_agent_velocity(history_positions_m: numpy.ndarray, future_positions_m: numpy.ndarray, step_time: float) → Tuple[numpy.ndarray, numpy.ndarray]

Compute estimated velocities by finite differentiation on future positions as(pos(T+t) - pos(T))/t. This simple approach gives less than 0.5% velocity difference compared to (pos(T+t) - pos(T-t))/2t on v1.1/sample.zarr.tar.

Parameters

  • history_positions_m (np.ndarray) – history XY positions in meters

  • future_positions_m (np.ndarray) – future XY positions in meters

  • step_time (np.ndarray) – length of a step in second

Returns

history and future XY speeds

Return type

Tuple[np.ndarray, np.ndarray]

l5kit.sampling.agent_sampling.generate_agent_sample(state_index: int, frames: numpy.ndarray, agents: numpy.ndarray, tl_faces: numpy.ndarray, selected_track_id: Optional[int], render_context: l5kit.rasterization.render_context.RenderContext, history_num_frames: int, future_num_frames: int, step_time: float, filter_agents_threshold: float, rasterizer: l5kit.rasterization.rasterizer.Rasterizer, perturbation: Optional[l5kit.kinematic.perturbation.Perturbation] = None) → dict

Generates the inputs and targets to train a deep prediction model. A deep prediction model takes as input the state of the world (here: an image we will call the “raster”), and outputs where that agent will be some seconds into the future.

This function has a lot of arguments and is intended for internal use, you should try to use higher level classes and partials that use this function.

Parameters

  • state_index (int) – The anchor frame index, i.e. the “current” timestep in the scene

  • frames (np.ndarray) – The scene frames array, can be numpy array or a zarr array

  • agents (np.ndarray) – The full agents array, can be numpy array or a zarr array

  • tl_faces (np.ndarray) – The full traffic light faces array, can be numpy array or a zarr array

  • selected_track_id (Optional[int]) – Either None for AV, or the ID of an agent that you want to

  • from (predict the future of. This agent is centered in the raster and the returned targets are derived) –

  • states. (their future) –

  • render_context (RenderContext) – The context for rasterisation

  • history_num_frames (int) – Amount of history frames to draw into the rasters

  • future_num_frames (int) – Amount of history frames to draw into the rasters

  • step_time (float) – seconds between consecutive steps

  • filter_agents_threshold (float) – Value between 0 and 1 to use as cutoff value for agent filtering

  • agent (based on their probability of being a relevant) –

  • Rasterizer (rasterizer) – Rasterizer of some sort that draws a map image

  • perturbation (Optional[Perturbation]) – Object that perturbs the input and targets, used

  • data (to train models that can recover from slight divergence from training set) –

Raises

  • IndexError – An IndexError is returned if the specified selected_track_id is not present in the scene

  • or was filtered by applying the filter_agent_threshold probability filtering. –

Returns

a dict object with the raster array, the future offset coordinates (meters), the future yaw angular offset, the future_availability as a binary mask

Return type

dict

l5kit.sampling.agent_sampling.get_agent_context(state_index: int, frames: numpy.ndarray, agents: numpy.ndarray, tl_faces: numpy.ndarray, history_num_frames: int, future_num_frames: int) → Tuple[numpy.ndarray, numpy.ndarray, List[numpy.ndarray], List[numpy.ndarray], List[numpy.ndarray], List[numpy.ndarray]]

Slice zarr or numpy arrays to get the context around the agent onf interest (both in space and time)

Parameters

  • state_index (int) – frame index inside the scene

  • frames (np.ndarray) – frames from the scene

  • agents (np.ndarray) – agents from the scene

  • tl_faces (np.ndarray) – tl_faces from the scene

  • history_num_frames (int) – how many frames in the past to slice

  • future_num_frames (int) – how many frames in the future to slice

Returns

Tuple[np.ndarray, np.ndarray, List[np.ndarray], List[np.ndarray], List[np.ndarray], List[np.ndarray]]

l5kit.sampling.agent_sampling.get_relative_poses(num_frames: int, frames: numpy.ndarray, selected_track_id: Optional[int], agents: List[numpy.ndarray], agent_from_world: numpy.ndarray, current_agent_yaw: float) → Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray, numpy.ndarray]

Internal function that creates the targets and availability masks for deep prediction-type models. The futures/history offset (in meters) are computed. When no info is available (e.g. agent not in frame) a 0 is set in the availability array (1 otherwise).

Note: return dtype is float32, even if the provided args are float64. Still, the transformation between space is performed in float64 to ensure precision

Parameters

  • num_frames (int) – number of offset we want in the future/history

  • frames (np.ndarray) – available frames. This may be less than num_frames

  • selected_track_id (Optional[int]) – agent track_id or AV (None)

  • agents (List[np.ndarray]) – list of agents arrays (same len of frames)

  • agent_from_world (np.ndarray) – local from world matrix

  • current_agent_yaw (float) – angle of the agent at timestep 0

Returns

position offsets, angle offsets, extent, availabilities

Return type

Tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray]