l5kit.environment.callbacks module

class l5kit.environment.callbacks.L5KitEvalCallback(eval_env: gym.core.Env, eval_freq: int = 10000, n_eval_episodes: int = 10, n_eval_envs: int = 4, metric_set: Optional[l5kit.cle.metric_set.L5MetricSet] = None, enable_scene_type_aggregation: Optional[bool] = False, scene_id_to_type_path: Optional[str] = None, prefix: str = 'eval', verbose: int = 0)

Bases: stable_baselines3.common.callbacks.EvalCallback

Callback for evaluating an agent using L5Kit evaluation metrics.

Parameters

  • eval_env – The environment used for initialization

  • n_eval_episodes – The number of episodes to test the agent

  • eval_freq – Evaluate the agent every eval_freq call of the callback.

  • metric_set – computes a set of metric parametrization for the L5Kit environment

  • enable_scene_type_aggregation – enable evaluation according to scene type

  • scene_id_to_type_path – path to the csv file mapping scene id to scene type

  • prefix – the prefix to save the computed metrics

  • verbose –

static compute_ade_fde(metric_set: l5kit.cle.metric_set.L5MetricSet) → Tuple[float, float]

Aggregate the Average displacement error (ADE) and Final displacement error (FDE) of the simulation outputs.

Returns

Tuple [ADE, FDE]

evaluate_scenes() → None

Evaluate the episode outputs for n_eval_episodes episodes.

static get_scene_types(scene_id_to_type_path: str) → List[List[str]]

Construct a list mapping scene types to their corresponding types.

Parameters

scene_id_to_type_path – path to the mapping.

Returns

list of scene type tags per scene