receiving sensor data associated with an environment;

determining, based at least in part on a first subset of the sensor data, a first object at a first physical location in the environment, and a first object type associated with the first object;

determining, based at least in part on a second subset of the sensor data, a second object at a second physical location in the environment, wherein the second object is a dynamic object;

receiving map data associated with the environment;

determining a map feature type associated with the first object, wherein the map feature type is based at least in part on a traffic control directive associated with the first object, and wherein the map feature type represents an object type different from the first object type;

generating modified map data, including combining a new map feature of the map feature type with the map data, at a first map location associated with the first physical location;

determining a representation of the environment, based at least in part on the second subset of the sensor data and the modified map data;

providing the representation as input to a machine learning model;

determining a predicted trajectory of the second object, based at least in part on an output of the machine learning model; and

controlling an autonomous vehicle in the environment, based at least in part on the predicted trajectory of the second object.