prl.transformers package

Submodules

prl.transformers.action_transformers module

class ActionTransformer

Bases: prl.typing.ActionTransformerABC, abc.ABC

Interface for raw action (original actions from agent) transformers. Object of this class are used by the classes implementing EnvironmentABC interface. Action transformers can use all episode history from the beginning of the episode up to the moment of transformation.

action_space(original_space)

Returns: action_space object of class gym.Space, which defines type and shape of transformed action.

Note

If transformed action is from the same action_space as original state, then action_space is None. Information contained within action_space can be important for agents, so it is important to properly define an action_space.

Return type:Space

id

State transformer UUID

Return type:str

reset()

Action transformer can be stateful, so it have to be reset after each episode.

transform(action, history)

Transforms action into another representation, which must be of the form defined by action_space object. Input action can be in a form of numpy array, list, tuple, int, etc.

Parameters:

• action (ndarray) – Action from the agent
• history (HistoryABC) – History object of an episode

Return type:

ndarray

Returns:

Transformed action in form defined by the action_space object.

class NoOpActionTransformer

Bases: prl.transformers.action_transformers.ActionTransformer

ActionTransformer doing nothing

action_space(original_space)

Returns: action_space object of class gym.Space, which defines type and shape of transformed action.

Note

If transformed action is from the same action_space as original state, then action_space is None. Information contained within action_space can be important for agents, so it is important to properly define an action_space.

Return type:Space

id

State transformer UUID

reset()

Action transformer can be stateful, so it have to be reset after each episode.

transform(action, history)

Transforms action into another representation, which must be of the form defined by action_space object. Input action can be in a form of numpy array, list, tuple, int, etc.

Parameters:

• action (ndarray) – Action from the agent
• history (HistoryABC) – History object of an episode

Return type:

ndarray

Returns:

Transformed action in form defined by the action_space object.

prl.transformers.reward_transformers module

class NoOpRewardTransformer

Bases: prl.transformers.reward_transformers.RewardTransformer

RewardTransformer doing nothing

id()

Reward transformer UUID

reset()

Reward transformer can be stateful, so it have to be reset after each episode.

transform(reward, history)

Transforms a reward.

Parameters:

• reward (Real) – Raw reward from the wrapped environment
• history (HistoryABC) – History object

Return type:

Number

Returns:

Transformed reward

class RewardShiftTransformer(shift)

Bases: prl.transformers.reward_transformers.RewardTransformer

RewardTransformer shifting reward by some constant value

id()

Reward transformer UUID

reset()

Reward transformer can be stateful, so it have to be reset after each episode.

transform(reward, history)

Transforms a reward.

Parameters:

• reward (Real) – Raw reward from the wrapped environment
• history (HistoryABC) – History object

Return type:

Number

Returns:

Transformed reward

class RewardTransformer

Bases: prl.typing.RewardTransformerABC, abc.ABC

Interface for classes for shaping the raw reward from wrapped environments. Object inherited from this class are used by the Environment class objects. Reward transformers can use all episode history from the beginning of the episode up to the moment of transformation.

id

Reward transformer UUID

Return type:str

reset()

Reward transformer can be stateful, so it have to be reset after each episode.

transform(reward, history)

Transforms a reward.

Parameters:

• reward (Real) – Raw reward from the wrapped environment
• history (HistoryABC) – History object

Return type:

Real

Returns:

Transformed reward

prl.transformers.state_transformers module

class NoOpStateTransformer

Bases: prl.transformers.state_transformers.StateTransformer

StateTransformer doing nothing

id

State transformer UUID

reset()

State transformer can be stateful, so it have to be reset after each episode.

transform(state, history)

Transforms observed state into another representation, which must be of the form defined by observation_space object. Input state must be in a form of numpy.ndarray.

Parameters:

• state (ndarray) – State from wrapped environment
• history (HistoryABC) – History object

Return type:

ndarray

Returns:

Transformed state in form defined by the observation_space object.

class PongTransformer(resize_factor=2, crop=True, flatten=False)

Bases: prl.transformers.state_transformers.StateTransformer

StateTransformer for Pong atari game

id

State transformer UUID

reset()

State transformer can be stateful, so it have to be reset after each episode.

transform(observation, history)

Transforms observed state into another representation, which must be of the form defined by observation_space object. Input state must be in a form of numpy.ndarray.

Parameters:

• state – State from wrapped environment
• history (HistoryABC) – History object

Return type:

ndarray

Returns:

Transformed state in form defined by the observation_space object.

class StateShiftTransformer(shift_tensor)

Bases: prl.transformers.state_transformers.StateTransformer

StateTransformer shifting reward by some constant vector

id

State transformer UUID

reset()

State transformer can be stateful, so it have to be reset after each episode.

transform(state, history)

Transforms observed state into another representation, which must be of the form defined by observation_space object. Input state must be in a form of numpy.ndarray.

Parameters:

• state (ndarray) – State from wrapped environment
• history (HistoryABC) – History object

Return type:

ndarray

Returns:

Transformed state in form defined by the observation_space object.

class StateTransformer

Bases: prl.typing.StateTransformerABC, abc.ABC

Interface for raw states (original states from wrapped environments) transformers. Object of this class are used by the classes implementing EnvironmentABC interface. State transformers can use all episode history from the beginning of the episode up to the moment of transformation.

id

State transformer UUID

Return type:str

reset()

State transformer can be stateful, so it have to be reset after each episode.

transform(state, history)

Transforms observed state into another representation, which must be of the form defined by observation_space object. Input state must be in a form of numpy.ndarray.

Parameters:

• state (ndarray) – State from wrapped environment
• history (HistoryABC) – History object

Return type:

ndarray

Returns:

Transformed state in form defined by the observation_space object.

Module contents