General utils

Some general utility functions that are used throughout the package.

check_parameter_types

 check_parameter_types (*args, parameter_type=<class 'numpy.ndarray'>)

Checks if each argument in args is of the specified type, defaulting to np.ndarray.

Example usage for the check_parameter_types function.

a = np.array([1, 2, 3])
b = [1, 2, 3]

try:
    check_parameter_types(a, b)
except TypeError as e:
    print(e)

Argument 2 of 2 is of type list, expected ndarray

source

Parameter

 Parameter ()

Legacy, not used in the current implementation.

source

MDPInfo

 MDPInfo (observation_space:gymnasium.spaces.space.Space,
          action_space:gymnasium.spaces.space.Space, gamma:float,
          horizon:int, dt:float=0.1, backend:Literal['numpy']='numpy')

*This class is used to store the information of the environment. It is based on MushroomRL (https://github.com/MushroomRL). It can be accessed by agents that need the information of the environment, such as the state and action spaces.

Key difference with MushroomRL is that the state and action spaces are gymnasium spaces.*

	Type	Default	Details
observation_space	Space
action_space	Space
gamma	float
horizon	int
dt	float	0.1
backend	Literal	numpy	Currently only numpy is supported
Returns	None

source

MDPInfo.size

 MDPInfo.size ()

Returns: The sum of the number of discrete states and discrete actions. Only works for discrete spaces.

source

MDPInfo.shape

 MDPInfo.shape ()

Returns: The concatenation of the shape tuple of the state and action spaces.

source

DatasetWrapper

 DatasetWrapper (dataloader:ddopai.dataloaders.base.BaseDataLoader,
                 obsprocessors:List=None)

This class is used to wrap a Pytorch Dataset around the ddopai dataloader to enable the usage of the Pytorch Dataloader during training. This way, agents that are trained using Pytorch without interacting with the environment can directly train on the data generated by the dataloader.

	Type	Default	Details
dataloader	BaseDataLoader		Any dataloader that inherits from BaseDataLoader
obsprocessors	List	None	processors (to mimic the environment processors)

source

DatasetWrapper.getitem

 DatasetWrapper.__getitem__ (idx)

Get the item at the provided idx.

source

DatasetWrapper.len

 DatasetWrapper.__len__ ()

Returns the length of the dataset. Depends on the state of the dataloader (train, val, test).

source

DatasetWrapperMeta

 DatasetWrapperMeta (dataloader:ddopai.dataloaders.base.BaseDataLoader,
                     draw_parameter_function:<built-
                     infunctioncallable>=None, distribution:Union[Literal[
                     'fixed','uniform'],List]='fixed',
                     parameter_names:List[str]=None,
                     bounds_low:Union[int,float,List]=0,
                     bounds_high:Union[int,float,List]=1,
                     obsprocessors:List=None)

	Type	Default	Details
dataloader	BaseDataLoader		Any dataloader that inherits from BaseDataLoader
draw_parameter_function	callable	None	function to draw parameters from distribution
distribution	Union	fixed	distribution for params during training, can be List for multiple parameters
parameter_names	List	None	names of the parameters
bounds_low	Union	0	lower bound for params during training, can be List for multiple parameters
bounds_high	Union	1	upper bound for params during training, can be List for multiple parameters
obsprocessors	List	None	processors (to mimic the environment processors)

source

merge_dictionaries

 merge_dictionaries (dict1, dict2)

Merge two dictionaries. If a key is found in both dictionaries, raise a KeyError.

source

set_param

 set_param (obj, name:str, input:Union[__main__.Parameter,int,float,numpy.
            ndarray,List,Dict,NoneType], shape:tuple=(1,), new:bool=False)

Set a parameter for the class. It converts scalar values to numpy arrays and ensures that environment parameters are either of the Parameter class of Numpy arrays. If new is set to True, the function will create a new parameter or update an existing one otherwise. If new is set to False, the function will raise an error if the parameter does not exist.

	Type	Default	Details
obj
name	str		name of the parameter (will become the attribute name)
input	Union		input value of the parameter
shape	tuple	(1,)	shape of the parameter
new	bool	False	whether to create a new parameter or update an existing one