Critic Networks

Specific models that can serve as critic networks for RL agents

RNNWrapper

 RNNWrapper (rnn_cell_class, *args, **kwargs)

*Base class for all neural network modules.

Your models should also subclass this class.

Modules can also contain other Modules, allowing to nest them in a tree structure. You can assign the submodules as regular attributes::

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
    def __init__(self) -> None:
        super().__init__()
        self.conv1 = nn.Conv2d(1, 20, 5)
        self.conv2 = nn.Conv2d(20, 20, 5)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        return F.relu(self.conv2(x))

Submodules assigned in this way will be registered, and will have their parameters converted too when you call :meth:to, etc.

.. note:: As per the example above, an __init__() call to the parent class must be made before assignment on the child.

:ivar training: Boolean represents whether this module is in training or evaluation mode. :vartype training: bool*

BaseApproximator

 BaseApproximator ()

Some basic functions for approximators

BaseApproximatorMLP

 BaseApproximatorMLP ()

Some basic functions for approximators

RNNMLPHybrid

 RNNMLPHybrid (RNN_input_size:int, MLP_input_size:int|None,
               output_size:int, num_hidden_units_RNN:int,
               hidden_layers_RNN:int, hidden_layers_MLP:List[int],
               hidden_layers_input_MLP:Optional[List[int]],
               RNN_cell:torch.nn.modules.module.Module,
               activation:torch.nn.modules.module.Module,
               final_activation:torch.nn.modules.module.Module,
               drop_prob:float, batch_norm:bool, init_method:str)

A hybrid model combining an RNN and an MLP

BaseApproximatorRNN

 BaseApproximatorRNN ()

Some basic functions for approximators

MLPStateAction

 MLPStateAction (input_shape:Union[Tuple,List[Tuple]], output_shape:Tuple,
                 hidden_layers:list, activation:str='relu',
                 drop_prob:float=0.0, batch_norm:bool=False,
                 final_activation:str='identity',
                 init_method:str='xavier_uniform', use_cuda:bool=False,
                 dropout:bool=False)

Multilayer perceptron model for critic networks that take both states and actions as inputs to output the q-value

	Type	Default	Details
input_shape	Union		number of features
output_shape	Tuple		number of outputs/actions
hidden_layers	list		list of number of neurons in each hidden layer
activation	str	relu
drop_prob	float	0.0	dropout probability
batch_norm	bool	False	whether to apply batch normalization
final_activation	str	identity	whether to apply ReLU activation to the output
init_method	str	xavier_uniform	Parameter for initialization
use_cuda	bool	False	handled by mushroomRL, not used here
dropout	bool	False	legacy parameter to ensure compatibility, use drop_prob instead

MLPState

 MLPState (input_shape:Tuple, output_shape:Tuple, hidden_layers:list,
           activation:str='relu', drop_prob:float=0.0,
           batch_norm:bool=False, final_activation:str='identity',
           init_method:str='xavier_uniform', use_cuda:bool=False,
           dropout:bool=False)

Multilayer perceptron model for critic networks that take both states and actions as inputs to output the q-value

	Type	Default	Details
input_shape	Tuple		number of features
output_shape	Tuple		number of outputs/actions
hidden_layers	list		list of number of neurons in each hidden layer
activation	str	relu
drop_prob	float	0.0	dropout probability
batch_norm	bool	False	whether to apply batch normalization
final_activation	str	identity	whether to apply ReLU activation to the output
init_method	str	xavier_uniform	Parameter for initialization
use_cuda	bool	False	handled by mushroomRL, not used here
dropout	bool	False	legacy parameter to ensure compatibility, use drop_prob instead

MLPActor

 MLPActor (input_shape:Tuple, output_shape:Tuple, hidden_layers:list,
           activation:str='relu', drop_prob:float=0.0,
           batch_norm:bool=False, final_activation:str='identity',
           init_method:str='xavier_uniform', use_cuda:bool=False,
           dropout:bool=False, **kwargs)

Multilayer perceptron model for critic networks that take both states and actions as inputs to output the q-value

	Type	Default	Details
input_shape	Tuple		number of features
output_shape	Tuple		number of outputs/actions
hidden_layers	list		list of number of neurons in each hidden layer
activation	str	relu
drop_prob	float	0.0	dropout probability
batch_norm	bool	False	whether to apply batch normalization
final_activation	str	identity	whether to apply ReLU activation to the output
init_method	str	xavier_uniform	Parameter for initialization
use_cuda	bool	False
dropout	bool	False	legacy parameter to ensure compatibility, use drop_prob instead
kwargs

RNNActor

 RNNActor (input_shape:List[Tuple], output_shape:Tuple,
           hidden_layers_RNN:int, num_hidden_units_RNN:int,
           hidden_layers_MLP:List,
           hidden_layers_input_MLP:Optional[List]=None,
           RNN_cell:str='GRU', activation:str='relu', drop_prob:float=0.0,
           batch_norm:bool=False, final_activation:str='identity',
           init_method:str='xavier_uniform', use_cuda:bool=False,
           dropout:bool=False, input_shape_:List[Tuple]=None, **kwargs)

Multilayer perceptron model for critic networks that take both states and actions as inputs to output the q-value

	Type	Default	Details
input_shape	List		input shape, must be exaclty as input shape into agent for mushroom_rl to work
output_shape	Tuple		number of outputs/actions
hidden_layers_RNN	int		number of initial hidden RNN layers
num_hidden_units_RNN	int		number of neurons in the RNN layers
hidden_layers_MLP	List		list of number of neurons in each hidden MLP layer, following the RNN layers
hidden_layers_input_MLP	Optional	None	If a separate MLP is used for (potential) MLP input
RNN_cell	str	GRU	RNN cell type
activation	str	relu
drop_prob	float	0.0	dropout probability
batch_norm	bool	False	whether to apply batch normalization
final_activation	str	identity	whether to apply ReLU activation to the output
init_method	str	xavier_uniform	Parameter for initialization
use_cuda	bool	False
dropout	bool	False	legacy parameter to ensure compatibility, use drop_prob instead
input_shape_	List	None	input shape for composite spaces
kwargs

RNNStateAction

 RNNStateAction (input_shape:List[Tuple], output_shape:Tuple,
                 hidden_layers_RNN:int, num_hidden_units_RNN:int,
                 hidden_layers_MLP:List,
                 hidden_layers_input_MLP:Optional[List]=None,
                 RNN_cell:str='GRU', activation:str='relu',
                 drop_prob:float=0.0, batch_norm:bool=False,
                 final_activation:str='identity',
                 init_method:str='xavier_uniform', use_cuda:bool=False,
                 dropout:bool=False, input_shape_:List[Tuple]=None,
                 **kwargs)

Multilayer perceptron model for critic networks that take both states and actions as inputs to output the q-value

	Type	Default	Details
input_shape	List		input shape, must be exaclty as input shape into agent for mushroom_rl to work
output_shape	Tuple		Output shape
hidden_layers_RNN	int		number of initial hidden RNN layers
num_hidden_units_RNN	int		number of neurons in the RNN layers
hidden_layers_MLP	List		list of number of neurons in each hidden MLP layer, following the RNN layers
hidden_layers_input_MLP	Optional	None	structure of MLP to speratly process non-RNN input
RNN_cell	str	GRU	RNN cell type
activation	str	relu
drop_prob	float	0.0	dropout probability
batch_norm	bool	False	whether to apply batch normalization
final_activation	str	identity	whether to apply ReLU activation to the output
init_method	str	xavier_uniform	Parameter for initialization
use_cuda	bool	False
dropout	bool	False	legacy parameter to ensure compatibility, use drop_prob instead
input_shape_	List	None	input shape for composite spaces
kwargs

import mushroom_rl
mushroom_rl.__file__

'/opt/hostedtoolcache/Python/3.10.15/x64/lib/python3.10/site-packages/mushroom_rl/__init__.py'