Class emulating the maze of digits environment. More...

#include <MazeOfDigits.hpp>

Inheritance diagram for mic::environments::MazeOfDigits:

Collaboration diagram for mic::environments::MazeOfDigits:

Public Member Functions
	MazeOfDigits (std::string node_name_="maze_of_digits")

	MazeOfDigits (const mic::environments::MazeOfDigits &md_)

virtual	~MazeOfDigits ()

mic::environments::MazeOfDigits &	operator= (const mic::environments::MazeOfDigits &md)

virtual size_t	getObservationSize ()

virtual void	initializePropertyDependentVariables ()

virtual void	initializeEnvironment ()

void	initExemplaryMaze ()

void	reRandomAgentPosition ()

void	initFullyRandomMaze ()

void	initRandomStructuredMaze ()

void	initRandomPathMaze ()

void	setBiggerDigit (size_t x_, size_t y_, size_t value_)

mic::types::TensorXfPtr	getObservation ()

virtual std::string	environmentToString ()

virtual std::string	observationToString ()

virtual mic::types::MatrixXfPtr	encodeEnvironment ()

virtual mic::types::MatrixXfPtr	encodeObservation ()

virtual mic::types::MatrixXfPtr	encodeAgentGrid ()
	Encode the current state of the reduced grid (only the agent position) as a matrix of size [1, width * height]. More...

virtual mic::types::Position2D	getAgentPosition ()

virtual bool	moveAgentToPosition (mic::types::Position2D pos_)

virtual float	getStateReward (mic::types::Position2D pos_)

virtual bool	isStateAllowed (mic::types::Position2D pos_)

virtual bool	isStateTerminal (mic::types::Position2D pos_)

unsigned int	optimalPathLength ()

Public Member Functions inherited from mic::environments::Environment
	Environment (std::string node_name_)

virtual	~Environment ()

mic::types::TensorXfPtr &	getEnvironment ()

virtual size_t	getEnvironmentWidth ()

virtual size_t	getEnvironmentHeight ()

virtual size_t	getEnvironmentSize ()

virtual size_t	getObservationWidth ()

virtual size_t	getObservationHeight ()

virtual size_t	getChannels ()

size_t	getROISize ()

bool	moveAgent (mic::types::Action2DInterface ac_)

virtual void	moveAgentToInitialPosition ()

virtual bool	isStateAllowed (long x_, long y_)

virtual bool	isStateTerminal (long x_, long y_)

virtual bool	isActionAllowed (long x_, long y_, size_t action_)

virtual bool	isActionAllowed (mic::types::Position2D pos_, mic::types::Action2DInterface ac_)

virtual bool	isActionAllowed (mic::types::Action2DInterface ac_)

Protected Member Functions
std::string	gridToString (mic::types::TensorXfPtr &grid_)

Protected Attributes
mic::configuration::Property < short >	type

unsigned int	optimal_path_length

Protected Attributes inherited from mic::environments::Environment
mic::configuration::Property < size_t >	width
	Property: width of the environment. More...

mic::configuration::Property < size_t >	height
	Property: height of the environment. More...

mic::configuration::Property < size_t >	roi_size
	Property: size of the ROI (region of interest). More...

size_t	channels
	Number of channels. More...

bool	pomdp_flag
	Flag related to. More...

mic::types::Position2D	initial_position
	Property: initial position of the agent. More...

mic::types::TensorXfPtr	environment_grid
	Tensor storing the environment. More...

mic::types::TensorXfPtr	observation_grid

Detailed Description

Class emulating the maze of digits environment.

Author: tkornuta

Definition at line 50 of file MazeOfDigits.hpp.

Constructor & Destructor Documentation

mic::environments::MazeOfDigits::MazeOfDigits ( std::string node_name_ = "maze_of_digits" )

Constructor. Registers properties.

Parameters

node_name_ Name of the node in configuration file.

Definition at line 29 of file MazeOfDigits.cpp.

References mic::environments::Environment::channels, mic::environments::Count, and type.

mic::environments::MazeOfDigits::MazeOfDigits ( const mic::environments::MazeOfDigits & md_ )

Copying constructor.

Parameters

md_	Maze of digits to be cloned.

Definition at line 38 of file MazeOfDigits.cpp.

References mic::environments::Environment::channels, mic::environments::Environment::environment_grid, mic::environments::Environment::height, mic::environments::Environment::initial_position, mic::environments::Environment::observation_grid, type, and mic::environments::Environment::width.

mic::environments::MazeOfDigits::~MazeOfDigits ( )

virtual

Destructor. Empty for now.

Definition at line 53 of file MazeOfDigits.cpp.

Member Function Documentation

mic::types::MatrixXfPtr mic::environments::MazeOfDigits::encodeAgentGrid ( )

virtual

Encode the current state of the reduced grid (only the agent position) as a matrix of size [1, width * height].

Definition at line 651 of file MazeOfDigits.cpp.

References mic::environments::Agent, mic::environments::Environment::environment_grid, mic::environments::Environment::height, and mic::environments::Environment::width.

mic::types::MatrixXfPtr mic::environments::MazeOfDigits::encodeEnvironment ( )

virtual

Encodes the current state of the environment in as a matrix of size [1, width * height * channels].

Returns: Matrix of size [1, width * height * channels].

Implements mic::environments::Environment.

Definition at line 585 of file MazeOfDigits.cpp.

References mic::environments::Environment::channels, mic::environments::Environment::environment_grid, mic::environments::Environment::height, and mic::environments::Environment::width.

Referenced by encodeObservation().

mic::types::MatrixXfPtr mic::environments::MazeOfDigits::encodeObservation ( )

virtual

Encodes the current observation taken in the environment in as a matrix of size [1, roi_size * roi_size * channels].

Returns: Matrix of size [1, roi_size * roi_size * channels].

Implements mic::environments::Environment.

Definition at line 597 of file MazeOfDigits.cpp.

References encodeEnvironment(), getAgentPosition(), getObservation(), mic::environments::Environment::pomdp_flag, and mic::environments::Environment::roi_size.

Referenced by mic::application::MazeOfDigitsDLRERPOMPD::getPredictedRewardsForGivenState(), mic::application::MazeOfDigitsDLRERPOMPD::performSingleStep(), and mic::application::MazeOfDigitsDLRERPOMPD::streamNetworkResponseTable().

std::string mic::environments::MazeOfDigits::environmentToString ( )

virtual

Returns the current state of the environment in the form of a string.

Returns: String with description of the environment.

Implements mic::environments::Environment.

Definition at line 571 of file MazeOfDigits.cpp.

References mic::environments::Environment::environment_grid, and gridToString().

Referenced by mic::application::MazeOfDigitsDLRERPOMPD::performSingleStep().

mic::types::Position2D mic::environments::MazeOfDigits::getAgentPosition ( )

virtual

Calculates the agent position.

Returns: Agent position.

Implements mic::environments::Environment.

Definition at line 673 of file MazeOfDigits.cpp.

References mic::environments::Agent, mic::environments::Environment::environment_grid, mic::environments::Environment::height, and mic::environments::Environment::width.

Referenced by encodeObservation(), getObservation(), mic::application::MazeOfDigitsDLRERPOMPD::getPredictedRewardsForGivenState(), moveAgentToPosition(), mic::application::MazeOfDigitsDLRERPOMPD::performSingleStep(), mic::application::MazeOfDigitsDLRERPOMPD::startNewEpisode(), and mic::application::MazeOfDigitsDLRERPOMPD::streamNetworkResponseTable().

mic::types::TensorXfPtr mic::environments::MazeOfDigits::getObservation ( )

Returns the tensor being the observation.

Returns: Observation tensor of size [roi_size, roi_size, channels].

Definition at line 620 of file MazeOfDigits.cpp.

References getAgentPosition(), mic::environments::Environment::observation_grid, and mic::environments::Environment::roi_size.

Referenced by encodeObservation(), mic::application::MazeOfDigitsDLRERPOMPD::initializePropertyDependentVariables(), observationToString(), mic::application::MazeOfDigitsDLRERPOMPD::performSingleStep(), and mic::application::MazeOfDigitsDLRERPOMPD::startNewEpisode().

virtual size_t mic::environments::MazeOfDigits::getObservationSize ( )

inlinevirtual

Returns the observation size, depending on the process type: FOMDP (width * height * channels) or POMDP (roi_size * roi_size * 1!) (an overridden method)

Returns: Size of the observation.

Reimplemented from mic::environments::Environment.

Definition at line 78 of file MazeOfDigits.hpp.

References mic::environments::Environment::channels, mic::environments::Environment::height, mic::environments::Environment::pomdp_flag, mic::environments::Environment::roi_size, and mic::environments::Environment::width.

Referenced by mic::application::MazeOfDigitsDLRERPOMPD::getPredictedRewardsForGivenState(), mic::application::MazeOfDigitsDLRERPOMPD::initializePropertyDependentVariables(), mic::application::MazeOfDigitsDLRERPOMPD::performSingleStep(), and mic::application::MazeOfDigitsDLRERPOMPD::streamNetworkResponseTable().

float mic::environments::MazeOfDigits::getStateReward ( mic::types::Position2D pos_ )

virtual

Returns the reward associated with the given state.

Parameters

pos_	Position (state).

Returns: Reward for being in given state (r).

Implements mic::environments::Environment.

Definition at line 705 of file MazeOfDigits.cpp.

References mic::environments::Environment::environment_grid, and mic::environments::Goals.

Referenced by mic::application::MazeOfDigitsDLRERPOMPD::performSingleStep().

std::string mic::environments::MazeOfDigits::gridToString ( mic::types::TensorXfPtr & grid_ )

protected

Returns the current state of the grid passed as an argument in the form of a string.

Parameters

grid_ Grid to be processed.

Returns: String with description of the grid.

Definition at line 534 of file MazeOfDigits.cpp.

References mic::environments::Agent, and mic::environments::Walls.

Referenced by environmentToString(), and observationToString().

void mic::environments::MazeOfDigits::initExemplaryMaze ( )

Method initializes the exemplary maze.

[['2','4','7','7'], ['1','5','7','9'], ['2','3','6','8'], ['A','2','5','6']]

Definition at line 95 of file MazeOfDigits.cpp.

References mic::environments::Environment::channels, mic::environments::Digits, mic::environments::Environment::environment_grid, mic::environments::Goals, mic::environments::Environment::height, mic::environments::Environment::initial_position, moveAgentToPosition(), optimal_path_length, and mic::environments::Environment::width.

Referenced by initializeEnvironment().

void mic::environments::MazeOfDigits::initFullyRandomMaze ( )

Generates a fully random maze of size (width x height), with spatially independent values of digits.

Definition at line 178 of file MazeOfDigits.cpp.

References mic::environments::Agent, mic::environments::Environment::channels, mic::environments::Digits, mic::environments::Environment::environment_grid, mic::environments::Goals, mic::environments::Environment::height, mic::environments::Environment::initial_position, moveAgentToPosition(), optimal_path_length, reRandomAgentPosition(), type, and mic::environments::Environment::width.

Referenced by initializeEnvironment().

void mic::environments::MazeOfDigits::initializeEnvironment ( )

virtual

(Re)initializes the environment - generates the maze of a required (defined by property) type, sets agent, goal etc.

Implements mic::environments::Environment.

Definition at line 73 of file MazeOfDigits.cpp.

References mic::environments::Environment::height, initExemplaryMaze(), initFullyRandomMaze(), initRandomPathMaze(), initRandomStructuredMaze(), mic::environments::Environment::observation_grid, mic::environments::Environment::pomdp_flag, mic::environments::Environment::roi_size, type, and mic::environments::Environment::width.

Referenced by mic::application::MazeOfDigitsDLRERPOMPD::initializePropertyDependentVariables(), and mic::application::MazeOfDigitsDLRERPOMPD::startNewEpisode().

void mic::environments::MazeOfDigits::initializePropertyDependentVariables ( )

virtual

Initializes all variables that are property-dependent.

Definition at line 69 of file MazeOfDigits.cpp.

void mic::environments::MazeOfDigits::initRandomPathMaze ( )

Generates a random maze of size (width x height), with spatially dependent values of digits, creating a path leading to the goal (9).

Definition at line 324 of file MazeOfDigits.cpp.

References mic::environments::Agent, mic::environments::Environment::channels, mic::environments::Digits, mic::environments::Environment::environment_grid, mic::environments::Goals, mic::environments::Environment::height, mic::environments::Environment::initial_position, moveAgentToPosition(), optimal_path_length, reRandomAgentPosition(), setBiggerDigit(), type, and mic::environments::Environment::width.

Referenced by initializeEnvironment().

void mic::environments::MazeOfDigits::initRandomStructuredMaze ( )

Generates a random maze of size (width x height), with spatially dependent values of digits, creating a heat map around the goal (9).

Definition at line 245 of file MazeOfDigits.cpp.

References mic::environments::Agent, mic::environments::Environment::channels, mic::environments::Digits, mic::environments::Environment::environment_grid, mic::environments::Goals, mic::environments::Environment::height, mic::environments::Environment::initial_position, moveAgentToPosition(), optimal_path_length, reRandomAgentPosition(), type, and mic::environments::Environment::width.

Referenced by initializeEnvironment().

bool mic::environments::MazeOfDigits::isStateAllowed ( mic::types::Position2D pos_ )

virtual

Checks if position is allowed, i.e. within the gridworld boundaries and there is no wall at that place.

Parameters

pos_	Position to be checked.

Returns: True if the position is allowed, false otherwise.

Implements mic::environments::Environment.

Definition at line 714 of file MazeOfDigits.cpp.

References mic::environments::Environment::environment_grid, and mic::environments::Walls.

Referenced by moveAgentToPosition(), and mic::application::MazeOfDigitsDLRERPOMPD::streamNetworkResponseTable().

bool mic::environments::MazeOfDigits::isStateTerminal ( mic::types::Position2D pos_ )

virtual

Checks if position is terminal, i.e. agent is standing in a pit or reached the goal. Returns reward associated with given state.

Parameters

pos_	Position (state) to be checked.

Returns: The reward associated with "final" action (might be positive or negative), equal to zero means that the position is not final.

Implements mic::environments::Environment.

Definition at line 729 of file MazeOfDigits.cpp.

References mic::environments::Environment::environment_grid, and mic::environments::Goals.

Referenced by mic::application::MazeOfDigitsDLRERPOMPD::performSingleStep(), and mic::application::MazeOfDigitsDLRERPOMPD::streamNetworkResponseTable().

bool mic::environments::MazeOfDigits::moveAgentToPosition ( mic::types::Position2D pos_ )

virtual

Moves the agent to the position.Type of move (deterministic vs stochastic) depends on the environment (the same goes to e.g. circular world assumption).

Parameters

pos_	Desired position of the agent.

Returns: True if position is valid and was reached, false otherwise.

Implements mic::environments::Environment.

Definition at line 688 of file MazeOfDigits.cpp.

References mic::environments::Agent, getAgentPosition(), and isStateAllowed().

Referenced by mic::application::MazeOfDigitsDLRERPOMPD::getPredictedRewardsForGivenState(), initExemplaryMaze(), initFullyRandomMaze(), initRandomPathMaze(), initRandomStructuredMaze(), mic::application::MazeOfDigitsDLRERPOMPD::performSingleStep(), reRandomAgentPosition(), and mic::application::MazeOfDigitsDLRERPOMPD::streamNetworkResponseTable().

std::string mic::environments::MazeOfDigits::observationToString ( )

virtual

Returns the current observation taken in the environment in the form of a string.

Returns: String with description of the observation.

Implements mic::environments::Environment.

Definition at line 575 of file MazeOfDigits.cpp.

References mic::environments::Environment::environment_grid, getObservation(), gridToString(), and mic::environments::Environment::pomdp_flag.

Referenced by mic::application::MazeOfDigitsDLRERPOMPD::performSingleStep().

mic::environments::MazeOfDigits & mic::environments::MazeOfDigits::operator= ( const mic::environments::MazeOfDigits & md )

Assign operator. Copies the gridworld state along with its properties.

Definition at line 57 of file MazeOfDigits.cpp.

References mic::environments::Environment::channels, mic::environments::Environment::environment_grid, mic::environments::Environment::height, mic::environments::Environment::initial_position, mic::environments::Environment::observation_grid, and mic::environments::Environment::width.

unsigned int mic::environments::MazeOfDigits::optimalPathLength ( )

inline

Returns the length of optimal path from agent initial position to goal.

Returns

Definition at line 205 of file MazeOfDigits.hpp.

References optimal_path_length.

Referenced by mic::application::MazeOfDigitsDLRERPOMPD::finishCurrentEpisode().

void mic::environments::MazeOfDigits::reRandomAgentPosition ( )

Generates only the agent new position, leaving the maze unchanged. Recalculates optimal path length to (unchanged) goal.

Definition at line 144 of file MazeOfDigits.cpp.

References mic::environments::Environment::environment_grid, mic::environments::Goals, mic::environments::Environment::height, mic::environments::Environment::initial_position, moveAgentToPosition(), optimal_path_length, and mic::environments::Environment::width.

Referenced by initFullyRandomMaze(), initRandomPathMaze(), and initRandomStructuredMaze().

void mic::environments::MazeOfDigits::setBiggerDigit	(	size_t	x_,
		size_t	y_,
		size_t	value_
	)

Sets the digit.

Parameters

point_
value_

Definition at line 525 of file MazeOfDigits.cpp.

References mic::environments::Digits, and mic::environments::Environment::environment_grid.

Referenced by initRandomPathMaze().

Member Data Documentation

unsigned int mic::environments::MazeOfDigits::optimal_path_length

protected

Optimal number of steps from initial agent position to goal.

Definition at line 231 of file MazeOfDigits.hpp.

Referenced by initExemplaryMaze(), initFullyRandomMaze(), initRandomPathMaze(), initRandomStructuredMaze(), optimalPathLength(), and reRandomAgentPosition().

mic::configuration::Property<short> mic::environments::MazeOfDigits::type

protected

Property: type of the generated gridworld. Currently available types: 0: the exemplary maze 4x4. -1 (or else): random maze - random maze generated, but generated only once, random initial agent position in each episode -2 (or else): random maze - all randomly generated each time

Definition at line 219 of file MazeOfDigits.hpp.

Referenced by initFullyRandomMaze(), initializeEnvironment(), initRandomPathMaze(), initRandomStructuredMaze(), and MazeOfDigits().

The documentation for this class was generated from the following files:

src/types/MazeOfDigits.hpp
src/types/MazeOfDigits.cpp

Public Member Functions

Protected Member Functions

Protected Attributes

Detailed Description

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation