Skip to content
Snippets Groups Projects
Commit eb42255e authored by Alexander Nasuta's avatar Alexander Nasuta
Browse files

documentation

parent c31999f5
No related branches found
No related tags found
No related merge requests found
...@@ -2,9 +2,9 @@ ...@@ -2,9 +2,9 @@
A Monte Carlo Tree Search Implementation for Gymnasium-style Environments. A Monte Carlo Tree Search Implementation for Gymnasium-style Environments.
- Github: [GYMCTS on Github](https://github.com/Alexander-Nasuta/GraphMatrixJobShopEnv) - Github: [GYMCTS on Github](https://github.com/Alexander-Nasuta/gymcts)
- Pypi: [GYMCTS on PyPi](https://pypi.org/project/graph-matrix-jsp-env/) - Pypi: [GYMCTS on PyPi](https://pypi.org/project/gymcts/)
- Documentation: [GYMCTS Docs](https://graphmatrixjobshopenv.readthedocs.io/en/latest/) - Documentation: [GYMCTS Docs](https://gymcts.readthedocs.io/en/latest/)
## Description ## Description
...@@ -48,10 +48,10 @@ env = NormalizeReward(env, gamma=0.99, epsilon=1e-8) ...@@ -48,10 +48,10 @@ env = NormalizeReward(env, gamma=0.99, epsilon=1e-8)
```python ```python
env = TransformReward(env, lambda r: r / n_steps_per_episode) env = TransformReward(env, lambda r: r / n_steps_per_episode)
``` ```
### FrozenLake Example (NaiveSoloMCTSGymEnvWrapper) ### FrozenLake Example (DeepCopyMCTSGymEnvWrapper)
A minimal example of how to use the package with the FrozenLake environment and the NaiveSoloMCTSGymEnvWrapper is provided in the following code snippet below. A minimal example of how to use the package with the FrozenLake environment and the NaiveSoloMCTSGymEnvWrapper is provided in the following code snippet below.
The NaiveSoloMCTSGymEnvWrapper can be used with non-deterministic environments, such as the FrozenLake environment with slippery ice. The DeepCopyMCTSGymEnvWrapper can be used with non-deterministic environments, such as the FrozenLake environment with slippery ice.
```python ```python
import gymnasium as gym import gymnasium as gym
...@@ -70,7 +70,7 @@ if __name__ == '__main__': ...@@ -70,7 +70,7 @@ if __name__ == '__main__':
env = gym.make('FrozenLake-v1', desc=None, map_name="4x4", is_slippery=True, render_mode="ansi") env = gym.make('FrozenLake-v1', desc=None, map_name="4x4", is_slippery=True, render_mode="ansi")
env.reset() env.reset()
# 1. wrap the environment with the naive wrapper or a custom gymcts wrapper # 1. wrap the environment with the deep copy wrapper or a custom gymcts wrapper
env = DeepCopyMCTSGymEnvWrapper(env) env = DeepCopyMCTSGymEnvWrapper(env)
# 2. create the agent # 2. create the agent
...@@ -93,7 +93,7 @@ if __name__ == '__main__': ...@@ -93,7 +93,7 @@ if __name__ == '__main__':
# 5. print the solution # 5. print the solution
# read the solution from the info provided by the RecordEpisodeStatistics wrapper # read the solution from the info provided by the RecordEpisodeStatistics wrapper
# (that NaiveSoloMCTSGymEnvWrapper uses internally) # (that DeepCopyMCTSGymEnvWrapper uses internally)
episode_length = info["episode"]["l"] episode_length = info["episode"]["l"]
episode_return = info["episode"]["r"] episode_return = info["episode"]["r"]
...@@ -186,7 +186,7 @@ if __name__ == '__main__': ...@@ -186,7 +186,7 @@ if __name__ == '__main__':
env = gym.make('FrozenLake-v1', desc=None, map_name="4x4", is_slippery=False, render_mode="rgb_array") env = gym.make('FrozenLake-v1', desc=None, map_name="4x4", is_slippery=False, render_mode="rgb_array")
env.reset() env.reset()
# 1. wrap the environment with the naive wrapper or a custom gymcts wrapper # 1. wrap the environment with the deep copy wrapper or a custom gymcts wrapper
env = DeepCopyMCTSGymEnvWrapper(env) env = DeepCopyMCTSGymEnvWrapper(env)
# 2. create the agent # 2. create the agent
...@@ -215,7 +215,7 @@ if __name__ == '__main__': ...@@ -215,7 +215,7 @@ if __name__ == '__main__':
env.close() env.close()
# 5. print the solution # 5. print the solution
# read the solution from the info provided by the RecordEpisodeStatistics wrapper (that NaiveSoloMCTSGymEnvWrapper wraps internally) # read the solution from the info provided by the RecordEpisodeStatistics wrapper (that DeepCopyMCTSGymEnvWrapper wraps internally)
episode_length = info["episode"]["l"] episode_length = info["episode"]["l"]
episode_return = info["episode"]["r"] episode_return = info["episode"]["r"]
...@@ -368,7 +368,7 @@ if __name__ == '__main__': ...@@ -368,7 +368,7 @@ if __name__ == '__main__':
env = gym.make('FrozenLake-v1', desc=None, map_name="4x4", is_slippery=False, render_mode="ansi") env = gym.make('FrozenLake-v1', desc=None, map_name="4x4", is_slippery=False, render_mode="ansi")
env.reset() env.reset()
# wrap the environment with the naive wrapper or a custom gymcts wrapper # wrap the environment with the wrapper or a custom gymcts wrapper
env = ActionHistoryMCTSGymEnvWrapper(env) env = ActionHistoryMCTSGymEnvWrapper(env)
# create the agent # create the agent
......
...@@ -9,7 +9,7 @@ ...@@ -9,7 +9,7 @@
project = 'gymcts' project = 'gymcts'
copyright = '2025, Alexander Nasuta' copyright = '2025, Alexander Nasuta'
author = 'Alexander Nasuta' author = 'Alexander Nasuta'
release = '1.0.0' release = '1.2.0'
# -- General configuration --------------------------------------------------- # -- General configuration ---------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration # https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
......
# Contributing # Contributing
If you want to check out the code and implement new features or fix bugs, you can set up the project as follows: If you want to check out the code and implement new features or fix bugs, you can set up the project as follows:
### Clone the Repository ### Clone the Repository
...@@ -8,11 +9,11 @@ clone the repository in your favorite code editor (for example PyCharm, VSCode, ...@@ -8,11 +9,11 @@ clone the repository in your favorite code editor (for example PyCharm, VSCode,
using https: using https:
```shell ```shell
git clone https://github.com/Alexander-Nasuta/todo git clone https://github.com/Alexander-Nasuta/gymcts.git
``` ```
or by using the GitHub CLI: or by using the GitHub CLI:
```shell ```shell
gh repo clone Alexander-Nasuta/todo gh repo clone Alexander-Nasuta/gymcts
``` ```
if you are using PyCharm, I recommend doing the following additional steps: if you are using PyCharm, I recommend doing the following additional steps:
...@@ -21,9 +22,6 @@ if you are using PyCharm, I recommend doing the following additional steps: ...@@ -21,9 +22,6 @@ if you are using PyCharm, I recommend doing the following additional steps:
- mark the `tests` folder as test root (by right-clicking on the folder and selecting `Mark Directory as` -> `Test Sources Root`) - mark the `tests` folder as test root (by right-clicking on the folder and selecting `Mark Directory as` -> `Test Sources Root`)
- mark the `resources` folder as resources root (by right-clicking on the folder and selecting `Mark Directory as` -> `Resources Root`) - mark the `resources` folder as resources root (by right-clicking on the folder and selecting `Mark Directory as` -> `Resources Root`)
at the end your project structure should look like this:
todo
### Create a Virtual Environment (optional) ### Create a Virtual Environment (optional)
...@@ -89,12 +87,6 @@ For testing with `tox` run the following command: ...@@ -89,12 +87,6 @@ For testing with `tox` run the following command:
tox tox
``` ```
Here is a screenshot of what the output might look like:
![](https://github.com/Alexander-Nasuta/GraphMatrixJobShopEnv/raw/master/resources/tox-screenshot.png)
Tox will run the tests in a separate environment and will also check if the requirements are installed correctly.
### Builing and Publishing the Project to PyPi ### Builing and Publishing the Project to PyPi
In order to publish the project to PyPi, the project needs to be built and then uploaded to PyPi. In order to publish the project to PyPi, the project needs to be built and then uploaded to PyPi.
...@@ -131,4 +123,3 @@ sphinx-autobuild ./docs/source/ ./docs/build/html/ ...@@ -131,4 +123,3 @@ sphinx-autobuild ./docs/source/ ./docs/build/html/
``` ```
This project features most of the extensions featured in this Tutorial: [Document Your Scientific Project With Markdown, Sphinx, and Read the Docs | PyData Global 2021](https://www.youtube.com/watch?v=qRSb299awB0). This project features most of the extensions featured in this Tutorial: [Document Your Scientific Project With Markdown, Sphinx, and Read the Docs | PyData Global 2021](https://www.youtube.com/watch?v=qRSb299awB0).
...@@ -83,7 +83,9 @@ agent.show_mcts_tree( ...@@ -83,7 +83,9 @@ agent.show_mcts_tree(
start_node=agent.search_root_node, start_node=agent.search_root_node,
) )
``` ```
![Visualisation Example](../_static/vis1.png)
![](https://github.com/Alexander-Nasuta/gymcts/raw/master/resources/vis1.png)
```python ```python
log.info(f"MCTS Tree starting at the pre-final state of the environment (actions: {agent.search_root_node.parent.state})") log.info(f"MCTS Tree starting at the pre-final state of the environment (actions: {agent.search_root_node.parent.state})")
...@@ -91,7 +93,9 @@ agent.show_mcts_tree( ...@@ -91,7 +93,9 @@ agent.show_mcts_tree(
start_node=agent.search_root_node.parent, start_node=agent.search_root_node.parent,
) )
``` ```
![Visualisation Example](../_static/vis2.png)
![](https://github.com/Alexander-Nasuta/gymcts/raw/master/resources/vis2.png)
```python ```python
log.info(f"MCTS Tree starting at the root state (actions: {agent.search_root_node.get_root().state})") log.info(f"MCTS Tree starting at the root state (actions: {agent.search_root_node.get_root().state})")
agent.show_mcts_tree( agent.show_mcts_tree(
...@@ -100,4 +104,5 @@ agent.show_mcts_tree( ...@@ -100,4 +104,5 @@ agent.show_mcts_tree(
tree_max_depth=1 tree_max_depth=1
) )
``` ```
![Visualisation Example](../_static/vis3.png)
\ No newline at end of file ![](https://github.com/Alexander-Nasuta/gymcts/raw/master/resources/vis3.png)
resources/vis1.png

161 KiB

resources/vis2.png

1020 KiB

resources/vis3.png

521 KiB

0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment