diff --git a/README.md b/README.md index 895d428c69fcebeb8b13fbd43974d69ace6fda28..12fe244f1092cc2edc265d0d831b56a4588b73be 100644 --- a/README.md +++ b/README.md @@ -2,9 +2,9 @@ A Monte Carlo Tree Search Implementation for Gymnasium-style Environments. -- Github: [GYMCTS on Github](https://github.com/Alexander-Nasuta/GraphMatrixJobShopEnv) -- Pypi: [GYMCTS on PyPi](https://pypi.org/project/graph-matrix-jsp-env/) -- Documentation: [GYMCTS Docs](https://graphmatrixjobshopenv.readthedocs.io/en/latest/) +- Github: [GYMCTS on Github](https://github.com/Alexander-Nasuta/gymcts) +- Pypi: [GYMCTS on PyPi](https://pypi.org/project/gymcts/) +- Documentation: [GYMCTS Docs](https://gymcts.readthedocs.io/en/latest/) ## Description @@ -48,10 +48,10 @@ env = NormalizeReward(env, gamma=0.99, epsilon=1e-8) ```python env = TransformReward(env, lambda r: r / n_steps_per_episode) ``` -### FrozenLake Example (NaiveSoloMCTSGymEnvWrapper) +### FrozenLake Example (DeepCopyMCTSGymEnvWrapper) A minimal example of how to use the package with the FrozenLake environment and the NaiveSoloMCTSGymEnvWrapper is provided in the following code snippet below. -The NaiveSoloMCTSGymEnvWrapper can be used with non-deterministic environments, such as the FrozenLake environment with slippery ice. +The DeepCopyMCTSGymEnvWrapper can be used with non-deterministic environments, such as the FrozenLake environment with slippery ice. ```python import gymnasium as gym @@ -70,7 +70,7 @@ if __name__ == '__main__': env = gym.make('FrozenLake-v1', desc=None, map_name="4x4", is_slippery=True, render_mode="ansi") env.reset() - # 1. wrap the environment with the naive wrapper or a custom gymcts wrapper + # 1. wrap the environment with the deep copy wrapper or a custom gymcts wrapper env = DeepCopyMCTSGymEnvWrapper(env) # 2. create the agent @@ -93,7 +93,7 @@ if __name__ == '__main__': # 5. print the solution # read the solution from the info provided by the RecordEpisodeStatistics wrapper - # (that NaiveSoloMCTSGymEnvWrapper uses internally) + # (that DeepCopyMCTSGymEnvWrapper uses internally) episode_length = info["episode"]["l"] episode_return = info["episode"]["r"] @@ -186,7 +186,7 @@ if __name__ == '__main__': env = gym.make('FrozenLake-v1', desc=None, map_name="4x4", is_slippery=False, render_mode="rgb_array") env.reset() - # 1. wrap the environment with the naive wrapper or a custom gymcts wrapper + # 1. wrap the environment with the deep copy wrapper or a custom gymcts wrapper env = DeepCopyMCTSGymEnvWrapper(env) # 2. create the agent @@ -215,7 +215,7 @@ if __name__ == '__main__': env.close() # 5. print the solution - # read the solution from the info provided by the RecordEpisodeStatistics wrapper (that NaiveSoloMCTSGymEnvWrapper wraps internally) + # read the solution from the info provided by the RecordEpisodeStatistics wrapper (that DeepCopyMCTSGymEnvWrapper wraps internally) episode_length = info["episode"]["l"] episode_return = info["episode"]["r"] @@ -368,7 +368,7 @@ if __name__ == '__main__': env = gym.make('FrozenLake-v1', desc=None, map_name="4x4", is_slippery=False, render_mode="ansi") env.reset() - # wrap the environment with the naive wrapper or a custom gymcts wrapper + # wrap the environment with the wrapper or a custom gymcts wrapper env = ActionHistoryMCTSGymEnvWrapper(env) # create the agent diff --git a/docs/source/conf.py b/docs/source/conf.py index 19f22d20102f2401634f0bb6b71cd718d1b8c087..7ff26cbc4d9efd754b54bf6cfc87e214aaa8b69e 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -9,7 +9,7 @@ project = 'gymcts' copyright = '2025, Alexander Nasuta' author = 'Alexander Nasuta' -release = '1.0.0' +release = '1.2.0' # -- General configuration --------------------------------------------------- # https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration diff --git a/docs/source/development-setup.md b/docs/source/development-setup.md index 2f03adc6d8f293caae8b659ddf9129df81b38b25..84bce84998224de3d8e5d86d044bceb631457d0c 100644 --- a/docs/source/development-setup.md +++ b/docs/source/development-setup.md @@ -1,5 +1,6 @@ # Contributing + If you want to check out the code and implement new features or fix bugs, you can set up the project as follows: ### Clone the Repository @@ -8,11 +9,11 @@ clone the repository in your favorite code editor (for example PyCharm, VSCode, using https: ```shell -git clone https://github.com/Alexander-Nasuta/todo +git clone https://github.com/Alexander-Nasuta/gymcts.git ``` or by using the GitHub CLI: ```shell -gh repo clone Alexander-Nasuta/todo +gh repo clone Alexander-Nasuta/gymcts ``` if you are using PyCharm, I recommend doing the following additional steps: @@ -21,13 +22,10 @@ if you are using PyCharm, I recommend doing the following additional steps: - mark the `tests` folder as test root (by right-clicking on the folder and selecting `Mark Directory as` -> `Test Sources Root`) - mark the `resources` folder as resources root (by right-clicking on the folder and selecting `Mark Directory as` -> `Resources Root`) -at the end your project structure should look like this: - -todo ### Create a Virtual Environment (optional) -Most Developers use a virtual environment to manage the dependencies of their projects. +Most Developers use a virtual environment to manage the dependencies of their projects. I personally use `conda` for this purpose. When using `conda`, you can create a new environment with the name 'my-graph-jsp-env' following command: @@ -56,8 +54,8 @@ pip install -r requirements_dev.txt pip install tox ``` -The testing package `tox` is not included in the `requirements_dev.txt` file, because it sometimes causes issues when -using github actions. +The testing package `tox` is not included in the `requirements_dev.txt` file, because it sometimes causes issues when +using github actions. Github Actions uses an own tox environment (namely 'tox-gh-actions'), which can cause conflicts with the tox environment on your local machine. Reference: [Automated Testing in Python with pytest, tox, and GitHub Actions](https://www.youtube.com/watch?v=DhUpxWjOhME). @@ -89,13 +87,7 @@ For testing with `tox` run the following command: tox ``` -Here is a screenshot of what the output might look like: - - - -Tox will run the tests in a separate environment and will also check if the requirements are installed correctly. - -### Builing and Publishing the Project to PyPi +### Builing and Publishing the Project to PyPi In order to publish the project to PyPi, the project needs to be built and then uploaded to PyPi. @@ -120,7 +112,7 @@ twine upload dist/** ``` ### Documentation -This project uses `sphinx` for generating the documentation. +This project uses `sphinx` for generating the documentation. It also uses a lot of sphinx extensions to make the documentation more readable and interactive. For example the extension `myst-parser` is used to enable markdown support in the documentation (instead of the usual .rst-files). It also uses the `sphinx-autobuild` extension to automatically rebuild the documentation when changes are made. @@ -131,4 +123,3 @@ sphinx-autobuild ./docs/source/ ./docs/build/html/ ``` This project features most of the extensions featured in this Tutorial: [Document Your Scientific Project With Markdown, Sphinx, and Read the Docs | PyData Global 2021](https://www.youtube.com/watch?v=qRSb299awB0). - diff --git a/docs/source/visualisation.md b/docs/source/visualisation.md index 7d74420921a08d3fa0a1db35fcc6b4ab25434baa..178dd6ba578cb834f52450055bfcb462b9cf0deb 100644 --- a/docs/source/visualisation.md +++ b/docs/source/visualisation.md @@ -83,7 +83,9 @@ agent.show_mcts_tree( start_node=agent.search_root_node, ) ``` - + + + ```python log.info(f"MCTS Tree starting at the pre-final state of the environment (actions: {agent.search_root_node.parent.state})") @@ -91,7 +93,9 @@ agent.show_mcts_tree( start_node=agent.search_root_node.parent, ) ``` - + + + ```python log.info(f"MCTS Tree starting at the root state (actions: {agent.search_root_node.get_root().state})") agent.show_mcts_tree( @@ -100,4 +104,5 @@ agent.show_mcts_tree( tree_max_depth=1 ) ``` - \ No newline at end of file + + diff --git a/resources/vis1.png b/resources/vis1.png new file mode 100644 index 0000000000000000000000000000000000000000..6a1ca5d96c0f43e667865f599c395236c8ae665c Binary files /dev/null and b/resources/vis1.png differ diff --git a/resources/vis2.png b/resources/vis2.png new file mode 100644 index 0000000000000000000000000000000000000000..17d413a47cda7629ec876cfc7620a0a235456181 Binary files /dev/null and b/resources/vis2.png differ diff --git a/resources/vis3.png b/resources/vis3.png new file mode 100644 index 0000000000000000000000000000000000000000..509c7f15ee6bc737f4bd8358c84d04eff8068bef Binary files /dev/null and b/resources/vis3.png differ