Eine Plattform für die Wissenschaft: Bauingenieurwesen, Architektur und Urbanistik
Eine Plattform für die Wissenschaft: Bauingenieurwesen, Architektur und Urbanistik
Efficient Policy Representation for Markov Decision Processes
https://orcid.org/http://orcid.org/0000-0002-6811-3955
https://orcid.org/http://orcid.org/0000-0002-4524-9737
Storing the optimal policy is an important challenge for autonomous agents and cyber-physical systems where the behavior is modeled by Markov decision processes. In this paper, we propose a symbolic approach to store an optimal policy compactly. We use decision trees for classifying optimal actions that are used to store a compressed representation of an optimal policy. To reduce memory consumption, we propose several approaches that keep the precision of the computed values or provide an approximation of them by considering a threshold for errors in the computed values. The first approach limits the depth of trees to reduce memory consumption. The second one detects and avoids non-important states to have smaller sets of states. The third approach prioritizes states by considering their impact on the precision of results. We use the PRISM case studies to investigate the effectiveness of our approach. Based on our results for the standard case studies, using decision trees and the proposed approaches, the needed memory for storing the optimal policies is reduced by several orders of magnitude, which is promising for embedded systems.
Efficient Policy Representation for Markov Decision Processes
https://orcid.org/http://orcid.org/0000-0002-6811-3955
https://orcid.org/http://orcid.org/0000-0002-4524-9737
Storing the optimal policy is an important challenge for autonomous agents and cyber-physical systems where the behavior is modeled by Markov decision processes. In this paper, we propose a symbolic approach to store an optimal policy compactly. We use decision trees for classifying optimal actions that are used to store a compressed representation of an optimal policy. To reduce memory consumption, we propose several approaches that keep the precision of the computed values or provide an approximation of them by considering a threshold for errors in the computed values. The first approach limits the depth of trees to reduce memory consumption. The second one detects and avoids non-important states to have smaller sets of states. The third approach prioritizes states by considering their impact on the precision of results. We use the PRISM case studies to investigate the effectiveness of our approach. Based on our results for the standard case studies, using decision trees and the proposed approaches, the needed memory for storing the optimal policies is reduced by several orders of magnitude, which is promising for embedded systems.
Efficient Policy Representation for Markov Decision Processes
https://orcid.org/http://orcid.org/0000-0002-6811-3955
https://orcid.org/http://orcid.org/0000-0002-4524-9737
Storing the optimal policy is an important challenge for autonomous agents and cyber-physical systems where the behavior is modeled by Markov decision processes. In this paper, we propose a symbolic approach to store an optimal policy compactly. We use decision trees for classifying optimal actions that are used to store a compressed representation of an optimal policy. To reduce memory consumption, we propose several approaches that keep the precision of the computed values or provide an approximation of them by considering a threshold for errors in the computed values. The first approach limits the depth of trees to reduce memory consumption. The second one detects and avoids non-important states to have smaller sets of states. The third approach prioritizes states by considering their impact on the precision of results. We use the PRISM case studies to investigate the effectiveness of our approach. Based on our results for the standard case studies, using decision trees and the proposed approaches, the needed memory for storing the optimal policies is reduced by several orders of magnitude, which is promising for embedded systems.
Efficient Policy Representation for Markov Decision Processes
Lect. Notes in Networks, Syst.
Arsenyeva, Olga (Herausgeber:in) / Romanova, Tatiana (Herausgeber:in) / Sukhonos, Maria (Herausgeber:in) / Tsegelnyk, Yevgen (Herausgeber:in) / Khademi, Anahita (Autor:in) / Khademian, Sepehr (Autor:in)
International Conference on Smart Technologies in Urban Engineering ; 2022 ; Kharkiv, Ukraine
29.11.2022
12 pages
Aufsatz/Kapitel (Buch)
Elektronische Ressource
Englisch
Efficient Policy Representation for Markov Decision Processes
| TIBKAT | 2023
Markov decision processes with fuzzy rewards
| British Library Online Contents | 2003
Incorporating Bayesian Networks in Markov Decision Processes
| Online Contents | 2013
Reliability-Based Structural Design with Markov Decision Processes
| Online Contents | 1995