Deep Q-Learning para o controle supervisório flexível de sistemas a eventos discretos em larga escala
Industrial Systems, as the manufacturing ones, can be tipically associated with a system profile that evolves with asynchronous events, in discrete time instants, which we name Discrete Event Systems (DES). Basically, a DES synthesizes the idea that the time between two actions (or events) in the sy...
| Autor principal: | Hendges, Lucas Volkmer |
|---|---|
| Formato: | Trabalho de Conclusão de Curso (Graduação) |
| Idioma: | Português |
| Publicado em: |
Universidade Tecnológica Federal do Paraná
2022
|
| Assuntos: | |
| Acesso em linha: |
http://repositorio.utfpr.edu.br/jspui/handle/1/29126 |
| Tags: |
Adicionar Tag
Sem tags, seja o primeiro a adicionar uma tag!
|
| Resumo: |
Industrial Systems, as the manufacturing ones, can be tipically associated with a system profile that evolves with asynchronous events, in discrete time instants, which we name Discrete Event Systems (DES). Basically, a DES synthesizes the idea that the time between two actions (or events) in the system can be despised due to the way the events occur. That means that DES prioritizes the logical mapping of the occurence of events in the system, and the operations on them consist only in processing logical outputs that orchestrate precisely the system’s physical structure. From the control perspective, one of the approaches that is dedicated on calculating optimal logical sequences to DES, taking into account properties such as the controllability and deadlock absence, is the Supervisory Control Theory (SCT), that is processed over DES models, expressed as Finite State Machines (FSM). As the SCT structures a formal method focused in the precision of the control logic’s computation, as a consequence, it benefits propertites such as safety and robustness of the control action, which is positive in many industrial environments. On the other hand, SCT fails on its limitation on totally or partially flexibilizing this logic, by processing events that are not necessarily exact. For these cases, i.e., when a DES has some events with probabilistic nature, intelligent processing approaches can be more suitable. For that, it is fundamental that the artificial perception be applied to certain events, but not to all of them, with the risk of compromising the exact control imposed by the SCT, which we want to preserve. Thus, this work proposes the partition of the event set of a DES on the deterministic and probabilistic subsets, in order to process them distinctly. While the deterministic ones are processed through SCT as usual, the ones that remain are treated through the conversion of a controlled model (FSM) of a DES in a Markov’s Decision Process (MDP), over which a Deep Q-Learning approach is applied. The choice of Deep Q-Learning is, over all, for its potential on processing big state sets, which is closer to the real applications of DES. The result of this integration is a more flexible control approach, in comparison to the classical SCT, making it possible to maximize certain system rewards while preserving the formality of the controller. |
|---|