| Definition | Deep Transformer Q-Networks for Partially Observable Reinforcement Learning |
| No of environments | 21 |
Deep Transformer Q-Network (DTQN) is an extension of DQN and DRQN designed to encode an agent’s history effectively for solving partially observable reinforcement learning tasks. Our architecture is built from a Transformer Decoder (like GPT). DTQN is a sequence-to-sequence model; that is, given a history of the agent’s interactions (either observation or action-observation) with the environment, DTQN outputs a sequence of Q-values. Each element in the output sequence represents the Q-values for each action, given the agent’s history up to that point. For instance, the 3rd vector of Q-values was generated based on only the first three interactions between the agent and the environment. This method allows us to train much more efficiently than a Sequence-to-One method, since we get Sequence more data points to use for training. Our results providence strong evidence indicating a transformer can solve partially observable domains faster than previous recurrent approaches.