Working memory networks: Augmenting memory networks with a relational reasoning module

Juan Pavez, Héctor Allende, Héctor Allende-Cid

Resultado de la investigación: Capítulo del libro/informe/acta de congresoContribución a la conferenciarevisión exhaustiva

6 Citas (Scopus)

Resumen

During the last years, there has been a lot of interest in achieving some kind of complex reasoning using deep neural networks. To do that, models like Memory Networks (MemNNs) have combined external memory storages and attention mechanisms. These architectures, however, lack of more complex reasoning mechanisms that could allow, for instance, relational reasoning. Relation Networks (RNs), on the other hand, have shown outstanding results in relational reasoning tasks. Unfortunately, their computational cost grows quadratically with the number of memories, something prohibitive for larger problems. To solve these issues, we introduce the Working Memory Network, a MemNN architecture with a novel working memory storage and reasoning module. Our model retains the relational reasoning abilities of the RN while reducing its computational complexity from quadratic to linear. We tested our model on the text QA dataset bAbI and the visual QA dataset NLVR. In the jointly trained bAbI-10k, we set a new state-of-the-art, achieving a mean error of less than 0.5%. Moreover, a simple ensemble of two of our models solves all 20 tasks in the joint version of the benchmark.

Idioma originalInglés
Título de la publicación alojadaACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers)
EditorialAssociation for Computational Linguistics (ACL)
Páginas1000-1009
Número de páginas10
ISBN (versión digital)9781948087322
DOI
EstadoPublicada - 2018
Publicado de forma externa
Evento56th Annual Meeting of the Association for Computational Linguistics, ACL 2018 - Melbourne, Australia
Duración: 15 jul. 201820 jul. 2018

Serie de la publicación

NombreACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers)
Volumen1

Conferencia

Conferencia56th Annual Meeting of the Association for Computational Linguistics, ACL 2018
País/TerritorioAustralia
CiudadMelbourne
Período15/07/1820/07/18

Huella

Profundice en los temas de investigación de 'Working memory networks: Augmenting memory networks with a relational reasoning module'. En conjunto forman una huella única.

Citar esto