All reviewers agree that this paper explores interesting territory, i.e., multi-agent Learning in the Diplomacy game. It is a well written and presented paper. The paper has generated quite some discussion after the rebuttal, discussing all pros and cons of the work. The major point in favor of the work (as also indicated by the authors themselves) seems to be that the work lays some ground work for future research in the Diplomacy game, that is known to be very hard and challenging. The biggest point of concern is that the paper presents little innovation in the techniques that it deploys but rather shows how the SOTA can be used/engineered to be successful in this domain to a certain extent, and illustrates the performance of known algorithms. The importance of this work is that it lays some ground work for future research to build on this initial study. There is consensus that this is indeed important. There are still many unanswered questions though about the performance of DipNet, which require a lot more work to be carried out. The authors have not responded to the question of code release raised by reviewers, and that remains also a point of concern as well. As a follow-up on the evaluation issues the authors discuss (feedback/paper). Some of the reviewers felt it would be nice if they would add a small discussion on the topic as it is gaining quite some interest, see e.g. Balduzzi et al., and Omidhshafiei et al. Note that the latter propose a method for evaluation that does apply to n-player games. - David Balduzzi et al. : Re-evaluating evaluation. NeurIPS 2018: 3272-3283 - Shayegan Omidshafiei et al.: α-Rank: Multi-Agent Evaluation by Evolution. Scientific Reports, 2019