The paper proposes a method that improves over the Hindsight Experience Replay (HER) method by prioritizing training experiences whose pseudo-goals are closer to the actual goals. Goals are sampled according to a score that balances between (1) proximity to desired goals and (2) diversity of achieved goals chosen. The paper is well-written, the proposed method is new and interesting. The experiments on simulated robotic manipulation tasks also support the claims for the paper.