This submission fits into a recent trend of developing more expressive, "layers" for deep learning, with the intent of adding more structure to data-driven models. A pivotal part of achieving these goals is making such layers differentiable in an efficient manner. Here, the authors develop a differentiable cloth simulator. Moreover, the authors also improve on a previously developed method for differentiating through optimization problems in order to achieve a significantly more efficient collision resolution. This paper takes ideas from existing works (such as differentiable optimization, differentiable physics simulations), applies them to a new domain and employ a previously unused factorization method to arrive at an original contribution. Also as a positive note for such a practical contribution, the authors have committed to releasing the source code openly. The paper is well written and organized. The concepts are exposed clearly and a good level of detail is provided. Nevertheless, the paper would probably benefit subtly by having a supplement with some extra details, such as: - The derivations mentioned but not written out in lines 136 and 163 - Full characterization of control task: what random initialization is performed? what are the specifics of the control network? are the reported results an average of N trials (or the best)? - Comparisons to other simulators (eg non differentiable ones), or to numerical methods (finite differences) - How are the interaction between the cloth simulation and the rigid bodies in the experimental scenes dealt with? Could this type of cloth simulation be easily integrated into broader (differentiable) physics simulations? The experiments performed demonstrate convincingly the strengths of the proposed method in a variety of settings. As mentioned before, for the sake of completeness, I would be interested in also seeing a comparison of the proposed simulator with numerical differentiation, if feasible for the problem sizes. Also, given significant efficiency gains in this method, it would be interesting to see the comparison of the differentiable simulator to a regular one.

This paper presents a differentiable cloth simulation method. It formulates cloth simulation as a QP with collision handling as constraints and it follows [1][7] to compute derivatives of linear systems of KKT conditions, but finds the high-dimensionality of cloth simulation leads to a impractical solution. --Originality The novelty of this paper lies in that the authors uses dynamic collision detection to figure out a sparse size of constraints so that a QR decomposition is used to solve linear systems of KKT conditions efficiently, which makes it practical to use in optimization and learning. --Quality The references are sufficient and evaluations are compelling. --Clarity The notations can be improved. In Algorithm 1, M, f and a are not explained. In Line 117, it is better to say explicitly k = 1,2,3. The whole description of differentiable cloth simulation seems to fall in parts. It is not clear to me how Algorithm 1 corresponds to the QP problem. Section 3.2 seems to be used to give an example of how one collision is handled, but its connection to the matrix formulation used later doesn't seem to be obvious. In Section 3.3, it will be great to use exact (physically meaningful) notations to describe the linear system in stead of using general Ax=b. The G and h also lacks of explanations of their physical meanings in Section 3.4. --Significance The reviewer finds this work tackles an important problem in deep learning and provides a practical solution to it. The public release of its code will further improve the impact of this paper.

Devising differentiable physics simulation has recently attracted many interests due to its potential to be combined with deep learning algorithms. This paper introduces a new type of differentiable physics simulation, specifically designed for cloth simulation. The key challenge in making cloth simulation differentiable is due to the complexity in handling external and self collisions. The paper proposed to solve this problem by formulating the collision handling as a constrained optimization problem and derived its derivatives using implicit differentiation. Furthermore, they leveraged the thin shape of the constraint matrix and used QR decomposition to accelerate the computation of the gradients. They evaluated the method on two applications: material estimation and motion control. The result shows that the proposed method can successfully achieve the given task and is more efficient than the baseline methods. The paper is well written and in general clear. I do have a few comments about the proposed method, as listed below. 1. For the material estimation experiment, the real-world data from Wang et al. was used. I’m wondering how was the target physical parameters obtained? Since Wang et al. also went through an optimization process to obtain the simulation model that fits the real-world data, it seems more suitable to compare the proposed gradient-based method to Wang et al. directly in terms of the per-vertex error, instead of the simplified baseline. 2. The main challenge and contribution of the paper in my opinion is about collision handling in cloth simulation. However, the examples shown in the video are fairly sparse in collisions. The ablation study in 4.1 is more collision rich, but wasn’t used to optimize anything. It would be desired to see examples of material estimation or motion control that are collision rich. 3. How well does the method handle frictional contact? It is not discussed in the paper and is not reflected in the experiments. ######################################## After reading the authors' rebuttal and other reviews, most of my concerns have been addressed. Thus I will raise my score for the submission.