Consider a smooth connected manifold *M* equipped with a bracket generating distribution *D*. Two sub-Riemannian metrics on (*M*, *D*) are said to be *projectively* (resp. *affinely*) *equivalent* if they have the same geodesics up to reparameterization (resp. up to affine reparameterization). A sub-Riemannian metric *g* is called *rigid* (resp. *conformally rigid*) with respect to projective/affine equivalence, if any sub-Riemannian metric which is projectively/affinely equivalent to *g* is constantly proportional to *g* (resp. conformal to *g*). In the Riemannian case the local classification of projectively (resp. affinely) equivalent metrics was done in the classical work (Levi-Civita in Ann Mat Ser 2a 24:255–300, 1896; resp. Eisenhart in Trans Am Math Soc 25(2):297–306, 1923). In particular, a Riemannian metric which is not rigid with respect to one of the above equivalences satisfies the following two special properties: its geodesic flow possesses a collection of nontrivial integrals of special type and the metric induces certain canonical product structure on the ambient manifold. The only proper sub-Riemannian cases to which these classification results were extended so far are sub-Riemannian metrics on contact and quasi-contact distributions (Zelenko in J Math Sci (NY) 135(4):3168–3194, 2006). The general goal is to extend these results to arbitrary sub-Riemannian manifolds. In this article we establish two types of results toward this goal: if a sub-Riemannian metric is not conformally rigid with respect to the projective equivalence, then, first, its flow of normal extremals has at least one nontrivial integral quadratic on the fibers of the cotangent bundle and, second, the nilpotent approximation of the underlying distribution at any point admits a product structure. As a consequence we obtain two types of genericity results for rigidity: first, we show that a generic sub-Riemannian metric on a fixed pair (*M*, *D*) is conformally rigid with respect to projective equivalence. Second, we prove that, except for special pairs (*m*, *n*), for a generic distribution *D* of rank *m* on an *n*-dimensional manifold, every sub-Riemannian metric on *D* is conformally rigid with respect to the projective equivalence. For the affine equivalence in both genericity results conformal rigidity can be replaced by usual rigidity.