A random mass of loose fibers interacting by fiber-fiber contact is considered. As proposed in a previous paper, the elastic response is modeled based on the statistical mechanics of bending and torsion of fiber segments between fiber-fiber contact points. Presently we show how the statistical approach can be used to account for a distribution of fiber diameters rather than just a single diameter. The resulting expression has the same form and the same set of parameters as its single-diameter counterpart, except for two dimensionless reduction factors, which depend on the fiber diameter distribution only and reduce to unity for monodisperse fibers. Uniaxial compressibility experiments are performed on several materials with different bimodal fiber diameter distributions and are compared to model predictions. Even though no additional parameters were introduced to model the effect of mixed fiber diameters, the behavior is accurately predicted. Notably, the effect of the nonuniform fiber diameter is strong: A mixture of two fiber diameters differing by a factor of 2 can reduce the response by an order of magnitude, compared to the case of uniform diameter.