The shear deformation of a composite comprising elastic particles in a single crystal elastic–plastic matrix is analyzed using a discrete dislocation plasticity (DDP) framework wherein dislocation motion occurs via climb-assisted glide. The topology of the reinforcement is such that dislocations cannot continuously transverse the matrix by glide-only without encountering the particles that are impenetrable to dislocations. When dislocation motion is via glide-only, the shear stress versus strain response is strongly strain hardening with the hardening rate increasing with decreasing particle size for a fixed volume fraction of particles. This is due to the formation of dislocation pile-ups at the particle/matrix interfaces. The back stresses associated with these pile-ups result in a size effect and a strong Bauschinger effect. By contrast, when dislocation climb is permitted, the dislocation pile-ups break up by forming lower energy dislocation wall structures at the particle/matrix interfaces. This results in a significantly reduced size effect and reduced strain hardening. In fact, with increasing climb mobility an “inverse size” effect is also predicted where the strength decreases with decreasing particle size. Mass transport along the matrix/particle interface by dislocation climb causes this change in the response and also results in a reduction in the lattice rotations and density of geometrically necessary dislocations (GNDs) compared to the case where dislocation motion is by glide-only.