We present a new approach for retrieving dark-field, phase shift, and attenuation images from speckle-based X-ray imaging data. Speckle-based X-ray imaging (SBXI) exploits sample-induced alterations to a reference near-field speckle pattern produced by a randomly structured mask. Attenuation images allow materials of different densities to be visualised. Phase-shift images are useful because they reveal how materials in a sample refract the X-ray beam, providing contrast between similar low-density structures that are difficult to reconstruct in attenuation images. Dark-field images convey information about structures that are smaller than the spatial resolution and thus invisible in both attenuation and phase-shift images. In previous works, we presented the Multimodal Intrinsic Speckle-Tracking (MIST) algorithm, which recovers the three complementary imaging modes from SBXI data by solving the associated Fokker--Planck equation. In this work, we present a variation of MIST, called ``gradient-flow MIST", which (1) reduces the amount of SBXI data required for image retrieval, (2) maintains the full generality of the X-ray Fokker--Planck equation, and (3) recovers dark-field images with higher quality than the previously proposed variants for weakly attenuating (i.e., low density) samples. We demonstrate the new gradient-flow MIST approach on experimental SBXI data of a knotted bundle of carbon fibres acquired at the Australian synchrotron. This approach is anticipated to be useful in phase-contrast and dark-field applications that require simplicity in experimentation and low sample X-ray exposure.