Magnetic tunnel junctions (MTJs), which are the fundamental building blocks of spintronic devices, have been used to build true random number generators (TRNGs) with different trade-offs between throughput, power, and area requirements. MTJs with high-barrier magnets (HBMs) have been used to generate random bitstreams with $\lesssim$ 200~Mb/s throughput and pJ/bit energy consumption. A high temperature sensitivity, however, adversely affects their performance as a TRNG. Superparamagnetic MTJs employing low-barrier magnets (LBMs) have also been used for TRNG operation. Although LBM-based MTJs can operate at low energy, they suffer from slow dynamics, sensitivity to process variations, and low fabrication yield. In this paper, we model a TRNG based on medium-barrier magnets (MBMs) with perpendicular magnetic anisotropy. The proposed MBM-based TRNG is driven with short voltage pulses to induce ballistic, yet stochastic, magnetization switching. We show that the proposed TRNG can operate at frequencies of about 500~MHz while consuming less than 100~fJ/bit of energy. In the short pulse ballistic limit, the switching probability of our device shows robustness to variations in temperature and material parameters relative to LBMs and HBMs. Our results suggest that MBM-based MTJs are suitable candidates for building fast and energy-efficient TRNG hardware units for probabilistic computing.