If you're OK with using a computer, here's a workaround:
Get it close, but leave a little "um".
Open it in "Lossless Cut" on Mac or Windows.
Extract the audio track as a separate file.
Edit in your fav editor: mute the remaining "um")
Use Lossless Cut to remove the original audio track, and add the new audio track.
Export the movie (container) with both tracks: video and audio.
This will not re-encode the video, and the export operation will happen as fast as your hard drive or SSD can copy the file.
(I do this for voice-over videos where I want to adjust the voice to sound more professional)
Alternatively, you could do this to the original file before you do any editing in LF. Use Lossless Cut to extract just the audio track, mute the "um"s, and recombine the A&V in Lossless Cut, then do your editing in LF. The video will not have been transcoded and the operation is fast.
Free download, scroll to the bottom: