Sounds like "of"...
Must have=must've not "must of"
Should have=should've not "should of"
Would have=would've not "would of"
Could have=could've not "could of"
Still, it's "coulda, woulda, shoulda" when you're complaining about betting the wrong horse.
And can I reiterate? Lose is how we write "not win/can't find." Loose is something altogether different.