Almond milk and sugar can be used as a cup-for-cup substitute for sweetened condensed milk. Oat milk, sugar, cornstarch, vanilla and butter Another dairy-free alternative is oat milk.