In a medium bowl, whisk together eggs, sugar, heavy cream, milk, and vanilla until combined. Pour into a shallow bowl that's big enough to dip a piece of bread into.
Soak the bread slices in the mixture for 20-30 seconds on each side (if it gets too mushy and break, soak it for only a few seconds). Meanwhile, over medium-low heat, melt some of the butter in a skillet pan (swirl it around so it coats the pan bottom evenly) and sprinkle some of the sugar, if using. The sugar will turn into caramel so work quickly so it won’t burn.
As soon as the sugar is added, place however many slices of bread that fit into the pan. Cook until golden brown, 1-3 minutes on each side (the first side usually takes longer). Clean the pan carefully (it’s hot!) with a kitchen paper towel so there’s no sugar left, and repeat the process; melt butter, sprinkle sugar, if using, and cook the bread slices.