Grease a 13x9" baking dish with cooking spray or butter and set aside.
Add the stale bread cubes to a large bowl. Drizzle with melted butter and gently toss until coat.
In a separate bowl, whisk together the half and half, eggs, ½ cup brown sugar, 2 teaspoons cinnamon, the nutmeg, and vanilla.
Pour the custard over the bread cubes in the mixing bowl. Stir until everything is coated and most of the liquid is absorbed.
Spoon the brioche mixture into the prepared baking dish and spread it into an even layer. Pour any extra custard from the bowl over the top.
Cover the baking dish and refrigerate for at least 6 hours—overnight is best.
The next morning, preheat the oven to 350°F and take the casserole out of the fridge.
Make the topping by mixing 3 tablespoons of brown sugar with ½ teaspoon of cinnamon. Sprinkle the topping over the casserole.
Bake the French toast uncovered for 45–60 minutes, until golden and puffed in the center. *Check the baking guide in the notes to decide how soft or crispy you want your casserole.
Once baked, let the casserole cool for 10 minutes and cut into slices. Serve warm with maple syrup or a dusting of powdered sugar.
Refrigerate any leftovers.