Preheat the oven to 375 degrees F / 190 degrees C / gas 5. Slice your bread into 1-inch cubes, leaving the crust on.
Lightly grease a 9x13 baking dish. Add about half of the bread cubes in an even layer on the bottom of the dish. Sprinkle half of the pecans on the top of the bread. Repeat with the remaining bread and pecans.
In a small bowl, make the custard. Start by whisking the eggs together, then whisk in the milk, sugar, orange zest, 1 teaspoon cinnamon, 1⁄4 teaspoon of nutmeg, and 1⁄4 teasoon salt. Pour evenly over the bread.
In another small bowl, make the topping by combining together the butter and brown sugar, then stir in 1 teaspoon cinnamon, 1⁄4 teaspoon nutmeg, and 1⁄8 teaspoon of salt. Drizzle evenly over the bread.
Place the casserole in the preheated oven and bake for 30-35 minutes, or until the top of the casserole is browned. Some of the tallest bread chunks will turn dark brown and all of the bread should be baked and crispy.
Let the casserole cool for 10 minutes before serving. Serve with maple syrup and powdered sugar on top, if desired.