Preheat the oven to 350°F. Spray an 8x8-inch baking dish with cooking spray and set aside.
Make the streusel topping: In a small bowl, whisk together the flour, brown sugar, cinnamon, and salt. Pour in the melted butter and stir until the mixture is evenly moistened and crumbly. Reserve a rounded ½ cup of the streusel and set it aside for topping the coffee cake. The remaining 1 cup of crumb topping will be used in the middle of the cake.
Make the coffee cake batter: In a large bowl, whisk together the flour, baking powder, baking soda, salt, and spices.
In a medium bowl, whisk together the sugar, oil, pumpkin, milk, eggs, and vanilla until smooth.
Pour the wet ingredients into the dry ingredients and gently fold until no streaks of flour remain. Do not overmix.
Spread half of the coffee cake batter into the prepared pan. Sprinkle evenly with the 1 cup of streusel.
Spoon the remaining batter over the streusel and spread it gently to the edges of the pan.
Sprinkle the top with the reserved ½ cup of streusel crumbs.
Bake for 45–55 minutes, or until a toothpick inserted in the center comes out clean.
Remove the coffee cake from the oven and let it cool to room temperature before icing.
Make the icing: While the cake is cooling, combine the powdered sugar, milk, and vanilla extract. Whisk until smooth. It should run off a spoon easily.
Once the cake is cooled, drizzle with icing if desired, and cut into squares. Serve and enjoy!