Preheat oven to 350F and grease a 9x9-inch baking pan. Make the streusel by combining gluten free flour, brown sugar, and cinnamon. Cut in cold butter with your fingers until crumbly. Set aside.
In a bowl, whisk together gluten free flour, baking powder, baking soda, and salt.
In a large bowl, beat softened butter and sugar together until light and fluffy, about 3 minutes. Add eggs one at a time, beating well after each. Mix in sour cream and vanilla extract until smooth.
Fold the dry ingredients into the wet ingredients until just combined. Spread half the batter into the prepared pan, sprinkle half the streusel over it, then spread the remaining batter on top. Finish with the rest of the streusel.
Bake for 38 to 42 minutes until a toothpick inserted in the center comes out clean. Let cool 15 minutes.
Whisk powdered sugar and milk together until smooth and drizzle over the warm coffee cake. Slice and serve.