Preheat your oven to 350 degrees F and line two baking sheets with parchment paper. In a medium bowl, whisk together gluten free all-purpose flour blend, baking powder, baking soda, and salt.
In a large bowl, beat softened butter and granulated sugar with an electric mixer until light and fluffy, about 3 minutes. Add the eggs one at a time, beating after each. Mix in the milk and vanilla extract.
Add the flour mixture and mix on low speed until a thick, cake-like batter forms. Using a 1/4-cup measuring scoop, drop mounds of batter onto the prepared baking sheets, spacing 3 inches apart. Gently smooth the tops into round discs with the back of a damp spoon.
Bake for 14 to 16 minutes until the tops spring back when lightly touched and the bottoms are lightly golden. Transfer to a wire rack and let cool completely, flat side up.
For the vanilla icing, whisk together 2 cups powdered sugar, 3 tablespoons milk, and 1 teaspoon vanilla until smooth and thick but spreadable.
For the chocolate icing, whisk together 2 cups powdered sugar, cocoa powder, 3 tablespoons milk, and 1 teaspoon vanilla until smooth. If too thick, add milk half a teaspoon at a time.
Flip each cooled cookie flat-side up. Spread vanilla icing on one half and chocolate icing on the other half using a small offset spatula or the back of a spoon, creating a clean line down the center. Let the icing set for 20 minutes before serving.