Preheat oven to 375F and line a baking sheet with parchment paper. In a large bowl, whisk together gluten free flour, sugar, baking powder, and salt. Cut in cold butter using a pastry cutter until the mixture resembles coarse crumbs.
In a small bowl, whisk together cold milk, egg, and vanilla extract. Pour into the flour mixture and stir until a soft dough forms. Turn dough out onto a surface dusted with gluten free flour.
Make the filling by beating cream cheese, sugar, egg yolk, and vanilla extract until smooth and creamy.
Divide dough into 8 equal portions. Flatten each into a rough 4-inch round and place on the prepared baking sheet. Use the back of a spoon to create an indent in the center of each round and fill with 1 to 2 tablespoons of the cream cheese filling.
Bake for 16 to 18 minutes until the edges are golden and the filling is set. Let cool on the pan for 10 minutes.
Whisk powdered sugar and milk together until smooth and drizzle over the warm pastries. Serve once the glaze sets.