Whisk together gluten free flour, baking powder, and salt in a bowl. In a large bowl, beat softened butter and sugar until light and fluffy, about 3 minutes. Add egg, vanilla extract, and almond extract and beat until combined.
Gradually mix the dry ingredients into the butter mixture until a soft dough forms. Divide dough in half, flatten each into a disc, wrap in plastic, and refrigerate for at least 1 hour.
Preheat oven to 350F and line baking sheets with parchment paper. Roll out one dough disc between two sheets of parchment paper to about 1/4-inch thickness. Cut into shapes with cookie cutters and place 1 inch apart on baking sheets.
Bake for 9 to 11 minutes until the edges are barely golden but the centers still look slightly underdone. Let cool on the baking sheet for 5 minutes, then transfer to wire racks to cool completely.
For the icing, whisk together powdered sugar, milk, and vanilla extract until smooth. Add food coloring if desired. Spread or pipe icing onto cooled cookies and let set for 20 minutes before serving.