Preheat oven to 350F and grease a 9x5-inch loaf pan, lining it with parchment paper. Whisk together gluten free flour, baking powder, and salt.
Beat softened butter and sugar until light and fluffy, about 3 minutes. Add eggs one at a time, beating well after each. Mix in lemon zest, lemon juice, milk, and vanilla extract.
Fold the dry ingredients into the wet mixture until just combined. Pour batter into the prepared pan and smooth the top. Bake for 45 to 50 minutes until a toothpick inserted in the center comes out clean.
While the cake bakes, make the lemon drizzle by stirring lemon juice and granulated sugar together until the sugar dissolves. As soon as the cake comes out of the oven, poke holes all over the top with a toothpick and pour the lemon drizzle slowly over the hot cake.
Let the cake cool completely in the pan. Once cooled, whisk together powdered sugar and lemon juice to make the icing and drizzle it over the top. Let the icing set before slicing.