In a large bowl, whisk together both boxes of instant vanilla pudding mix and the cold whole milk for 2 minutes, until the pudding begins to thicken. Place in the refrigerator for 5 minutes to set slightly.
Once the pudding has firmed up, fold in the thawed whipped topping with a spatula until the mixture is completely smooth and uniform.
Arrange a single layer of gluten free graham crackers across the bottom of a 9x13-inch dish, breaking them as needed to fit and cover the bottom evenly.
Spread half of the pudding mixture over the first layer of graham crackers. Add another single layer of graham crackers on top, then spread the remaining pudding mixture over that. Finish with one final layer of graham crackers on top.
For the chocolate topping, whisk together the cocoa powder, powdered sugar, melted butter, milk, and vanilla extract until completely smooth and glossy. If the mixture is too thick, add milk one teaspoon at a time.
Pour the chocolate topping over the final layer of graham crackers and spread it evenly with an offset spatula or the back of a spoon.
Cover the dish with plastic wrap and refrigerate for at least 8 hours or overnight. The graham crackers will soften into a cake-like texture. Slice into squares and serve chilled.