Melt the butter in a medium saucepan over medium-low heat. Add the brown sugar and cook for about 3 minutes, stirring constantly, until smooth. Remove from heat and add vanilla. Stir to combine.
Preheat the oven to 325 degrees.
Beat the remaining butter and brown sugar until creamy. Add the eggs and milk and beat until incorporated. Stir in the flour, baking powder, and spices until incorporated. Fold in the sea salt caramel chips.
Pour caramel sauce into a 9 inch round cake pan. Spread a layer of apples over caramel. Spoon cake batter over apples and spread gently with a spatula.
Place cake pan on a baking sheet and bake for 40 - 45 minutes and let cool in pan for 10 minutes. Place a cake plate over the top and invert.