To make the cupcakes:
Preheat oven to 350°F/175°C. Line muffin tin with cupcake liners. Set aside.
In a standing mixer fitted with the paddle attachment, beat together butter and sugar on medium speed until light and fluffy, about 3 minutes. Scrape down the sides and bottom of the bowl occasionally. Beat in eggs one at a time, beating well after each addition. Add mashed bananas and vanilla extract and beat. With the mixer on low speed, add half of the flour mixture, followed by the milk, then add the rest of the flour mixture and beat just until combined. Do not overmix—the less you mix, the lighter the cupcakes will be. Fold in chocolate chips.
Fill the cupcakes:
Core each cupcake with a cupcake or apple corer, or use a teaspoon to scoop out holes, leaving a well in the center of the cupcake. Place dulce de leche (or caramel sauce) in a a pastry bag, cut off the corner of the bag, and fill the holes (or just use a teaspoon to fill it instead).
*If you want to give the frosting a caramely flavor, you can add a few teaspoons of dulce de leche or caramel sauce to the frosting.