Preheat oven to 320 degrees Fahrenheit.
Add brown sugar and butter to a stand mixer or mix by hand until light and fluffy.
Pour cold espresso and one egg into the butter and sugar mixture, then mix on medium speed until the mixture become lighter again.
Add the remaining egg and repeat until light and fluffy.
Remove from the mixer and sieve in the baking powder, cinnamon, cayenne pepper, and allspice into the cake batter.
Fold together with a metal spoon, adding the milk until a thick consistency is met.
Grease a loaf tin and transfer the mixture.
Bake in the preheated oven for 40 minutes or until a skewer comes out clean.
Allow to cool for 10 minutes before transferring to a wire rack to cool completely.