There are a few other things to change in the web application. I noticed that on every page request my web app makes some unique calls to the content service. Recording those calls makes no sense since they are unique for every page request, and even worse, in simulation mode those requests fail since they were never recorded. Those unique calls origin from the Ambient Data Framework. While I like ADF and you will see it in any site which uses Experience Manager or Experience Optimization anyway, it needs to be disabled while working with Hoverfly.
In the Web.config comment the AmbientFrameworkModule.