Take a look at how this # header renders:
I don't think it should have the permalink on it.
Yes, this is a known issue with the markdown rendering being misused. Specifically, we're currently using a whole-article rendering mode, but instead a segment-with-context mode should be used.
This should be resolved in future enhancements to the markdown rendering package.