When the Dark Elves were destroyed by the Frost Giants, the realm was now littered with their remains and filled with evil souls of the dead Dark Elves. This is also the reason why Algrim sought revenge on Odin when Thor was a teenager.
As Loki had conquered all of the realms except for Midgard, Svartalfheim was among them. In Svartelfheim, Black Panther had been pulled into there and was chased by the Dark Elves' tortured souls after examing the bones of the dead elves. After a long chase, he finally found his way out through the well which leds to Alfheim where he bumped into Hawkeye and Faradei.
When Loki is defeated, all of the realms were free including Svartalfheim.