In my Norsca campaign, the party of norsca characters is currently in the town Gardariborg. Just south of this city, there is a ruin of a dwarf hold marked "Kraka Ungrim, The Foul Fortress".

Here is the sparse fluff I have found about this place:

IC 2100 :
The Norse Dwarfs establish new outpost called Kraka-Azkahr, which could be translated as the Fortress City . Site has rich veins. Only problem is very aggressive neighbours (the Vargs, the Trolls and the Ogres) and that the new outpost is not protected by the magical Dwarfen Stones (Dwarf year 5100)."

IC 2300 :
The Dwarf stronghold/outpost of Kraka-Azkahr is overrun by the Chaos Horde sent by the Valmir Aesling, the Emperor of Chaos, and his Varg allies (Dwarf year c.5300).

IC 2304 :
The Imperial Dwarfs help the Norse Dwarfs to create expedition to travel the lost outpost of Kraka-Azkahr. They found ruined outpost full of foul creatures of the darkness. Small tribe of the Chaos Ogres are now living in the upper levels of the hold. Lower levels have Chaos Trolls and other foul creatures. The Hold is lost and the Dwarfs probably killed. The Imperial Dwarfs give the place new name - Karak-Ungrim (the Foul Fortress ), which becomes Kraka-Ungrim for the Norse Dwarfs. The Dwarfs leave the hold and try to forget this dark place. But there are strange rumours after this about the fallen hold. Rumours about horribly mutated Chaos Dwarfs (Dwarf year 5304).

Now, there is a chance that the characters will try to explore the depths of this fortress looking for a mcguffin, so I was wondering if anyone got any inspirations from the text above about what they might encounter.

The obvious answer is Chaos Dwarfs. However, CDs are not just "evil dwarfs". They are a specific dwarf culture situated in one specific location that was corrupted by worshipping Hastur.

These dwarfs seem to be a separate case however. So what has happened to them while living isolated deep below ground in the mountains of Norsca?

I'd probably just use standard Chaos Dwarfs.

While technically, Kraka Ungrim "should" ("logically") be a different culture than the Hastur-worshiping Chaos Dwarfs of Zharr-Naggrund, very few campaigns will ever globe-trot enough to get to both Norsca and the Dark Lands of the Chaos Dwarfs. If you're ever going to use the basic Chaos Dwarfs content, now's the time to do it. They have a rich iconography (love those hats), and there's plenty of info and ideas about them available for conversion from WHB or previous editions of the WFRP.

It's certainly easier to use all that existing content (or use it as a template but "norse-ify" it a little) than to create a whole new corrupted dwarven subculture from scratch.

Even if you had a player (or two) that knew the setting inside and out, and were real sticklers for canon-accuracy, you could probably placate them by suggesting that Kraka Ungrim and Zharr-Naggrund had some sort of fealty or trade agreement that included a yearly caravan exchange. They'll probably accept that. I would imagine most die-hard fans of the setting would be pleased to get a little screen-time dealing with one of the rarer cultures/badguys of the setting. Given that opportunity as a player, I for one would not be inclined to waste it arguing over whether or not this was the most realistic interpretation of the undocumented, but no doubt subtly-nuanced, culture of a fictional place called The Foul Fortress.

EDIT: The above assumes that you're just talking about the PCs spending a few quick sessions there chasing after a McGuffin, as you indicated. If instead you're looking at running six months of gaming immersed in the location, then it might be worth putting some more effort into creating a more unique culture or situation. It's all about time-management. Don't bite off more than you can chew, especially if there's another unclaimed plate with food that's already been cut into bite-sized morsels.

None of my players are really well-versed in Warhammer (which makes playing a norscan campaign pretty handy. The characters started out not knowing much outside their own village.) so being true to the CD canon is not all that important as they will not be able to tell the difference. It is mostly for my own part. I like to be internally consistent.

I think I will work out from the concept that the CD are an example of what happens when dwarfs fall to chaos: they keep the culture, but it's twisted. But find some other reason for their fall. The texts I posted above refer to horribly mutated dwarfs, so I am thinking that the chaos invasion of the fort left something that corrupted the dwarfs.

If the players wind up going there, they will probably spend between 1 and 3 sessions there. I mostly just need to flesh out the events between "The dwarf hold is sacked by a chaos horde" and "The deepest levels of the hold are inhabited by mutated dwarfs". The fortress also hold a tribe of mutated ogres and some chaos trolls, which implies that the place mutates anything that lives there, but I am not sure why exactly. Perhaps it's tied to the mcguffin. Or perhaps the dwarfs "dug too deep"...

Cool location. If you want to use something different than Chaos Dwarfs, how about something like this. The Dwarfs were somewhat renegades when compared to their southern brethren. And they had strong tradition in innovative, and questionable, methods in finding and processing ore. The veins in these mountains were rich in some material (not warpstone) that is not so well-known in the south and they had for a long time been doing research based on alchemy and "northern" runes.

This research led to them becoming renegade (not chaotic) and shunned by other dwarfs. Their research methods had them create these monstrous alchemical apparatus that were inscribed with runes. They needed manpower to operate these machines and dealt with the northmen to get slaves, or conquered etc. a tribe of ogres.

The problem was that the strange ore was addictive. They inscribed runes on their skin with it, used it internally etc. It was basically a drug and they all became addicts who needed more and more potent staff. This led to their destruction, and mutated their slaves (they might have used methods in trying to make more efficient workers that mutated them). Now, only mutated servants and the monstrous machines remain. And at the lowest levels the laboratories that tell the sad story.

Well, thought I would update this with what I finally ended up with:

When Valmir sacked the hold of Kraka-Azkahr, a group of dwarf runesmiths fled into the deepest recesses of the mines below the hold.

After Valmirs horde left, the dwarfs returned from the deep and saw their homes devestated. They went mad from the sight and swore to rebuild the hold stronger than ever. No matter what it took.

Among the defilements done by Valmirs horde, one of the worst was that Valmir split the great runic anvil of the hold with a deamon-possessed weapon.

They deployed all their knowledge and skill into re-forging the anvil, and it became a symbol of the unbreakable will of the dwarfs of the hold. They built new fortification inside the hold, with the anvil as its heart, insulating themselves with stone, metal and runes.

But the anvil was no longer what it appeared. The deamon of the weapon that broke it had seeped into the metal and it worked its corruptive influence on the runesmiths. Making them paranoid and insular and inspiring them to find new and more effective runes to protect themselves.

Now, after centuries of the anvils corrupting influences, the dwarfs have sealed off a part of the hold as a hold-within-the-hold. Bridges have been collapsed and tunnels have been sealed with stone and runes. Within the hold, the dwarfs create new runes that they burn into the stone, their weapons, their armour and even their flesh to protect against the next invasion.

At the heart of the hold sits the deamon forge, whispering inspiration into the runesmiths minds. No moral boundry is unbreakable in the pursuit of strengthening the hold, and they have long since discovered that the strongest runes are forged with blood sacrifices.

They capture any creatures that try to move into the hold and make them slaves and sacrifices. However, a tribe of ogres have moved into the upper level of the hold now. The ogres are too strong and numerous to be captured, so the dwarfs have grudgingly entered into a agreement with the ogres; slaves for weapons.

Recently, the slave trade has dried up however. As people and beastmen have learned to avoid the area of Kraka-Ungrim. And relationships have turned sour as the ogres keep demanding weapons and armour for less and less slaves.

Into this, wanders a group of adventurers with a dwarven key and a map of the hold.

Back in the day's of WFB 3rd ed (the last hardcover version) there were actually army lists for Norsca Dwarves. More recently there have been a few Norse related fanmade army lists as well, some available on Scrib'd. they may provide some ideas for classes and even some background fluff if you are interested