Bramble Scramble from DKC2, was really infuriating back in the day when I was still getting into video games, and it was pretty much an enorumous road block for me.
That entire game is a roadblock, the roadblock for me in that game was boss 2 for some reason. Partially because he's the line where every level moves from pretty easy to quite hard, partially because he himself is hard.
The final chapter in Fire Emblem: Blazing Sword, but this was very largely due to PEBCAK and the level being hard, my lords were very underleveled, and promoted very early (and thus had horrible stats), I had overused Marcus so most of my team was pretty weak, I never used anything stronger than Iron weapons because the game was too forgiving up to that point. Now imagine my frustration when all my n00b mistake came back to bite me. I had to abuse the overpowered Luna tome to win.
Fire Emblem games are all ridiculous, they're good, but their level systems are annoying. To play through the game you more or less have to redo every battle and priorities things with foresight. I don't like it at all. No grind points, fatal decisions masked early on. It's not a good balance, the game relies too much on knowing what will happen later on, which both adds an infinite learning curve, and proceeds to make the difficulty curve go in reverse. I'd much prefer it be balanced in a more forgivable way, let people just play the game without having to worry so much.
Dreadnaught Purple Coins from Super Mario Galaxy 1, for many people, people raged long and hard about Luigi's Purple Coins, but for me it was Dreadnaught's. It took me several hours to complete considering a sinlge misstep screws you over completely.
That was hard, and took me awhile, but it was also my favorite of the levels. Heck, that level on it's own wasn't really all that easy. The worst challenges were that ice levels purple coins, because it took forever in a level that was hell, and of course, daredevil lava level. Daredevil boulder was also annoying. Mario Galaxy 2's challenges were far less annoying, for it the clockwork double time was my worst.

Wily Stage 3, from MegaMan 9. Ookay, lets take level that has a few areas where you're constantly floating upwards, sound okay right? WRONG, the floating areas are chock full of spikes, and you can only move MegaMan by shooting the buster, which increases his inertia the more you shoot in one direction, which is a lot harder than it sound as you have to move at the correct speeds or you will get spiked and die. And if that wasn't enough, there's several spots where an enemy will drop down and DRAG you right into the very spike you thought you narrowly avoided. Oh, and the boss of that stage is both the Yellow AND the Green devil at the same fracking time. RAGE. Megaman games are the devil. They sit at by far one of the most difficult game series's in existence, and for some reason, no one seems to remember them as such. Half of them have situations that if you screw up once guarantee a game over. Their bosses are about the easiest part of them, and they still require a fair bit of memorization.

Metal Shark's Stage. It's basically a death course with a giant crusher, and it's full of ice and instakill spike, and enemies love to knock you back into these things at the most inconvenient moments. Oh and don't take the Nightmare route route if you value your sanity, cause now you have to deal with all that AND a forced scrolling level. Fun.
DEVIL
Shiver Factory from Kirby 64. Lots of insta-kill crushers, conveyer belts, the works.
I love that level
World 4 of Donkey Kong Returns, yes I'm serious, and ENTIRE WORLD full of aggravating levels. The Rocketbarrel level are among the most hated for many people, and the mine carts are pure trail-and-error levels.
That bat was quite annoying. Overall that game wasn't all that hard though, but I never took the time to deal with the secret world (other then cheating to play the levels, which are quite good). My largest frustration with that game was easily the final boss. He's not really all that hard, but he requires rolling, and for some reason, DK felt like to make the battle more challenging, 1/2 of the time he would refuse to roll. That and the boss decided that it would be a good idea to repeatably do patterns that didn't let me hit it, making a somewhat easy boss, annoying as hell. I bashed my controller into the ground a few times, and it decided to play nice. Seriously though why was roll not a button, there are a ton of unused buttons under my fingers.

Metal Shark's Stage. It's basically a death course with a giant crusher, and it's full of ice and instakill spike, and enemies love to knock you back into these things at the most inconvenient moments. Oh and don't take the Nightmare route route if you value your sanity, cause now you have to deal with all that AND a forced scrolling level. Fun.

DEVIL

...am I the only one who actually liked X6? Sure seems to be that way.

I used to like that game but then I played it again recently and realized it wasn't really that good. Pretty bad/annoying design in some/a lot of sections.