This is gonna be long, so my apologies in advance, but there's a lot of info I need to get across..
killer of a new motor is excessive heat being generated during break-in. The moving surfaces haven't mated in yet (i.e., pistons and rings rubbing against the cylinder walls, etc.), and running clearances (i.e., piston and ring diameter vs cylinder diameter) are the tightest they're ever going to be, if only because those clearances are naturally going to increase as the surfaces eventually wear. Letting an engine get too hot will run the risk of reducing those clearances below safe minimums, because the piston diameter increases as it gets hotter (because the metal expands outward) and the cylinder diameter is reduced (because the metal in the walls expands inward, just as a hole in a metal block will decrease in size as the block is heated). Also, the gap in the rings is reduced when the rings get too hot, and if the gap is completely eliminated the rings must grow in diameter as the temperature rises further, because the ring material continues to expand and grow in length but there's no room left to expand. So what was an acceptable fit at normal operating temperatures can become an unacceptably tight fit (or even a press-fit) at elevated temperatures, such that the sliding action of the rings in the cylinder now become more of a cutting action, chewing away the cylinder walls.
So you can see why it's very important to never let the engine get too hot, but it's even more critical during break-in.
That being said, you DO want to subject the engine to varying loads (and varying rpms) during break-in, because you do want the various moving surfaces to scrub-in together. That's particularly important at the wear interface between the rings and the cylinders, as the rings are designed to press harder against the cylinder walls as the load on the engine is increased (i.e., the more you open the throttle). So it's important not to "baby" the engine during break-in, but as I stated earlier it's also important not to overheat it either.
While we're on the subject of heat, here's another thing to remember -- NEVER run a cold engine hard, because things don't fit together correctly until they're up to normal operating temperature. For example, a piston isn't round when it's cold...there are thick sections of the piston (the wrist pin bosses) and there are thin sections (the skirts), and those different sections expand at different rates, so it's not until the piston is at operating temperature that it becomes truly "round".
That being said, here's what I do during break-in of a fresh motor...
I'll take the vehicle out onto the highway, running at highway speeds in top gear, and I'll look around to make sure there aren't any cops around, and I'll drop it down a couple of gears down and open the throttle wide open and hold the throttle open until I either run out of nerve or reach the redline. Then I close the throttle, snick it back into high gear and let out the clutch and trail-throttle back to legal cruising speed and run at legal speed for a couple of minutes again, letting the accumulated engine heat dissipate through the radiator. Then I'll repeat the process again, keeping an eye on the temperature guage, and never letting it get appreciably warmer than normal without letting the engine cool fully back down to normal operating temperature again.
I guarantee if you spend an hour doing that, your bike will be fully broken in.
Regarding "redline", it's perfectly okay to run a brand new engine up to the factory-designated redline (assuming the engine's warmed up fully) and hold it at that rpm indefinitely. The "redline" is simply an operational limit placed on the owner by the manufacturer...it's the manufacturer's way of protecting themselves. Basically, they're saying that as long as you don't exceed the redline, the manufacturer will guarantee that the engine will not come flying apart. Of course, if you do exceed that limit, then the guarantee is no longer valid, lol.
And you DO want to at least occasionally take the engine to redline, or at least to the highest rpm that it will ever see at any time during it's service life. Here's the reason for that: at very high rpms, the connecting rod gets elastically stretched because it has to stop the piston from it's high-speed travel towards the top of the cylinder and yank it back down in the reverse direction again...and keep in mind this is all happening at extremely high rates of piston acceleration and deceleration, so very high stretching forces are being applied to the connecting rod. And because the rod is elastically stretched at the top of the stroke, the piston (and more importantly, the piston rings) are "visiting" an area higher up in the cylinder than they would normally be visiting at lower rpms. Remember that as a cylinder wears, it naturally grows in diameter due to the wear. However, it's not the piston that's wearing the cylinder, it's the outward pressure of the rings against the cylinder wall. So the area of the cylinder that's higher than the level of the topmost ring actually never wears at all (and thus never grows in diameter) during the service life of the engine. Normally that wouldn't be a problem, unless you have a worn cylinder in an engine that was never subjected to high-rpm operation in its younger days, then one day late in its service life the owner decided to run the engine up to redline! And guess what happens then -- the connecting rod will stretch due to the high rpm, and the topmost ring will slam into the underside of a "ridge" at the top of the cylinder that has never been visited before (and thus has never been worn away), with the result that the topmost "ring land" on the piston gets crushed and the ring gets destroyed, which will then destroy the cylinder wall in the following few seconds (i.e., a destroyed engine), all because the owner never ran the rpm up to redline when the engine was still young.
And regarding the owner's manual stating that "engine speed should be kept below xxxx rpm for the first xxxx miles", that restriction is in there because the manufacturer has no idea who will be buying that engine or how it will be used...if the brand new engine is used for nothing except jack-rabbit starts from red lights, and immediately stops at the next light, and jumps again from that light, ad nauseum, then even in such a situation, as long as the owner never exceeds the stated rpm then there is very little likelihood that the motor will overheat...effectively, the manufacturer is simply protecting the idiot owner against his own stupidity (and protecting themselves, because they're also protecting their guarantee that the motor will also survive his stupidity).
Sorry again for the very wordy response.