I stumbled across a blog post about Scrum's origins on Jeff Sutherland's blog recently. The whole thing is worth a read in order to get a perspective on where the methodology came from, but especially interesting was the following comment:
"Few implementations of Scrum achieve the hyperproductive state for which Scrum was designed (5-10 times normal performance). Those that do all implement variations on XP engineering practices..."This fits with my - and others' - experience over the past few years. Scrum, with a few tweaks based on personal experience, provides a good starting point for a lightweight management framework. But that's just it - it's a management framework. You need wrap something to be managed in it. And if that something is a lardy, slow mini-waterfall process then you will simply end up delivering stale garbage more efficiently.
XP has the opposite problem. It provides tools and techniques to develop quality software efficiently, but it does not provide any guidance on tracking or how to control requirements. It benefits from being wrapped in something to manage its external dependencies.
Combining Scrum and XP may not be the only way to develop software but it does seem to be a damned good starting point for further iteration. Do it - you know it makes sense.