Out of many advantages cited in Linux v/s Windows is that Linux is usually faster. After using both, I would say it is not the whole truth. The whole truth is that Linux is more customizable because of which it can be faster than Windows. It is not difficult to end up with a bloated and slow Linux machine. But you have more ability and probability of fine tuning your OS for your hardware with Linux.
Martyn Honeyford illustrates by showing how Linux memory footprint can be reduced (via E@zyVG). Not to say that Windows does not allow customization, it does, but much lesser. The ability to choose a Window manager is the first step towards fine-tuning.
I have come across a philosophy which says design is about eliminating options. I partially agree, I think design is about making decisions. There is a subtle difference. Let me illustrate. Ubuntu can decide to go with Gnome as the default window manager, but it allows for installation and selection of other window managers. Design is not about eliminating KDE and Xfce, it is about deciding to go with Gnome as the default option.
This is why I think configuration is important, it gives us a chance to customize without having to change the code. Configuration does not make it complex, the way it is exposed to the user makes it so. In fact configuration can help in making the software simpler. Imagine if we were not allowed to change the desktop resolution, and were always forced to use the 800 x 600 resolution? Would it be simpler or difficult?
I think Linux can be faster than Windows because it lets us configure the various aspects involved. We can customize it for performance, for visual appeal or for sturdiness. It is not that every user has to, but if required, Linux does not stop you.