Generally speaking, it is not considered to be a good idea to do this if you are used to drving yout amplifier very hard.
The reason is that you could end up going to the limitds of the amplifier and the result is DC being pushed into the speakers.
That said, you really would have to be pushing the amp right to the limits and somewhat beyond. If your normal listening levels are more moderate, such as mid range, then you will not have a problem.
Personally I would have no problem doing this, and to me it is better than using speakers that cannot handle the power.