The Trouble with Fast Fitness