Tiny Steps Make Big Strides