If You Only Focus On What You Left Behind