State Dependent Forgetting The Former Things