Catastrophic forgetting, rehearsal and pseudorehearsal

摘要: This paper reviews the problem of catastrophic forgetting (the loss or disruption previously learned information when new is learned) in neural networks, and explores rehearsal mechanisms retraining some as added) a potential solution. We replicate experiments described by Ratcliff (1990), including those relating to simple 'recency' based regime. then develop further regimes which are more effective than recency rehearsal. In particular, 'sweep rehearsal' very successful at minimizing forgetting. One possible limitation general, however, that may not be available for retraining. describe solution this problem, 'pseudorehearsal', method provides advantages without actually requiring any access original training population) itself. sugge...