% AR leidet nur unter signifikantem Forgetting bei CIFAR-10; T1 (Klasse 4-9) nimmt stark ab nach T5 (Klasse 2)
\par\noindent\textbf{Latent replay/latent AR: }
For latent replay (SVHN and CIFAR), the results (see \cref{tab:short_resul, upper part}) show that DGR universally suffers from catastrophic forgetting although having the same baseline performance $\alpha^{\text{base}}$ as latent ER and AR. Forgetting for AR seems to only be significant for CIFAR D5-$1^5$B after task $T_5$, due to a high overlap with classes from initial task $T_1$.
For latent replay (SVHN and CIFAR), the results (\cref{tab:short_results}, upper part) show that DGR universally suffers from catastrophic forgetting although having the same baseline performance $\alpha^{\text{base}}$ as latent ER and AR. Forgetting for AR seems to only be significant for CIFAR D5-$1^5$B after task $T_5$, due to a high overlap with classes from initial task $T_1$.
% Argument: ER nur schlecht weil budget zu klein
Moreover, it is surprising to see that latent AR is able to achieve generally better results than latent ER. It could be argued that the budget per class for a complex dataset like SVHN and CIFAR-10 is rather small, and it can be assumed that increasing the budget would increase CL performance. However, we stress again that this is not trivially applicable in scenarios with a constrained memory budget.
%
\par\noindent\textbf{CF and selective replay:}
AR shows promising results in terms of knowledge retention, or prevention of forgetting, for sequentially learned classes, as reflected by generally lower average forgetting. We observed very little loss of knowledge on the first task $T_1$ after full training, suggesting that AR's ability to handle small incremental additions/updates to the internal knowledge base over a sequence of tasks is an intrinsic property, due to the selective replay mechanism.
AR shows promising results in terms of knowledge retention, or prevention of forgetting, for sequentially learned classes, as reflected by generally lower average forgetting.
In virtually all of the experiments conducted we observed a very moderate loss of information about the first task $T_1$ after full training, suggesting that AR's ability to handle small incremental additions/updates to the internal knowledge base over a sequence of tasks is an intrinsic property, due to the selective replay mechanism.
Moreover, AR demonstrates its intrinsic ability to limit unnecessary overwrites of past knowledge by performing efficient \textit{selective updates}, instead of having to replay the entire accumulated knowledge each time.
%
\par\noindent\textbf{Selective updates:}
As performed by AR training, are mainly characterized by matching GMM components with arriving input. Therefore, performance on previous tasks only decreases moderately through the adaptation of selected/similar units, as shown by low forgetting rates on almost every investigated CIL-problem in \cref{tab:short_results}. This implies that the GMM tends to converge towards a \textit{trade-off} between past knowledge and new data. This effect is observed in successive (replay-)training for two classes sharing a high similarity in the input space, as e.g. seen for, FMNIST D5-$1^5$A, where task $T_2$ (class: \enquote{sandals}) and task $T_4$ (class: \enquote{sneakers}) compete for internal capacity.
As performed by AR training, are mainly characterized by matching GMM components with arriving input. Therefore, performance on previous tasks generally decreases only slightly by the adaptation of selected/similar units, as shown by the low forgetting rates for almost all CIL-problems studied in \cref{tab:short_results}. This implies that the GMM tends to converge towards a \textit{trade-off} between past knowledge and new data. This effect is most notable when there is successive (replay-)training for two classes with high similarity in the input space, such as in, F-MNIST D5-$1^5$A, where task $T_2$ (class: \enquote{sandals}) and task $T_4$ (class: \enquote{sneakers}) compete for internal capacity.
% ---------------------
\section{Discussion}
In summary, we can state that our AR approach clearly surpasses VAE-based DGR in the evaluated CIL-P when constraining replay to a constant-time strategy. This is remarkable because the AR scholar performs the tasks of both solver and generator, while at the same time having less parameters. The advantage of AR becomes even more pronounced when considering forgetting prevention instead of simply looking at the classification accuracy results.
