All experiments are run on a cluster of 30 machines equipped with single RTX3070Ti GPUs.
% General experimental setup -> ML domain
Replay is investigated in a supervised CIL-scenario, assuming known task-boundaries and disjoint classes. All of the following details apply to all investigated CL algorithms, namely AR, ER and DGR with VAEs.
Replay is investigated in a supervised CIL-scenario, assuming known task-boundaries and disjoint classes. All of the following details apply to all investigated CL algorithms, namely MIR, MerGAN, AR, ER and DGR with VAEs.
% Balancing of Tasks/Classes
The CIL problems used for all experiments are described in \cref{sec:data}.
...
...
@@ -340,9 +340,10 @@
%\end{figure}
First, we demonstrate the ability of a trained GMM to query its internal representation through data samples and selectively generate artificial data that \enquote{best match} those defining the query. To illustrate this, we train a GMM layer of $K=25$ components on MNIST classes 0, 4 and 6 for 50 epochs using the best-practice rules described in \cref{app:ar}. Then, we query the trained GMM with samples from class 9 uniquely, as described in \cref{sec:gmm}. The resulting samples are all from class 4, since it is the class that is \enquote{most similar} to the query class. These results are visualized in \cref{fig:var}. Variant generation results for deep convolutional extensions of GMMs can be found in \cite{gepperth2021new}, emphasizing that the AR approach can be scaled to more complex problems.
\subsection{Comparison: AR, ER, MIR,DGR-MerGAN and DGR-VAE}
%
In this main experiment, we evaluate the CL performance of AR w.r.t. measures given in \cref{sec:exppeval}, and compare its performance to DGR and ER since these represent principled approaches to replay-based CL. Results are tabulated in \cref{tab:short_results}.
In this main experiment, we evaluate the CL performance of AR w.r.t. measures given in \cref{sec:exppeval}, and compare its performance to MIR (see \cref{app:mir}), DGR-MerGAN (see \cref{app:mergan}, DGR-VAE (see \cref{app:dgr} and ER (see \cref{app:er}, since these represent principled approaches to replay-based CL.
Results are tabulated in \cref{tab:short_results}.
We use Gen-MIR with the parameter settings for the SplitMNIST problem as described in \cite{mir}. In order to have a fair comparison w.r.t. AR, we set the ratio of new to generated samples (n$\_$mem) to 1, and the samples per task to 5500. For perfoming the experiments, we adapted the software provided by the authors to work with different MNIST splits, as well as the other datasets.
% ------------------------------------------
\section{DGR-MerGAN training}\label{app:mergan}
Our DGR-MerGAN implementation is analogous to our DGR-VAE implementation, with the exception that the generator is implemented by MerGAN instances \cite{wu2018memory}. For the generators, we used the network topology and the experimental settings from \cite{wu2018memory}. For every training mini-batch, we create a "noise" mini-batch of the same size and use it for generating samples for the discriminator and the distillation loss term. The solver is trained exactly as for DGR-VAE, see \cref{app:dgr}.
