Results of anomaly detection using AutoEncoderMLP-256

This notebook presents experimental results of anomaly detection on the Tennessee Eastman Process dataset using the model AutoEncoderMLP-256.

Importing libraries.

import numpy as np
import torch
from sklearn.preprocessing import StandardScaler
from tqdm.auto import trange
from ice.anomaly_detection.datasets import AnomalyDetectionReinartzTEP, AnomalyDetectionSmallTEP
from ice.anomaly_detection.models import AutoEncoderMLP

Downloading the TEP dataset.

dataset = AnomalyDetectionReinartzTEP()

Training the model and calculation of metrics.

metrics = []
for i in trange(5):
    torch.random.manual_seed(i)
    model = AutoEncoderMLP(
        window_size=32,
        batch_size=512,
        lr=0.001,
        num_epochs=20,
        verbose=False,
        device='cuda'
    )
    model.fit(
        dataset.df[dataset.train_mask])
    metrics.append(
        model.evaluate(
            dataset.df[dataset.test_mask], dataset.target[dataset.test_mask]))

Printing metrics.

acc = []
tpr = []
fpr = []
for metrics_i in metrics:
    acc.append(metrics_i["accuracy"])
    tpr.append(metrics_i["true_positive_rate"])
    fpr.append(metrics_i["false_positive_rate"])
tpr_mean, tpr_std = np.mean(tpr, axis=0), np.std(tpr, axis=0)
fpr_mean, fpr_std = np.mean(fpr, axis=0), np.std(fpr, axis=0)

print(f'Accuracy: {np.mean(acc):.4f} ± {2*np.std(acc):.4f}')
for i in range(len(tpr_mean)):
    print(f'TPR/FPR: {tpr_mean[i]:.4f} ± {2*tpr_std[i]:.4f} / {fpr_mean[i]:.4f} ± {2*fpr_std[i]:.4f}')

Accuracy: 0.8028 ± 0.0007
TPR/FPR: 0.7378 ± 0.0022 / 0.0366 ± 0.0034