CookieSensei

CookieSensei Home Learn Free Workshops Live Guidance About FAQ Contact Launch Lab

Learn/Programming & AI/Teaching Computers to Learn/Deep Learning

Teaching Computers to Learn•Interactive notebook lesson•Free

mnist eda notebook

No account is required. Learn the material, try the examples, and mark it complete locally when you are ready to move on.

MNIST Exploratory Data Analysis (EDA)

This notebook performs exploratory data analysis on the MNIST handwritten digit dataset.

MNIST contains 70,000 grayscale images of digits (0–9). Each image is 28×28 pixels.

Dataset split:

Training set: 60,000 images
Test set: 10,000 images

Goals of this notebook:

Load the dataset
Understand its structure
Visualize sample digits
Analyze pixel distributions
Check label distribution
Normalize image data
Explore summary statistics

1. Import Required Libraries

We import the libraries required for data analysis and visualization.

Codepython

import tensorflow as tf
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

Output

2026-03-15 19:07:22.456438: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2026-03-15 19:07:22.670204: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2026-03-15 19:07:22.670223: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2026-03-15 19:07:22.704776: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2026-03-15 19:07:22.778931: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-03-15 19:07:23.366083: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT

Codepython

(X_train, y_train), (X_test, y_test) = tf.keras.datasets.mnist.load_data()

print('Training images shape:', X_train.shape)
print('Training labels shape:', y_train.shape)
print('Test images shape:', X_test.shape)
print('Test labels shape:', y_test.shape)

Output

Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz

11490434/11490434 [==============================] - 3s 0us/step
Training images shape: (60000, 28, 28)
Training labels shape: (60000,)
Test images shape: (10000, 28, 28)
Test labels shape: (10000,)

Codepython

df_summary = pd.DataFrame({
    'pixel_mean':[X_train.mean()],
    'pixel_std':[X_train.std()],
    'pixel_min':[X_train.min()],
    'pixel_max':[X_train.max()]
})

df_summary

Output

   pixel_mean  pixel_std  pixel_min  pixel_max
0   33.318421   78.56749          0        255

Codepython

import tensorflow as tf

(X_train, y_train), (X_test, y_test) = tf.keras.datasets.mnist.load_data()

print('Training images shape:', X_train.shape)
print('Training labels shape:', y_train.shape)
print('Test images shape:', X_test.shape)
print('Test labels shape:', y_test.shape)

Output

Training images shape: (60000, 28, 28)
Training labels shape: (60000,)
Test images shape: (10000, 28, 28)
Test labels shape: (10000,)

Codepython

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense

model = Sequential([
    
    Dense(128, activation='relu', input_shape=(784,)),
    
    Dense(64, activation='relu'),
    
    Dense(10, activation='softmax')
])

Output

2026-03-15 19:08:19.357256: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2026-03-15 19:08:19.485773: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2026-03-15 19:08:19.488257: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2026-03-15 19:08:19.491534: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2026-03-15 19:08:19.493811: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2026-03-15 19:08:19.495986: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2026-03-15 19:08:19.589486: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2026-03-15 19:08:19.590362: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2026-03-15 19:08:19.591115: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355
2026-03-15 19:08:19.591927: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1929] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 10105 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 3060, pci bus id: 0000:01:00.0, compute capability: 8.6

Codepython

history = model.fit(
    X_train,
    y_train,
    epochs=10,
    batch_size=32,
    validation_split=0.2
)

Output

Epoch 1/10

2026-03-15 19:08:39.290369: I external/local_xla/xla/service/service.cc:168] XLA service 0x7ebad833a6d0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2026-03-15 19:08:39.290380: I external/local_xla/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA GeForce RTX 3060, Compute Capability 8.6
2026-03-15 19:08:39.300807: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:269] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
2026-03-15 19:08:39.327139: I external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:454] Loaded cuDNN version 8900
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1773601719.394029  198014 device_compiler.h:186] Compiled cluster using XLA!  This line is logged at most once for the lifetime of the process.

1500/1500 [==============================] - 3s 1ms/step - loss: 0.2680 - accuracy: 0.9215 - val_loss: 0.1466 - val_accuracy: 0.9581
Epoch 2/10
1500/1500 [==============================] - 2s 1ms/step - loss: 0.1105 - accuracy: 0.9659 - val_loss: 0.1094 - val_accuracy: 0.9674
Epoch 3/10
1500/1500 [==============================] - 1s 994us/step - loss: 0.0770 - accuracy: 0.9767 - val_loss: 0.0959 - val_accuracy: 0.9703
Epoch 4/10
1500/1500 [==============================] - 2s 1ms/step - loss: 0.0582 - accuracy: 0.9816 - val_loss: 0.0898 - val_accuracy: 0.9728
Epoch 5/10
1500/1500 [==============================] - 1s 992us/step - loss: 0.0439 - accuracy: 0.9858 - val_loss: 0.0913 - val_accuracy: 0.9743
Epoch 6/10
1500/1500 [==============================] - 2s 1ms/step - loss: 0.0374 - accuracy: 0.9875 - val_loss: 0.1021 - val_accuracy: 0.9733
Epoch 7/10
1500/1500 [==============================] - 1s 988us/step - loss: 0.0293 - accuracy: 0.9902 - val_loss: 0.1003 - val_accuracy: 0.9741
Epoch 8/10
1500/1500 [==============================] - 2s 1ms/step - loss: 0.0232 - accuracy: 0.9923 - val_loss: 0.0986 - val_accuracy: 0.9770
Epoch 9/10
1500/1500 [==============================] - 2s 1ms/step - loss: 0.0243 - accuracy: 0.9920 - val_loss: 0.1129 - val_accuracy: 0.9729
Epoch 10/10
1500/1500 [==============================] - 2s 1ms/step - loss: 0.0184 - accuracy: 0.9938 - val_loss: 0.1112 - val_accuracy: 0.9758

Codepython

test_loss, test_accuracy = model.evaluate(X_test, y_test)

print("Test Accuracy:", test_accuracy)

Output

313/313 [==============================] - 0s 632us/step - loss: 0.1072 - accuracy: 0.9750
Test Accuracy: 0.9750000238418579

Codepython

model.summary()

Output

Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense (Dense)               (None, 128)               100480    
                                                                 
 dense_1 (Dense)             (None, 64)                8256      
                                                                 
 dense_2 (Dense)             (None, 10)                650       
                                                                 
=================================================================
Total params: 109386 (427.29 KB)
Trainable params: 109386 (427.29 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________

Codepython

predictions = model.predict(X_test)

import numpy as np
predicted_label = np.argmax(predictions[0])

print("Predicted:", predicted_label)
print("Actual:", y_test[0])

Output

313/313 [==============================] - 0s 508us/step
Predicted: 7
Actual: 7

Codepython

import numpy as np

y_pred_probs = model.predict(X_test)

# Convert probabilities to class labels
y_pred = np.argmax(y_pred_probs, axis=1)

Output

313/313 [==============================] - 0s 526us/step

Codepython

from sklearn.metrics import classification_report

print(classification_report(y_test, y_pred))

Output

              precision    recall  f1-score   support

           0       0.99      0.98      0.99       980
           1       0.99      0.99      0.99      1135
           2       0.95      0.99      0.97      1032
           3       0.96      0.98      0.97      1010
           4       0.97      0.97      0.97       982
           5       0.99      0.96      0.98       892
           6       0.98      0.98      0.98       958
           7       0.98      0.96      0.97      1028
           8       0.98      0.96      0.97       974
           9       0.97      0.96      0.97      1009

    accuracy                           0.97     10000
   macro avg       0.98      0.97      0.97     10000
weighted avg       0.98      0.97      0.98     10000

Codepython

from sklearn.metrics import confusion_matrix
import seaborn as sns
import matplotlib.pyplot as plt

cm = confusion_matrix(y_test, y_pred)

plt.figure(figsize=(8,6))
sns.heatmap(cm, annot=True, fmt='d', cmap="Blues")

plt.xlabel("Predicted")
plt.ylabel("Actual")
plt.title("Confusion Matrix - MNIST ANN")
plt.show()

Output

[1;31mThe Kernel crashed while executing code in the current cell or a previous cell. 
[1;31mPlease review the code in the cell(s) to identify a possible cause of the failure. 
[1;31mClick <a href='https://aka.ms/vscodeJupyterKernelCrash'>here</a> for more info. 
[1;31mView Jupyter <a href='command:jupyter.viewOutput'>log</a> for further details.

CookieSensei

mnist eda notebook

MNIST Exploratory Data Analysis (EDA)

1. Import Required Libraries

2. Load MNIST Dataset

3. Inspect Data Types

4. Visualize a Sample Digit

5. Display Multiple Digits

6. Pixel Value Distribution

7. Label Distribution

8. Normalize Pixel Values

9. Visualize Normalized Image

10. Average Digit Visualization

11. Dataset Summary Statistics

Conclusion

Complete basic flow

Lesson resources

Finished this lesson?