Building Software•Interactive notebook lesson•Free

embedding vectors basics

No account is required. Learn the material, try the examples, and mark it complete locally when you are ready to move on.

Embedding vectors for images

Codepython

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline

Codepython

arr = np.zeros(shape=(7,5))
arr

Output

array([[0., 0., 0., 0., 0.],
       [0., 0., 0., 0., 0.],
       [0., 0., 0., 0., 0.],
       [0., 0., 0., 0., 0.],
       [0., 0., 0., 0., 0.],
       [0., 0., 0., 0., 0.],
       [0., 0., 0., 0., 0.]])

Codepython

three = arr.copy()
three[1:6, 3] = 255
three[range(1, 6, 2), 1] = 255
three[range(1, 6, 2), 2] = 255
three

Output

array([[  0.,   0.,   0.,   0.,   0.],
       [  0., 255., 255., 255.,   0.],
       [  0.,   0.,   0., 255.,   0.],
       [  0., 255., 255., 255.,   0.],
       [  0.,   0.,   0., 255.,   0.],
       [  0., 255., 255., 255.,   0.],
       [  0.,   0.,   0.,   0.,   0.]])

Codepython

plt.imshow(three, cmap="gray")
plt.show()

Output

Codepython

three.reshape(-1)

Output

array([  0.,   0.,   0.,   0.,   0.,   0., 255., 255., 255.,   0.,   0.,
         0.,   0., 255.,   0.,   0., 255., 255., 255.,   0.,   0.,   0.,
         0., 255.,   0.,   0., 255., 255., 255.,   0.,   0.,   0.,   0.,
         0.,   0.])

Codepython

small_three = three[1:-1, 1:-1].copy()
small_three

Output

array([[255., 255., 255.],
       [  0.,   0., 255.],
       [255., 255., 255.],
       [  0.,   0., 255.],
       [255., 255., 255.]])

Codepython

plt.imshow(small_three, cmap="gray")
plt.show()

Output

Codepython

small_three.shape

Output

(5, 3)

Codepython

small_three.reshape(-1)

Output

array([255., 255., 255.,   0.,   0., 255., 255., 255., 255.,   0.,   0.,
       255., 255., 255., 255.])

Codepython

small_three.reshape(-1).shape

Output

(15,)

Embedding vectors for TEXT

Codepython

import numpy as np
import pandas as pd
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.feature_extraction.text import CountVectorizer

Codepython

corpus = ['Early to bed', 'Early to rise', 'Makes a person', 'Healthy wealthy and wise']
corpus

Output

['Early to bed', 'Early to rise', 'Makes a person', 'Healthy wealthy and wise']

Call the CountVectorizer class from scikit-learn (alias sklearn)

Codepython

cv = CountVectorizer()
cv

Output

CountVectorizer()

We will get a sparse matrix with this

To convert this numpy sparse matrix to a row and column array, we will use toarray() method of numpy sparse matrix.

Codepython

X1 = cv.fit_transform(corpus)
X1

Output

<4x10 sparse matrix of type '<class 'numpy.int64'>'
	with 12 stored elements in Compressed Sparse Row format>

Codepython

X1 = X1.toarray()
X1

Output

array([[0, 1, 1, 0, 0, 0, 0, 1, 0, 0],
       [0, 0, 1, 0, 0, 0, 1, 1, 0, 0],
       [0, 0, 0, 0, 1, 1, 0, 0, 0, 0],
       [1, 0, 0, 1, 0, 0, 0, 0, 1, 1]])

Let's understand what this array of features actually convey.

But before that, take a look at the feature names now.

Codepython

cv.get_feature_names_out()

Output

array(['and', 'bed', 'early', 'healthy', 'makes', 'person', 'rise', 'to',
       'wealthy', 'wise'], dtype=object)

Now, lets see how count vectorizer works

Codepython

corpus

Output

['Early to bed', 'Early to rise', 'Makes a person', 'Healthy wealthy and wise']

Transformed corpus using count vectorizer

Codepython

transformed_cv = pd.DataFrame(X1, columns=cv.get_feature_names_out())
transformed_cv["whole string"] = corpus

Codepython

transformed_cv

Output

   and  bed  early  healthy  makes  person  rise  to  wealthy  wise  \
0    0    1      1        0      0       0     0   1        0     0   
1    0    0      1        0      0       0     1   1        0     0   
2    0    0      0        0      1       1     0   0        0     0   
3    1    0      0        1      0       0     0   0        1     1   

               whole string  
0              Early to bed  
1             Early to rise  
2            Makes a person  
3  Healthy wealthy and wise

Codepython

transformed_cv.head(1)

Output

   and  bed  early  healthy  makes  person  rise  to  wealthy  wise  \
0    0    1      1        0      0       0     0   1        0     0   

   whole string  
0  Early to bed

Codepython

# early to bed

# 0 1 1 0 0 0 0 1 0 0 
# bed early to

# this is good
# good is this

Codepython

cv2 = CountVectorizer(ngram_range=(1, 2))

Codepython

X2 = cv2.fit_transform(corpus).toarray()
X2

Output

array([[0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0],
       [0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0],
       [1, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1]])

Codepython

transformed_cv2 = pd.DataFrame(X2, columns=cv2.get_feature_names_out())
transformed_cv2["whole string"] = corpus

Codepython

transformed_cv2

Output

   and  and wise  bed  early  early to  healthy  healthy wealthy  makes  \
0    0         0    1      1         1        0                0      0   
1    0         0    0      1         1        0                0      0   
2    0         0    0      0         0        0                0      1   
3    1         1    0      0         0        1                1      0   

   makes person  person  rise  to  to bed  to rise  wealthy  wealthy and  \
0             0       0     0   1       1        0        0            0   
1             0       0     1   1       0        1        0            0   
2             1       1     0   0       0        0        0            0   
3             0       0     0   0       0        0        1            1   

   wise              whole string  
0     0              Early to bed  
1     0             Early to rise  
2     0            Makes a person  
3     1  Healthy wealthy and wise

Codepython

cv3 = CountVectorizer(ngram_range=(1,3))
X3 = cv3.fit_transform(corpus).toarray()
X3

Output

array([[0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0],
       [0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0],
       [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0],
       [1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1]])

Codepython

transformed_cv3 = pd.DataFrame(X3, columns=cv3.get_feature_names_out())
transformed_cv3["whole string"] = corpus

Codepython

import pandas as pd
pd.set_option('display.max_rows', 500)
pd.set_option('display.max_columns', 500)
pd.set_option('display.width', 1000)

Codepython

transformed_cv3

Output

   and  and wise  bed  early  early to  early to bed  early to rise  healthy  healthy wealthy  healthy wealthy and  makes  makes person  person  rise  to  to bed  to rise  wealthy  wealthy and  wealthy and wise  wise              whole string
0    0         0    1      1         1             1              0        0                0                    0      0             0       0     0   1       1        0        0            0                 0     0              Early to bed
1    0         0    0      1         1             0              1        0                0                    0      0             0       0     1   1       0        1        0            0                 0     0             Early to rise
2    0         0    0      0         0             0              0        0                0                    0      1             1       1     0   0       0        0        0            0                 0     0            Makes a person
3    1         1    0      0         0             0              0        1                1                    1      0             0       0     0   0       0        0        1            1                 1     1  Healthy wealthy and wise

Tf-Idf vectorization

Codepython

tf_trigram = TfidfVectorizer(ngram_range=(1,3))
X4 = tf_trigram.fit_transform(corpus).toarray()
transformed_tf = pd.DataFrame(X4, columns=tf_trigram.get_feature_names_out())
transformed_tf["whole string"] = corpus
transformed_tf

Output

        and  and wise       bed     early  early to  early to bed  early to rise   healthy  healthy wealthy  healthy wealthy and    makes  makes person   person      rise        to    to bed   to rise   wealthy  wealthy and  wealthy and wise      wise              whole string
0  0.000000  0.000000  0.453386  0.357455  0.357455      0.453386       0.000000  0.000000         0.000000             0.000000  0.00000       0.00000  0.00000  0.000000  0.357455  0.453386  0.000000  0.000000     0.000000          0.000000  0.000000              Early to bed
1  0.000000  0.000000  0.000000  0.357455  0.357455      0.000000       0.453386  0.000000         0.000000             0.000000  0.00000       0.00000  0.00000  0.453386  0.357455  0.000000  0.453386  0.000000     0.000000          0.000000  0.000000             Early to rise
2  0.000000  0.000000  0.000000  0.000000  0.000000      0.000000       0.000000  0.000000         0.000000             0.000000  0.57735       0.57735  0.57735  0.000000  0.000000  0.000000  0.000000  0.000000     0.000000          0.000000  0.000000            Makes a person
3  0.333333  0.333333  0.000000  0.000000  0.000000      0.000000       0.000000  0.333333         0.333333             0.333333  0.00000       0.00000  0.00000  0.000000  0.000000  0.000000  0.000000  0.333333     0.333333          0.333333  0.333333  Healthy wealthy and wise

EMBEDDINGS & GloVe TUTORIAL

Codepython

# =========================================
# EMBEDDINGS & GloVe TUTORIAL (COMPLETE)
# =========================================

# This tutorial walks step-by-step through:
# 1. What embeddings are
# 2. How text is converted into vectors
# 3. Simple embedding creation
# 4. Using pre-trained embeddings (GloVe)
# 5. Understanding each transformation step

# -----------------------------------------
# STEP 1: IMPORT LIBRARIES
# -----------------------------------------

import numpy as np
from collections import Counter

# -----------------------------------------
# STEP 2: SAMPLE TEXT DATA
# -----------------------------------------

sentences = [
    "I love machine learning",
    "I love deep learning",
    "machine learning is fun"
]

# -----------------------------------------
# STEP 3: TOKENIZATION
# -----------------------------------------

# Convert sentences into words (tokens)
# This is the FIRST step in converting text to vectors

tokenized_sentences = [sentence.lower().split() for sentence in sentences]
print("Tokenized Sentences:")
print(tokenized_sentences)

Output

Tokenized Sentences:
[['i', 'love', 'machine', 'learning'], ['i', 'love', 'deep', 'learning'], ['machine', 'learning', 'is', 'fun']]

Codepython

# -----------------------------------------
# STEP 4: BUILD VOCABULARY
# -----------------------------------------

# Create a vocabulary (unique words)

all_words = [word for sentence in tokenized_sentences for word in sentence]
vocab = sorted(set(all_words))

# Assign index to each word
word_to_index = {word: i for i, word in enumerate(vocab)}
index_to_word = {i: word for word, i in word_to_index.items()}

print("\nVocabulary:")
print(word_to_index)

Output


Vocabulary:
{'deep': 0, 'fun': 1, 'i': 2, 'is': 3, 'learning': 4, 'love': 5, 'machine': 6}

Codepython

# -----------------------------------------
# STEP 5: ONE-HOT ENCODING (BASIC VECTORS)
# -----------------------------------------

# Each word becomes a vector of size = vocab size
# Only one position = 1, rest = 0

def one_hot(word):
    vector = np.zeros(len(vocab))
    vector[word_to_index[word]] = 1
    return vector

print("\nOne-hot example for 'learning':")
print(one_hot("learning"))

Output


One-hot example for 'learning':
[0. 0. 0. 0. 1. 0. 0.]

Codepython

# -----------------------------------------
# HOW A SENTENCE BECOMES VECTORS
# -----------------------------------------

# Step-by-step transformation:
# "I love machine learning"
# -> ["i", "love", "machine", "learning"]
# -> each word → one-hot vector
# -> sentence = list of vectors

import pandas as pd

sentence_words = tokenized_sentences[0]
sentence_vectors = [one_hot(word) for word in sentence_words]

# Create a DataFrame where:
# - Each ROW = a word in the sentence
# - Each COLUMN = a position in the vocabulary vector
# This helps visualize how a sentence is represented numerically

df = pd.DataFrame(sentence_vectors, columns=vocab, index=sentence_words)

print("Sentence as DataFrame (rows = words, columns = vocab positions):")
df

Output

Sentence as DataFrame (rows = words, columns = vocab positions):

          deep  fun    i   is  learning  love  machine
i          0.0  0.0  1.0  0.0       0.0   0.0      0.0
love       0.0  0.0  0.0  0.0       0.0   1.0      0.0
machine    0.0  0.0  0.0  0.0       0.0   0.0      1.0
learning   0.0  0.0  0.0  0.0       1.0   0.0      0.0

Codepython

# -----------------------------------------
# INTERPRETATION
# -----------------------------------------

# Each row = one word
# Each column = a word from the vocabulary
# Value = 1 means that word is active at that position

# So the sentence is NOT a single vector yet -
# it is a SEQUENCE of vectors (one per word)

# To make a single sentence vector:
# we usually combine them (e.g., average)

sentence_vector_avg = df.values.mean(axis=0)

print("Single sentence vector (average of rows):")
print(sentence_vector_avg)

Output

Single sentence vector (average of rows):
[0.   0.   0.25 0.   0.25 0.25 0.25]

Codepython

# -----------------------------------------
# LIMITATION OF ONE-HOT
# -----------------------------------------

# - Sparse (mostly zeros)
# - No semantic meaning ("love" and "like" are unrelated numerically)

# -----------------------------------------
# STEP 6: SIMPLE EMBEDDINGS (LEARNED)
# -----------------------------------------

# Instead of large sparse vectors, we use dense vectors
# Example: embedding size = 4

embedding_dim = 4

# Randomly initialize embedding matrix
embedding_matrix = np.random.rand(len(vocab), embedding_dim)

# Function to get embedding for a word

def get_embedding(word):
    return embedding_matrix[word_to_index[word]]

print("\nEmbedding for 'learning':")
print(get_embedding("learning"))

Output


Embedding for 'learning':
[0.54329461 0.57475506 0.84584712 0.18102721]

Codepython

# -----------------------------------------
# SENTENCE → EMBEDDING PROCESS
# -----------------------------------------

# "I love machine learning"
# -> tokenize
# -> lookup embeddings
# -> combine (average/sum)

sentence = tokenized_sentences[0]
embeddings = np.array([get_embedding(word) for word in sentence])

# Average embedding (common technique)
sentence_embedding = np.mean(embeddings, axis=0)

print("\nSentence embedding (average):")
print(sentence_embedding)

Output


Sentence embedding (average):
[0.59757869 0.75001016 0.72049372 0.30183945]

Codepython

# -----------------------------------------
# STEP 7: INTRODUCTION TO GloVe
# -----------------------------------------

# GloVe = Global Vectors for Word Representation
# Pre-trained on large corpora
# Captures semantic relationships

# Example:
# king - man + woman ≈ queen

# -----------------------------------------
# STEP 8: LOAD GLOVE EMBEDDINGS
# -----------------------------------------

# NOTE: You must download GloVe file manually into ../datasets 
# https://nlp.stanford.edu/projects/glove/

# Example file: glove.6B.50d.txt

# We'll simulate loading a few vectors for demo

# Format of GloVe file:
# word val1 val2 val3 ...

# Example loader:

import numpy as np

def load_glove(file_path):
    glove_dict = {}
    with open(file_path, 'r', encoding='utf-8') as f:
        for line in f:
            values = line.split()
            word = values[0]
            vector = np.array(values[1:], dtype='float32')
            glove_dict[word] = vector
    return glove_dict

# Uncomment when file available:
glove = load_glove("../datasets/glove.6B.50d.txt")

Output

[0;31m---------------------------------------------------------------------------[0m[0;31mFileNotFoundError[0m                         Traceback (most recent call last)Cell [0;32mIn[38], line 41[0m
[1;32m     38[0m     [38;5;28;01mreturn[39;00m glove_dict
[1;32m     40[0m [38;5;66;03m# Uncomment when file available:[39;00m
[0;32m---> 41[0m glove [38;5;241m=[39m [43mload_glove[49m[43m([49m[38;5;124;43m"[39;49m[38;5;124;43m../datasets/glove.6B.50d.txt[39;49m[38;5;124;43m"[39;49m[43m)[49m
Cell [0;32mIn[38], line 32[0m, in [0;36mload_glove[0;34m(file_path)[0m
[1;32m     30[0m [38;5;28;01mdef[39;00m[38;5;250m [39m[38;5;21mload_glove[39m(file_path):
[1;32m     31[0m     glove_dict [38;5;241m=[39m {}
[0;32m---> 32[0m     [38;5;28;01mwith[39;00m [38;5;28;43mopen[39;49m[43m([49m[43mfile_path[49m[43m,[49m[43m [49m[38;5;124;43m'[39;49m[38;5;124;43mr[39;49m[38;5;124;43m'[39;49m[43m,[49m[43m [49m[43mencoding[49m[38;5;241;43m=[39;49m[38;5;124;43m'[39;49m[38;5;124;43mutf-8[39;49m[38;5;124;43m'[39;49m[43m)[49m [38;5;28;01mas[39;00m f:
[1;32m     33[0m         [38;5;28;01mfor[39;00m line [38;5;129;01min[39;00m f:
[1;32m     34[0m             values [38;5;241m=[39m line[38;5;241m.[39msplit()
File [0;32m/opt/envs/ds/lib/python3.10/site-packages/IPython/core/interactiveshell.py:324[0m, in [0;36m_modified_open[0;34m(file, *args, **kwargs)[0m
[1;32m    317[0m [38;5;28;01mif[39;00m file [38;5;129;01min[39;00m {[38;5;241m0[39m, [38;5;241m1[39m, [38;5;241m2[39m}:
[1;32m    318[0m     [38;5;28;01mraise[39;00m [38;5;167;01mValueError[39;00m(
[1;32m    319[0m         [38;5;124mf[39m[38;5;124m"[39m[38;5;124mIPython won[39m[38;5;124m'[39m[38;5;124mt let you open fd=[39m[38;5;132;01m{[39;00mfile[38;5;132;01m}[39;00m[38;5;124m by default [39m[38;5;124m"[39m
[1;32m    320[0m         [38;5;124m"[39m[38;5;124mas it is likely to crash IPython. If you know what you are doing, [39m[38;5;124m"[39m
[1;32m    321[0m         [38;5;124m"[39m[38;5;124myou can use builtins[39m[38;5;124m'[39m[38;5;124m open.[39m[38;5;124m"[39m
[1;32m    322[0m     )
[0;32m--> 324[0m [38;5;28;01mreturn[39;00m [43mio_open[49m[43m([49m[43mfile[49m[43m,[49m[43m [49m[38;5;241;43m*[39;49m[43margs[49m[43m,[49m[43m [49m[38;5;241;43m*[39;49m[38;5;241;43m*[39;49m[43mkwargs[49m[43m)[49m
[0;31mFileNotFoundError[0m: [Errno 2] No such file or directory: '../datasets/glove.6B.50d.txt'

Codepython

tokenized_sentences[0]

Output

['i', 'love', 'machine', 'learning']

Codepython

# Example (simulated dictionary)

glove = {
    "i": np.random.rand(50),
    "love": np.random.rand(50),
    "machine": np.random.rand(50),
    "learning": np.random.rand(50)
}

# Convert sentence using GloVe

def sentence_to_glove(sentence):
    vectors = [glove[word] for word in sentence if word in glove]
    return np.mean(vectors, axis=0)

print("\nGloVe sentence embedding:")
print(sentence_to_glove(tokenized_sentences[0]))

Output


GloVe sentence embedding:
[0.69563138 0.34140288 0.38813717 0.69906133 0.64981008 0.65982411
 0.30781142 0.66403958 0.51324121 0.63790584 0.3036328  0.50807332
 0.65598617 0.58524639 0.31849782 0.3680781  0.43269395 0.63531403
 0.5415867  0.37661252 0.49360365 0.36666192 0.19096357 0.49929238
 0.54098938 0.47281504 0.60295804 0.61350962 0.37739279 0.32263523
 0.64404827 0.30198929 0.50835469 0.51012688 0.76326134 0.39191583
 0.56962997 0.3070738  0.3962676  0.50432343 0.46421006 0.51214749
 0.47486568 0.38812231 0.54032576 0.55593843 0.7859546  0.47067767
 0.52249425 0.45768273]

Codepython

# -----------------------------------------
# STEP 10: WHY GLOVE IS BETTER
# -----------------------------------------

# - Captures meaning
# - Words with similar meaning have similar vectors
# - Pre-trained on huge datasets

# -----------------------------------------
# FULL PIPELINE SUMMARY
# -----------------------------------------

# Sentence → Lowercase → Tokenize
# → Vocabulary lookup
# → Convert to vectors (one-hot or embeddings)
# → Combine vectors (average, sum, etc.)
# → Final numeric representation

# -----------------------------------------
# END OF TUTORIAL
# -----------------------------------------

print("\nTutorial Complete!")

Output


Tutorial Complete!

Codepython

Lesson resources

Supporting code and files from this part of the curriculum.

Chatbot-1py Chatbot-2py Chatbot-3py

Finished this lesson?

Completion is saved in this browser without creating an account.