A simple tiny GPT model
📌 Note: This model is for educational purposes only. It is trained on a small dataset and is not intended for production use.
Tensorflow 2.15.0
doesn't support3.12
and needs3.11
-
Setup the poetry shell with the following command:
poetry env use /usr/local/bin/python3.11
-
Use the newly created shell
poetry shell
-
Running the model with character embedding.
python model_char_embedding.py
-
Running the model with word embedding.
python model_word_embedding.py
Visual Explaination
flowchart TD
A[Text Corpus] --> B[Tokenization]
B --> C[Vocabulary & Sequence Generation]
C --> D[Pad Sequences]
D --> E[Define GPT Model]
E --> F[Embedding Layer]
F --> G[Self-Attention Block]
G --> H[Feedforward Layers]
H --> I[Compile Model]
I --> J[Train Model]
J --> K[Generate Text]
K --> L[Output Result]