I put together a list of things I want to learn, including retroactively some stuff I’ve already taught myself to get started. I’m currently about halfway through part 2, and keeping notes as I go; once I’m done (for whatever value of “done” that turns out to be) I may write this up as a resource for anyone else who wants to go on a similar journey.

1. Use AI: Learn to communicate with hosted and locally running models, and integrate them into software.
2. Modify AI: Learn how to provide models with external knowledge, fine-tune them, and serve adapted models.
3. Build AI: Understand neural networks and transformers from the ground up, then build an artificial ant that learns through experience.

Part I — Using Existing Models

Goal: Understand how to access models, communicate with them programmatically, and choose where they run.

1. Hosted Models and APIs

1. Create and configure an OpenRouter account.
2. Understand the API request/response cycle.
3. Make API calls from Python.
4. Explore model names, providers, parameters, and response formats.
5. Understand tokens, context windows, temperature, and token limits.
6. Handle API errors, rate limits, and usage costs.
7. Store API keys safely in a .env file.
8. Understand environment variables and why secrets should never go into source control.
9. Understand the different categories of models available: generative, embedding, classification, reranking, vision-language, audio, and multimodal.
10. Understand the distinction between a model’s architecture, training objective, and capabilities.

Project: Build a small Python command-line chat client that talks to OpenRouter.

2. Running Models Locally with Ollama

1. Install Ollama and understand what it does.
2. Understand the difference between a model and a model server.
3. Download, list, run, and remove models.
4. Communicate with Ollama through its API.
5. Compare hosted and local models using the same prompts: quality, speed, privacy, hardware, and cost.
6. Understand quantization: floating-point versus integer representations, and the tradeoffs among memory, speed, and model quality.
7. Understand why different model sizes have different hardware requirements.
8. Understand model formats such as GGUF and how they relate to quantization and model serving.
9. Understand the distinction between model weights, model files, and the runtime serving them.

Project: Extend the chat client so you can switch between a hosted model and a local model without rewriting the application.

3. Structured Model Outputs

1. Understand the difference between ordinary text responses and structured responses.
2. Constrain model output to JSON.
3. Define and provide a JSON Schema.
4. Require specific fields, types, enums, and nested structures.
5. Understand schema-constrained generation versus asking the model to “please return JSON.”
6. Validate responses in Python.
7. Handle malformed responses, missing fields, refusals, and API errors.
8. Explore provider differences in structured-output support.

Project: Build an information-extraction tool that takes arbitrary text and returns validated JSON matching a schema you define.

4. Giving Models Tools

1. Understand function calling (also called tool calling).
2. Define a tool using a JSON Schema.
3. Send tool definitions along with a model request.
4. Understand the model’s tool-call response.
5. Understand why the model requests a tool but your application—not the model—actually executes it.
6. Execute the requested function in Python.
7. Return the function’s result to the model.
8. Build the tool-calling loop: model → tool → result → model.
9. Give the model multiple tools and let it choose between them.
10. Handle invalid arguments, tool errors, and unavailable tools.

Project: Build a model that can call Python functions to perform calculations, look up information, and manipulate data.

5. Designing an AI Application

1. Understand the roles of system, developer, and user messages.
2. Manage conversation history and context windows.
3. Understand stateless API calls versus application-managed conversation state.
4. Combine structured outputs and tool calling.
5. Design multi-step workflows.
6. Add guardrails and permission checks before executing consequential actions.
7. Log requests, tool calls, results, latency, and token usage.
8. Design a provider-independent interface that can work with both OpenRouter and Ollama.

Project: Build an agent harness with a loop that manages task state, tool execution, stopping limits, errors, approvals, and logs.

6. Model Behavior, Inference, and Generation

Goal: Understand what happens inside a model when it generates a response.

1. Understand how a model generates text one token at a time.
2. Explore greedy decoding and sampling.
3. Understand probability distributions, conditional probability, and sampling.
4. Understand temperature, top-k, and top-p.
5. Explore deterministic versus nondeterministic responses, and why the same prompt can produce different answers.
6. Understand expected values and why results vary between runs.
7. Understand streaming responses and handling partial output.
8. Explore stop sequences and maximum generation length.
9. Experiment with generation parameters and compare outputs under the same conditions.
10. Understand the relationship between probability distributions and the likelihood of different outputs.
11. Compare repeated generations to measure output consistency.
12. Understand semantic agreement versus exact textual agreement.
13. Explore prompt sensitivity: how small changes in wording affect generated results.
14. Understand why reproducibility is different from correctness.

Project: Build a small playground that lets you change generation parameters and compare the resulting outputs.

7. Security and Trust Boundaries

Goal: Understand the security implications of giving models access to tools, data, and external systems.

1. Understand prompt injection and how it works.
2. Distinguish trusted instructions from untrusted input.
3. Understand why tool outputs and retrieved documents must be treated as untrusted.
4. Limit tool permissions and validate arguments.
5. Protect API keys, secrets, and sensitive data.
6. Understand the principle of least privilege.
7. Implement human approval for consequential actions.
8. Explore common security mistakes in AI applications.
9. Understand indirect prompt injection through retrieved documents and other external content.

Project: Add security controls to the AI assistant built in Section 5. Test it with malicious instructions and unexpected tool responses.

---

Part II — Modifying Existing Models

Goal: Understand how to take a pretrained model, give it access to new information, and adapt it to a particular task or body of material.

1. Retrieval-Augmented Generation (RAG)

Goal: Learn how to provide a model with relevant information at query time without changing its weights.

1. Understand the difference between training a model and providing information at inference time.
2. Understand embeddings: turning text into numerical representations.
3. Generate embeddings for documents.
4. Break documents into chunks.
5. Store embeddings in a vector database.
6. Perform similarity searches.
7. Retrieve relevant passages in response to a question.
8. Pass retrieved content into a model’s context.
9. Understand chunk size, overlap, and retrieval quality.
10. Evaluate whether retrieval actually improves answers.
11. Explore limitations such as irrelevant retrieval, missing information, and context-window constraints.
12. Choose RAG when the task needs information from a changing or external knowledge source, especially when answers should be grounded in retrieved passages.
13. Choose fine-tuning when the goal is to change a model’s behavior, style, or response patterns—not simply to add a store of factual knowledge.
14. Consider combining RAG and fine-tuning when an application needs both current or source-grounded information and a consistently adapted response style or task behavior.
15. Compare the simplest suitable approach with alternatives using the same evaluation examples.
16. Understand embedding similarity and common distance metrics.
17. Compare semantic search with keyword search.
18. Explore hybrid retrieval and reranking.
19. Understand basic RAG variations, including query expansion, iterative retrieval, and agentic approaches.
20. Evaluate retrieval quality separately from answer quality.
21. Understand answer grounding, faithfulness, and the limitations of retrieved context.

Project: Build a local question-answering system over a collection of documents.

• Retrieve content within a specified date range.
• Filter by topic, source, or seFntiment.
• Ask questions about changes over time.
• Cite the underlying articles or posts supporting an answer.

2. Embeddings and Unsupervised Learning

Goal: Understand how embeddings can be used beyond similarity search.

1. Understand clustering and implement K-means.
2. Explore choosing K and evaluating cluster quality with silhouette scores.
3. Compare discovered clusters against known categories using ARI or NMI.
4. Explore stability by repeating clustering with different samples and random seeds.
5. Explore density-based clustering with HDBSCAN.
6. Investigate HDBSCAN’s parameter sensitivity.
7. Evaluate stability by repeating HDBSCAN with different min_samples and min_cluster_size.

Project: Cluster passages from public-domain books and compare the results against their known sources.

3. Fine-Tuning Fundamentals

Goal: Understand how to change a pretrained model’s behavior through training.

1. What does it mean to train a model?
2. Pretraining versus fine-tuning.
3. What are model weights, and what changes during training?
4. What is supervised fine-tuning (SFT)?
5. What is a training example?
6. Understand prompts, completions, and labels.
7. Preview epochs, batches, model size and depth, training duration, learning rates, gradient accumulation, optimizers, and learning-rate schedules; revisit these during the LoRA and Unsloth projects.
8. Understand training versus validation performance, and what overfitting looks like.
9. Preview regularization, dropout, weight decay, early stopping, and the effects of dataset size and diversity; revisit them during the neural-network and fine-tuning projects.
10. Evaluate whether fine-tuning improved a model by using a held-out evaluation set and comparing it with a baseline.
11. Preview transfer learning: how pretrained models reuse learned representations, when fine-tuning is appropriate, and its limitations; revisit it after the neural-network foundations.
12. Understand the difference between teaching a model new behavior and giving it access to new information.

4. Supervised Learning: Text Classification

Goal: Practice the supervised-learning workflow with a small, interpretable model before fine-tuning a language model.

1. Choose or create a small, separate text dataset with labeled categories; it need not be the content prepared for fine-tuning later.
2. Split the data into training, validation, and test sets without allowing examples from one split to leak into another.
3. Build a simple baseline, such as always predicting the most common category.
4. Convert text into features with TF-IDF and train a logistic-regression classifier.
5. Evaluate predictions with accuracy and per-class precision, recall, and F1; inspect a confusion matrix and misclassified examples.
6. Compare the classifier with the baseline, and discuss what the test results do—and do not—show.
7. Observe overfitting by comparing training and validation performance; try one small change and evaluate it without tuning against the test set.

Project: Build a sentiment classifier for a collection of dated articles or posts.

• Classify each item as positive, negative, or neutral.
• Use an existing pretrained sentiment model as a baseline.
• Train a simple TF-IDF + logistic regression classifier on labeled examples.
• Compare the two approaches using precision, recall, F1, and a confusion matrix.
• Investigate misclassifications, ambiguous language, sarcasm, and domain-specific sentiment.

5. Preparing Content for Training

Goal: Learn how to turn a collection of source material into a useful training dataset.

1. Identify the goal of a training corpus.
2. Extract content from your source.
3. Clean HTML, boilerplate, duplicates, and irrelevant material.
4. Preserve the original author’s voice rather than over-editing the material.
5. Structure articles and metadata into a consistent corpus.
6. Tokenize text and understand what tokens represent.
7. Create training, validation, and test splits.
8. Explore different training formats:
   • Raw text for continued language-model training.
   • Prompt/completion examples for instruction fine-tuning.
9. Understand dataset quality checks, deduplication, and data provenance.
10. Understand data contamination, train/test and evaluation leakage, and how to avoid them.
11. Consider copyright, licensing, data ownership, and personal information.
12. Recognize when a dataset is too small, unrepresentative, or has poor coverage.
13. Measure dataset coverage and representativeness.
14. Build a small corpus inspection and analysis tool.

Project: Produce a clean, documented, versioned dataset for the first fine-tune.

6. Your First Fine-Tune with Unsloth

Goal: Get a guided, end-to-end experience training an adapter before studying the underlying mechanics in detail.

1. Understand what Unsloth does and how it relates to PyTorch, Transformers, PEFT, and TRL.
2. Set up a supported training environment and choose a small model and dataset that fit available hardware.
3. Load the model and a prepared dataset using a guided Unsloth workflow; treat the framework’s defaults as tools, not as concepts already mastered.
4. Configure the basic training settings, such as the number of examples, batch size, learning rate, and training duration.
5. Run a short adapter fine-tune and monitor training loss.
6. Generate a few outputs before and after training using the same prompts and settings.
7. Save the adapter and verify that it can be loaded again with its base model.
8. Record the model, dataset, settings, hardware, and observed results; note questions about what the framework did behind the scenes.

Project: Complete a small guided fine-tune and produce a brief before-and-after comparison. The aim is to observe the workflow and outcomes, not yet to implement LoRA or explain every setting.

7. Unsloth: QLoRA and Deployment

Goal: Build on the first guided fine-tune by exploring a quantized adapter workflow, evaluating the result, and serving it.

1. Revisit the earlier run and inspect its configuration, including adapter rank, target modules, dropout, and trainable parameter count; record questions to revisit during the LoRA section in Part III.
2. Load a model using QLoRA and observe how quantized base weights and trainable adapter weights work together.
3. Change one setting at a time, keep the dataset and evaluation prompts fixed, and compare the results with the earlier run.
4. Track loss and evaluate generated outputs on held-out examples; recognize that lower training loss alone does not establish improved behavior.
5. Save the adapter and its configuration, then reload it with the compatible base checkpoint.
6. Understand adapter compatibility: architecture, base checkpoint, target modules, and format.
7. Explore merging an adapter into the base model and the tradeoffs between merged and separate weights.
8. Understand quantization and deployment formats, including memory, speed, and quality tradeoffs.
9. Serve the adapted model through an inference interface; explore how Ollama can serve compatible adapters or converted models.
10. Compare the base and adapted models using the same prompts and generation settings, and document limitations and results.
11. Set random seeds where supported, compare multiple runs when resources allow, and keep datasets, code, model versions, and settings organized.

Project: Adapt the prepared content with QLoRA, evaluate the result against the unchanged base model, save and reload the adapter, and serve it locally where supported.

Part III — The Deep Magic

Goal: Understand how neural networks learn, how language models work internally, and how to build an agent that learns to survive and behave in a simulated world.

1. Foundations of Neural Networks

Goal: Understand the basic machinery behind modern machine learning.

1. What is a neural network, conceptually?
2. Tensors, vectors, matrices, and dimensions.
3. Neurons, weights, biases, and activation functions.
4. Forward propagation: turning inputs into outputs.
5. Loss functions: measuring how wrong a prediction is.
6. Backpropagation: figuring out which weights contributed to the error.
7. Gradient descent and optimizers: changing weights to improve predictions.
8. Training versus validation.
9. Generalization versus memorization.
10. Explore regularization techniques, dropout, weight decay, and early stopping.
11. Build a tiny neural network in PyTorch, initially without a high-level training framework.
12. Inspect weights, activations, predictions, loss, and learning behavior.
13. Understand the common mathematical foundations shared by different neural-network architectures.
14. Understand how different training objectives produce different model capabilities.

Project: Train a small neural network from random initialization and understand every major step in its training loop.

2. How Language Models Work: Tokenization, Attention, and Transformers

Goal: Understand the architecture behind modern large language models.

1. Tokenization: converting text into token IDs.
2. Embeddings: converting token IDs into vectors.
3. Positional information: how a model represents token order.
4. Attention: how tokens relate to other tokens.
5. Self-attention and causal masking.
6. Multi-head attention.
7. Feed-forward layers and residual connections.
8. Layer normalization.
9. Understand the transformer block.
10. How a transformer predicts the next token.
11. How multiple transformer blocks form a language model.
12. How the model turns predictions into generated text.
13. Visualize attention and investigate what hidden layers represent.
14. Examine why a model produced a particular output, while recognizing the limits of interpretability.

Project: Build a tiny character-level or token-level language model in PyTorch, then gradually implement a miniature transformer.

3. LoRA: Understanding the Machinery

Goal: Use your experience with Unsloth and the neural-network and transformer foundations to understand adapters, then implement a simplified LoRA layer yourself.

1. Review linear layers: weight matrices, dimensions, and how a forward pass transforms inputs.
2. Understand what full fine-tuning changes and why updating every model weight can be expensive in memory and compute.
3. Understand parameter-efficient fine-tuning (PEFT): updating a small number of parameters while keeping the pretrained model mostly fixed.
4. Build intuition for matrix rank and low-rank approximations.
5. Understand the LoRA idea: represent a weight update as the product of two smaller trainable matrices instead of updating the full weight matrix.
6. Follow the LoRA equation: for a layer with weight matrix (W \in \mathbb{R}^{d_{\text{out}} \times d_{\text{in}}}), use (W’ = W + \frac{\alpha}{r}BA), where (A \in \mathbb{R}^{r \times d_{\text{in}}}) and (B \in \mathbb{R}^{d_{\text{out}} \times r}); understand how the dimensions make the update valid.
7. Understand the scaling factor, rank, and alpha, and how rank affects adapter size and capacity.
8. Understand how initialization makes the adapter’s initial contribution small or zero, so adding it does not disrupt the pretrained model at the start of training.
9. Keep the base weights frozen while gradients update only the adapter parameters.
10. Trace a forward pass through a linear layer with a LoRA update, and verify that gradients reach the adapter but not the frozen base weights.
11. Implement a LoRA-wrapped linear layer from scratch in PyTorch and test its dimensions, outputs, and gradients.
12. Attach LoRA to selected layers of a small pretrained model; understand common target modules in attention and feed-forward layers.
13. Explore how target-module choice, rank, scaling, and adapter dropout affect trainable parameter count and results.
14. Prepare examples in the model’s expected chat or prompt/completion format, and understand how tokenization and labels determine which tokens contribute to the loss.
15. Train the adapter with a minimal loop; track loss and watch for overfitting or unstable training.
16. Compare the adapted model against the unchanged base model using the same held-out examples, prompts, and generation settings.
17. Change one setting at a time, record results, and compare LoRA with full fine-tuning conceptually or on a small model when resources allow.
18. Recognize LoRA’s limitations: it does not guarantee better results, depends on suitable data and configuration, and may be less effective when a task requires broad changes to the model.

Project: Implement a LoRA-wrapped linear layer, then adapt a small pretrained model with your simplified implementation. Compare it with the unchanged base model and the earlier Unsloth result using held-out examples and consistent generation settings.

4. Build an Ant World — The Training Harness

Before the ant gets a brain, it needs a world to live in.

Goal: Build a simulation environment in which an agent can act and learn.

1. Represent a world as a two-dimensional grid.
2. Define terrain: dirt, open space, food, water, and nest.
3. Create the ant and define its position and state.
4. Define actions: move, turn, dig, pick up, carry, and drop.
5. Implement the simulation loop.
6. Define what the ant can observe.
7. Track time, energy, food, and other world state.
8. Add reset and reproducible random world generation.
9. Build a visual display so we can watch the simulation.
10. Separate the world simulation from the agent’s decision-making code.

Project: A working ant simulation where a human can control the ant and inspect the world.

5. Give the Ant a Brain — The First Neural Network

Goal: Connect a neural network to the ant simulation.

1. Translate the ant’s observations into numerical inputs.
2. Decide what the network needs to see: position, nearby cells, food, nest, and so on.
3. Define the network’s outputs: probabilities for each available action.
4. Understand action selection and random exploration.
5. Give the ant a reward signal.
6. Record observations, actions, rewards, and outcomes.
7. Understand the difference between supervised learning and reinforcement learning.
8. Connect the neural network to the simulation.
9. Experiment with an untrained network and observe its behavior.

Project: An ant controlled by a neural network, initially behaving randomly.

6. Reinforcement-Learning Fundamentals

Goal: Understand how an agent learns from experience rather than from labeled examples.

1. Agents, environments, states, and actions.
2. Rewards and cumulative rewards.
3. The Markov decision process (MDP).
4. Policies and value functions.
5. Exploration versus exploitation.
6. Discount factors and long-term rewards.
7. Understand Q-learning as an introductory algorithm.
8. Understand the difference between value-based and policy-based learning.
9. Introduce policy-gradient learning.
10. Understand PPO (Proximal Policy Optimization).
11. Compare simple policy-gradient methods with PPO.
12. Evaluate learning progress across multiple training runs.
13. Understand probability distributions, expected values, mean, variance, and standard deviation when analyzing rewards and training results.
14. Use correlation carefully; distinguish it from causation when interpreting experiments.
15. Establish baselines, change one variable at a time, and record settings and results.

Project: Train an initial ant agent using a simple reinforcement-learning algorithm.

7. Make It Behave Like an Ant

Goal: Develop increasingly sophisticated behavior through learning, memory, and interaction.

1. Improve the reward system and investigate unintended behavior.
2. Introduce partial observation: the ant can only see a small area around itself.
3. Add memory so it can use information from earlier observations.
4. Introduce recurrent neural networks or other memory mechanisms.
5. Introduce pheromone trails and their decay over time.
6. Experiment with different ant roles and behaviors.
7. Introduce multiple ants sharing the same world.
8. Explore how simple individual rules can produce collective behavior.
9. Experiment with obstacles, changing environments, and more complex goals.
10. Visualize learning progress and compare different agents.

Project: A colony of artificial ants that develops useful foraging behavior through learning and interaction.

8. Build a Tiny Language Model from Scratch

Goal: Bring together neural-network foundations and transformer architecture to build a working language model without starting from pretrained weights.

1. Prepare a small text corpus.
2. Implement or choose a simple tokenizer.
3. Convert text into training sequences.
4. Create input/target pairs for next-token prediction.
5. Build an embedding layer.
6. Implement a simple neural language model.
7. Understand n-gram language models.
8. Understand how conditional probability determines the next-token distribution.
9. Understand the relationship between n-gram models and Markov chains.
10. Compare n-gram models with neural language models.
11. Implement attention and a transformer block.
12. Build the training loop.
13. Train the model from random initialization.
14. Monitor loss and inspect generated text.
15. Build an evaluation set and compare model versions under the same conditions.
16. Experiment with model size, context length, and training data; record settings and compare multiple runs.
17. Explore why small models have limitations and how training data affects their behavior.

Project: Implement a bigram model. Extend it to trigram and higher-order models.

Project: Train a miniature language model from scratch and generate text with it.