TLDR: Kinda bad day, am meta-learning how to learn ML better.
Details
Stayed up till midnight last night, got up late today (~9:00). Only started work by 11am because of my overly long morning routine (and procrastinating exercise)
Read sections 3.2 - 3.7 (~2h) of d2l.ai, 3.2 was interesting, I got to see how they structured the high level API.
Spent some time (1h) writing a convnet for cifar10. ran into a weird bug and spent some time debugging. Kinda a waste of time
Spent ~2h messing with Autoencoders and eventually refactored because my existing monolithic Jupyter notebook was getting hard to handle. Kinda a waste? I got more familiar witht Pytorch.
There's a fine balance between too much theory and too much practice. I think today was too much practice, I got bogged down into the nitty-gritty details.
Theory and Practice Rambling
Judging by Today and Yesterday's experiments, theory is more important for me to learn than practice. The logic is simple: Good theoretical understanding helps practical implementation, but good practical implementation skills don't help theoretical understanding (Or at least, not as much.)
There's something to direct practice and building "Useful" things though. I need to strike a balance. Maybe 75% theory 25% practice? It's likely beneficial to start by making useful things with pretrained models, then fine-tune them or do transfer learning. Reading code teaches
For learning the theory d2l.ai, the deep learning book, distill, and YouTube paper explanations seem like the best resources. Once I truly understand something implementing it should be easy.
Plan for tomorrow (and future)
Ideal plan for ML stuff I want to have a deep understanding of (early stopping along this list depending on how deep I want to go)
- Focus primarily on understanding models and algorithms
- Once I have a good understanding read code from a PyTorch implementation
- Wait some time, then try implementing it from scratch to check my knowledge
- Implement something cool and/or useful with my new knowledge, or write a blog post.
Things I want to go deep on: (I'm sure I'll add more later)
- Transformers / GPT
- Autoencoders (practical idea: compression program)