Hello everyone! It has been a busy week, but I’ve made some exciting progress on my machine learning journey. Here is what I’ve been up to:
Kaggle Orbit Wars & AWS
I completed the baseline implementation for the Kaggle Orbit Wars competition and initially hit a score of around 1030. My score has dipped slightly over the past few days, so I am currently brainstorming ways to improve it.
This week also marked my very first time using AWS! I used it to extract data for reinforcement learning. Transparency check: I spent exactly $7.58 USD on AWS resources during the process.
Paper Reading & RL Insights
I spent a lot of time reading research papers this week.
- AlphaZero: I was initially excited about using the self-play mechanism from AlphaZero. However, because this specific game has rock-paper-scissors dynamics, standard self-play might not work effectively.
- AlphaStar: This led me to the AlphaStar paper, which uses self-play combined with League Training.
The engineering behind AlphaStar is incredible. Two specific concepts really stood out to me: Pointer Networks and V-trace off-policy correction. I was also impressed by their use of an LSTM core to handle long-term memory.
Next Steps
Moving forward, I plan to leverage Kaggle, AWS, and GCP credits to train different components of my model. I am giving myself total freedom to experiment, imagine, and test unconventional solutions.
Random life update to close out the week: I used to have long hair because I was insecure about my forehead, but I finally decided to shave it all off at home by myself. It honestly feels really weird right now, but it’s a fresh start!
