At its core, the Lukebox is music recommendation service. Under the hood, most modern recommendation services, like those at Google or Pinterest, are very complex machine learning systems. To be honest with everyone, I understand very little about machine learning. I’ve been a backend/infrastructure engineer for the last few years. I love learning by doing slightly more than I enjoy learning by reading, so the Lukebox is a great opportunity for me to dive into a new field of computer science and engineering.
For the forseeable future, this blog post is going to be pinned to the top of my website, so people who visit this site can keep up with what I’m reading. There’s going to be a list of articles, videos, and papers that discuss interesting topics in machine learning, and if I really feel like it, I’ll put a summary of my learnings for each listing. I hope you all enjoy and learning something new!
Doing My Homework
As the hero section of this personal website says, I’m finding joy in learning (or doing my best to do so).
AI/ML/LLMs
- BERT Research paper – https://arxiv.org/abs/1810.04805
- Gotta shoutout one of the heros of the AI community, Andrej Karpathy. He has an awesome Youtube channel blessing the people with ML education. His Intro to LLMs video is fantastic for any beginner.
- More of a math and science channel, but 3Blue1Brown has been releasing some great content about transformers and neural networks.
- Learning about encoder/decoder in neural networks – https://arxiv.org/pdf/1706.03762.pdf
- Research into the future of foundational model development. They claim that their new model outperforms Transformers – https://arxiv.org/abs/2312.00752
- Contextual Multi-Armed Bandits in ad systems. This deserves a blog post of its own. – https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/37042.pdf
- Article highlighting the challenges of LLMs – https://www.honeycomb.io/blog/hard-stuff-nobody-talks-about-llm
- Very practical video about building a simple RAG application – https://www.youtube.com/watch?v=BrsocJb-fAo
Recommendation Systems
- Paper that goes into how new and old recommender systems are built and how LLMs with zero-shot prompting are effective recommenders – https://arxiv.org/html/2305.08845v2#:~:text=LLMs%20outperform%20existing%20zero%2Dshot,models%20with%20different%20practical%20strategies.
- Very, very good article – https://aman.ai/recsys/LLM/
- Great high level overview about how LLMs can be used as recsys – https://blog.reachsumit.com/posts/2023/04/llm-for-recsys/
- A paper that talks about a bunch of LLM Recommender systems – https://arxiv.org/html/2402.18590v1#:~:text=Unlike%20conventional%20systems%20lacking%20direct,in%20the%20realm%20of%20recommendations.
- Another paper highlighting their experiment results of zero-shot recommendation with LLMs - https://arxiv.org/pdf/2304.03153.pdf
- A survey of LLMs recommendation systems – https://arxiv.org/pdf/2305.19860.pdf
ML Ops
- Great book about ML Ops written by Chip Huyen – https://github.com/chiphuyen/dmls-book
Evals
LLM evals is a space I’ve become more interested in lately. Lots of cool companies building tooling in this space too.
Cover Photo: #science
