How to Build Intuition in AI

"It is by logic that we prove, but by intuition that we discover." - Henri Poincaré

Having strong intuition and “feel” on ML concepts (architectures, training techniques, etc.) is a non-negotiable prerequisite for being productive in ML research. But how do you build it?

The answer isn't as simple as it may seem, and from my experience, I don't think people talk about how to do this efficiently. As an undergrad who's sunken probably over 200 hours into this and made many mistakes along the way, I wanted to share 3 pitfalls I've made in my journey, as well as the current 3-step framework I've arrived at that I find to be most effective (and fun!) for building a robust, practical intuition in AI/ML efficiently.

3 Common Mistakes

1. Don't Go Straight to the Papers

Papers, especially frontier ones, assume strong intuition and feel for a topic to be properly understood and appreciated. Treating these as an introductory resource for understanding a concept sets you up for failure. Seems simple, but I know I've made this mistake several times and have wasted countless hours trying to understand papers I simply didn't have the foundations to fully understand.

2. Don't Go to the Survey Papers, Either

While definitely a step in the right direction from reading frontier papers, in my experience, survey papers might assume too much prior knowledge (e.g. math, theory, etc) or are too general and don't provide enough detail to be useful. Building robust intuition quickly requires a resource that meets you exactly where you are in terms of what you know and what you don't know, and can answer your specific questions. Survey papers are static and frequently quite long, which makes it difficult to find the information you need quickly.

3. Online Courses/Lecture Series Are Inefficient

The other popular alternative to papers is to take an online course or view a lecture series. While this might be helpful, completing a course/series frequently takes tens of hours to complete and especially for courses requires a significant amount of energy. If you're in general a pretty busy person (student/working job), these typically take too long and are inconvenient.

My 3-Step Framework for Building Intuition

What we want for building intuition is something that can be done quickly (< 30 min), be completely tailored to your current understanding, and builds a robust and practical foundation for you to then apply however you'd like (reading papers, implementing papers, etc). Obviously, we will use LLMs to do this. But exactly how to do this efficiently is less clear. After sinking probably 200+ hours into experimenting with this, here's the framework I've arrived at that I find to be most effective (and fun!)

Step 0: Create a Project

Create a project (on GPT/Claude) and title it the concept you want to learn (e.g. "Variational Inference" or "Transformers"). This allows you to both organize your understanding neatly but also to enable project-local memory (in case one chat gets too long, Claude/ChatGPT will remember it for you in future conversations).

Step 1: Build High Level Intuition

Your first goal is to gain a high level intuition of the concept. Throw out all math, logic, and technical jargon out of the window. Ask the model to explain a concept to you like you're twelve. Here's a prompt I like to use with some prompting tricks I've found useful highlighted:

Step 2: Rapidly Identify Holes in Your Intuition and Fill Them

Ask any follow-up questions you have. Then, once you feel like you've gotten a basic grasp, try re-explaining the concept back to the LLM from scratch. Ask the LLM to identify any inaccuracies in your explanation or key missing pieces of the intuition that you missed. You might also find that in re-explaining this concept, a series of new follow up questions might come up. Save these and maybe even ask them first before resuming your explanation of the concept.

Step 3: Make Learning Multimodal

After going through the Feynman loop a couple of times (explaining the concept, realizing what parts of the concept you don't understand, then explaining again), it's time to go one step further by making your learning multimodal. Ask the model to implement the concept in annotated, interpretable code. Or ask it to introduce you to the core mathematical derivations / formulas. Doing this has helped me bridge the gap between high level concept and practical, working level understanding that makes my understanding of the concept even more robust.

Conclusion

That's it! Now you are ready to move forward as you please: read the frontier papers (they should now seem 10x less intimidating and 10x more insightful!), implement the concept/papers yourself from scratch, or start thinking about research tests you'd be curious to run given any ideas that popped up into your head.

Many thanks to Aarush Sah for his thoughts and feedback while drafting this post.