Quit to Learn: My Large Language Model Journey

A little over a month ago, I quit my job at Robinhood and spend a lot of time exploring the wonders of generative AI. Naturally, my friends were curious about my reasoning, so I decided to write this blog post to unveil the inner workings of my mind. Now, to grab attention like those clickbait wizards, I couldn't resist jazzing up the title a bit. How about "Why I Ditched My $500K+ Robinhood Job!"? Sounds intriguing, doesn't it?

The Beginning

In September 2022, my journey into the world of generative AI began with the release of stable diffusion. Fascinated by this technology, I collaborated with a group of colleagues to participate in an internal hackathon hosted by Robinhood. Our project utilized stable diffusion to create a large number of Robinhood-style NFTs for our loyal customers. Although the project did not go public, it solidified my belief in the immense possibilities of GenAI. In my opinion, the release of stable diffusion allowed many individuals to directly experience the power of GenAI.

Soon after, the launch of ChatGPT revolutionized the field and gained widespread adoption due to its versatility in performing various tasks. Unlike previous machine learning models, Large Language Models (LLMs) excel in zero-shot learning. By modifying the prompt, these models can effortlessly tackle different tasks. This feature proves invaluable for prototyping, as product managers can conduct proof-of-concept solely through prompt engineering. Based on the outcomes, they can then make informed decisions on whether to further invest in the ML modeling aspect.

A few months after the launch of ChatGPT, I had the opportunity to explore the use of LLMs within Robinhood. I initially developed an LLM application that converts text into SQL, which helped me gain a better understanding of the limitations of LLMs. Even there were a lot of limitations, the impressive capabilities demonstrated by LLMs constantly distract me from my daily responsibilities. I find it difficult to resist dedicating all my time to working on LLMs. Unfortunately, I still have my full-time job leading the maintenance and development of the crucial ML infrastructure, specifically the feature store. Given the small size of my team compared to the various components in the system, it becomes extremely challenging for me to fully focus on LLMs, not to mention landing a job in the LLM space or building LLM products.

Upon recognizing this issue, the thought of quitting my job and dedicating myself to learning LLMs emerged.

Couldn't you do it while having a full time job?

Starting from stable diffusion, I was amazed at how quickly the GenAI field is advancing. Every day, there are countless papers, code, and products being published. While most of these don't hold much value, the remaining 0.1% of innovative work pushes us forward. It's easy to get lost in the details, but if we dedicate enough time, we can develop effective ways to filter and process the information.

One technique I often use is to read as much as possible and imagine creating a textbook for this field. I try to summarize the information into 10 clear chapters. When I come across new information, I try to fit it into one of these chapters. By consistently doing this, we create a structure that helps us process new information quickly.

Considering how rapidly the field of LLM evolves, I find that having a full-time job unrelated to this field results in a sense of wasting valuable time. Thus, after some deliberation, I made the decision to leave my position at Robinhood in order to dedicate myself completely to pursuing my goals in LLM.

The Down Side

I want to make clear that I am not advocating for anyone to quit their jobs at this moment, nor am I suggesting that this is the sole method to enter the LLM field. However, I have found it to be an effective approach in maximizing my personal potential. While it has ultimately led to me securing a job in the LLM domain, I also want to emphasize the drawbacks one may encounter.

Firstly, there are practical financial pressures that arise when you are not employed or have not secured enough funds to support your pursuit of this dream. It is vital to ensure that you have sufficient savings if you choose to go with this path. Additionally, make sure you carefully plan for your medical insurance to be prepared for any emergencies that may arise.

Secondly, it is important to note that this is not a conventional route, so it is possible that interviewers may question your background and motives (e.g. attributing your unemployment to performance issues). I personally have not encountered this issue frequently, but I understand the concerns interviewers may have, and I am willing to accept the consequences of choosing a less conventional path.