The Stargate Project is a new company which intends to invest $500 billion over the next four years building new AI infrastructure for OpenAI in the United States. We will begin deploying $100 billion immediately. This infrastructure will secure American leadership in AI, create hundreds of thousands of American jobs, and generate massive economic benefit for the entire world. This project will not only support the re-industrialization of the United States but also provide a strategic capability to protect the national security of America and its allies

I walk through how a transformer-based Large Language Model (LLM) generates text. From tokenization to embeddings, self-attention, logits, softmax, sampling, the autoregressive loop, and detokenization, I break down each step visually so you can see how it all fits together

In this video CJ explains chat bots, neural networks, tokenization, embeddings, transformers and more. He shows real working code for everything and gives historical context along the way. View the code here