Large language models such as ChatGPT translate and converse so fluently that they seem to understand language. This talk opens them up: how text becomes tokens and is generated one token at a time, how post-training makes answers more typical, how chatbots "remember" a conversation, and how models compare with humans, from implicature to spoken stress. Because these models learn mainly from written text, they tend to pull accents, dialects, low-resource languages, and local culture toward the dominant standard. Drawing on our recent LREC and EMNLP work on Taiwan Hakka and Taigi speech recognition, I will show what this means for building and evaluating speech and language technology.