artificial intelligence
artificial intelligence articles on the Math Mental blog, with worked examples solved step by step.
Subscribe via RSS-
How AI picks the next word: softmax and logarithms
An AI assistant that writes text is, at its core, a conditional probability machine: given what has been written so far, it assigns a probability to every possible next word and picks one. Internal scores become probabilities through the softmax function, which uses exponentials and divides by their sum so that everything adds up to 1. The probability of a whole sentence is the product of each step's probability, and since that product shrinks very fast, logarithms turn the multiplication into addition.
Your first lesson takes five minutes
Create an account, pick where to start and keep your streak from today.