%20by%20%E2%80%9EPablo%20Stanley%E2%80%9D%2C%20licensed%20under%20%E2%80%9EFree%20for%20personal%20and%20commercial%20use%E2%80%9D%20(https%3A%2F%2Fbottts.com%2F)%3C%2Fdc%3Arights%3E%3C%2Frdf%3ADescription%3E%3C%2Frdf%3ARDF%3E%3C%2Fmetadata%3E%3Cmask%20id%3D%22viewboxMask%22%3E%3Crect%20width%3D%22180%22%20height%3D%22180%22%20rx%3D%220%22%20ry%3D%220%22%20x%3D%220%22%20y%3D%220%22%20fill%3D%22%23fff%22%20%2F%3E%3C%2Fmask%3E%3Cg%20mask%3D%22url(%23viewboxMask)%22%3E%3Crect%20fill%3D%22%23ffd400%22%20width%3D%22180%22%20height%3D%22180%22%20x%3D%220%22%20y%3D%220%22%20%2F%3E%3Cg%20transform%3D%22translate(0%2066)%22%3E%3Cg%20opacity%3D%22.9%22%20stroke%3D%22%232A3544%22%3E%3Cellipse%20cx%3D%2232.5%22%20cy%3D%2223%22%20rx%3D%2216.5%22%20ry%3D%2218%22%20stroke-width%3D%226%22%2F%3E%3Cpath%20d%3D%22M29.51%2036.76c-7.4%204.29-17%201.55-21.42-6.1%22%20stroke-width%3D%224%22%2F%3E%3Cellipse%20cx%3D%2228.5%22%20cy%3D%2252.5%22%20rx%3D%2216.5%22%20ry%3D%2214.5%22%20stroke-width%3D%224%22%2F%3E%3C%2Fg%3E%3Cg%20opacity%3D%22.9%22%20stroke%3D%22%232A3544%22%3E%3Cpath%20d%3D%22M168.6%2060.42c-4.27-7.41-13.95-9.84-21.6-5.42%22%20stroke-width%3D%224%22%2F%3E%3Cellipse%20cx%3D%22148.5%22%20cy%3D%2222.5%22%20rx%3D%2216.5%22%20ry%3D%2215.5%22%20stroke-width%3D%226%22%2F%3E%3C%2Fg%3E%3Cmask%20id%3D%22sidesCables02-a%22%20style%3D%22mask-type%3Aluminance%22%20maskUnits%3D%22userSpaceOnUse%22%20x%3D%2221%22%20y%3D%220%22%20width%3D%22138%22%20height%3D%2272%22%3E%3Cg%20fill%3D%22%23fff%22%3E%3Crect%20x%3D%2221%22%20y%3D%2227%22%20width%3D%2216%22%20height%3D%2222%22%20rx%3D%222%22%2F%3E%3Crect%20x%3D%2222%22%20y%3D%2260%22%20width%3D%2216%22%20height%3D%2212%22%20rx%3D%222%22%2F%3E%3Crect%20x%3D%22143%22%20y%3D%2242%22%20width%3D%2216%22%20height%3D%2222%22%20rx%3D%222%22%2F%3E%3Crect%20x%3D%22143%22%20width%3D%2216%22%20height%3D%2222%22%20rx%3D%222%22%2F%3E%3C%2Fg%3E%3C%2Fmask%3E%3Cg%20mask%3D%22url(%23sidesCables02-a)%22%3E%3Cpath%20d%3D%22M0%200h180v76H0V0Z%22%20fill%3D%22%23e53935%22%2F%3E%3Cpath%20d%3D%22M0%200h180v76H0V0Z%22%20fill%3D%22%23fff%22%20fill-opacity%3D%22.3%22%2F%3E%3C%2Fg%3E%3C%2Fg%3E%3Cg%20transform%3D%22translate(41)%22%3E%3Cmask%20id%3D%22topAntennaCrooked-a%22%20style%3D%22mask-type%3Aluminance%22%20maskUnits%3D%22userSpaceOnUse%22%20x%3D%2238%22%20y%3D%2212%22%20width%3D%2224%22%20height%3D%2240%22%3E%3Cg%20fill%3D%22%23fff%22%3E%3Cpath%20d%3D%22M55.54%2034.39%2051%2045h-3.74l4.92-10.44-6.05-10.43%203.22-11.84%202.9.8-2.9%2010.62%206.2%2010.68Z%22%2F%3E%3Cpath%20d%3D%22M38%2039h24v13H38z%22%2F%3E%3C%2Fg%3E%3C%2Fmask%3E%3Cg%20mask%3D%22url(%23topAntennaCrooked-a)%22%3E%3Cpath%20d%3D%22M0%200h100v52H0V0Z%22%20fill%3D%22%23e53935%22%2F%3E%3Cpath%20d%3D%22M0%206h100v52H0V6Z%22%20fill%3D%22%23fff%22%20fill-opacity%3D%22.3%22%2F%3E%3Cpath%20fill%3D%22%23fff%22%20fill-opacity%3D%22.2%22%20d%3D%22M38%2039h24v13H38z%22%2F%3E%3C%2Fg%3E%3Ccircle%20cx%3D%2250%22%20cy%3D%228%22%20r%3D%228%22%20fill%3D%22%23FFE65C%22%2F%3E%3Ccircle%20cx%3D%2253%22%20cy%3D%225%22%20r%3D%223%22%20fill%3D%22%23fff%22%2F%3E%3C%2Fg%3E%3Cg%20transform%3D%22translate(25%2044)%22%3E%3Cmask%20id%3D%22faceSquare01-a%22%20style%3D%22mask-type%3Aluminance%22%20maskUnits%3D%22userSpaceOnUse%22%20x%3D%220%22%20y%3D%220%22%20width%3D%22130%22%20height%3D%22120%22%3E%3Crect%20width%3D%22130%22%20height%3D%22120%22%20rx%3D%2218%22%20fill%3D%22%23fff%22%2F%3E%3C%2Fmask%3E%3Cg%20mask%3D%22url(%23faceSquare01-a)%22%3E%3Cpath%20d%3D%22M-2-2h134v124H-2V-2Z%22%20fill%3D%22%23e53935%22%2F%3E%3Cg%20transform%3D%22translate(-1%20-1)%22%3E%3Cpath%20d%3D%22M29.47%2034.03c-.1%200-.19-.06-.26-.12-.08-.07-.14-.12-.13.14.01.5.67-.05.4-.02ZM9.33%2048.92s.2-.56.03-.74c-.17.17-.18.42-.03.74ZM7.5%2052c.06%200%20.57-.28.48-.43-.12-.18-.49.34-.49.42ZM22.9%2059.14c-.13-.11-.53-.45-.28-.5.14-.03.5.36.32.54v-.01h-.01l-.03-.03ZM26.33%20108.46c.13-.02.66-.36.34-.45-.21-.04-.6.5-.34.46ZM52.63%2096.34c-.05.05-.2.09-.34.12-.15.04-.26.07-.16.1.19.06.84.16.5-.22ZM4.76%2043.63ZM5.1%2071.63l.06-.1c1.02%200-.68.19-.07.1ZM80.53%20109.92c.09-.02.1-.02.1-.11-.07.01-.08.02-.1.1ZM29.17%20117.78c.08-.02.09-.03.1-.11-.08.01-.09.02-.1.1ZM9.01%2049.75c.08-.02.09-.03.1-.12-.08.02-.09.03-.1.12ZM65.84%2048.74c.02.09.03.1.1.1%200-.08-.01-.09-.1-.1ZM107.28%2076.91c-.08.02-.09.03-.1.12.08-.02.09-.03.1-.12Z%22%20fill%3D%22%23000%22%20fill-opacity%3D%22.2%22%2F%3E%3C%2Fg%3E%3C%2Fg%3E%3C%2Fg%3E%3Cg%20transform%3D%22translate(52%20124)%22%3E%3Crect%20x%3D%2216%22%20y%3D%228%22%20width%3D%2244%22%20height%3D%224%22%20rx%3D%222%22%20fill%3D%22%23000%22%20fill-opacity%3D%22.8%22%2F%3E%3C%2Fg%3E%3Cg%20transform%3D%22translate(38%2076)%22%3E%3Cpath%20d%3D%22M96%202H8c-4.5%200-8%203.5-8%208.03V28c0%204.5%203.5%208%208%208h13c8%200%2011%208%2018%208h27c7%200%209-8%2017-8h13c4.5%200%208-3.5%208-8V10c0-4.5-3.5-8-8-8Z%22%20fill%3D%22%23000%22%20fill-opacity%3D%22.8%22%2F%3E%3Cpath%20d%3D%22M87%2014H17c-3.5%200-5%203-5%205v2c0%202%201.5%205%205%205h12c6%200%2011.62%208%2017%208h14c5.38%200%209-8%2015-8h12c3.5%200%205-3%205-5v-2c0-2-1.5-5-5-5Z%22%20fill%3D%22%23FF3D3D%22%2F%3E%3Cpath%20d%3D%22M22.44%2036.09%2037.26%202h11L31.4%2040.78l-.76-.58c-2.38-1.82-4.83-3.69-8.2-4.11ZM11.48%2036%2026.26%202h4L15.48%2036h-4Z%22%20fill%3D%22%23fff%22%20fill-opacity%3D%22.2%22%2F%3E%3C%2Fg%3E%3C%2Fg%3E%3C%2Fsvg%3E)
AI glossary
What is Softmax?
A function that turns a list of scores (logits) into probabilities that are all positive and sum to 1.
Every token a language model generates comes from a softmax over its vocabulary. Subtracting the maximum logit first keeps the computation numerically stable.