My original idea was to make a modern supercharged version of SAM, the old-school text-to-speech. Then I remembered Pink Trombone, one of my favorite creative coding projects. In the end, I decided to just make it a model that mimics speech - it would be easy to make a text-to-speech by chaining an existing TTS with a Samuel filter.The phonetics work quite well for vowels - you can try making a sound and then seeing if the predicted tongue prediction matches where your tongue was. It's less accurate for consonants; it doesn't do plosives or nasals (I tried).How the machine learning works:It's a speech autoencoder in which the decoder is Pink Trombone. A couple of issues: PT is not differentiable, so I use Gumbel softmax to get a differentiable approximation. PT also generates audio sample-by-sample (apparently it's called a "Kelly-Lochbaum vocal tract model"), which would make training extremely slow. To do that, I approximate it using FIR filters that change each 10ms. The fundamental...