Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

It's actually an effect that happens in the (re-)alignment process due to harmonic properties of the positional encoding in the attention matrix.

(I recommend reading and implementing the Attention is all you need paper. By hand. Otherwise you won't learn anything from it.)



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: