This visualizer explores non-uniform resampling of Bézier curves, driven by two distinct importance metrics. The Sharpness resampling prioritizes points in regions of high curvature, approximated by the second derivative, which focuses samples on visually complex sections. In contrast, the Attention resampling uses the total attention weight received by each point within a self-attention mechanism as its importance score, concentrating samples on points identified as contextually significant based on their relationship with all other points.
To calculate the non-uniform sample points, which are `t` values from 0 to 1, the chosen importance scores, either sharpness or attention scores, are treated like a discrete Probability Density Function (PDF). Areas with high scores have higher density. We then compute the Cumulative Distribution Function (CDF) by taking the normalized cumulative sum of these scores. This CDF maps the original uniform parameter `t` to the accumulated importance. To get the new `t` values, we invert this process: we sample uniformly along the CDF's output range from 0 to 1 and find the corresponding input `t` values by looking up or interpolating on the empirical CDF. This effectively clusters the new `t` values in regions where the original importance scores were high.
The self-attention process calculates how strongly each point influences others. Here, we've modified the standard calculation: a bias based on calculated sharpness, scaled by the 'Sharpness Emphasis Factor', is added to raw attention scores before the final activation function is applied. This injection of local geometric properties into the attention computation guides the mechanism to assign higher importance to points in sharp turns.
Note: This demonstration illustrates the mechanics of self-attention and resampling. It does not involve training a neural network or using gradient descent to optimize weights. The entire app contents and code were generated by a large language model using the authors agent. README for more, or contact the author: folkstack@gmail.com.