<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Token Reduction | About Hyeongheon</title><link>https://chahh9808.github.io/tags/token-reduction/</link><atom:link href="https://chahh9808.github.io/tags/token-reduction/index.xml" rel="self" type="application/rss+xml"/><description>Token Reduction</description><generator>Hugo Blox Builder (https://hugoblox.com)</generator><language>en-us</language><lastBuildDate>Tue, 06 Oct 2026 04:54:06 +0000</lastBuildDate><image><url>https://chahh9808.github.io/media/icon_hu68170e94a17a2a43d6dcb45cf0e8e589_3079_512x512_fill_lanczos_center_3.png</url><title>Token Reduction</title><link>https://chahh9808.github.io/tags/token-reduction/</link></image><item><title>Later Is Better: Token Reduction for ViTs Under Distribution Shift</title><link>https://chahh9808.github.io/post/later-is-better/</link><pubDate>Tue, 06 Oct 2026 04:54:06 +0000</pubDate><guid>https://chahh9808.github.io/post/later-is-better/</guid><description>&lt;h2 id="position">Position&lt;/h2>
&lt;p>First author&lt;/p>
&lt;h2 id="project-goals--works">Project Goals &amp;amp; Works&lt;/h2>
&lt;ol>
&lt;li>Study how the depth of token removal affects vision-transformer robustness under distribution shift.&lt;/li>
&lt;li>Use a one-parameter power-law schedule to preserve tokens in early layers and concentrate removal in later layers.&lt;/li>
&lt;li>Match the compute budget of a uniform removal schedule without retraining or input-specific tuning.&lt;/li>
&lt;/ol>
&lt;h2 id="key-results">Key Results&lt;/h2>
&lt;ol>
&lt;li>On ImageNet-C at severity 5, improved DeiT-S with ToMe by 1.17 percentage points over uniform removal at matched compute, while using about 26% less compute than the unreduced model.&lt;/li>
&lt;li>Demonstrated improvements across five token-reduction methods and nine vision-transformer backbones.&lt;/li>
&lt;li>Combined the schedule with six test-time adaptation methods, with further evaluations on video and vision-language tasks.&lt;/li>
&lt;/ol>
&lt;h2 id="links">Links&lt;/h2>
&lt;ul>
&lt;li>&lt;a href="https://chahh9808.github.io/publication/later-is-better/">Publication details&lt;/a>&lt;/li>
&lt;li>&lt;a href="https://arxiv.org/abs/2610.07758">arXiv&lt;/a>&lt;/li>
&lt;li>&lt;a href="https://arxiv.org/pdf/2610.07758">Paper PDF&lt;/a>&lt;/li>
&lt;li>&lt;a href="https://github.com/chahh9808/LaterIsBetter">Code&lt;/a>&lt;/li>
&lt;/ul></description></item><item><title>Later Is Better: Token Reduction for ViTs Under Distribution Shift</title><link>https://chahh9808.github.io/publication/later-is-better/</link><pubDate>Tue, 06 Oct 2026 04:54:06 +0000</pubDate><guid>https://chahh9808.github.io/publication/later-is-better/</guid><description/></item></channel></rss>