aiha-lab/BeaconKV: BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference What problem? CoT (Chain-of-Thought) brings memory bottleneck (K...
When you see a string like RG8gdSBrbm93IEJBU0U2ND8=, random letters and numbers with = at the end, you should guess it might be a Base64 string. How does it work? How to write you own BaseShittyThi...
We know that Discord has a sound board, which allows you to play various weird sounds in game. But sadly it doesn’t work in full screen game like ELDEN RING:NIGHTREGIN. I used to voice chat with m...
Reset to specific commit hash: git reset --hard <commit-hash> Force push to remote: git push --force origin <branch>
Agent System Trilemma performance ($i.e.$, task success and accuracy) efficiency ($i.e.$, time or steps required to complete tasks) cost ($i.e.$., computational and monetary resources consu...
这是我第一篇碎碎念博客，其实我偶尔会有记日记，但是这是我第一次发到网站上 大胆发邮件进组干杂活，在几个星期的无所事事（几乎）后，总算是把我扔个一个 Phd 手下了，而且我必须得说情况不是很好，教授似乎对本校硕士非常不满意，想全招 remote intern。我寻思着我还没开始呢就要被踢走吗，压力爆大（ 还有这教授最开始疑似对我期望过高（，给我画大饼，实际上我是纯区，教授发现我是区...
参考文章：Fast Inference from Transformers via Speculative Decoding Speculative Decoding Overview We have a slow, big target model $M_p$, inference from which we’re trying to accelerate (distribu...
Summary FreeKV, a training-free algorithm-system co-optimization framework that boosts the efficiency of KV retrieval, while maintaining near-lossless model accuracy across diverse scenarios. Why...
问各路 ai 半天也没怎么特别搞清楚，最后总算找到一个讲的不错的 🤪。 别问我怎么 KV Cache 都看不懂，问就是我很想知道具体细节操作是怎么个事，但是好多讲解都只讲个概念列列公式，很烦 😡。刚好这个教程里还附了 GPT2 的代码链接，可以过去瞅瞅怎么个事。 参考文章：看图学 KV Cache - 知乎 KV Cache 和注意力机制有关，首先介绍注意力机制. 对...
Here is the Puzzle: Nadya Nayme hsvvrp.zjsvzlsfo.paola.lluayhujlp..pzy.bolyl. Caesar → alooki.scloselyh.ithet.eentrancei..isr.uhere. → look closely the entrance is here → aishiteiru → https://a...