Is speculative decoding possible across 1/4/12/27 B Gemma 3 variants? LM Studio ... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		werediver 11 months ago \| parent \| context \| favorite \| on: Gemma 3 Technical Report [pdf] Is speculative decoding possible across 1/4/12/27 B Gemma 3 variants? LM Studio doesn't allow that (yet), but maybe the s/w requires some adjustments to support speculative decoding with Gemma 3.

pinglin 10 months ago [–]

It's reported working but not with LM Studio: https://www.reddit.com/r/LocalLLaMA/comments/1j9reim/comment...

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact