Would it be possible to run something like vLLM or TensortRT-llm with tinfoil? | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		tealcod 8 months ago \| parent \| context \| favorite \| on: Launch HN: Tinfoil (YC X25): Verifiable Privacy fo... Would it be possible to run something like vLLM or TensortRT-llm with tinfoil?

FrasiertheLion 8 months ago [–]

We’re already using vllm as our inference server for our standard models. We can run whatever inference server for custom deployments.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact