Fine-tuning for specific tasks. I'm hoping to see some good examples of that soon - the blog entry mentions things like structured text extraction, so maybe something like "turn this text about an event into an iCal document" might work?
Fine tuning messes with instruction following and RL'd behavior. I think this is mostly going to be useful for high volume pipelines doing some sort of mundane extraction or transformation.
I feel like the blog post, and GP comment, does a good job of explaining how it's built to be a small model easily fine tuned for narrow tasks, rather than used for general tasks out of the box. The latter is guaranteed to hallucinate heavily at this size, that doesn't mean every specific task it's fine tuned to would be. Some examples given were fine tuning it to efficiently and quickly route a query to the right place to actually be handled or tuning it to do sentiment analysis of content.
An easily fine tunable tiny model might actually be one of the better uses of local LLMs I've seen yet. Rather than try to be a small model that's great at everything it's a tiny model you can quickly tune to do one specific thing decently, extremely fast, and locally on pretty much anything.
This sounds like bro science. Having boring sessions is not the point, Z2 training is designed to build endurance, improve cardiovascular efficiency, and increase your body's ability to burn fat as a fuel source at moderate intensities. It’s not about enduring boredom or embracing “mental pain” but rather about consistently training at a level that is sustainable for extended periods.
I think they are a good middle ground, but you're still left with some of the busy work. Further, you're a bit more at the mercy of the maintainer. Likely, Lazyvim isn't going anywhere, but it isn't out of the realm of possibilities either.
I get frustrated seeing this go into the iPad and knowing that we can't get a shell, and run our own binaries there. Not even as a VM like [UserLAnd](https://userland.tech). I could effectively travel with one device less in my backpack but instead I have to carry two M chips, two displays, batteries, and so on...
It's great to see this tech moving forward but it's frustrating to not see it translate into a more significant impact in the ways we work, travel and develop software.
> A leaker has claimed that Apple is working on a version of macOS exclusive for the M2 iPad Pro ... the exclusivity to M2 iPad Pro could be a marketing push. If the feature is only available on that iPad, more people would buy it.
Based on the M4 announcement, vMacOS could be exclusive to the 1TB/2TB iPad Pro with 16GB RAM that would be helpful for VMs.
gpu passthrough for VMs is not supported on apple silicon period afaik. there may be some "native" renderer built on top of metal but apple doesn't support SR-IOV or "headless passthrough".
otoh no, it is not "more or less [automatic]" in other hardware either, SR-IOV has been on the enthusiast wishlist for a ridiculously long time now because basically nobody implements it (or, they restrict it to the most datacenter-y of products).
intel iGPUs from the HD/UHD Intel Graphics Technology era have a concept called GVT-g which isn't quite SR-IOV but generally does the thing. Newer Xe-based iGPUs do not support this, nor do the discrete graphics cards.
AMD's iGPUs do not have anything at all afaik. Their dGPUs don't even implement reset properly, which is becoming a big problem with people trying to set up GPU clouds for AI stuff - a lot of times the AMD machines will need a hard power reset to come back.
NVIDIA GPUs do work properly, and do implement SR-IOV properly... but they only started letting you do passthrough recently, and only 1 VM instance per card (so, 1 real + 1 virtual).
Curious what you're using (I'm guessing intel iGPU or nvidia dGPU) but generally this is still something that gets Wendell Level1techs hot and bothered about the mere possibility of this feature being in something without a five-figure subscription attached.
It does suck that Apple refuses to implement vulkan support (or sign graphics drivers), I think that's de-facto how people interact with most "hardware accelerated graphics" solutions in vmware or virtualbox, but SR-IOV is actually quite a rare feature, and "passthrough" is not sufficient here since the outer machine still needs to use the GPU as well. The feature point is SR-IOV not just passthrough.
This is not an embedding model though. Yes you can always extract some embeddings from somewhere, but for most LLMs those won't perform well for retrieval (which makes sense as it's not what the models are optimizing for)
This isn't an embedding model, but it is a group of people working in this general area in a language other than English. Maybe they'll get to an embedding model next?
reply