Ash Lewis¶
Co-founder and CEO of fastino (building task-specific small models as an alternative to large general-purpose LLMs). Previously ran DevGPT (built on GPT-3; "went viral… then we spent $2 million on OpenAI's system, which wasn't so great").
Grounded quotes¶
- "Choosing the right model is really hard… once it's in production maintaining the model accuracy is even harder."
- "It's really hard to actually fine-tune these models. Not because it's necessarily difficult to do, but because it's time consuming. It's very difficult to get the data to fine-tune the models, which — it turns out — you have that data. It's in your inference logs, but we just don't get it from the LLM providers."
- Pioneer thesis: "deploying an open-source language model is actually the first step as opposed to that being kind of the end state."
- On the outcome: "you end up with the situation in which you've deployed Llama and it's actually getting more accurate over time as opposed to the alternative, which would be that because of model drift it's actually getting less accurate over time."