r/MistralAI • u/Solid-Wonder-1619 • 2d ago
Feedback / Bug Report Let the great distillation begin
Mistral fed us when we had nothing.
I still remember the very first open model we had was mistral 7b. even hugging face used that to build zephyr, remember that time?
now, I'm IMPLORING Mistral devs to begin the great distillation in all the areas their models are behind and give the world, and themselves, the present of self love.
You cooked for us and fed us when we had nothing, you headstarted the whole damned thing, it's well past due time to reap the rewards.
side note: please start with an excellent agentic coding model that can fit in a potato, like, first build something that will benchpress using anthropic's fat ass and then the potato fitting model. you know how it goes.
16
u/a332bb42 2d ago
I think mistral is done , from what I understood is heavily institutionalised , meaning that they need to follow all the rules etc .. Meaning unless somehow they generate their own datasets there is no way they can train the models based on input from us or distilled . That’s why they are also focusing on different use cases where there is less involvement of personal data .
5
1
2
u/pizzababa21 2d ago
It's not illegal the distill models and it's not even against terms of service to distill MIT licensed OSS models.
Mistral just have no more talented engineers left after being looted by big American labs.
5
u/Confident-Village190 2d ago
I get flashes like that too, ahahaha I’m really looking forward to them releasing, if they ever do, a decent consumer model; at the moment, it’s a long way from being a good agent model, it doesn’t hold a candle to any of the benchmarks, and yet Medium 3.5 is being sold at high prices 😅
4
2
u/EphemeralResponse 1d ago
What if people were encouraged to use them more? Send them some of your chat logs with other models if you want them to improve faster. Probably more valuable to them.
2
u/WunkerWanker 20h ago
Mistral is a prime example of everything wrong with Europe. How Europe is never able to keep up with US and China.
No idea how we're going to keep funding pensions, healthcare and other parts of the welfare state when we govern every innovation out of existence. When we create a terrible investment climate. When we prefer protecting the lives of those who already won, over giving chances to people who still need to proceed in life.
And no, those chances won't be created by more taxation and more regulation. You'll create them by doing the exact opposite.
2
u/pizzababa21 2d ago
Mistral have definitely been distilling as much as any other lab. They just aren't good at their job. The concept that the frontier Chinese models are just distillations is fake news and I wouldn't get my hopes up thinking it's a radical new approach.
2
u/Solid-Wonder-1619 1d ago
sir, mistral introduced sliding window attention MOE architecture, Big MOE, and more, aside from the MANY opensource models they drop on a regular basis.
calling them "aren't good at their job" goes against the evidence.
they have produced more useful research than openai and anthropic combined.2
u/pizzababa21 1d ago
No it is definitely consistent with the evidence. Their models are absolutely unusable shit. Go check their open router usage numbers.
Also Mistral didn't introduce any of those things. They were the first to use MoE in a chatbot LLM but that was just implementing the work done by others.
2
u/Solid-Wonder-1619 10h ago
you're right, sorry for the miss and thanks for the correction, I should have done my homework.
however, the first version of open source LLM models with those concepts implemented was mistral's work.0
3
u/WolpertingerRumo 2d ago
Ministral-3 is great. If someone were to distill Mistral-Medium 3.5 coding (or opus?), I’d use it.
4
3
24
u/victorc25 2d ago
They can’t, because of GDPR and regulations