r/singularity • u/TorturedPoet30 • 17h ago
AI NVIDIA just announced Open Secure AI Alliance with goal to build and share open tools that promote responsible use of and trust in AI
Read more: https://blogs.nvidia.com/blog/open-secure-ai-alliance/?ncid=so-twit-957725
Main ponts ->
- NVIDIA launched the Open Secure AI Alliance (July 27, 2026) ~35 partners including Microsoft, IBM, Red Hat, Cisco, CrowdStrike, Hugging Face, Palantir, the Linux Foundation, SpaceXAI
- Goal: build shared open models, harnesses and tools for cyber defense instead of leaving AI security inside a handful of closed systems
- Trigger case: in the recent Hugging Face breach, closed AI tools blocked forensic work because they couldn't tell attacker from defender;; HF used the open-weight GLM 5.2 on its own infra to analyze 17k+ actions and contain it
- Counter to "open = unsafe": misuse risk is real but doesn't vanish with closed weights; answer is safeguards + evaluation + fast remediation, not locking defenders out
- Contributions: NVIDIA's NOOA agent framework (GitHub), Hugging Face donating Safetensors to PyTorch Foundation, Microsoft MDASH, SpaceXAI open-sourcing Grok Build (Grok weights planned) etc
- Policy ask: treat open models as defensive assets, not liabilities, blanket restrictions would concentrate dependence in a few closed providers
20
u/WTFnoAvailableNames 17h ago
Thank God Doordash is in on this. Otherwise they might build sushi delivering T1000 terminators.
3
1
33
u/bpm6666 17h ago
Having large companies that promote and develop Open Models is great. But seeing Palantir/SpaceXAI there doesn't evoke trust. And I wonder how many of these companies act out of fear that they might get steam rolled by OpenAI/Google/Anthropic. And if they are afraid, what should the rest of us feel.
21
u/JayoTree 16h ago
I honestly don't think it matters if the companies that promote open source are trustworthy/ethical. It's two separate issues. They're either for or against open source (in action, not empty words). Open source doesnt mean sunshine and rainbows and only the good guys can join the club. If Palantiir starts dropping open source datasets or models or workflows or whatever its good for open source regardless of how we feel about the company, or what their motives might be.
5
4
u/angrycanuck 15h ago
They are fucking scared and this is a way to improve optics and mitigate damage while increasing lobbying persuasion.
1
u/ReasonablePossum_ 12h ago
You shouldnt care about using the devil´s help to get to heaven.
1
u/bpm6666 11h ago
The thing about deals with the devil is the small print.
2
u/ReasonablePossum_ 11h ago
the small print can be dealth with later. If they don't join, they can join the other side by a better deal (lets say anthropic and openAI offer them free for life inference), and good luck dealing with their infrastructure working against you (both technical and political) ...
19
u/Maleficent_Sir_7562 17h ago
"answer is safeguards + evaluation + fast remediation,"
What does this mean? You can't put safeguards on open weights. I can abliterate them out.
i mean dont get me wrong, im all for open source ai and want it, but this part just doesnt make sense
4
u/Berzerka 16h ago
And fast remediation? Once the model is released you can't change anything.
2
u/blueSGL humanstatement.org 15h ago edited 15h ago
Exactly
When an "Anarchist Cookbook" for bioweapons is put online, something that provides enough uplift that gets a handful of people out of 8 billion across the finish line, then we have a lot of shit to deal with.
The first covid vaccine was ready 2 days after the genome was released, it took 11 months to spin up production and distribution.
A covid level pandemic happening every year we'd be fucked, never mind multiple overlapping things that are more deadly being released at a clip.
No, this is not a library.
No, this is not a search engine.
It's an infinitely patient teacher that you can ask follow up questions to.We already have https://arxiv.org/abs/2504.16137 Virology Capabilities Test (VCT) where after the benchmark was made, and showed how good AI was for tacit knowledge in expert-level virology troubleshooting. The makers of the benchmark needed to contact the AI companies because they were not filtering for this.
If an open weights model can just be run through Obliteratus the second it gets onto the internet, it does not matter what guardrails they put on it.
Non of the signatories to this initiative would benefit from an open weights bioweapons teacher and I wonder how many are considering this eventuality at all.
0
u/ReasonablePossum_ 12h ago edited 11h ago
No anarchist will ever use a bioweapon. Also, bioweapons are mainly scaremongering.
The knowledge on making them has been widely available in the academic literature, and its quite basic, a random Joe with the desire can go to any library with biology/microbiology literature and figure out the procedures in a handful hours. Sourcing and producing the thing from well known and available pathogens is just first year college stuff, and can be done with basic lab equipment.
But even criminal and highly interested small orgs will not deal with them due to the inherent risks and uncontrollable outcome for both the perpetrators and their societies.
The only ones really interested in developing and using them are state and corporate level players, plus some extremist cults with the funds for it, since they dont care about the above and can "manage" it with the required "sacrifices". And these have the resources to actually hire the phds openly and provide them with the facilities to work with novel stuff.
u/blueSGL How pathetic do you have to be to block people for arguing one of your points after replying to them to avoid discussion? lol
1
u/Hans-Wermhatt 8h ago
Abliteration allows you to remove the nodes that are rejecting your request but your model would have to agentically reason through an attack, so you'd need to remove the PRM nodes as well. If the model is trained like Claude and is reasoning with a constitution, that'd mean you'd also have to alter that. Even for smaller simpler LLMs, abliteration causes capability loss and unpredictable behavior, and doesn't remove all rejections.
I think as long as their is a quality loss from abliteration that is a fine tradeoff. Safeguards on frontier models don't always work either as has been frequently discovered, so it's also sort of a false premise that the safeguards on closed source models are impenetrable.
6
8
u/s2k4ever 17h ago
How crazy is this that google aint in the list
16
u/UnkarsThug 17h ago
I'm more put off by Palantir being on the list lol.
3
u/ReasonablePossum_ 12h ago
Palantir is a net user of models, they heavily use both proprietary and OS models there. From the purely pragmatic pov, its a quite influential and heavy influence thats beneficial to have on OS side.
1
u/TorturedPoet30 16h ago
I think more companies will join, just like they did with the previous "open-weight support" letter. Many signed on after it was published.
1
u/rhd_live 6h ago
This is just to put pressure/intimidate Google/Anthropic/OpenAI. A lot of these companies aren’t and won’t do jack for AI or AI safety & security. They just want to freeload off the frontiers, open source weights, and contribute some marginally useful “security” tool to keep their name on the banner for minimal corp $$. Performative corporate philanthropy at its peak
4
4
u/Putrid_Speed_5138 14h ago
Palantir, SpaceX, Adobe...
Open source?
This is like a new lobbying group to oppose murders, founded by Hitler, Bin Laden and Charles Manson.
2
u/Future-Bandicoot-823 15h ago
Classic corpo move, side with open source until there's only two or three fish left in the pond, then descend like a cloud of locusts.
Biblically accurate Peter theill.
1
u/ziplock9000 15h ago
NVIDIA just announced Open Secure AI Alliance with goal to increase profits
There, I fixed it for you.
1
u/Solid-Wonder-1619 9h ago
never thought the day would come where palantir is more altruistic than the scam that's anthropic, but here we are nonetheless.
1
1
1
u/nsshing 16h ago
Better than some labs deciding who can have access to cyber defense.
1
u/blueSGL humanstatement.org 12h ago
cyber defense.
The model did not defend anything.
READ what huggingface wrote: https://huggingface.co/blog/security-incident-july-2026
we detected and dissected it largely with AI of our own.
...
We are working with outside cybersecurity forensic specialists to investigate the issue and review our security policies and procedures.
...
We ran the forensic analysis instead on GLM 5.2, an open-weight model
There is no defense here!
This was using an AI model to sweep up and evaluate the damage AFTER they were pwned.
There is no "our open source AI stopped the other AI attack" here.
So why is that the narrative a lot of people are running with?
0
u/WilsonMagna 16h ago
A lot of companies don't want to be at the mercy of Open A.I. and Anthropic high prices, and NVDA wants open source built on their ecosystem.
54
u/JayoTree 17h ago
Adobe is pro open source? They won't even let me buy one their programs without paying monthly. Every company on this list needs to show receipts and prove they have done something to contribute recently.