But there’s so many more dimensions to safety and security with AI - that’s what we’re trying to cover here.” “I’m sure there are other AI events that have happened, but they’ve probably been very targeted, like finding great prompt injection. “That’s what makes the scale of this unique,” he said.
Scale AI’s Levinson said that while others have run red-team exercises with one model, the scale of the challenge with so many testers and so many models becomes far more complex - as well as the fact that the organizers want to make sure to cover various principles in the AI Bill of Rights. The AI Village organizers said in a press release that they are bringing in hundreds of students from “overlooked institutions and communities” to be among the thousands who will experience the hands-on LLM red-teaming for the first time. “It’s very difficult to try to create the kinds of safeguards that will perform consistently across a wide range of issues,” she explained.
Rumman Chowdhury, cofounder of the nonprofit Humane Intelligence, which offers safety, ethics and subject-specific expertise to AI model owners, said in a media briefing that the AI companies providing their models are most excited about the kind of feedback they will get, particularly about the embedded harms and emergent risks that come from automating these new technologies at scale.Ĭhowdhury pointed to challenges focusing on multilingual harms of AI models: “If you can imagine the breadth of complexity in not just identifying trust and safety mechanisms in English for every kind of nuance, but then trying to translate that into many many languages - that’s something that is quite difficult thing to do,” she said.Īnother challenge, she said, is internal consistency of the models. AI companies seeking feedback on embedded harms The challenge also provides a capture-the-flag (CTF) style point system to promote testing a wide range of harms.Īnd there’s a not-too-shabby grand prize at the end: The individual who gets the highest number of points wins a high-end Nvidia GPU (which sells for over $40,000). The challenge will provide 150 laptop stations and timed access to multiple LLMs from the vendors - the models and AI companies will not be identified in the challenge. “This is really an exercise to identify what wasn’t known before - it’s that unpredictability and being able to say we never thought of that,” he said. “Every one of these companies develops their models in different ways - they have secret sauces.” But, he cautioned, the challenge is not a competition between the models. “in this context, what we’re trying to do is actually emulate behaviors that people might take and identify weaknesses in the models and how they work,” he explained. >Follow VentureBeat’s ongoing generative AI coverage<< DEF CON hackers seek to identify AI model weaknessesĪ red-teamer’s job is to simulate an adversary, and to do adversarial emulation and simulation against the systems that they’re trying to red team, said Alex Levinson, Scale AI’s head of security, who has over a decade of experience running red-teaming exercises and events. “Not only does it allow us to gather valuable feedback that can make our models stronger and safer, red-teaming also provides different perspectives and more voices to help guide the development of AI.” “Red-teaming has long been a critical part of deployment at OpenAI and we’re pleased to see it becoming a norm across the industry,” the spokesperson said.
0 kommentar(er)
