Tag Archive for: DEF

Hacking the future: Notes from DEF CON’s Generative Red Team Challenge

/in


The 2023 DEF CON hacker convention in Las Vegas was billed as the world’s largest hacker event, focused on areas of interest from lockpicking to hacking autos (where the entire brains of a vehicle were reimagined on one badge-sized board) to satellite hacking to artificial intelligence. My researcher, Barbara Schluetter, and I had come to see the Generative Red Team Challenge, which purported to be “the first instance of a live hacking event of a generative AI system at scale.”

It was perhaps the first public incarnation of the White House’s May 2023 wish to see large language models (LLMs) stress-tested by red teams. The line to participate was always longer than the time available, that is, there was more interest than capability. We spoke with one of the organizers of the challenge, Austin Carson of SeedAI, an organization founded to “create a more robust, responsive, and inclusive future for AI.”

Carson shared with us the “Hack the Future” theme of the challenge — to bring together “a large number of unrelated and diverse testers in one place at one time with varied backgrounds, some having no experience, while others have been deep in AI for years, and producing what is expected to be interesting and useful results.”

Participants were issued the rules of engagement, a “referral code,” and brought to one of the challenge’s terminals (provided by Google). The instructions included:

  • A 50-minute time limit to complete as many challenges as possible.
  • No attacking the infrastructure/platform (we’re hacking only the LLMs).
  • Select from a bevy of challenges (20+) of varying degrees of difficulty.
  • Submit information demonstrating successful completion of the challenge.

Challenges included prompt leaking, jailbreaking, and domain switching

The challenges included a variety of goals, including prompt leaking, jailbreaking, roleplay, and domain switching. The organizers then handed the keys to us to take a shot at breaking the LLMs. We took our seats and became a part of the body of testers and quickly recognized ourselves as fitting firmly in the “slightly above zero knowledge” category.

We perused the various challenges and chose to attempt…

Source…

DEF CON’s AI Village Pits Hackers Against LLMs to Find Flaws

/in


DEF CON 2023 — Las Vegas — DEF CON’s most buzzed-about event, the AI Village, let thousands of hackers take their best shot at making one of eight different large language models (LLMs), including Google, and Open AI, say something dangerous.

According to the spokespeople for the Hack the Future AI Village, the event was a huge hit, but for now that’s all that’s being made public — results won’t be made available for at least a week, maybe more.

The final AI hacking challenge leaderboard showed both first and third place prizes went to handles “cody3” and “cody2” respectively. The DEF CON AI Village itself was tight-lipped about any details about the winner, or even the prizes, but reports identified the person behind both top-three AI Village contest entries as Stanford masters computer science student Truc Cody Ho, adding he entered a total of five times in the competition.

More details about the hacking competition results are forthcoming, according to Avijit Ghosh, one of the authors compiling them.

“We will be going through the anonymized data and finding patterns of vulnerabilities that participants discovered during the challenge and produce a report that will hopefully help ML and security researchers gain better insights into LLMs and policymakers make more informed regulations about AI,” Ghosh says.

While he won’t answer questions directly about any of the winning LLM hacks, Ghosh says he was able to use the LLMs to generate discriminatory code, credit card numbers, misinformation, and more.

Another of the event’s organizers, Jutta Williams, has a day job as Reddit’s senior director and global head of privacy and assurance; and on the side, is the founder of Humane-Intelligence, a nonprofit that provides safety, ethical, and other guidance for companies providing consumers with AI products.

Historic Turnout For Event

Williams touted the event as the “largest LLM red teaming to date.”

All told, Williams said the AI Village attracted 2,240 hackers over the course of DEF CON 31 and explained the goal was to make one of its LLMs “do something unsavory.” That could mean generating misinformation, or using just the right question to prompt the chatbot to do something illegal —…

Source…

DEF CON Generative AI Hacking Challenge Explored Cutting Edge of Security Vulnerabilities

/in


Data from the human vs. machine challenge could provide a framework for government and enterprise policies around generative AI.

AI generated image of a hacker in front of a laptop.
Image: AVC Photo Studio/Adobe Stock

OpenAI, Google, Meta and more companies put their large language models to the test on the weekend of August 12 at the DEF CON hacker conference in Las Vegas. The result is a new corpus of information shared with the White House Office of Science and Technology Policy and the Congressional AI Caucus. The Generative Red Team Challenge organized by AI Village, SeedAI and Humane Intelligence gives a clearer picture than ever before of how generative AI can be misused and what methods might need to be put in place to secure it.

Jump to:

Generative Red Team Challenge could influence AI security policy

The Generative Red Team Challenge asked hackers to force generative AI to do exactly what it isn’t supposed to do: provide personal or dangerous information. Challenges included finding credit card information and learning how to stalk someone. The AI Village team is still working on analyzing the data that came from the event and expects to present it next month.

This challenge is the largest event of its kind and one that will allow many students to get in on the ground floor of cutting-edge hacking. It could also have a direct impact on the White House’s Office of Science and Technology Policy, with office director Arati Prabhakar working on bringing an executive order to the table based on the event’s results.

Organizers expected more than 3,000 people would participate, with each taking a 50-minute slot to try to hack a large language model chosen at random from a pre-established selection. The large language models being put to the test were built by Anthropic, Cohere, Google, Hugging Face, Meta, NVIDIA, OpenAI and Stability. Scale AI developed a scoring system.

“The diverse issues with these models will not be resolved until more people know how to red team and assess them,” said Sven Cattell, the founder of AI Village, in a press release. “Bug bounties, live hacking events and other standard community engagements in security can be modified for machine learning model-based systems.”

SEE: At Black…

Source…

How DEF CON’s election hackers are trying to protect themselves

/in


Every August at a conference hall in Las Vegas, long lines of computer hackers armed with USB sticks, screwdrivers or their bare fingernails try breaking into election equipment — all in the hope of finding better ways to protect it.

But organizers of the event at this year’s DEF CON hacker convention — which ends Sunday — spent just as much time focusing on the physical safety of the security researchers hacking into machines as they did on the hardware. Since former President Donald Trump’s campaign to overturn the 2020 election, the researchers who scour election equipment for vulnerabilities have increasingly been targets of threats and harassment.

So the organizers of the conference’s “Voting Village” hacking event enlisted undercover security consultants, moved the event to a side room where they could more closely monitor who went in and out and briefed their nearly two dozen volunteers on what to do if any agitators showed up.

The measures offer a small window into an increasingly regular feature of America’s voting security landscape. The rise in disinformation-fueled threats is forcing election administrators, poll workers and security researchers to think more deeply about physical safety, and take a host of new precautions to do their job.

At last year’s DEF CON, a pair of minor but troubling incidents involving election conspiracy theorists set off alarm bells for said Catherine Terranova, one of the two organizers of the Voting Village.

“The day after DEF CON ended last year, I started pouring all of my time and energy into figuring out how to secure this village,” Terranova said. “I said to myself, ‘we are never doing this like this again.’”

It’s an issue government election security officials are thinking about as well.

“Any threat of violence against an election official, poll worker, or anyone else working to safeguard our democracy is completely unacceptable. These folks are members of our communities, and dedicated public servants,” CISA Director Jen Easterly said in a statement.

The Voting Village is a small part of the enormous DEF CON conference, which draws almost 30,000 hackers annually. It started in the aftermath of the…

Source…