Tag Archive for: Challenge

Hacking the future: Notes from DEF CON’s Generative Red Team Challenge

/in


The 2023 DEF CON hacker convention in Las Vegas was billed as the world’s largest hacker event, focused on areas of interest from lockpicking to hacking autos (where the entire brains of a vehicle were reimagined on one badge-sized board) to satellite hacking to artificial intelligence. My researcher, Barbara Schluetter, and I had come to see the Generative Red Team Challenge, which purported to be “the first instance of a live hacking event of a generative AI system at scale.”

It was perhaps the first public incarnation of the White House’s May 2023 wish to see large language models (LLMs) stress-tested by red teams. The line to participate was always longer than the time available, that is, there was more interest than capability. We spoke with one of the organizers of the challenge, Austin Carson of SeedAI, an organization founded to “create a more robust, responsive, and inclusive future for AI.”

Carson shared with us the “Hack the Future” theme of the challenge — to bring together “a large number of unrelated and diverse testers in one place at one time with varied backgrounds, some having no experience, while others have been deep in AI for years, and producing what is expected to be interesting and useful results.”

Participants were issued the rules of engagement, a “referral code,” and brought to one of the challenge’s terminals (provided by Google). The instructions included:

  • A 50-minute time limit to complete as many challenges as possible.
  • No attacking the infrastructure/platform (we’re hacking only the LLMs).
  • Select from a bevy of challenges (20+) of varying degrees of difficulty.
  • Submit information demonstrating successful completion of the challenge.

Challenges included prompt leaking, jailbreaking, and domain switching

The challenges included a variety of goals, including prompt leaking, jailbreaking, roleplay, and domain switching. The organizers then handed the keys to us to take a shot at breaking the LLMs. We took our seats and became a part of the body of testers and quickly recognized ourselves as fitting firmly in the “slightly above zero knowledge” category.

We perused the various challenges and chose to attempt…

Source…

DEF CON Generative AI Hacking Challenge Explored Cutting Edge of Security Vulnerabilities

/in


Data from the human vs. machine challenge could provide a framework for government and enterprise policies around generative AI.

AI generated image of a hacker in front of a laptop.
Image: AVC Photo Studio/Adobe Stock

OpenAI, Google, Meta and more companies put their large language models to the test on the weekend of August 12 at the DEF CON hacker conference in Las Vegas. The result is a new corpus of information shared with the White House Office of Science and Technology Policy and the Congressional AI Caucus. The Generative Red Team Challenge organized by AI Village, SeedAI and Humane Intelligence gives a clearer picture than ever before of how generative AI can be misused and what methods might need to be put in place to secure it.

Jump to:

Generative Red Team Challenge could influence AI security policy

The Generative Red Team Challenge asked hackers to force generative AI to do exactly what it isn’t supposed to do: provide personal or dangerous information. Challenges included finding credit card information and learning how to stalk someone. The AI Village team is still working on analyzing the data that came from the event and expects to present it next month.

This challenge is the largest event of its kind and one that will allow many students to get in on the ground floor of cutting-edge hacking. It could also have a direct impact on the White House’s Office of Science and Technology Policy, with office director Arati Prabhakar working on bringing an executive order to the table based on the event’s results.

Organizers expected more than 3,000 people would participate, with each taking a 50-minute slot to try to hack a large language model chosen at random from a pre-established selection. The large language models being put to the test were built by Anthropic, Cohere, Google, Hugging Face, Meta, NVIDIA, OpenAI and Stability. Scale AI developed a scoring system.

“The diverse issues with these models will not be resolved until more people know how to red team and assess them,” said Sven Cattell, the founder of AI Village, in a press release. “Bug bounties, live hacking events and other standard community engagements in security can be modified for machine learning model-based systems.”

SEE: At Black…

Source…

Computer scientist confronts worldwide challenge of online security and privacy – News Center

/in



Thursday, Apr 06, 2023
• Herb Booth :
Contact

A University of Texas at Arlington computer security researcher has received a prestigious federal grant to determine what technologies and methods work best to attain and retain online security and privacy.

Shirin Nilizadeh
Shirin Nilizadeh

Shirin Nilizadeh, assistant professor in the Department of Computer Science and Engineering, received a $200,000 National Science Foundation grant to study social media discussions and better understand what concerns are about online security and privacy, what technologies and tools they suggest to each other to use and whether they are effective. Nilizadeh called this a “worldwide challenge.”

“People care about their online security and privacy everywhere,” she said. “And sometimes, due to societal and political movements, they become more cautious or aware of the problems, where they go online and on social media, and proactively discuss their concerns and ask for tools and methods that can help protect them.

“We can help as a research community to see what’s working and what isn’t. We can take these research findings to design and develop better online safeguards and to improve the existing security and privacy-preserving systems if they are not secure, effective and efficient.”

Hong Jiang, chair of the Department of Computer Science and Engineering, said Nilizadeh’s research could further the security of social network tools.

“Everyone is connected to social networks,” Jiang said. “Studying social networks’ discussions and understanding what security measures people are looking for and using allow researchers to develop and provide such measures to improve online security and privacy.”

Previous Nilizadeh work showed that social media users extensively discussed the security and privacy threats of video communication tools more people started working from home due to the COVID-19 pandemic. This work showed how misinformation about security and privacy spread on social media platforms.

Nilizadeh…

Source…

In-orbit satellite Moonlighter features in Hack-A-Sat 4 security challenge

/in


On-orbit satellite Moonlighter features in Hack-A-Sat 4 space security challenge

The annual event, which is run by the U.S. Air Force Research Laboratory (AFRL), is offering the winner a cash prize of $50,000.

Hack-A-Sat, now in its fourth iteration, is described as an opportunity for hackers, researchers and space enthusiasts to focus their thinking on solving space systems cyber security challenges.

“What we are trying to do is build trust, and doing it in a way that is cool and attracts the best talent to work on real-world cyber security issues for space security,” said Steve Colenzo, a computer scientist at the AFRL Information Directorate.

“This type of competitive environment brings together the hacker and research communities to take on challenges we are facing with future capabilities in space and provide critical data and information for building out more resilient capabilities for our warfighters.”

Format

Specifically, registration was opened for the virtual qualification round of the competition, running over the weekend. The teams will have competed in a jeopardy-style format, earning points based on speed and accuracy

The top five teams from the qualification event will advance to the Hack-A-Sat 4 finals in the Aerospace Village at DEF CON 31, an annual hacker’s convention in Las Vegas, running 11-13 August 2023.

Teams will then compete in an attack/defend style capture-the-flag competition that incorporates communication with an actual in-orbit satellite named Moonlighter.

The top three ranked teams at the final event will receive $50,000 for first place, $30,000 for second place and $20,000 for third place.

Moonlighter

The Moonlighter satellite, right, was designed to advance security researcher knowledge and skills in securing space systems. It is scheduled to launch in early summer 2023 ready for the Hack-A-Sat finals.

“Moonlighter is an excellent example of industry-government collaboration and a major step forward in our quest to advance space cybersecurity,” said Col. Kenny Decker, division chief, Integration and Futures Directorate at the U.S. Space Force. “It is purpose-built to provide security researchers access and opportunities like Hack-A-Sat to gain experience with…

Source…