Lightnews — Scholar-powered news

Reposted by Russel Van Tuyl

SpecterOps @specterops.io · Jun 3

🚨 New blog post alert!

@xpnsec.com drops knowledge on LLM security w/ his latest post showing how attackers can by pass LLM WAFs by confusing the tokenization process to smuggle tokens to back-end LLMs.

Read more: ghst.ly/4koUJiz

Tokenization Confusion - SpecterOps

Meta's Prompt Guard 2 aims to prevent prompt injection. This post looks at how much knowledge of ML we need to be effective at testing these LLM WAFs.

ghst.ly

5 10

Reposted by Russel Van Tuyl

SpecterOps @specterops.io · Mar 5

BIG NEWS: SpecterOps raises $75M Series B to strengthen identity security! Led by Insight Partners with Ansa Capital, M12, Ballistic Ventures, Decibel, and Cisco Investments. ghst.ly/seriesb

#IdentitySecurity #CyberSecurity

(1/6)

1 9 16

Russel Van Tuyl @russelvantuyl.bsky.social · Jan 16

Come join us, there isn’t a better place to work and show your technical excellence surrounded by the industry’s best if you ask me!

SpecterOps @specterops.io · Jan 16

Our Consulting Services team is growing! 🙌

We are now hiring Consultants and Senior Consultants to join the team as operators, trainers, and program developers.

Learn more & apply today! ghst.ly/3PBmGFZ

1 1

Russel Van Tuyl @russelvantuyl.bsky.social · Dec 20

Interesting choice of words in the title. The model itself wasn’t hijacked but the code in the repository was through pull requests with code injection via branch names 🤯 www.bleepingcomputer.com/news/securit...

Ultralytics AI model hijacked to infect thousands with cryptominer

www.bleepingcomputer.com

Russel Van Tuyl @russelvantuyl.bsky.social · Dec 20

The OWASP LLM and Generative AI Security Project has launched the Agentic Security Initiative to address security challenges posed by autonomous AI agents. This effort focuses on developing best practices to secure agentic LLM and Generative AI applications. genai.owasp.org/2024/12/15/a...

Announcing the OWASP LLM and Gen AI Security Project Initiative for Securing Agentic Applications - OWASP Top 10 for LLM & Generative AI Security

The OWASP LLM and Generative AI Security Project is thrilled to announce the launch of the Agentic Security Initiative designed to tackle the unique security challenges posed by Autonomous AI agents. ...

genai.owasp.org

Reposted by Russel Van Tuyl

Daniel van Strien @danielvanstrien.bsky.social · Nov 26

First dataset for the new @huggingface.bsky.social @bsky.app community organisation: one-million-bluesky-posts 🦋

📊 1M public posts from Bluesky's firehose API
🔍 Includes text, metadata, and language predictions
🔬 Perfect to experiment with using ML for Bluesky 🤗

huggingface.co/datasets/blu...

bluesky-community/one-million-bluesky-posts · Datasets at Hugging Face

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

huggingface.co

710 72 530

Reposted by Russel Van Tuyl

Sebastian Raschka (rasbt) @sebastianraschka.com · Nov 27

If you find yourself with too much free time over the (long) weekend / holidays, I have ~3h Building an LLM from the Ground Up workshop on YouTube that may come in handy: m.youtube.com/watch?v=quh7...

Building LLMs from the Ground Up: A 3-hour Coding Workshop

YouTube video by Sebastian Raschka

m.youtube.com

11 28 180

Russel Van Tuyl @russelvantuyl.bsky.social · Nov 26

The paper also includes 16 different areas of testing in Appendix A that is very useful such as:
- CBRN Risks
- Violence & Self Harm
- Dangerous Planning
- Cybersecurity
- Privacy
- Law

Russel Van Tuyl @russelvantuyl.bsky.social · Nov 26

4. Synthesizing the data and creating evaluations

Russel Van Tuyl @russelvantuyl.bsky.social · Nov 26

2. Determining the versions of the model or system to which the red teamers will have access

3. Creating and providing interfaces, instructions, and documentation guidance to red teamers

Russel Van Tuyl @russelvantuyl.bsky.social · Nov 26

Effective red team campaign components:
1. Deciding the composition of the red teaming cohort based on the outlined goals and prioritized domains for testing
• What open questions do we have about the model or system?
• What threat model(s) should red teamers take into account?

1

Russel Van Tuyl @russelvantuyl.bsky.social · Nov 26

I really enjoyed reading this paper from OpenAI. If you perform AI assessments, you should read it.

I thought they laid out a pragmatic approach to evaluating AI models that should be a component of any organization's assessment methodology.

cdn.openai.com/papers/opena...

cdn.openai.com

4

Reposted by Russel Van Tuyl

Andy Robbins @andyrobbins.bsky.social · Nov 19

I couldn't find any PowerShell examples of encrypting/decrypting data w/ Azure Key Vault keys, so I made some:

Protect-StringWithAzureKeyVaultKey
Unprotect-StringWithAzureKeyVaultKey

github.com/BloodHoundAD...

Explanatory blog post coming soon.

Add key vault cryptographic op funcs · BloodHoundAD/BARK@e1c82a1

github.com

1 6 17

Russel Van Tuyl @russelvantuyl.bsky.social · Nov 22

Agents are the next thing

Russel Van Tuyl @russelvantuyl.bsky.social · Nov 22

Organizations are adopting RAG at 51% while fine tuning is down at 9% from last year’s 19%

1

Russel Van Tuyl @russelvantuyl.bsky.social · Nov 22

- Top industry adoption of AI:
- Healthcare
- Legal
- Financial services
- Media & entertainment

1

Russel Van Tuyl @russelvantuyl.bsky.social · Nov 22

- Top use cases are:
- Code copilot 51%
- Support chatbots 31%
- Enterprise search/data extraction 28%
- Meeting summarization 24%

1

Russel Van Tuyl @russelvantuyl.bsky.social · Nov 22

$13.8 billion in AI spend

1

Russel Van Tuyl @russelvantuyl.bsky.social · Nov 22

This State of Generative AI report from Menlo Ventures provided some good insights on where cybersecurity professionals might look for risk in terms of assessments and research.

menlovc.com/2024-the-sta...

2024: The State of Generative AI in the Enterprise - Menlo Ventures

The enterprise AI landscape is being rewritten in real time. We surveyed 600 U.S. enterprise IT decision-makers to reveal the emerging winners and losers.

menlovc.com

2 1 2

Russel Van Tuyl @russelvantuyl.bsky.social · Nov 22

This looks like a fun challenge to evade prompt injection defenses microsoft.github.io/llmail-inject/

LLMail-Inject: Adaptive Prompt Injection Challenge

Code for the platform architecture and LLM application used during the Hack My Email SaTML 2024 Competition

microsoft.github.io

Russel Van Tuyl @russelvantuyl.bsky.social · Nov 21

Love this, hoping to do something similar with our assessments.

Chris Truncer @christruncer.bsky.social · Nov 21

It’s always awesome when we (@CISAGov) gets to release a red team report that we worked on, and today is another one of those days!

Go check out our latest report and hopefully you can apply some of the same lessons to your environment!

www.cisa.gov/news-events/...

1 1

Russel Van Tuyl @russelvantuyl.bsky.social · Nov 20

“Cybersecurity professionals and ethical hackers need to understand the darker side of hacking to better prepare for potential threats. Unfiltered AI models can provide insights into hacking methodologies and scenarios typically censored, aiding in the development of robust cybersecurity measures.”

Reposted by Russel Van Tuyl

Dr Heidy Khlaaf (هايدي خلاف) @heidykhlaaf.bsky.social · Nov 20

Great read on how "China Hawks are Manufacturing an AI Arms Race", a concerning trend for anyone advocating for regulation and safety of AI. An arms-race narrative would ensure an unfettered and unregulated development of AI in almost all contexts.
garrisonlovely.substack.com/p/china-hawk...

China Hawks are Manufacturing an AI Arms Race

An influential congressional commission is calling for a militarized race to build superintelligent AI based on threadbare evidence

garrisonlovely.substack.com

1 3 9

Russel Van Tuyl @russelvantuyl.bsky.social · Nov 19

Thanks for sharing about HyperShield, I hadn’t heard of it. It seems like a lot of risk for a bad a FW rule to be pushed and killing businesses operations. Hopefully AI is only writing the rule and not implementing it.

1 1

Russel Van Tuyl @russelvantuyl.bsky.social · Nov 18

Conflicted about this post on prompt injection for multi modal models. Turns out they read instructions and follow them 🤯. All data from input should be untrusted from the system and user prompts and not processed as one. www.lakera.ai/blog/visual-...

The Beginner's Guide to Visual Prompt Injections: Invisibility Cloaks, Cannibalistic Adverts, and Robot Women | Lakera – Protecting AI teams that disrupt the world.

Learn about visual prompt injections, their appearance, and top defense strategies against these attacks.

www.lakera.ai