arXiv.org / 2025

Smoothed Embeddings for Robust Language Models

Ryo Hase, Md. Rafi Ur Rashid, Ashley Lewis, Jing Liu, T. Koike-Akino, K. Parsons, Ye Wang

AI SafetyFoundation ModelsGenerative AILarge Language Models

Improving the safety and reliability of large language models (LLMs) is a crucial aspect of realizing trustworthy AI systems. Although alignment methods aim to suppress harmful content generation, LLMs are often still vulnerable to jailbreaking attacks that employ adversarial inputs that subvert alignment and induce harmful outputs. We propose the Randomized Embedding Smoothing and Token Aggregation (RESTA) defense, which adds random noise to the embedding vectors and performs aggregation during the generation of each output token, with the aim of better preserving semantic information. Our experiments demonstrate that our approach achieves superior robustness versus utility tradeoffs compared to the baseline defenses.

8 citations0 influential

Full paper

Read the original paper

Source page

A direct open-access PDF is not available in the database yet. Use the source page or learning resources below to open the complete paper from the publisher or index.

Learning resources

Google Scholar referencesGoogle Scholar Papers with Code searchPapers with Code Semantic Scholar paper pageSemantic Scholar YouTube explanationsYouTube

Reading state

Discuss in ChatGPT

Uses your own ChatGPT account. The paper context is copied into a tutor prompt before ChatGPT opens.

Preview prompt

You are my AI/ML research paper instructor. I want to deeply understand the paper below.

First, teach it in layers:
1. One-paragraph intuition.
2. Problem statement and why it mattered.
3. Key method, architecture, or algorithm.
4. Important equations or mechanisms, explained intuitively.
5. Experiments and evidence.
6. Limitations, assumptions, and failure modes.
7. How this paper influenced later AI/ML/Deep Learning/GenAI work.
8. A 30-minute study plan with checkpoints.
9. Quiz me with 5 questions and wait for my answers.

When something is not available in the attached context, say what is missing and infer carefully.

### Paper attached as context
Title: Smoothed Embeddings for Robust Language Models
Authors: Ryo Hase, Md. Rafi Ur Rashid, Ashley Lewis, Jing Liu, T. Koike-Akino, K. Parsons, Ye Wang
Year: 2025
Venue: arXiv.org
Categories: AI Safety, Foundation Models, Generative AI, Large Language Models
Citations: 8
Paper URL: https://www.semanticscholar.org/paper/e4529b1e1458ea8ac5af96837757f3457e1e2817
Open PDF: Not available

Abstract:
Improving the safety and reliability of large language models (LLMs) is a crucial aspect of realizing trustworthy AI systems. Although alignment methods aim to suppress harmful content generation, LLMs are often still vulnerable to jailbreaking attacks that employ adversarial inputs that subvert alignment and induce harmful outputs. We propose the Randomized Embedding Smoothing and Token Aggregation (RESTA) defense, which adds random noise to the embedding vectors and performs aggregation during the generation of each output token, with the aim of better preserving semantic information. Our experiments demonstrate that our approach achieves superior robustness versus utility tradeoffs compared to the baseline defenses.

Learning resources:
- Google Scholar: Google Scholar references (https://scholar.google.com/scholar?q=Smoothed%20Embeddings%20for%20Robust%20Language%20Models)
- Papers with Code: Papers with Code search (https://paperswithcode.com/search?q=Smoothed%20Embeddings%20for%20Robust%20Language%20Models)
- Semantic Scholar: Semantic Scholar paper page (https://www.semanticscholar.org/paper/e4529b1e1458ea8ac5af96837757f3457e1e2817)
- YouTube: YouTube explanations (https://www.youtube.com/results?search_query=Smoothed%20Embeddings%20for%20Robust%20Language%20Models+paper+explained)