Eli Chien
LightNews LightNews
banner
elichien.bsky.social
Eli Chien
@elichien.bsky.social
52 followers 97 following 33 posts
Incoming assistant professor at National Taiwan University. Postdoc at GeorgiaTech. Ph.D. from UofIllinois. Focus on privacy + graph learning. #MachineUnlearning #DifferentialPrivacy #DP #GNN #LLM

Homepage: https://sites.google.com/view/eli-chien/home
Posts Replies Media Videos
Pinned
elichien.bsky.social
Eli Chien @elichien.bsky.social · Apr 9
Life Update: I am happy to share the news that I will be an Assistant Professor at the National Taiwan University EE department! I am very grateful for this opportunity to be back in my home country, especially at the university where I was an undergrad! 1/3
elichien.bsky.social
Some random thoughts after chatting with multiple friends: I do feel that one reason the general ML research community is getting worse (imo, maybe not for others) is that we don't share the bad things we found with others more often. 1/n
August 16, 2025 at 4:20 AM
elichien.bsky.social
I will be at #icml2025 next week to present our work on LLM unlearning evaluation [https://arxiv.org/abs/2412.08559]. We also have a work on AI copyright to be presented at the MemFM and R2FM workshop. Please let me know if you're also around! I will be around 7/15-7/17.
July 8, 2025 at 5:12 AM
elichien.bsky.social
This is a great paper! It resonate with one of our recent work (a short version to appear at ICML MemFM workshop!). We really need to be careful on "defining meaningful" copyright measure.
dppapers.bsky.social Differential Privacy Papers @dppapers.bsky.social · Jun 26
Blameless Users in a Clean Room: Defining Copyright Protection for Generative Models

Aloni Cohen

http://arxiv.org/abs/2506.19881
Blameless Users in a Clean Room: Defining Copyright Protection for Generative Models Aloni Cohen http://arxiv.org/abs/2506.19881 Are there any conditions under which a generative model's outputs are guaranteed not to infringe the copyrights of its training data? This is the question of "provable copyright protection" first posed by Vyas, Kakade, and Barak (ICML 2023). They define near access-freeness (NAF) and propose it as sufficient for protection. This paper revisits the question and establishes new foundations for provable copyright protection -- foundations that are firmer both technically and legally. First, we show that NAF alone does not prevent infringement. In fact, NAF models can enable verbatim copying, a blatant failure of copy protection that we dub being tainted. Then, we introduce our blameless copy protection framework for defining meaningful guarantees, and instantiate it with clean-room copy protection. Clean-room copy protection allows a user to control their risk of copying by behaving in a way that is unlikely to copy in a counterfactual clean-room setting. Finally, we formalize a common intuition about differential privacy and copyright by proving that DP implies clean-room copy protection when the dataset is golden, a copyright deduplication requirement.
June 26, 2025 at 5:41 AM
elichien.bsky.social
What needs to be take care of when applying privacy amplification by iteration to zeroth-order optimization? Can it even be done? What's the "good design" for DP zeroth-order method? Check out our latest work! It's so nice to collaborate with Wei-Ning (as usual) and Pan!
dppapers.bsky.social Differential Privacy Papers @dppapers.bsky.social · Jun 4
Privacy Amplification in Differentially Private Zeroth-Order Optimization with Hidden States

Eli Chien, Wei-Ning Chen, Pan Li

http://arxiv.org/abs/2506.00158
Privacy Amplification in Differentially Private Zeroth-Order Optimization with Hidden States Eli Chien, Wei-Ning Chen, Pan Li http://arxiv.org/abs/2506.00158 Zeroth-order optimization has emerged as a promising approach for fine-tuning large language models on domain-specific data, particularly under differential privacy (DP) and memory constraints. While first-order methods have been extensively studied from a privacy perspective, the privacy analysis and algorithmic design for zeroth-order methods remain significantly underexplored. A critical open question concerns hidden-state DP analysis: although convergent privacy bounds are known for first-order methods, it has remained unclear whether similar guarantees can be established for zeroth-order methods. In this work, we provide an affirmative answer by proving a convergent DP bound for zeroth-order optimization. Our analysis generalizes the celebrated privacy amplification-by-iteration framework to the setting of smooth loss functions in zeroth-order optimization. Furthermore, it induces better DP zeroth-order algorithmic designs that are previously unknown to the literature.
June 4, 2025 at 5:49 PM
Reposted by Eli Chien
dppapers.bsky.social
Open Problem: Selection via Low-Sensitivity Queries
Open Problem: Selection via Low-Sensitivity Queries
Two of the basic tools for building differentially private algorithms are noise addition for answering low-sensitivity queries and the exponential mechanism for selection. Could we do away with the e...
differentialprivacy.org
May 2, 2025 at 1:21 PM
elichien.bsky.social
Our paper about LLM unlearning evaluation is accepted by #icml2025 !

Thanks to the leading author Rongzhe, and my collaborators
@mufei-li.bsky.social @xiangyue96.bsky.social
(and others may not be on Bluesky).

It's my first "last" author paper. Feels quite special :p 1/n
May 1, 2025 at 2:08 PM
elichien.bsky.social
I wonder how well this result can be applied to convert the KL-based result in the sampling literature (i.e., LMC convergence) to Renyi divergence, compared to those results that directly bound the Renyi divergence (i.e., the results in Sinho Chewi's book or the paper by Vempala and Wibisono 😂)
stein.ke Thomas Steinke @stein.ke · Apr 26
You can bound Rényi divergences in terms of KL divergences for tilted distributions. This is useful e.g. for Gaussians, where tilting just corresponds to shifting the distribution.
LaTeX source: https://pastebin.com/m9RwBcCi
April 27, 2025 at 1:06 AM
Reposted by Eli Chien
stein.ke
I wrote a post explaining why, in practice, privacy amplification by subsampling doesn't quite work as well as promised. This is a significant problem for differentially private machine learning applications, but I don't know if this is as widely known as it should be.
April 21, 2025 at 7:22 PM
Reposted by Eli Chien
mraginsky.bsky.social
PSA — if you’re interested in learning about statistical aspects of optimal transport, check out this new monograph by Sinho Chewi, Jonathan Niles-Weed, and Philippe Rigollet: link.springer.com/book/10.1007...
Statistical Optimal Transport
This monograph aims to offer a concise introduction to optimal transport, quickly transitioning to its applications in statistics and machine learning.
link.springer.com
April 14, 2025 at 11:03 PM
Reposted by Eli Chien
dppapers.bsky.social
Privacy Amplification by Subsampling
Privacy Amplification by Subsampling
Privacy Amplification by Subsampling is an important property of differential privacy. It is key to making many algorithms efficient – particularly in machine learning applications. Thus a lot of wor...
differentialprivacy.org
April 13, 2025 at 4:15 PM
elichien.bsky.social
Life Update: I am happy to share the news that I will be an Assistant Professor at the National Taiwan University EE department! I am very grateful for this opportunity to be back in my home country, especially at the university where I was an undergrad! 1/3
April 9, 2025 at 7:54 PM
elichien.bsky.social
I am so shocked to learn that Poisson (in French) means fish...... As a person who constantly deals with Poisson distribution, Poissonization, etc I now have a completely different feeling about Poisson 🤣. I guess we always learn something unexpected on the internet 🤣
April 9, 2025 at 7:52 PM
elichien.bsky.social
I will give a talk at GaTech CSE seminar this Friday on the topic: "Machine Unlearning: The General Theory and LLM Practice of Privacy".

Please join if you are around :)

cse.gatech.edu/events/2025/...
School of CSE Seminar Series: Eli Chien | School of Computational Science and Engineering
School of CSE hosts a seminar from Georgia Tech Postdoctoral Fellow Eli Chien
cse.gatech.edu
March 23, 2025 at 7:23 PM
elichien.bsky.social
I am glad to share that our paper on hidden-state Noisy SGD DP analysis for non-convex non-smooth problems has been accepted at #ICLR2025! I really appreciate the effort from reviewers, AC, and all my friends who provided valuable comments and feedback!
January 22, 2025 at 7:39 PM
Reposted by Eli Chien
gautamkamath.com
With @adamsmith.xyz and @thejonullman.bsky.social, we have compiled a set of profiles of 29 people in the "foundations of responsible computing" community ("mathematical research in computation and society writ large") who are on the faculty job market.

Link: drive.google.com/file/d/1Hyvg... 1/3
December 24, 2024 at 7:50 PM
elichien.bsky.social
Why do we need "theoretical guarantees" for trustworthy AI? We need to prevent the worst-case scenario, where theory in AI truly shines and is necessary, in my opinion. That's also why my work with theoretical guarantees for machine unlearning and DP matters! 😉
December 20, 2024 at 4:53 AM
elichien.bsky.social
It's my great pleasure to contribute to the great A3D3 community. Congrats to all #A3D3 members!
December 19, 2024 at 8:55 PM
elichien.bsky.social
The last time when I attended NeurIPS in 2019 Vancouver, I missed my flight back to Urbana due to a border check. Today after NeurIPS 2024 I got stuck in Dallas due to a flight cancellation...🥲🥲🥲
December 17, 2024 at 6:46 AM
elichien.bsky.social
I will present 1 spotlight in the noon and 1 poster in the evening tomorrow, both in West building about unlearning theory! Feel free to come and chat if you’re around 😀 #NeurIPS2024
December 11, 2024 at 11:02 PM
Reposted by Eli Chien
dppapers.bsky.social
Privacy-Preserving Retrieval Augmented Generation with Differential Privacy
Tatsuki Koga, Ruihan Wu, Kamalika Chaudhuri
http://arxiv.org/abs/2412.04697
Privacy-Preserving Retrieval Augmented Generation with Differential Privacy Tatsuki Koga, Ruihan Wu, Kamalika Chaudhuri http://arxiv.org/abs/2412.04697 With the recent remarkable advancement of large language models (LLMs), there has been a growing interest in utilizing them in the domains with highly sensitive data that lies outside their training data. For this purpose, retrieval augmented generation (RAG) is particularly effective -- it assists LLMs by directly providing relevant information from the external knowledge sources. However, without extra privacy safeguards, RAG outputs risk leaking sensitive information from the external data source. In this work, we explore RAG under differential privacy (DP), a formal guarantee of data privacy. The main challenge with differentially private RAG is how to generate long accurate answers within a moderate privacy budget. We address this by proposing an algorithm that smartly spends privacy budget only for the tokens that require the sensitive information and uses the non-private LLM for other tokens. Our extensive empirical evaluations reveal that our algorithm outperforms the non-RAG baseline under a reasonable privacy budget of $\epsilon\approx 10$ across different models and datasets.
December 9, 2024 at 4:34 AM
elichien.bsky.social
[NeurIPS 2024]

I will be at NeurIPS this year to present our works on privacy! Two papers on the theory of machine unlearning (including a spotlight) and one on DP-PageRank with hidden-state DP analysis! Let me know if you are also around and would like to chat about privacy/trustworthy AI :)
December 3, 2024 at 3:10 PM
Reposted by Eli Chien
ccanonne.github.io
On the TCS job market: Eli Chien!

Eli works on TCS∩ML: e.g., Machine Unlearning, hidden-state DP analysis and their application to graph ML. He recently generalized the celebrated shifted-divergence analysis to non-convex non-smooth setting for hidden-state DP-SGD.

1/2 #TCSSky #AcademicJobMarket
November 26, 2024 at 8:09 PM
elichien.bsky.social
🤔: How's your lemma different from the lemma of Chien et al.?
Me thinking: You are asking the right person 😆
(after some explanation)
Me: We agree that there are some similarities and we will give Chien et al. more credit in our revision. 🤣
November 21, 2024 at 3:50 AM