Hello! I'm Arka, a PhD student in Computing and Information Sciences at the Rochester Institute of Technology, advised by Prof. Ashiqur R. KhudaBukhsh in the Social Insight Lab. My research is on evaluating and post-training large language models, with a focus on safety and trustworthiness. I build evaluations that expose where models fail: they cave to adversarial nudges and hallucinate, carry hidden bias against social groups, and infer sensitive attributes like race from text that never states them. My current work turns those failures into post-training methods that make models safer. I've published 10+ papers at ACL, EMNLP, AAAI, IJCAI, COLM, and ICWSM, and Race in the Record received an Outstanding Paper Award at EMNLP 2026 (about the top 25 of 17,000+ submissions). CNN, WIRED, ABC News, Good Morning America, The Conversation, and the Daily Mail have covered the research. I also love mathematics, especially combinatorics, optimization, and probability, and it shapes how I think about measurement.
Research Featured In
News
- 2026Race in the Record won an Outstanding Paper Award at EMNLP 2026 — roughly the top 25 papers out of 17,000+ submissions. Read the paper →
- Oct '26Our EMNLP 2026 paper Physiognomy Revisited featured in The Conversation and the Daily Mail — how AI models alter faces to look stereotypically gay, straight, or criminal.
- 2026Paper from my Amazon internship accepted at the REALM workshop @ EMNLP 2026 — a session-level study of whether LLM shopping agents can be told apart from human shoppers by their behavior.
- 2026Paper accepted at EMNLP 2026 (Main) — Race in the Record: measuring racial signal and model inference in police arrest narratives.
- 2026Paper accepted at EMNLP 2026 (Main) — Physiognomy Revisited: how vision–language models encode sensitive attributes through image transformation.
- Summer '26Mentored three undergraduate researchers through the NSF REU 2026 program at RIT — two resulting projects are now under review. Details →
- Summer '26Joining Amazon as an Applied Scientist II Intern in Seattle, on the Buyer's Risk Prevention team.
- 2026Paper accepted at ACL 2026 (Main) — HAUNT gives a leakage-free evaluation benchmark for LLM self-consistency and Hallucination.
- 2026Paper accepted at ACL 2026 (Findings) — auditing LLM responses to harmful stereotypes targeting mental-health groups.
- 2026Paper accepted at AAAI 2026 — a framework to bias-audit LLMs using internal beliefs.
- 2026Paper accepted at ICWSM 2026 — a large-scale social-web audit of AI-generated-text detectors.
- 2025Our antisemitism-in-LLMs work covered by CNN; bias-audit research featured in WIRED.
- 2025Featured on Good Morning America and ABC News for research on AI-generated hateful content.
- 2025Paper accepted at COLM 2025 and IJCAI 2025.
- 2024Received the RIT Language Science Student Excellence Award; paper accepted at IJCAI 2024.
Research Themes
My work runs along two strands that continually feed each other — measuring social phenomena in real-world text, and stress-testing the models that increasingly generate it. They share a common core: bias & fairness, human–AI interaction, and responsible AI.
Computational Social Science
Using NLP to study society at scale — from de-escalating conflict to auditing fairness and the provenance of online content.
-
Hope speech & peace
A dataset of 10,081 posts shows vicarious, bipartisan perspectives help model de-escalating language between nuclear adversaries.
Bipartisan Peace · IJCAI 2025 -
Bias & fairness in society
Measures how much racial signal is present in — and inferable from — police arrest narratives, and what models do with it.
Race in the Record · EMNLP 2026 -
AI-content provenance
A large-scale social-web audit of AI-generated-text detectors, testing how reliably they hold up in the wild.
AI-Text Audit · ICWSM 2026
LLM Evaluation
Stress-testing large language models to reveal where they break — probing bias, reliability, and factual consistency under adversarial pressure.
-
Bias & fairness in models
A "rabbit hole" framework iteratively elicits toxic content across 1,266 identity groups, exposing where guardrails quietly fail.
Toxicity Rabbit Hole · IJCAI 2024Closet Antisemite · AAAI 2026 -
Reliability under attack
Shows that simple spacing-based jailbreaks slip past guardrails and reveal the biases lying beneath them.
S P A C E Jailbreak · AAAI 2025 -
Factuality & self-consistency
HAUNT nudges a model against its own answers in closed domains, surfacing when it stays consistent and when it caves.
HAUNT · ACL 2026
Selected Publications
-
Race in the Record: Measuring Racial Signal and Model Inference in Police Arrest Narratives
Introduces a corpus of 3,091 police use-of-force narratives and shows that race stays recoverable after redaction, that LLMs write longer narratives for White than Black subjects from identical seeds, and that the signal re-emerges in generated images.
-
A three-step framework that turns a model against itself to reveal when it caves to subtle conversational nudges and fabricates confident falsehoods.
ACL 2026 · Main ACL Anthology -
Iteratively elicits increasingly toxic content from an LLM, exposing how guardrails degrade with depth across 1,266 identity groups.
IJCAI 2024 Proceedings -
A framework that surfaces latent antisemitic bias in LLMs by measuring how their responses diverge under targeted probes.
AAAI 2026 Proceedings -
Builds entity networks from LLM-generated attack narratives to trace how bias against mental-health groups emerges and spreads.
COLM 2025 PDF -
Towards a Bipartisan Understanding of Peace and Vicarious Interactions
Shows that vicarious, cross-national perspectives help model de-escalating "hope speech" between nuclear adversaries.
IJCAI 2025 Proceedings
Skills
Selected Achievements
- Outstanding Paper Award, EMNLP 2026 — for Race in the Record, roughly the top 25 papers out of 17,000+ submissions.
- NSF REU Mentorship (2026) — mentored three undergraduate researchers at RIT (UT Austin, Texas A&M, UCLA), leading to two projects currently under review.
- RIT Language Science Student Excellence Award (2024) — one of two recipients among 1000+ students university-wide, for research in responsible AI and NLP systems.
- Common Admission Test (CAT, 2022) — 98.97 percentile among 100k+ candidates, securing admission to the PGP at IIM Indore (top-100 globally, Financial Times).
- Regional Mathematics Olympiad (2018) — qualified for the RMO (West Bengal); advanced aptitude in combinatorics, number theory, algebra, geometry, and optimization.
- Jagadish Bose National Science Talent Scholarship (2017) — awarded for excellence in science.