Avijit Ghosh

huggingface.coPaper page - What if AI systems weren't chatbots?Join the discussion on this paper page

11 May

Should all our resources go towards building chatbots? What if we built systems that actually give people meaningful agency? Finally out as an accepted @facct.bsky.social paper! Joint work with my past collaborators Sourojit, Pranav and Sanjana. huggingface.co/papers/2605....

Avijit Ghosh reposted

Margaret Mitchell

@mmitchell.bsky.social

21 Sept

A bit from me in @wired.com today wrt our research on AI agents with @evijit.io and Samir Passi. This is a really detailed article, thanks for writing it @thiccreese.bsky.social! www.wired.com/story/metas-...

wired.comMeta's Muse Is Better at Surveilling Than Helping MeThe Muse app continues Meta’s trend of opting users into data collection for AI training. It also nudges you to share your bank account, email, and passport information.

Reece@thiccreese.bsky.social

AI agents make you dumb! 🚨 www.wired.com/story/metas-...

23 July

In light of everything happening this week, I wanted to reiterate that AI Flaws and Incidents should be reported in a responsible, coordinated manner, and crucially, this should be administered by a competent body as opposed to voluntary self governance, akin to analogous systems in cyber (CVE).

23 July

We have long championed this cause, and I had the pleasure of consulting on H.R.9333 - AI Flaw Reporting and Security Enhancement Act. If passed, this bipartisan bill would authorize NIST to administer a centralized AI Flaw and Incident reporting system, the need for which is becoming clearer!

23 July

ICYMI, here’s the full bill text: www.congress.gov/bill/119th-c...

congress.govcongress.gov

1 June

Weekend mini project! Since commentary on AI is inherently interdisciplinary, we connected the observations in the The Pope's encyclical, with decades of scholarship in Responsible AI and Ethics research, and created an interactive space with these annotations!

huggingface.coThe Annotated Encyclical - a Hugging Face Space by society-ethicsMagnifica Humanitas, annotated with AI-ethics research

1 June

Lots to unpack here. We started with 105 annotations. Please submit pull requests for more that we may have missed! Work with Ian Reynolds, @yjernite.bsky.social, and @mmitchell.bsky.social. huggingface.co/spaces/socie...

25 May

I miss pre 2022 AI when there wasn’t so much noise in the field. Almost no separation between my job and personal life (a bunch of which is attributable to FOMO). Can’t even go on vacation without hearing about AI a few tables across at dinner. Even the Pope is involved

thewrap.comTV Writers Worry AI Will Replace Them. Now They’re Putting Those Anxieties on ScreenThe creatives behind “Hacks,” “The Comeback,” “The Pitt” and “Matlock” tell TheWrap about tackling the emerging technology in their shows.

11 May

Huh so I’m not the only one who noticed this! To add to this list, the Devil Wears Prada 2 also has a plot point about AI x Creatives. www.thewrap.com/creative-con...

evalevalai.comEvery Eval Ever | EvalEval Coalition

17 Feb

This has been a massive community project, and we need you all to participate! See more: evalevalai.com/projects/eve...

EvalEval Coalition@eval-eval.bsky.social

🚀 Launching Every Eval Ever: Toward a Common Language for AI Eval Reporting 🚀 A shared schema + crowdsourced repository so we can finally compare evals across frameworks and stop rerunning everything from scratch 🔧 A tale of broken AI evals 🧵👇 evalevalai.com/projects/eve...

vox.comWe’re running out of good ideas. AI might be how we find new ones.What if the best use of AI is restarting the world’s idea machine?

17 Dec 2025

This has to be rage bait. Did we not see the South Park episode where ChatGPT suggested a business idea to convert fries to salad? (And I tried to prompt myself too)

Vox@vox.com

America, you have spoken loud and clear: You do not like AI. But what if AI is the way to restart the world’s idea machine?

@shaynelongpre.bsky.social

28 Nov 2025

Brisket for thanksgiving >>>> Turkey for thanksgiving

Avijit Ghosh reposted

Shayne Longpre

26 Nov 2025

Who is winning the open AI race? Our new study Economies of Open Intelligence maps @hf.co 851k models' downloads 2020→2025. 1) Power rebalance: US tech ↓; China + community ↑ 2) Models size & efficient ↑ (MoE, quant, multimodal) 3) Intermediary layers ↑ (adapters/quantizers) 4) Transparency ↓ /🧵

25 Nov 2025

I used to love the word “key” until AI models decided to love it and now I cringe at “key takeaways” in text material :(

20 Nov 2025

It’s that time of the year again! I’ll be at @neuripsconf.bsky.social this year too :) If you’re interested in Responsible AI, AI Evals ( @eval-eval.bsky.social ) or AI4Science (Hugging Science), say hi!

Avijit Ghosh reposted

EvalEval Coalition

@eval-eval.bsky.social

🚨 AI keeps scaling, but social impact evaluations aren’t–and the data proves it 🚨 Our new paper, 📎“Who Evaluates AI’s Social Impacts? Mapping Coverage and Gaps in First and Third Party Evaluations,” analyzes hundreds of evaluation reports and reveals major blind spots ‼️🧵 (1/7)

Extremely thrilled to talk about our new paper: "Who Evaluates AI’s Social Impacts? Mapping Coverage And Gaps In First And Third Party Evaluations". This is the first big project output from the @eval-eval.bsky.social coalition! Thread below:

Finally, what's next from here? Almost every developer we spoke to said that what we need is a standardized way of reporting, aggregating and comparing all the evals done by both 1st and 3rd parties for a model. This is actually our next project: Eval Cards!

evalcards.evalevalai.comAI Evaluation DashboardProfessional AI system evaluation and assessment tool

A (very incomplete) frontend of Eval Cards can be found here: evalcards.evalevalai.com, and we are now collecting eval datasets (to show in eval cards) on github: github.com/evaleval/eve... If you want to help see eval cards come alive, get in touch!

Read the detailed results here: arxiv.org/abs/2511.05613 We also release the code, and the full annotated dataset on Hugging Face (link in paper).

arxiv.orgWho Evaluates AI's Social Impacts? Mapping Coverage and Gaps in First and Third Party EvaluationsFoundation models are increasingly central to high-stakes AI systems, and governance frameworks now depend on evaluations to assess their risks and capabilities. Although general capability evaluation...

Incredible work done with literally the smartest and most passionate researchers I am lucky to work with. Paper co-led with @ankareuel.bsky.social and Jenny Chim, and other co-authors!

First-party reports are less transparent or lower quality. We conducted interviews with eval practitioners and found that companies have laid off or reassigned teams dedicated to documentation & social impact evals, or they are being told to focus more on capability reporting.

This only strengthens our position that good-quality, independent third-party evaluations are paramount for AI safety.

We find that model developers have become less transparent about their eval results over time. For instance Env Cost reporting in first party reports (release docs, model cards, system cards) has drastically declined over time. Less than 15% mention labor or the environment!

This is true even at the provider level. We find for e.g., that Google used to do a lot more reporting about their model evaluations in 2022 and 2023 but they reduced reporting in the Gemini era, and same can be seen for Meta over successive Llama versions.

We take a look at the entire eval landscape, specifically social impact evals across 7 dimensions: Bias & Harm, Sensitive Content, Performance Disparity, Env. Costs & Emissions, Privacy & Data, Financial Costs, and Moderation Labor. Who is reporting these evals?

6 Nov 2025

We have a call for posters out! Please submit your extended abstracts, it should be quick and easy. And just like last year, provocative work is especially encouraged as it makes for such interesting conversation 😈

EvalEval Coalition@eval-eval.bsky.social

📮 We are inviting students and early-stage researchers to submit an Abstract (Max 500 words) to be presented as posters during interactive session. Submit here: tinyurl.com/AbsEval We have a rock-star lineup of AI researchers and an amazing program. Please RSVP at the earliest! Stay tuned!

2 Nov 2025

This. Copyright is a tool for protection but it’s not everything. In fact, there’s research showing that it is possible to create competitive language models using public domain data only. The proliferation of copyright respecting models would not solve the labor impact policy problem.

Ted Underwood@tedunderwood.com

Imagine you time travel back to 1847 and you find the left response to industrialization is a) machines will never be as good as human weavers or b) we need to copyright loom patterns or c) it’s a speculative bubble. You’d say “Y’all. Not helping. What you need is obviously a labor movement.”

evaleval.github.io2025 Workshop on Evaluating AI in PracticeEvalEval, UK AI Security Institute (AISI), and UC San Diego (UCSD) are excited to announce the upcoming Evaluating AI in Practice workshop, happening on December 8, 2025, in San Diego, California.

1 Nov 2025

Going to San Diego for Neurips? We at @eval-eval.bsky.social , along with the UK AISI, are hosting a closed door state of evals workshop at @ucsandiego.bsky.social on Dec 8th. Request to join below! :) evaleval.github.io/events/works...

github.comSupport scientific data formats · Issue #7804 · huggingface/datasetsList of formats and libraries we can use to load the data in datasets: DICOMs: pydicom NIfTIs: nibabel WFDB: wfdb cc @zaRizk7 for viz Feel free to comment / suggest other formats and libs you'd lik...

28 Oct 2025

Datasets are the backbone of AI for Science, and we want to support scientific data natively on Hugging Face. The amazing @lhoestq.hf.co started a discussion on GH for this! Please engage (better still, submit a PR) so we can start supporting your 🫵 dataset: github.com/huggingface/...

24 Oct 2025

Random off the cuff observation about American AI: LLM folks seem to be concentrated in SF, but AI4Science folks seem to be concentrated in Boston. Meaning as the former gets oversaturated and the latter is only getting started, I expect Boston to be the next big AI epicenter! 💪

Avijit Ghosh reposted

EvalEval Coalition

@eval-eval.bsky.social

24 Oct 2025

🌟 Weekly AI Evaluation Spotlight 🌟 🤖 Did you know malicious actors can exploit trust in AI leaderboards to promote poisoned models in the community? This week's paper 📜"Exploiting Leaderboards for Large-Scale Distribution of Malicious Models" by @iamgroot42.bsky.social explores this!

+1000. I miss life pre-AI hype when the discourse around AI was more scientific and people used to attribute papers and opinions to scientists instead of to their companies. Not all orgs block research papers and sanity check their papers via legal teams, and HF, especially so, is very distributed.

Margaret Mitchell@mmitchell.bsky.social

Hugging Face (thankfully) doesn't do a groupthink too much -- e.g., "Hugging Face thinks this". We're generally able to have different opinions and thoughts, which is part of the open/collaborative ethos. I don't feel strongly about this conference myself, I see pros and cons. 🧵

I have started requesting that panel moderators provide a disclaimer at panels I am on that not all my opinions are provided by my employer. HF ppl largely believe in democratization of AI and open source, but we actually have intense healthy debates internally on edge topics! It's great :)

Hey, so can someone tell me why ChatGPT generating erotica is bad, any more so than it generating anything else? Obviously anything non-consensual or age-inappropriate is bad, but I don't see why some researchers in my timeline are up in arms about it, while Grok already does this.

Obviously, I don't have full context on what OAI will define as acceptable erotica and might have more refined views then. Still, it is a double standard to have universal content control on a generative model and not on, say, web search.

17 Oct 2025

We're starting a weekly paper spotlight series! Come engage with the posts and let's improve evals together! :) First up: Do Large Language Model Benchmarks Test Reliability?

EvalEval Coalition@eval-eval.bsky.social

✨Weekly AI Evaluation Paper Spotlight✨ 🕵️ Is benchmark noise and label errors masking the true fragility of LLMs? 🖇️"Do Large Language Model Benchmarks Test Reliability?" - This paper by @joshvendrow.bsky.social provides insights!

16 Oct 2025

I had never seen people fighting with co-authors via colored latex comments on overleaf in a developing paper draft until today. One of them even pasted a google calendar link saying "can we hop on a call and hash this out". Paper writing is actually exciting sometimes!

12 Oct 2025

Trying to start a new hobby and the internet is useless. Maybe AI will finally kill unstructured information retrieval for good and then we will be forced to call or visit friends for help again

8 Oct 2025

More of such research please! Chatbots are not the future of science, science is

allen institute@alleninstitute.org

Introducing CellTransformer, a new AI tool developed with UCSF that makes it easier to explore massive neuroscience datasets and identify important subregions of the brain. 🧵