Luca

Luca

@sciencialab.com

36 followers124 following65 posts

At #TPDL2026 in Faro ☀️: 🔹 Today, Michael Paris presents our Common Crawl paper on what web crawlers actually see 🔹 Thu afternoon I demo the INRIA DataLake: HAL papers → structured knowledge + software mentions Come say hi! #DigitalLibraries #CommonCrawlFoundation #Grobid

After releasing Grobid 0.9.1 we released the companion Python client optimised for processing PDF documents at scale. What's new: - Stream PDFs directly from ZIP archives — local or S3, no unzip needed - In-memory processing, lighter on disk I/O - Flexible file selection with glob patterns 1/2

🚀 GROBID 0.9.1 is out! 📄 Large PDF parsing (pdfalto 0.6.2): reduced bounded memory e.g. on large docs including dissertation of 1Gb size. 🔒 Security: command-injection, zip-slip, Jackson, OpenNLP 2.5 & Jetty CVEs fixed 📊 Monitoring: Prometheus/OTLP metrics + Grafana docs, 1/3

GROBID just passed 5,000 GitHub stars ⭐ From a 2008 hobby project to the engine behind Semantic Scholar, OpenAlex, ResearchGate, Internet Archive Scholar, CERN, and more. 18 years of steady open-source development. No hype, just reliable PDF→structured data at scale. 1/2

View full thread

Finally, I was positively surprised to see that such a large number of people using and talking about #Grobid. 3/4

Moreover, we met in person after working remotely for a few years. Tall people don't look that tall on video conference. 2/4

1/ 📣 Three GROBID community updates in one go: 📬 New mailing list 💬 New Discord server 🎨 Logo vote is open Details below 👇 1/5

View full thread

1/ 📣 Three GROBID community updates in one go: 📬 New mailing list 💬 New Discord server 🎨 Logo vote is open Details below 👇 1/5

To validate the data quality, we pre-trained SciLaD-M — a RoBERTa encoder trained from scratch on SciLaD. Result: performance comparable to or better than SciBERT across scientific NLP benchmarks, using only openly available data. Open data → open models. 💡 4/5

This builds on the foundational harvesting work by Patrice Lopez & James Howison (SoftCite project), and is a collaboration with @DFKI, @HUBerlin, @CommonCrawl & Uni Mannheim. Attending LREC? Let's connect!👋 #NLP #ScientificNLP #MultilingualNLP #SciLaD #ScienciaLAB #grobid 5/5

Scientific NLP needs large, open, high-quality data. SciLaD is our answer: a corpus of Open Access publications harvested from Unpaywall, converted into structured TEI XML using open-source tools. 35M+ multilingual publications across languages and scientific domains. 🌍 2/5

We also release a curated English split of 10M+ documents — filtered and deduplicated — along with the full open-source pipeline so anyone can reproduce or extend the dataset. No black boxes. Full transparency. 🔓 3/5

View full thread

4/ New pluggable NLP engines: Lingua for language identification, Blingfire for sentence segmentation — both available as drop-in alternatives to the existing defaults. 5/6

5/ Infrastructure: JDK 21, Gradle 9, TensorFlow 2.17 (Python 3.10–3.11), pdfalto 0.6.0, wapiti 1.5.1, virtualenv/conda support for DeLFT. 6/ Full release notes → github.com/kermitt2/... #GROBID #OpenSource #NLP #ScholarlyInfrastructure 6/6

github.comRelease 0.9.0 · grobidOrg/grobidWhat's Changed Added Conflict of interest and author contributions statement extraction in header and segmentation models #1319 Extract figures, tables and equations from back/annex sections #...
View full thread

Multi-architecture Docker images (amd64 + arm64) are now available, enabling native deployment on Apple Silicon and ARM cloud instances. 3/6

3/ Revised Crossref consolidation, developed in collaboration with the Crossref team — improved rate limit handling, better error recovery, and more robust reference matching. 4/6

1/ New extraction coverage: conflict of interest & author contribution statements; figures, tables and equations from back/annex sections; URLs from PDF annotations; ORCID identifiers fetched via Crossref when absent from the source document. 2/ Native Linux ARM64 support. 2/6

You found the paper. Now where's the BibTeX? 🧵 It happens every time! You track down the right reference — some older journal article, a conference paper, a preprint — scroll down to "Cite this paper", and get… a plain text citation. 1/7

View full thread

Results are automatically extracted, so always give the output a quick review before dropping it into your .bib file — especially on older or inconsistently formatted references. 6/7

You found the paper. Now where's the BibTeX? 🧵 It happens every time! You track down the right reference — some older journal article, a conference paper, a preprint — scroll down to "Cite this paper", and get… a plain text citation. 1/7

View full thread

Under the hood it's powered by #Grobid, a battle-tested machine learning library for extracting structured data from scientific documents. The same technology used to process millions of PDFs at scale — now doing one job really well, in one click. 5/7

You found the paper. Now where's the BibTeX? 🧵 It happens every time! You track down the right reference — some older journal article, a conference paper, a preprint — scroll down to "Cite this paper", and get… a plain text citation. 1/7

But the real magic is the bookmarklet. Drag it once to your bookmarks bar. Then, on any webpage — a journal site, a preprint server, a bibliography — highlight the citation, click the bookmarklet, and the BibTeX is already in your clipboard. 3/7

Luca reposted

@thomvaughan.bsky.social did a WCAG colour contrast audit of 240 top domains using Common Crawl's February 2026 archive. You can read more about his study in the thread below

Thom@thomvaughan.bsky.social

What can we measure about accessibility on the web using web archives? TL;DR Large-scale accessibility research can be conducted reproducibly from open crawl archives alone, and colours on the web are mostly quite ugly. thunderpoot.github.io/wcag-audit/ commoncrawl.org/blog/measuri...

Luca reposted

On the 26-27 November we held the #Grobid Camp at the Centre de #Inria Paris. The goal was to have a meeting with the major players in the French community which spaces from government institutes, to companies and large scale projects. 1/4

View full thread

It also enabled us to better coordinate our roadmap, integrating the benefits of advanced Large Language Models while preserving our commitment to enabling processing on standard consumer-grade hardware. 3/4

We tried to carry on without the sharp mind of Patrice Lopez, who is currently immersed in new and exciting challenges. And on a lighter note, we might finally have a logo for Grobid! 4/4

On the 26-27 November we held the #Grobid Camp at the Centre de #Inria Paris. The goal was to have a meeting with the major players in the French community which spaces from government institutes, to companies and large scale projects. 1/4

The meeting was extremely productive and helped us consolidate our community-driven vision for Grobid’s development in the coming years. 2/4

Exactly! There is a common misconception that by throwing any kind of crap into a vector it will magically work. Still at the age of AI, metadata information cannot still be ignored.

Danny Ki@dannykii.bsky.social

Vector search is good, but it's not enough. Metadata adds a layer of logic on top of your text chunks. This lets you filter by date or source first, then retrieve all the related pieces at once. #RAG #AI #VectorSearch