Common Crawl
The latest news and announcements from Common Crawl
- Website
- commoncrawl.org · all feeds from this site
- Feed URL
http://commoncrawl.org/feed/
Read it in your FeedListing reader
Or add to: Feedly Inoreader Raw feed
Latest posts
-
Host- and Domain-Level Web Graphs August, September, and October 2026
We are extremely pleased to announce the release of our October 2026 Web Graph, comprising data from August, September, and October 2026.
-
October 2026 Crawl Archive Now Available
We are pleased to announce the release of the October 2026 crawl archive, consisting of 2.16 billion pages (or 371.19 TiB of uncompressed content)
-
The Common Crawl Foundation at the TPDL Conference
The Common Crawl Foundation participated in the 30th TPDL (Theory and Practice for Digital Archives) conference in Faro, Portugal presenting two contributions on Crawl theory and on archival data enrichment.
-
Host- and Domain-Level Web Graphs July, August, and September 2026
We are absolutely thrilled to announce the release of our September 2026 Web Graph, comprising data from July, August, and September 2026.
-
September 2026 Crawl Archive Now Available
We are pleased to announce the release of the September 2026 crawl archive, consisting of 2.17 billion pages (or 361.4 TiB of uncompressed content.)
-
Getting Started with Common Crawl Data on Hugging Face
We made parts of our crawl archive available experimentally on a Hugging Face Storage Bucket. This blog post explains how to use the data on Hugging Face, with tools and examples on which you can build.
-
CommonsDB, ISCC, Spaghetti, and Meatballs
Notes from the CommonsDB final conference in Alicante: a registry of 6.5 million open works keyed on ISCC, and the EUIPO's push for federated copyright infrastructure.
-
Web Graph Embeddings: An Experimental Dataset Release
We are publishing an experimental dataset of graph embeddings built from the Common Crawl Web Graph: a 128-dimensional vector for each of 52.9 million web hosts, learned from hyperlinks alone, along with two interactive Hugging Face Spaces.
-
A Content Analysis of llms.txt Files from the July 2026 Crawl Archive
We analyzed the contents of 584,107 llms.txt files from the July 2026 crawl. Two thirds are produced by a plugin, half follow the structure the specification defines, and a few files even contain prompt injections.
-
Host- and Domain-Level Web Graphs June, July, and August 2026
We are pleased to announce a new release of host-level and domain-level web graphs based on the crawls of June, July, and August 2026, consisting of 235.0 million nodes and 3.1 billion edges at the host level, and 119.7 million nodes and 2.5 billion edges at the domain level.
-
August 2026 Crawl Archive Now Available
We are pleased to announce that the crawl archive for August 2026 is now available, containing 2.14 billion web pages or 360 TiB of uncompressed content.
-
Common Crawl Foundation at ACL 2026
The Common Crawl team attended the 64th Annual Meeting of the Association for Computational Linguistics in San Diego, California, presenting recent published work, and strengthening ties with the research community.
-
Announcing the First Stable Release of CC-Downloader
Over a year ago we released cc-downloader, an experimental tool to politely download Common Crawl data. Today we're releasing its first stable version, with a Rust library and Python bindings.