Repositories list

• python-crfsuite

  Public

  scrapinghub/python-crfsuite’s past year of commit activity
  A python binding for crfsuite

  data-sciencecrfcrfsuite

  Python

  •

  MIT License

  •224•772•46•3•Updated Dec 23, 2025Dec 23, 2025

• shub-workflow

  Public
  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •14•15•2•1•Updated Dec 16, 2025Dec 16, 2025

• web-poet

  Public

  scrapinghub/web-poet’s past year of commit activity
  Web scraping Page Objects core library

  pythonweb-scrapinghacktoberfest+ 1page-objects

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •18•104•16•11•Updated Dec 15, 2025Dec 15, 2025

• andi

  Public
  Library for annotation-based dependency injection

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •6•24•4•1•Updated Dec 9, 2025Dec 9, 2025

• marathon-apps-collectd-plugin

  Public

  scrapinghub/marathon-apps-collectd-plugin’s past year of commit activity
  marathon-apps-collectd-plugin

  Python

  •

  GNU General Public License v2.0

  •96•2•0•1•Updated Nov 27, 2025Nov 27, 2025

• shub

  Public
  Scrapinghub Command Line Client

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •80•130•46•13•Updated Nov 6, 2025Nov 6, 2025

• scrapinghub-entrypoint-scrapy

  Public

  scrapinghub/scrapinghub-entrypoint-scrapy’s past year of commit activity
  Scrapy entrypoint for Scrapinghub job runner

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •16•26•6•0•Updated Nov 3, 2025Nov 3, 2025

• hcf-backend

  Public

  scrapinghub/hcf-backend’s past year of commit activity
  Crawl Frontier HCF backend

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •6•8•2•1•Updated Oct 31, 2025Oct 31, 2025

• dateparser

  Public

  scrapinghub/dateparser’s past year of commit activity
  python parser for human readable dates

  hacktoberfest

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •486•2.8k•301•55•Updated Oct 28, 2025Oct 28, 2025

• docker-images

  Public

  scrapinghub/docker-images’s past year of commit activity
  Dockerfile

  •8•33•0•5•Updated Oct 20, 2025Oct 20, 2025

• scrapy-poet

  Public
  Page Object pattern for Scrapy

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •28•125•13•5•Updated Oct 17, 2025Oct 17, 2025

• price-parser

  Public
  Extract price amount and currency symbol from a raw text string

  hacktoberfest

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •52•346•17•9•Updated Oct 6, 2025Oct 6, 2025

• python-scrapinghub

  Public
  A client interface for Scrapinghub's API

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •62•205•23•2•Updated Oct 3, 2025Oct 3, 2025

• extruct

  Public
  Extract embedded metadata from HTML markup

  microformatssemantic-webrdfa+ 4json-ldmicrodataopengraphhacktoberfest

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •120•935•39•15•Updated Oct 1, 2025Oct 1, 2025

• article-extraction-benchmark

  Public
  Article extraction benchmark: dataset and evaluation scripts

  Python

  •

  MIT License

  •31•343•1•1•Updated Sep 23, 2025Sep 23, 2025

• scrapyrt

  Public
  HTTP API for Scrapy spiders

  pythoncrawlerscraper+ 7crawlingtwistedscrapywebcrawlerhacktoberfestwebcrawlinghacktoberfest2021

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •161•872•24•6•Updated Sep 22, 2025Sep 22, 2025

• Formasaurus

  Public
  Formasaurus tells you the type of an HTML form and its fields using machine learning

  HTML

  •46•8•1•1•Updated Sep 4, 2025Sep 4, 2025

• sklearn-crfsuite

  Public
  scikit-learn inspired API for CRFsuite

  Python

  •213•1•0•1•Updated Sep 4, 2025Sep 4, 2025

• scrapinghub-stack-scrapy

  Public
  Software stack with latest Scrapy and updated deps

  Dockerfile

  •

  BSD 3-Clause "New" or "Revised" License

  •20•65•1•2•Updated Aug 2, 2025Aug 2, 2025

• scrapy-frontera

  Public
  More flexible and featured Frontera scheduler for Scrapy

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •5•36•2•1•Updated Jun 6, 2025Jun 6, 2025

• frontera

  Public
  A scalable frontier for web crawlers

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •217•1.3k•78•17•Updated Jun 6, 2025Jun 6, 2025

• spidermon

  Public
  Scrapy Extension for monitoring spiders execution.

  testingmonitoringscraping+ 5crawlingspidershacktoberfestmonitoring-toolscrapinghub

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •101•552•42•5•Updated Apr 11, 2025Apr 11, 2025

• social-app-django

  Public
  Python Social Auth - Application - Django

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •395•2•0•1•Updated Nov 18, 2024Nov 18, 2024

• number-parser

  Public
  Parse numbers written in natural language

  hacktoberfest

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •25•124•13•6•Updated Oct 23, 2024Oct 23, 2024

• streamparse

  Public
  streamparse lets you run Python code against real-time streams of data. Integrates with Apache Storm.

  Python

  •

  Apache License 2.0

  •221•2•0•1•Updated Sep 20, 2024Sep 20, 2024

• splash

  Public
  Lightweight, scriptable browser as a service with an HTTP API

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •518•4.2k•373•26•Updated Aug 2, 2024Aug 2, 2024

• pgcontents

  Public
  A Postgres-backed ContentsManager implementation for IPython

  Python

  •

  Apache License 2.0

  •87•2•0•1•Updated Jul 18, 2024Jul 18, 2024

• shublang

  Public
  Pluggable DSL that uses pipes to perform a series of linear transformations to extract data

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •8•16•23•6•Updated Jul 9, 2024Jul 9, 2024

• woodpecker

  Public
  An opinionated fork of the Drone CI system

  Go

  •

  Other

  •504•0•0•5•Updated Jul 7, 2024Jul 7, 2024

• varanus

  Public
  A command line spider monitoring tool

  spidermonitoringpython36

  Python

  •7•8•2•2•Updated Jul 6, 2024Jul 6, 2024