Skip to content
@crwlrsoft

crwlr.software

PHP Packages for Rapid Crawler and Scraper Development
README.md

crwlr.software logo

crwlr.software - PHP Packages for Rapid Crawler and Scraper Development

crwlr.software is a collection of open source PHP composer packages that provide the necessary tools for web crawling and scraping tasks. The crawler package contains everything and helps you build crawlers as fast as possible. And there are also parts of it that you can use standalone.

Popular repositories Loading

  1. crawler crawler Public

    Library for Rapid (Web) Crawler and Scraper Development

    PHP 309 11

  2. url url Public

    Swiss Army knife for urls.

    PHP 101 7

  3. query-string query-string Public

    A library for convenient handling of query strings used in HTTP requests.

    PHP 16 3

  4. schema-org schema-org Public

    Extract schema.org objects from HTML documents

    PHP 11 2

  5. robots-txt robots-txt Public

    Robots Exclusion Standard/Protocol Parser for Web Crawling/Scraping

    PHP 8 2

  6. utils utils Public

    Utilities that are needed in multiple crwler packages.

    PHP 2 1

Repositories

Showing 10 of 14 repositories
  • crawler-ext-browser Public

    Extension for the crwlr/crawler package containing steps utilizing a headless browser.

    crwlrsoft/crawler-ext-browser’s past year of commit activity
    PHP 0 MIT 0 0 0 Updated Jun 18, 2024
  • crwl-extension-utils Public

    Utils for extension packages for the crwl.io app.

    crwlrsoft/crwl-extension-utils’s past year of commit activity
    PHP 0 0 0 0 Updated Jun 18, 2024
  • crawler Public

    Library for Rapid (Web) Crawler and Scraper Development

    crwlrsoft/crawler’s past year of commit activity
    PHP 309 MIT 11 1 0 Updated Jun 17, 2024
  • crwl-ext-browser Public

    Extension configurations for integration of crwlr/crawler-ext-browser into the crwl.io app.

    crwlrsoft/crwl-ext-browser’s past year of commit activity
    PHP 0 MIT 0 0 0 Updated Feb 26, 2024
  • html-2-text Public

    Convert HTML to formatted plain text.

    crwlrsoft/html-2-text’s past year of commit activity
    PHP 2 MIT 0 0 0 Updated Feb 21, 2024
  • package-template Public template

    Template repository for new crwlr packages

    crwlrsoft/package-template’s past year of commit activity
    PHP 1 MIT 0 0 0 Updated Feb 5, 2024
  • url Public

    Swiss Army knife for urls.

    crwlrsoft/url’s past year of commit activity
    PHP 101 MIT 7 0 0 Updated Jan 31, 2024
  • schema-org Public

    Extract schema.org objects from HTML documents

    crwlrsoft/schema-org’s past year of commit activity
    PHP 11 MIT 2 0 0 Updated Nov 30, 2023
  • utils Public

    Utilities that are needed in multiple crwler packages.

    crwlrsoft/utils’s past year of commit activity
    PHP 2 MIT 1 0 0 Updated Oct 29, 2023
  • robots-txt Public

    Robots Exclusion Standard/Protocol Parser for Web Crawling/Scraping

    crwlrsoft/robots-txt’s past year of commit activity
    PHP 8 MIT 2 0 0 Updated Oct 29, 2023
View all repositories

Top languages

Loading…

Most used topics

Loading…