deepnoc bot

network optimized crawling

Follow us on linkedin or twitter to get the latest news!

Goals

- reduce traffic load for webservers
- collect up to 100% of all web pages worldwide
- replace up to 250 bots worldwide

Currently we are testing our bot worldwide in the revised version 3.0

General notes

Our bot does NOT search for email addresses or personal data. It just parses and saves the website content for research purposes.

Our goal is to save lots of Internet traffic and cpu power for web servers in the future.

Follow us on linkedin or twitter to get the latest news!

History

We started test crawling with our bot on 12/2019.

Cloud based crawler active since 03/2020.

Enhanced database system since 05/2021.

Thank you for your attention.

In case you want to stop the deepnoc bot crawling your website: Just enter the following lines into your robots.txt:

User-agent: deepnoc
Disallow: /

Last update: June 2021
Homepage - deepnoc bot

deepnoc GmbH - Speditionstraße 15A - 40221 Düsseldorf - Germany
CEO: Sebastian Karpp & Timm Bauten - Reg.: Amtsgericht Düsseldorf, HRB 89931

Find us on linkedin