← All tools

xcrawl3r

https://github.com/hueristiq/xcrawl3r

Open Source

Categories

Reconnaissance Web Application

Description

A command-line utility designed to recursively spider webpages for URLs. It works by actively traversing websites - following links embedded in webpages, parsing resources like sitemaps and robots.txt files, and even processing local files - to uncover every URL.

Keywords

bug-bounty bug-bounty-tools contentdiscovery crawler ethical-hacking ethical-hacking-tools go golang penetration-testing penetration-testing-tools reconnaissance red-teaming red-teaming-tools web-security

Example Usage

xcrawl3r -h

See also

GraphQL Voyager / InQL

Open Source

InQL is a Burp Suite and standalone GraphQL security scanner. Analyzes introspection queries, genera…

Pentest-Swarm-AI

Open Source

Autonomous penetration testing using a swarm of AI agents. Orchestrates recon, classification, explo…

xurlfind3r

Open Source

A command-line utility designed to discover URLs for a given domain in a simple, efficient way. It w…

BurpSuite-Xkeys

Free

A Burp Suite Extension to extract interesting strings (key, secret, token, or etc.) from a webpage.…

darkdump

Open Source

Open Source Intelligence Interface for Deep Web Scraping…

EyeWitness

Open Source Kali

Takes screenshots of web pages, RDP, and VNC services. Reports on default credentials and interestin…