← All tools

xcrawl3r

https://github.com/hueristiq/xcrawl3r

Open Source

Categories

Reconnaissance Web Application

Description

A command-line utility designed to recursively spider webpages for URLs. It works by actively traversing websites - following links embedded in webpages, parsing resources like sitemaps and robots.txt files, and even processing local files - to uncover every URL.

Keywords

bug-bounty bug-bounty-tools contentdiscovery crawler ethical-hacking ethical-hacking-tools go golang penetration-testing penetration-testing-tools reconnaissance red-teaming red-teaming-tools web-security

Example Usage

xcrawl3r -h

See also

public-skills-builder

Free

Generate Claude Code bug bounty skills from public HackerOne reports and GitHub writeups — 18 vuln c…

GraphQL Voyager / InQL

Open Source

InQL is a Burp Suite and standalone GraphQL security scanner. Analyzes introspection queries, genera…

username-anarchy

Open Source Kali

Username tools for penetration testing…

EyeWitness

Open Source Kali

Takes screenshots of web pages, RDP, and VNC services. Reports on default credentials and interestin…

Burp Suite

Freemium Kali

Leading platform for web application security testing. The Community Edition includes an interceptin…

Advanced-WAF-WAFinity

Free

An Advanced Web Application Firewall that protects against threats like SQL injection and XSS by fil…