gigirassy/riptide
1
0
Fork 0
mirror of https://codeberg.org/gigirassy/riptide synced 2026-06-22 13:46:53 -04:00
Code Issues Projects Releases Packages Wiki Activity
No description
18 commits 1 branch 2 tags 56 KiB
  • Rust 68%
  • Shell 23.3%
  • Dockerfile 8.7%
Find a file
Exact
gigirassy 5edfd48231 Update Dockerfile
2026-06-17 07:36:35 +02:00
.forgejo/workflows Add .forgejo/workflows/docker.yml 2026-06-17 07:31:36 +02:00
misc Update misc/riptideinstaller.sh 2026-06-17 02:14:37 +02:00
src Add src/main.rs 2026-06-14 00:34:52 +02:00
Cargo.lock init part 1 2026-06-14 00:34:10 +02:00
Cargo.toml init part 1 2026-06-14 00:34:10 +02:00
Dockerfile Update Dockerfile 2026-06-17 07:36:35 +02:00
LICENSE Initial commit 2026-06-14 00:29:29 +02:00
README.md Update README.md 2026-06-17 02:15:46 +02:00

README.md

riptide

quick, recursive, sitemap url extractor

riptide, with 16 workers, can rip through a 121,000+ link sitemap in less than 6 seconds.

nune@nunes-pc ~/sitemaps2> time riptide \
                                                              --sitemap https://www.************.com/sitemap.axd \
                                                              --output ****.txt \
                                                              --scheme https \
                                                              --workers 16 \
________________________________________________________
Executed in    5.99 secs      fish           external
   usr time  528.67 millis    1.14 millis  527.53 millis
   sys time  201.95 millis    0.21 millis  201.74 millis

installation and usage:

install (linux static binary, x86_64)

this assumes /usr/local/bin is part of your path, youll need sha256sum

concerned about what the script does? check the url!

curl https://codeberg.org/gigirassy/riptide/raw/branch/main/misc/riptideinstaller.sh | bash

install (cargo)

cargo install --git https://codeberg.org/gigirassy/riptide/

usage

linux/bsd

riptide \
                               --sitemap https://example.com/sitemap.xml \
                               --output examplecomurls.txt \
                               --scheme https \
                               --workers 16 \
                               --user-agent "Mozilla/5.0 (X11; Linux i686; rv:151.0) Gecko/20100101 Firefox/151.0"

windows in same directory as riptide.

.\riptide.exe --sitemap https://example.com/sitemap.xml --output examplecomurls.txt --scheme https --workers 16 --user-agent "Mozilla/5.0 (X11; Linux i686; rv:151.0) Gecko/20100101 Firefox/151.0"

for searching for terms in urls after output is completed, i recommend ripgrep.

by default, riptide identifies itself in user agent, the above example overrides it.

license

isc