Compare commits

...

20 commits
0.9.0 ... main

Author SHA1 Message Date
7cc2ad9117
chore: use Database::Files.file_count to get the files hosted count
All checks were successful
File-uploader-crystal CI / build (push) Successful in 2m40s
2025-04-28 18:01:41 -04:00
44d7afddfc
fix: select all fields on a row for old_files 2025-04-28 18:01:00 -04:00
c0f73bab0a
chore: change the name of properties inside the config class 2025-04-28 17:59:37 -04:00
b65acac27d
update dockerfile and ci
Some checks failed
File-uploader-crystal CI / build (push) Failing after 28s
2025-04-22 19:00:53 -04:00
b19c423648
0.9.6: Re-enable IP rate limits, add ips database logic and idk what more 2025-04-22 18:59:54 -04:00
8995f023ac
0.9.5: Rewrite
Some checks failed
File-uploader-crystal CI / build (push) Failing after 15s
2025-04-21 00:35:29 -04:00
a4562ca005
feat(webserver): add host option to the configuration
All checks were successful
File-uploader-crystal CI / build (push) Successful in 1m41s
2025-01-02 19:06:05 -03:00
c554b772c8
0.9.3.5: only generate thumbnails on known extensions, remove trailing '/' from config.files and config.thumbnails
All checks were successful
File-uploader-crystal CI / build (push) Successful in 2m20s
2024-11-26 20:56:58 -03:00
bb9ecee67b
0.9.3.4: Fix what I did yesterday
All checks were successful
File-uploader-crystal CI / build (push) Successful in 1m49s
2024-11-21 13:30:19 -03:00
cb75b97520
0.9.3.3: Better handling when retrieving files, move rate limiter
All checks were successful
File-uploader-crystal CI / build (push) Successful in 1m47s
2024-11-21 04:02:12 -03:00
fdfa782e91
0.9.3.2-1: Update docker compose file
All checks were successful
File-uploader-crystal CI / build (push) Successful in 1m48s
2024-11-21 03:23:56 -03:00
99c22095f9
0.9.3.2: Delete entry from the DB is the file doesn't exists on the filesystem 2024-11-21 03:23:29 -03:00
9de4960932
0.9.3.1: Update Dockerfile and add compose file
All checks were successful
File-uploader-crystal CI / build (push) Successful in 1m49s
2024-11-19 23:16:32 -03:00
0002c81429
0.9.3: BUGFIX! Fix deletion of thumbnails on check_old_files job.
All checks were successful
File-uploader-crystal CI / build (push) Successful in 1m52s
- Add colors to logs
- Use static table names instead of config provided ones, it's kinda
  stupid to give the user an option to set the name of the table if I'm
  developing it for sqlite
2024-11-19 22:39:23 -03:00
b51513339c
0.9.2: Fix thumbnail folder generation, better chatterino config generation and better error hadling
All checks were successful
File-uploader-crystal CI / build (push) Successful in 2m15s
2024-10-21 13:54:51 -03:00
4803700cab
0.9.1: Upload file from URL with GET request
All checks were successful
File-uploader-crystal CI / build (push) Successful in 4m34s
2024-09-11 01:50:17 -03:00
493322039d
0.9.0-4: ... kms
All checks were successful
File-uploader-crystal CI / build (push) Successful in 4m53s
2024-09-11 00:10:38 -03:00
c7f30c8245
0.9.0-3: .
Some checks failed
File-uploader-crystal CI / build (push) Failing after 21s
2024-09-10 23:51:40 -03:00
db51190e5f
0.9.0-2: Build please
Some checks failed
File-uploader-crystal CI / build (push) Failing after 2m48s
2024-09-10 23:34:39 -03:00
e1a64b225d
0.9.0-1: Add Docker file and automated builds using forgejo actions 2024-09-10 23:26:32 -03:00
38 changed files with 1245 additions and 946 deletions

46
.forgejo/workflows/ci.yml Normal file
View file

@ -0,0 +1,46 @@
name: 'File-uploader-crystal CI'
on:
workflow_dispatch:
schedule:
- cron: '0 7 * * 0'
push:
branches:
- "main"
jobs:
build:
runs-on: runner
steps:
- uses: https://code.forgejo.org/actions/checkout@v2
- uses: https://code.forgejo.org/docker/setup-buildx-action@v3
name: Setup Docker BuildX system
- name: Login to Docker Container Registry
uses: https://code.forgejo.org/docker/login-action@v3.1.0
with:
registry: git.nadeko.net
username: ${{ secrets.USERNAME }}
password: ${{ secrets.TOKEN }}
- name: Docker meta
id: meta
uses: https://github.com/docker/metadata-action@v6
with:
images: git.nadeko.net/fijxu/file-uploader-crystal
tags: |
type=sha,format=short,prefix={{date 'YYYY.MM.DD'}}-,enable=${{ github.ref == format('refs/heads/{0}', 'main') }}
type=raw,value=latest,enable=${{ github.ref == format('refs/heads/{0}', 'main') }}
- uses: https://code.forgejo.org/docker/build-push-action@v6
name: Build images
with:
context: .
file: Dockerfile
tags: ${{ steps.meta.outputs.tags }}
platforms: linux/amd64
push: true
cache-from: type=gha
cache-to: type=gha,mode=max

4
.gitignore vendored
View file

@ -5,3 +5,7 @@
*.dwarf
data
torexitnodes.txt
files
thumbnails
db.sqlite3
config/config.yml

42
Dockerfile Normal file
View file

@ -0,0 +1,42 @@
# Based on https://github.com/iv-org/invidious/blob/master/docker/Dockerfile
FROM crystallang/crystal:1.16.1-alpine AS builder
RUN apk add --no-cache sqlite-static yaml-static
WORKDIR /file-uploader-crystal
COPY ./shard.yml ./shard.yml
COPY ./shard.lock ./shard.lock
RUN shards install --production
COPY ./src/ ./src/
# TODO: .git folder is required for building this is destructive.
# See definition of CURRENT_BRANCH, CURRENT_COMMIT and CURRENT_VERSION.
COPY ./.git/ ./.git/
RUN --mount=type=cache,target=/root/.cache/crystal \
crystal build ./src/file-uploader-crystal.cr \
--release \
--static --warnings all
# 2nd stage
FROM alpine:3.21
RUN apk add --no-cache tini ffmpeg
WORKDIR /file-uploader-crystal
RUN addgroup -g 1000 -S file-uploader-crystal && \
adduser -u 1000 -S file-uploader-crystal -G file-uploader-crystal
COPY --chown=file-uploader-crystal ./config/config.example.yml ./config/
RUN mv -n config/config.example.yml config/config.yml
COPY --from=builder /file-uploader-crystal/file-uploader-crystal /file-uploader-crystal
RUN chmod o+rX -R /file-uploader-crystal/file-uploader-crystal
RUN chown file-uploader-crystal: -R /file-uploader-crystal
EXPOSE 8080
USER file-uploader-crystal
ENTRYPOINT ["/sbin/tini", "--"]
CMD [ "/file-uploader-crystal/file-uploader-crystal" ]

View file

@ -1,5 +1,8 @@
# file-uploader
> [!WARNING]
> Project being rewritten, some features like the admin API and some upload endpoints are unavailable on 0.9.5
Simple file uploader made on Crystal.
~~I'm making this to replace my current File uploader hosted on https://ayaya.beauty which uses https://github.com/nokonoko/uguu~~
Already replaced lol.
@ -9,7 +12,7 @@ Already replaced lol.
- Temporary file uploads like Uguu
- File deletion link (not available in frontend for now)
- Chatterino and ShareX support
- Video Thumbnails for Chatterino and FrankerFaceZ (Requires `ffmpeg` to be installed, can be disabled.)
- Video Thumbnails for Chatterino and FrankerFaceZ (Requires `ffmpeg` to be installed, disabled by default)
- Rate Limiting
- [Small Admin API](./src/handling/admin.cr) that allows you to delete files, reset rate limits and more (Needs to be enabled in the configuration)
- Unix socket support if you don't want to deal with all the TCP overhead
@ -37,7 +40,7 @@ server {
proxy_pass http://127.0.0.1:8080;
# This if you want to use a UNIX socket instead
#proxy_pass http://unix:/tmp/file-uploader.sock;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
proxy_pass_request_headers on;
@ -69,18 +72,3 @@ WorkingDirectory=%h/file-uploader-crystal/
[Install]
WantedBy=default.target
```
## TODO
- ~~Add file size limit~~ ADDED
- ~~Fix error when accessing `http://127.0.0.1:8080` with an empty DB.~~ Fixed somehow.
- Better frontend...
- ~~Disable file deletion if `deleteFilesCheck` or `deleteFilesAfter` is set to `0`~~ DONE
- ~~Disable delete key if `deleteKeyLength` is `0`~~ DONE (But I think there is a better way to do it)
- ~~Exit if `fileameLength` is `0`~~ DONE
- ~~Disable file limit if `size_limit` is `0`~~ DONE
- ~~Prevent files from being overwritten in the event of a name collision~~ DONE
- Dockerfile and Docker image (Crystal doesn't has dependency hell like other languages so is not really necessary to do, but useful for people that want instant deploy)
- Custom file expiration using headers (Like rustypaste)
- Small CLI to upload files (like `rpaste` from rustypaste)
- Add more endpoints to Admin API

43
config/config.example.yml Normal file
View file

@ -0,0 +1,43 @@
colorize_logs: true
log_level: "debug"
# File paths
files: "./files"
thumbnails: "./thumbnails"
db: "./db.sqlite3"
# Tor
block_tor_addresses: true
tor_exit_nodes_check: 1600
tor_exit_nodes_url: "https://check.torproject.org/exit-addresses"
tor_message: "TOR IS BLOCKED!"
generate_thumbnails: true
admin_enabled: true
admin_api_key: "asd"
size_limit: 512
enable_checksums: false
port: 8080
files_per_ip: 2
rate_limit_period: 20
rate_limit_message: ""
filename_length: 3
delete_files_after: 7
delete_files_check: 1800
delete_key_length: 4
site_info: "Whatever you want to put here"
site_warning: "WARNING!"
blocked_extensions:
- "exe"
opengraph_useragents:
- "chatterino-api-cache/"
- "FFZBot/"
- "Twitterbot/"
- "Synapse/"
- "Mastodon/"
# alternative_domains:
# - "example.com"

View file

@ -1,44 +0,0 @@
files: "./files"
thumbnails: "./thumbnails"
generateThumbnails: true
db: "./db.sqlite3"
dbTableName: "files"
adminEnabled: true
adminApiKey: "asd"
fileameLength: 3
# In MiB
size_limit: 512
port: 8080
blockTorAddresses: true
# Every hour
torExitNodesCheck: 1600
torExitNodesUrl: "https://check.torproject.org/exit-addresses"
torExitNodesFile: "./torexitnodes.txt"
torMessage: "TOR IS BLOCKED!"
filesPerIP: 2
ipTableName: "ips"
rateLimitPeriod: 20
rateLimitMessage: ""
# If you define the unix socket, it will only listen on the socket and not the port.
#unix_socket: "/tmp/file-uploader.sock"
# In days
deleteFilesAfter: 1
# In seconds
deleteFilesCheck: 1600
deleteKeyLength: 4
siteInfo: "Whatever you want to put here"
siteWarning: "WARNING!"
log_level: "debug"
blockedExtensions:
- "exe"
# List of useragents that use OpenGraph to gather file information
opengraphUseragents:
- "chatterino-api-cache/"
- "FFZBot/"
- "Twitterbot/"
alternativeDomains:
- "ayaya.beauty"
- "lamartina.gay"

17
docker-compose.yml Normal file
View file

@ -0,0 +1,17 @@
services:
file-uploader:
image: git.nadeko.net/fijxu/file-uploader-crystal:latest
# This program should never use that many memory and more than 50% of the CPU
mem_limit: 512MB
cpus: 0.5
# If you want to use a custom config file, you can mount it here.
volumes:
# - ./config/config.yml:/file-uploader-crystal/config/config.yml
- ./public:/file-uploader-crystal/public
- ./files:/file-uploader-crystal/files
- ./thumbnails:/file-uploader-crystal/thumbnails
- ./db:/file-uploader-crystal/db
- ./torexitnodes.txt:/file-uploader-crystal/torexitnodes.txt
ports:
- 127.0.0.1:8080:8080

Binary file not shown.

Before

Width:  |  Height:  |  Size: 46 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 52 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 17 KiB

After

Width:  |  Height:  |  Size: 66 KiB

BIN
public/favicon.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.2 KiB

View file

@ -89,18 +89,18 @@ document.addEventListener("DOMContentLoaded", () => {
uploadContainer.className = "upload-status"; // Use the existing CSS class for styling
uploadContainer.appendChild(uploadText);
uploadContainer.appendChild(statusLink);
buttons.appendChild(copyButton)
buttons.appendChild(deleteButton)
uploadContainer.appendChild(buttons)
buttons.appendChild(copyButton)
buttons.appendChild(deleteButton)
uploadContainer.appendChild(buttons)
uploadStatus.appendChild(uploadContainer);
// Update upload text
uploadText.innerHTML = "0%";
uploadText.className = "percent";
statusLink.className = "status";
copyButton.className = "copy-button"; // Add class for styling
copyButton.className = "button copy-button"; // Add class for styling
copyButton.innerHTML = "Copiar"; // Set button text
deleteButton.className = "delete-button";
deleteButton.className = "button delete-button";
deleteButton.innerHTML = "Borrar";
copyButton.style.display = "none";
deleteButton.style.display = "none";

View file

@ -18,10 +18,7 @@
html {
font-family: "FG";
background-image: linear-gradient(to bottom,
rgba(11, 11, 11, 0.92),
rgba(11, 11, 11, 0.92)),
url(./bliss-small.avif);
background: #111111;
background-attachment: fixed;
background-repeat: no-repeat;
background-size: cover;
@ -29,8 +26,6 @@ html {
body {
/* font-family: Arial, sans-serif; */
/* background-color: #111; */
margin: 0;
padding: 20px;
}
@ -53,11 +48,11 @@ h1 {
a {
text-decoration: none;
color: #ffb6c1
}
.bottom {
font-size: 0.9em;
/* margin-top: 1ch;*/
flex: 1;
text-align: center;
}
@ -73,31 +68,25 @@ a {
.container {
max-width: 800px;
margin: auto;
/* background: white; */
/*! padding: 20px; */
border-radius: 0px;
/*! box-shadow: 0 0 10px rgba(0, 0, 0, 0.1); */
}
.container>img {
max-width: 100%;
max-height: 500px
}
#drop-area {
/*! border: 2px solid #00ff00; */
/*! border-radius: 6px; */
/*! padding-left: 10px; */
/*! padding-right: 10px; */
text-align: center;
position: relative;
width: fit-content;
margin: 0 auto;
/* Center the element */
display: block;
/* Ensure it behaves as a block-level element */
background: rgba(202, 230, 190, .75);
border: 1px solid #b7d1a0;
border-radius: 4px;
color: #468847;
cursor: pointer;
/*! display: inline-block; */
font-size: 24px;
padding: 28px 48px;
text-shadow: 0 1px hsla(0, 0%, 100%, .5);
@ -107,11 +96,9 @@ a {
.button {
display: inline-block;
padding: 10px 20px;
/* background: #; */
color: white;
border-radius: 5px;
cursor: pointer;
/* margin-top: 10px; */
}
.upload-status {
@ -128,101 +115,59 @@ nav>ul {
#upload-status {
margin: 20px;
/* Adjust as needed */
}
.upload-status {
display: flex;
align-items: center;
justify-content: space-between;
border: 2px solid #999;
/* Optional styling for the status box */
border: 1px solid #ffffff;
padding: 5px;
/* Optional padding */
/*! border-radius: 6px; */
/* Optional rounded corners */
/*! background-color: #f9f9f9; */
/* Optional background color */
}
.link-container {
display: flex;
align-items: center;
margin-left: auto;
/* Pushes the link and button to the right */
}
.link {
color: #ffb6c1;
text-decoration: none;
/* Remove underline from link */
margin-right: 5px;
/* Space between link and button */
}
.link:hover {
text-decoration: underline;
/* Optional: underline on hover */
}
.button {
display: inline;
color: rgb(255, 255, 255);
border: none;
border-radius: 3px;
padding: 5px 10px;
cursor: pointer;
font-weight: bold;
}
.copy-button {
display: inline;
background-color: #7a6fff;
/* Button background color */
color: white;
/* Button text color */
border: none;
/* Remove border */
border-radius: 3px;
/* Rounded corners for the button */
padding: 5px 10px;
/* Button padding */
cursor: pointer;
/* Pointer cursor on hover */
font-weight: bold;
background-color: #1e8a1a;
}
.delete-button {
display: inline;
background-color: #ff6f6f;
/* Button background color */
color: white;
/* Button text color */
border: none;
/* Remove border */
border-radius: 3px;
/* Rounded corners for the button */
padding: 5px 10px;
/* Button padding */
cursor: pointer;
/* Pointer cursor on hover */
margin-left: 6px;
font-weight: bold;
background-color: #b83434;
margin-left: 6px
}
.copy-button:hover {
background-color: #6057ce;
/* Darker shade on hover */
background-color: #156412;
}
.delete-button:hover {
background-color: #ce5757;
/* Darker shade on hover */
background-color: #912a2a;
}
.status {
color: rgb(255, 132, 0);
}
a:link {
color: #ffb6c1
}
a:visited {
color: #ffb6c1
}
a:hover {
color: #ffb6c1
}

View file

@ -1,4 +1,4 @@
name: file-uploader
name: file-uploader-crystal
version: 0.8.7
authors:
@ -6,7 +6,7 @@ authors:
targets:
file-uploader:
main: src/file-uploader.cr
main: src/file-uploader-crystal.cr
dependencies:
kemal:

View file

@ -2,6 +2,15 @@ require "yaml"
class Config
include YAML::Serializable
# Colorize logs
property colorize_logs : Bool = true
# Log level
property log_level : LogLevel = LogLevel::Info
# Port on which the uploader will bind
property port : Int32 = 8080
# IP address on which the uploader will bind
property host : String = "127.0.0.1"
# Where the uploaded files will be located
property files : String = "./files"
@ -9,83 +18,91 @@ class Config
property thumbnails : String = "./thumbnails"
# Generate thumbnails for OpenGraph compatible platforms like Chatterino
# Whatsapp, Facebook, Discord, etc.
property generateThumbnails : Bool = false
property generate_thumbnails : Bool = false
# Where the SQLITE3 database will be located
property db : String = "./db.sqlite3"
# Name of the table that will be used for file information
property dbTableName : String = "files"
# Enable or disable the admin API
property adminEnabled : Bool = false
property admin_enabled : Bool = false
# The API key for admin routes. It's passed as a "X-Api-Key" header to the
# request
property adminApiKey : String? = ""
property admin_api_key : String? = nil
# Not implemented
property incrementalFileameLength : Bool = true
property incrementalfilename_length : Bool = true
# Filename length
property fileameLength : Int32 = 3
property filename_length : Int32 = 3
# In MiB
property size_limit : Int16 = 512
# TCP port
property port : Int32 = 8080
property enable_checksums : Bool = true
# A file path where do you want to place a unix socket (THIS WILL DISABLE ACCESS
# BY IP ADDRESS)
property unix_socket : String?
# True if you want this program to block IP addresses coming from the Tor
# network
property blockTorAddresses : Bool? = false
property block_tor_addresses : Bool = false
# How often (in seconds) should this program download the exit nodes list
property torExitNodesCheck : Int32 = 3600
property tor_exit_nodes_check : Int32 = 3600
# Only https://check.torproject.org/exit-addresses is supported
property torExitNodesUrl : String = "https://check.torproject.org/exit-addresses"
# Where the file of the exit nodes will be located, can be placed anywhere
property torExitNodesFile : String = "./torexitnodes.txt"
property tor_exit_nodes_url : String = "https://check.torproject.org/exit-addresses"
# Message that will be displayed to the Tor user.
# It will be shown on the Frontend and shown in the error 401 when a user
# tries to upload a file using curl or any other tool
property torMessage : String? = "Tor is blocked!"
# How many files an IP address can upload to the server
property filesPerIP : Int32 = 32
# Name of the table that will be used for rate limit information
property ipTableName : String = "ips"
property tor_message : String? = "Tor is blocked!"
# How many files an IP address can upload to the server. Setting this to 0
# disables rate limits in the rate limit period
property files_per_ip : Int32 = 32
# How often is the file limit per IP reset? (in seconds)
property rateLimitPeriod : Int32 = 600
property rate_limit_period : Int32 = 600
# TODO: UNUSED CONSTANT
property rateLimitMessage : String = ""
property rate_limit_message : String = ""
# Delete the files after how many days?
property deleteFilesAfter : Int32 = 7
property delete_files_after : Int32 = 14
# How often should the check of old files be performed? (in seconds)
property deleteFilesCheck : Int32 = 1800
property delete_files_check : Int32 = 1800
# The lenght of the delete key
property deleteKeyLength : Int32 = 4
property siteInfo : String = "xd"
property delete_key_length : Int32 = 6
property site_info : String = "xd"
# TODO: UNUSED CONSTANT
property siteWarning : String? = ""
# Log level
property log_level : LogLevel = LogLevel::Info
property site_warning : String? = ""
# Blocked extensions that are not allowed to be uploaded to the server
property blockedExtensions : Array(String) = [] of String
property blocked_extensions : Array(String) = [] of String
# A list of OpenGraph user agents. If the request contains one of those User
# agents when trying to retrieve a file from the server; the server will
# reply with an HTML with OpenGraph tags, pointing to the media thumbnail
# (if it was generated successfully) and the name of the file as title
property opengraphUseragents : Array(String) = [] of String
property opengraph_useragents : Array(String) = [] of String
# Since this program detects the Host header of the client it can be used
# with multiple domains. You can display the domains in the frontend
# and in `/api/stats`
property alternativeDomains : Array(String) = [] of String
property alternative_domains : Array(String) = [] of String
def self.load
config_file = "config/config.yml"
def self.check_config(config : Config)
if config.filename_length <= 0
puts "Config: filename_length cannot be less or equal to 0"
exit(1)
end
if config.files.ends_with?('/')
config.files = config.files.chomp('/')
end
if config.thumbnails.ends_with?('/')
config.thumbnails = config.thumbnails.chomp('/')
end
end
def self.load(config_file : String = "config/config.yml")
config_yaml = File.read(config_file)
config = Config.from_yaml(config_yaml)
check_config(config)
config
end
def self.check_config(config : Config)
if config.fileameLength <= 0
puts "Config: fileameLength cannot be #{config.fileameLength}"
exit(1)
end
end
end

83
src/database/files.cr Normal file
View file

@ -0,0 +1,83 @@
module Database::Files
extend self
# -------------------
# Insert / Delete
# -------------------
def insert(file : UFile) : Nil
request = <<-SQL
INSERT INTO files
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
ON CONFLICT DO NOTHING
SQL
SQL.exec(request, *file.to_tuple)
end
def delete(filename : String) : Nil
request = <<-SQL
DELETE
FROM files
WHERE filename = ?
SQL
SQL.exec(request, filename)
end
def delete_with_key(key : String) : Nil
request = <<-SQL
DELETE FROM files
WHERE delete_key = ?
SQL
SQL.exec(request, key)
end
# -------------------
# Select
# -------------------
def select(filename : String) : UFile?
request = <<-SQL
SELECT *
FROM files
WHERE filename = ?
SQL
SQL.query_one?(request, filename, as: UFile)
end
def select_with_key(delete_key : String) : UFile?
request = <<-SQL
SELECT *
FROM files
WHERE delete_key = ?
SQL
SQL.query_one?(request, delete_key, as: UFile)
end
# -------------------
# Misc
# -------------------
def old_files : Array(UFile)
request = <<-SQL
SELECT *
FROM files
WHERE uploaded_at < strftime('%s', 'now') - #{CONFIG.delete_files_after * 3600}
SQL
SQL.query_all(request, as: UFile)
end
def file_count : Int32
request = <<-SQL
SELECT COUNT (filename)
FROM files
SQL
SQL.query_one(request, as: Int32)
end
end

55
src/database/ip.cr Normal file
View file

@ -0,0 +1,55 @@
module Database::IP
extend self
# -------------------
# Insert / Delete
# -------------------
def insert(ip : UIP) : DB::ExecResult
request = <<-SQL
INSERT OR IGNORE
INTO ips
VALUES ($1, $2, $3)
SQL
SQL.exec(request, *ip.to_tuple)
end
def delete(ip : String) : Nil
request = <<-SQL
DELETE
FROM ips
WHERE ip = ?
SQL
SQL.exec(request, ip)
end
# -------------------
# Select
# -------------------
def select(ip : String) : UIP?
request = <<-SQL
SELECT *
FROM ips
WHERE ip = ?
SQL
SQL.query_one?(request, ip, as: UIP)
end
# -------------------
# Update
# -------------------
def increase_count(ip : UIP) : Nil
request = <<-SQL
UPDATE ips
SET count = count + 1
WHERE ip = $1
SQL
SQL.exec(request, ip.ip)
end
end

View file

@ -7,20 +7,22 @@ require "digest"
require "./logger"
require "./routing"
require "./utils"
require "./handling/**"
require "./config"
require "./jobs"
require "./lib/**"
require "./utils/*"
require "./lib/*"
require "./types/*"
require "./database/*"
CONFIG = Config.load
Kemal.config.port = CONFIG.port
Kemal.config.host_binding = CONFIG.host
Kemal.config.shutdown_message = false
Kemal.config.app_name = "file-uploader-crystal"
# https://github.com/iv-org/invidious/blob/90e94d4e6cc126a8b7a091d12d7a5556bfe369d5/src/invidious.cr#L136C1-L136C61
LOGGER = LogHandler.new(STDOUT, CONFIG.log_level)
LOGGER = LogHandler.new(STDOUT, CONFIG.log_level, CONFIG.colorize_logs)
# Give me a 128 bit CPU
# MAX_FILES = 58**CONFIG.fileameLength
# MAX_FILES = 58**CONFIG.filename_length
SQL = DB.open("sqlite3://#{CONFIG.db}")
# https://github.com/iv-org/invidious/blob/90e94d4e6cc126a8b7a091d12d7a5556bfe369d5/src/invidious.cr#L78
@ -32,6 +34,7 @@ CURRENT_TAG = {{ "#{`git describe --long --abbrev=7 --tags | sed 's/([^-]*)-
Utils.check_dependencies
Utils.create_db
Utils.create_files_dir
Utils.create_thumbnails_dir
Routing.register_all
Utils.delete_socket

View file

@ -1,301 +0,0 @@
require "../http-errors"
require "http/client"
require "benchmark"
module Handling
extend self
def upload(env)
env.response.content_type = "application/json"
ip_address = Utils.ip_address(env)
protocol = Utils.protocol(env)
host = Utils.host(env)
# You can modify this if you want to allow files smaller than 1MiB.
# This is generally a good way to check the filesize but there is a better way to do it
# which is inspecting the file directly (If I'm not wrong).
if CONFIG.size_limit > 0
if env.request.headers["Content-Length"].to_i > 1048576*CONFIG.size_limit
return error413("File is too big. The maximum size allowed is #{CONFIG.size_limit}MiB")
end
end
filename = ""
extension = ""
original_filename = ""
uploaded_at = ""
checksum = ""
if CONFIG.deleteKeyLength > 0
delete_key = Random.base58(CONFIG.deleteKeyLength)
end
# TODO: Return the file that matches a checksum inside the database
HTTP::FormData.parse(env.request) do |upload|
if upload.filename.nil? || upload.filename.to_s.empty?
LOGGER.debug "No file provided by the user"
return error403("No file provided")
end
# TODO: upload.body is emptied when is copied or read
# Utils.check_duplicate(upload.dup)
extension = File.extname("#{upload.filename}")
if CONFIG.blockedExtensions.includes?(extension.split(".")[1])
return error401("Extension '#{extension}' is not allowed")
end
filename = Utils.generate_filename
file_path = ::File.join ["#{CONFIG.files}", filename + extension]
File.open(file_path, "w") do |output|
IO.copy(upload.body, output)
end
original_filename = upload.filename
uploaded_at = Time.utc
checksum = Utils.hash_file(file_path)
end
# X-Forwarded-For if behind a reverse proxy and the header is set in the reverse
# proxy configuration.
begin
spawn { Utils.generate_thumbnail(filename, extension) }
rescue ex
LOGGER.error "An error ocurred when trying to generate a thumbnail: #{ex.message}"
end
begin
# Insert SQL data just before returning the upload information
SQL.exec "INSERT INTO #{CONFIG.dbTableName} VALUES (?, ?, ?, ?, ?, ?, ?, ?)",
original_filename, filename, extension, uploaded_at, checksum, ip_address, delete_key, nil
SQL.exec "INSERT OR IGNORE INTO #{CONFIG.ipTableName} (ip, date) VALUES (?, ?)", ip_address, Time.utc.to_unix
# SQL.exec "INSERT OR IGNORE INTO #{CONFIG.ipTableName} (ip) VALUES ('#{ip_address}')"
SQL.exec "UPDATE #{CONFIG.ipTableName} SET count = count + 1 WHERE ip = ('#{ip_address}')"
rescue ex
LOGGER.error "An error ocurred when trying to insert the data into the DB: #{ex.message}"
return error500("An error ocurred when trying to insert the data into the DB")
end
json = JSON.build do |j|
j.object do
j.field "link", "#{protocol}://#{host}/#{filename}"
j.field "linkExt", "#{protocol}://#{host}/#{filename}#{extension}"
j.field "id", filename
j.field "ext", extension
j.field "name", original_filename
j.field "checksum", checksum
if CONFIG.deleteKeyLength > 0
j.field "deleteKey", delete_key
j.field "deleteLink", "#{protocol}://#{host}/delete?key=#{delete_key}"
end
end
end
json
end
# The most unoptimized and unstable feature lol
# TODO: Support batch upload via JSON array
def upload_url(env)
env.response.content_type = "application/json"
ip_address = Utils.ip_address(env)
protocol = Utils.protocol(env)
host = Utils.host(env)
files = env.params.json["files"].as((Array(JSON::Any)))
successfull_files = [] of NamedTuple(filename: String, extension: String, original_filename: String, checksum: String, delete_key: String | Nil)
failed_files = [] of String
# X-Forwarded-For if behind a reverse proxy and the header is set in the reverse
# proxy configuration.
if files.empty?
end
files.each do |url|
url = url.to_s
filename = Utils.generate_filename
original_filename = ""
extension = ""
checksum = ""
uploaded_at = Time.utc
extension = File.extname(URI.parse(url).path)
delete_key = nil
file_path = ::File.join ["#{CONFIG.files}", filename + extension]
File.open(file_path, "w") do |output|
begin
HTTP::Client.get(url) do |res|
IO.copy(res.body_io, output)
end
rescue ex
LOGGER.debug "Failed to download file '#{url}': #{ex.message}"
return error403("Failed to download file '#{url}'")
failed_files << url
end
end
# successfull_files << url
# end
if extension.empty?
extension = Utils.detect_extension(file_path)
File.rename(file_path, file_path + extension)
file_path = ::File.join ["#{CONFIG.files}", filename + extension]
end
# The second one is faster and it uses less memory
# original_filename = URI.parse("https://ayaya.beauty/PqC").path.split("/").last
original_filename = url.split("/").last
checksum = Utils.hash_file(file_path)
begin
spawn { Utils.generate_thumbnail(filename, extension) }
rescue ex
LOGGER.error "An error ocurred when trying to generate a thumbnail: #{ex.message}"
end
begin
# Insert SQL data just before returning the upload information
SQL.exec("INSERT INTO #{CONFIG.dbTableName} VALUES (?, ?, ?, ?, ?, ?, ?, ?)",
original_filename, filename, extension, uploaded_at, checksum, ip_address, delete_key, nil)
successfull_files << {filename: filename,
original_filename: original_filename,
extension: extension,
delete_key: delete_key,
checksum: checksum}
rescue ex
LOGGER.error "An error ocurred when trying to insert the data into the DB: #{ex.message}"
return error500("An error ocurred when trying to insert the data into the DB")
end
end
json = JSON.build do |j|
j.array do
successfull_files.each do |fileinfo|
j.object do
j.field "link", "#{protocol}://#{host}/#{fileinfo[:filename]}"
j.field "linkExt", "#{protocol}://#{host}/#{fileinfo[:filename]}#{fileinfo[:extension]}"
j.field "id", fileinfo[:filename]
j.field "ext", fileinfo[:extension]
j.field "name", fileinfo[:original_filename]
j.field "checksum", fileinfo[:checksum]
if CONFIG.deleteKeyLength > 0
delete_key = Random.base58(CONFIG.deleteKeyLength)
j.field "deleteKey", fileinfo[:delete_key]
j.field "deleteLink", "#{protocol}://#{host}/delete?key=#{fileinfo[:delete_key]}"
end
end
end
end
end
json
end
def retrieve_file(env)
begin
protocol = Utils.protocol(env)
host = Utils.host(env)
fileinfo = SQL.query_all("SELECT filename, original_filename, uploaded_at, extension, checksum, thumbnail
FROM #{CONFIG.dbTableName}
WHERE filename = ?",
env.params.url["filename"].split(".").first,
as: {filename: String, ofilename: String, up_at: String, ext: String, checksum: String, thumbnail: String | Nil})[0]
# Benchmark.ips do |x|
# x.report("header multiple") { headers(env, {"Content-Disposition" => "inline; filename*=UTF-8''#{fileinfo[:ofilename]}",
# "Last-Modified" => "#{fileinfo[:up_at]}",
# "ETag" => "#{fileinfo[:checksum]}"}) }
# x.report("shorter sleep") do
# env.response.headers["Content-Disposition"] = "inline; filename*=UTF-8''#{fileinfo[:ofilename]}"
# env.response.headers["Last-Modified"] = "#{fileinfo[:up_at]}"
# env.response.headers["ETag"] = "#{fileinfo[:checksum]}"
# end
# end
# `env.response.headers` is faster than `headers(env, Hash(String, String))`
# https://github.com/kemalcr/kemal/blob/3243b8e0e03568ad3bd9f0ad6f445c871605b821/src/kemal/helpers/helpers.cr#L102C1-L104C4
env.response.headers["Content-Disposition"] = "inline; filename*=UTF-8''#{fileinfo[:ofilename]}"
# env.response.headers["Last-Modified"] = "#{fileinfo[:up_at]}"
env.response.headers["ETag"] = "#{fileinfo[:checksum]}"
CONFIG.opengraphUseragents.each do |useragent|
if env.request.headers.try &.["User-Agent"].includes?(useragent)
env.response.content_type = "text/html"
return %(
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta property="og:title" content="#{fileinfo[:ofilename]}">
<meta property="og:url" content="#{protocol}://#{host}/#{fileinfo[:filename]}">
#{if fileinfo[:thumbnail]
%(<meta property="og:image" content="#{protocol}://#{host}/thumbnail/#{fileinfo[:filename]}.jpg">)
end}
</head>
</html>
)
end
end
send_file env, "#{CONFIG.files}/#{fileinfo[:filename]}#{fileinfo[:ext]}"
rescue ex
LOGGER.debug "File '#{env.params.url["filename"]}' does not exist: #{ex.message}"
return error403("File '#{env.params.url["filename"]}' does not exist")
end
end
def retrieve_thumbnail(env)
begin
send_file env, "#{CONFIG.thumbnails}/#{env.params.url["thumbnail"]}"
rescue ex
LOGGER.debug "Thumbnail '#{env.params.url["thumbnail"]}' does not exist: #{ex.message}"
return error403("Thumbnail '#{env.params.url["thumbnail"]}' does not exist")
end
end
def stats(env)
env.response.content_type = "application/json"
begin
json_data = JSON.build do |json|
json.object do
json.field "stats" do
json.object do
json.field "filesHosted", SQL.query_one "SELECT COUNT (filename) FROM #{CONFIG.dbTableName}", as: Int32
json.field "maxUploadSize", CONFIG.size_limit
json.field "thumbnailGeneration", CONFIG.generateThumbnails
json.field "filenameLength", CONFIG.fileameLength
json.field "alternativeDomains", CONFIG.alternativeDomains
end
end
end
end
rescue ex
LOGGER.error "Unknown error: #{ex.message}"
return error500("Unknown error")
end
json_data
end
def delete_file(env)
if SQL.query_one "SELECT EXISTS(SELECT 1 FROM #{CONFIG.dbTableName} WHERE delete_key = ?)", env.params.query["key"], as: Bool
begin
fileinfo = SQL.query_all("SELECT filename, extension, thumbnail
FROM #{CONFIG.dbTableName}
WHERE delete_key = ?",
env.params.query["key"],
as: {filename: String, extension: String, thumbnail: String | Nil})[0]
# Delete file
File.delete("#{CONFIG.files}/#{fileinfo[:filename]}#{fileinfo[:extension]}")
if fileinfo[:thumbnail]
# Delete thumbnail
File.delete("#{CONFIG.thumbnails}/#{fileinfo[:thumbnail]}")
end
# Delete entry from db
SQL.exec "DELETE FROM #{CONFIG.dbTableName} WHERE delete_key = ?", env.params.query["key"]
LOGGER.debug "File '#{fileinfo[:filename]}' was deleted using key '#{env.params.query["key"]}'}"
return msg("File '#{fileinfo[:filename]}' deleted successfully")
rescue ex
LOGGER.error("Unknown error: #{ex.message}")
return error500("Unknown error")
end
else
LOGGER.debug "Key '#{env.params.query["key"]}' does not exist"
return error401("Delete key '#{env.params.query["key"]}' does not exist. No files were deleted")
end
end
def sharex_config(env)
host = Utils.host(env)
protocol = Utils.protocol(env)
env.response.content_type = "application/json"
# So it's able to download the file instead of displaying it
env.response.headers["Content-Disposition"] = "attachment; filename=\"#{host}.sxcu\""
return %({
"Version": "14.0.1",
"DestinationType": "ImageUploader, FileUploader",
"RequestMethod": "POST",
"RequestURL": "#{protocol}://#{host}/upload",
"Body": "MultipartFormData",
"FileFormName": "file",
"URL": "{json:link}",
"DeletionURL": "{json:deleteLink}",
"ErrorMessage": "{json:error}"
})
end
end

View file

@ -1,40 +0,0 @@
macro error401(message)
env.response.content_type = "application/json"
env.response.status_code = 401
error_message = {"error" => {{message}}}.to_json
error_message
end
macro error403(message)
env.response.content_type = "application/json"
env.response.status_code = 403
error_message = {"error" => {{message}}}.to_json
error_message
end
macro error404(message)
env.response.content_type = "application/json"
env.response.status_code = 404
error_message = {"error" => {{message}}}.to_json
error_message
end
macro error413(message)
env.response.content_type = "application/json"
env.response.status_code = 413
error_message = {"error" => {{message}}}.to_json
error_message
end
macro error500(message)
env.response.content_type = "application/json"
env.response.status_code = 500
error_message = {"error" => {{message}}}.to_json
error_message
end
macro msg(message)
env.response.content_type = "application/json"
msg = {"message" => {{message}}}.to_json
msg
end

View file

@ -1,28 +1,27 @@
# Pretty cool way to write background jobs! :)
module Jobs
def self.check_old_files
if CONFIG.deleteFilesCheck <= 0
if CONFIG.delete_files_check <= 0
LOGGER.info "File deletion is disabled"
return
end
spawn do
loop do
Utils.check_old_files
sleep CONFIG.deleteFilesCheck
sleep CONFIG.delete_files_check.seconds
end
end
end
def self.retrieve_tor_exit_nodes
if !CONFIG.blockTorAddresses
if !CONFIG.block_tor_addresses
return
end
LOGGER.info("Blocking Tor exit nodes")
spawn do
loop do
Utils.retrieve_tor_exit_nodes
# Updates the @@exit_nodes array instantly
Routing.reload_exit_nodes
sleep CONFIG.torExitNodesCheck
Utils::Tor.refresh_exit_nodes
sleep CONFIG.tor_exit_nodes_check.seconds
end
end
end

View file

@ -1,4 +1,6 @@
# https://github.com/iv-org/invidious/blob/master/src/invidious/helpers/logger.cr
require "colorize"
enum LogLevel
All = 0
Trace = 1
@ -11,7 +13,9 @@ enum LogLevel
end
class LogHandler < Kemal::BaseLogHandler
def initialize(@io : IO = STDOUT, @level = LogLevel::Debug)
def initialize(@io : IO = STDOUT, @level = LogLevel::Debug, use_color : Bool = true)
Colorize.enabled = use_color
Colorize.on_tty_only!
end
def call(context : HTTP::Server::Context)
@ -21,42 +25,32 @@ class LogHandler < Kemal::BaseLogHandler
# Default: full path with parameters
requested_url = context.request.resource
# Try not to log search queries passed as GET parameters during normal use
# (They will still be logged if log level is 'Debug' or 'Trace')
if @level > LogLevel::Debug && (
requested_url.downcase.includes?("search") || requested_url.downcase.includes?("q=")
)
# Log only the path
requested_url = context.request.path
end
info("#{context.response.status_code} #{context.request.method} #{requested_url} #{elapsed_text}")
context
end
def puts(message : String)
@io << message << '\n'
@io.flush
end
def write(message : String)
@io << message
@io.flush
end
def set_log_level(level : String)
@level = LogLevel.parse(level)
end
def set_log_level(level : LogLevel)
@level = level
def color(level)
case level
when LogLevel::Trace then :cyan
when LogLevel::Debug then :green
when LogLevel::Info then :white
when LogLevel::Warn then :yellow
when LogLevel::Error then :red
when LogLevel::Fatal then :magenta
else :default
end
end
{% for level in %w(trace debug info warn error fatal) %}
def {{level.id}}(message : String)
if LogLevel::{{level.id.capitalize}} >= @level
puts("#{Time.utc} [{{level.id}}] #{message}")
puts("#{Time.utc} [{{level.id}}] #{message}".colorize(color(LogLevel::{{level.id.capitalize}})))
end
end
{% end %}

26
src/macros.cr Normal file
View file

@ -0,0 +1,26 @@
macro ee(status_code, message)
env.response.content_type = "application/json"
env.response.status_code = {{status_code}}
msg = {"error" => {{message}}}.to_json
return msg
end
macro msg(message)
env.response.content_type = "application/json"
msg = {"message" => {{message}}}.to_json
return msg
end
module Headers
macro host
env.request.headers["X-Forwarded-Host"]? || env.request.headers["Host"]?
end
macro scheme
env.request.headers["X-Forwarded-Proto"]? || "http"
end
macro ip_addr
env.request.headers["X-Real-IP"]? || env.request.remote_address.as?(Socket::IPAddress).try &.address
end
end

View file

@ -1,6 +1,4 @@
require "../http-errors"
module Handling::Admin
module Routes::Admin
extend self
# private macro json_fill(named_tuple, field_name)
@ -17,7 +15,7 @@ module Handling::Admin
file = file.to_s
begin
fileinfo = SQL.query_one("SELECT filename, extension, thumbnail
FROM #{CONFIG.dbTableName}
FROM files
WHERE filename = ?",
file,
as: {filename: String, extension: String, thumbnail: String | Nil})
@ -29,7 +27,7 @@ module Handling::Admin
File.delete("#{CONFIG.thumbnails}/#{fileinfo[:thumbnail]}")
end
# Delete entry from db
SQL.exec "DELETE FROM #{CONFIG.dbTableName} WHERE filename = ?", file
SQL.exec "DELETE FROM files WHERE filename = ?", file
LOGGER.debug "File '#{fileinfo[:filename]}' was deleted"
successfull_files << file
rescue ex : DB::NoResultsError
@ -37,7 +35,7 @@ module Handling::Admin
failed_files << file
rescue ex
LOGGER.error "Unknown error: #{ex.message}"
error500 "Unknown error: #{ex.message}"
http_error 500, "Unknown error: #{ex.message}"
end
end
json = JSON.build do |j|
@ -61,7 +59,7 @@ module Handling::Admin
item = item.to_s
begin
# Delete entry from db
SQL.exec "DELETE FROM #{CONFIG.ipTableName} WHERE ip = ?", item
SQL.exec "DELETE FROM ips WHERE ip = ?", item
LOGGER.debug "Rate limit for '#{item}' was deleted"
successfull << item
rescue ex : DB::NoResultsError
@ -69,7 +67,7 @@ module Handling::Admin
failed << item
rescue ex
LOGGER.error "Unknown error: #{ex.message}"
error500 "Unknown error: #{ex.message}"
Macros.ee 500, "Unknown error: #{ex.message}"
end
end
json = JSON.build do |j|
@ -95,7 +93,7 @@ module Handling::Admin
begin
fileinfo = SQL.query_one("SELECT original_filename, filename, extension,
uploaded_at, checksum, ip, delete_key, thumbnail
FROM #{CONFIG.dbTableName}
FROM files
WHERE filename = ?",
item,
as: {original_filename: String, filename: String, extension: String,
@ -107,7 +105,7 @@ module Handling::Admin
failed << item
rescue ex
LOGGER.error "Unknown error: #{ex.message}"
error500 "Unknown error: #{ex.message}"
Macros.ee 500, "Unknown error: #{ex.message}"
end
end
json = JSON.build do |j|
@ -152,13 +150,13 @@ module Handling::Admin
# /api/admin/whitelist
# curl -X GET -H "X-Api-Key: asd" http://localhost:8080/api/admin/torexitnodes | jq
# def add_ip_to_whitelist(env, nodes)
# json = JSON.build do |j|
# j.object do
# j.field "ips", nodes
# end
# end
# end
# def add_ip_to_whitelist(env, nodes)
# json = JSON.build do |j|
# j.object do
# j.field "ips", nodes
# end
# end
# end
# /api/admin/blacklist
# curl -X GET -H "X-Api-Key: asd" http://localhost:8080/api/admin/torexitnodes | jq

39
src/routes/delete.cr Normal file
View file

@ -0,0 +1,39 @@
module Routes::Deletion
extend self
def delete_file(env)
key = env.params.query["key"]?
if !key || key.empty?
ee 400, "No delete key supplied"
end
file = Database::Files.select_with_key(key)
if file
full_filename = file.filename + file.extension
thumbnail = file.thumbnail
begin
# Delete file
File.delete("#{CONFIG.files}/#{full_filename}")
if file.thumbnail
File.delete("#{CONFIG.thumbnails}/#{thumbnail}")
end
# Delete entry from db
Database::Files.delete_with_key(key)
LOGGER.debug "File '#{full_filename}' was deleted using key '#{key}'}"
msg("File '#{full_filename}' deleted successfully")
rescue ex
LOGGER.error("Unknown error: #{ex.message}")
ee 500, "Unknown error"
end
else
LOGGER.debug "Key '#{env.params.query["key"]}' does not exist"
ee 401, "Delete key '#{env.params.query["key"]}' does not exist. No files were deleted"
end
end
end

66
src/routes/misc.cr Normal file
View file

@ -0,0 +1,66 @@
require "http/client"
module Routing::Misc
extend self
struct Stats
include JSON::Serializable
@[JSON::Field(key: "filesHosted")]
property files_hosted : Int32
@[JSON::Field(key: "maxUploadSize")]
property max_upload_size : String
@[JSON::Field(key: "thumbnailGeneration")]
property thumbnail_generation : Bool
@[JSON::Field(key: "filenameLength")]
property filename_length : Int32
@[JSON::Field(key: "alternativeDomains")]
property alternative_domains : Array(String)
def initialize
@files_hosted = Database::Files.file_count
@max_upload_size = CONFIG.size_limit.to_s
@thumbnail_generation = CONFIG.generate_thumbnails
@filename_length = CONFIG.filename_length
@alternative_domains = CONFIG.alternative_domains
end
end
def stats(env)
env.response.content_type = "application/json"
Stats.new.to_json
end
def sharex_config(env)
host = env.request.headers["X-Forwarded-Host"]? || env.request.headers["Host"]?
scheme = env.request.headers["X-Forwarded-Proto"]? || "http"
env.response.content_type = "application/json"
# So it's able to download the file instead of displaying it
env.response.headers["Content-Disposition"] = "attachment; filename=\"#{host}.sxcu\""
return %({
"Version": "14.0.1",
"DestinationType": "ImageUploader, FileUploader",
"RequestMethod": "POST",
"RequestURL": "#{scheme}://#{host}/upload",
"Body": "MultipartFormData",
"FileFormName": "file",
"URL": "{json:link}",
"DeletionURL": "{json:deleteLink}",
"ErrorMessage": "{json:error}"
})
end
def chatterino_config(env)
host = env.request.headers["X-Forwarded-Host"]? || env.request.headers["Host"]?
scheme = env.request.headers["X-Forwarded-Proto"]? || "http"
env.response.content_type = "application/json"
return %({
"requestUrl": "#{scheme}://#{host}/upload",
formField": "data",
imageLink": "{link}",
deleteLink": "{deleteLink}"
})
end
end

51
src/routes/retrieve.cr Normal file
View file

@ -0,0 +1,51 @@
module Routes::Retrieve
extend self
def retrieve_file(env)
host = env.request.headers["X-Forwarded-Host"]? || env.request.headers["Host"]?
scheme = env.request.headers["X-Forwarded-Proto"]? || "http"
ip_addr = env.request.headers["X-Real-IP"]? || env.request.remote_address.as?(Socket::IPAddress).try &.address
filename = env.params.url["filename"].split(".").first
begin
file = Database::Files.select(filename)
if file.nil?
ee 404, "File '#{filename}' does not exist"
end
rescue ex
LOGGER.debug "Error when retrieving file '#{filename}': #{ex.message}"
ee 500, "Error when retrieving file '#{filename}'"
end
env.response.headers["Content-Disposition"] = "inline; filename*=UTF-8''#{file.original_filename}"
env.response.headers["ETag"] = "#{file.checksum}"
CONFIG.opengraph_useragents.each do |useragent|
env.response.content_type = "text/html"
if env.request.headers["User-Agent"]?.try &.includes?(useragent)
return %(<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta property="og:title" content="#{file.original_filename}">
<meta property="og:url" content="#{scheme}://#{host}/#{file.filename}">
#{%(<meta property="og:image" content="#{scheme}://#{host}/thumbnail/#{file.filename}.jpg">) if file.thumbnail}
</head>
</html>)
end
end
send_file env, "#{CONFIG.files}/#{file.filename}#{file.extension}"
end
def retrieve_thumbnail(env)
thumbnail = env.params.url["thumbnail"]?
begin
send_file env, "#{CONFIG.thumbnails}/#{thumbnail}"
rescue ex
LOGGER.debug "Thumbnail '#{thumbnail}' does not exist: #{ex.message}"
ee 403, "Thumbnail '#{thumbnail}' does not exist"
end
end
end

112
src/routes/upload.cr Normal file
View file

@ -0,0 +1,112 @@
module Routes::Upload
extend self
struct Response
include JSON::Serializable
property link : String
@[JSON::Field(key: "linkExt")]
property link_ext : String
property id : String
property ext : String
property name : String
property checksum : String?
@[JSON::Field(key: "deleteKey")]
property delete_key : String
@[JSON::Field(key: "deleteLink")]
property delete_link : String
def initialize(file : UFile, scheme : String, host : String?)
@link = "#{scheme}://#{host}/#{file.filename}"
@link_ext = "#{scheme}://#{host}/#{file.filename}#{file.extension}"
@id = file.filename
@ext = file.extension
@name = file.original_filename
@checksum = file.checksum
@delete_key = file.delete_key
@delete_link = "#{scheme}://#{host}/delete?key=#{file.delete_key}"
end
end
def upload(env)
host = env.request.headers["X-Forwarded-Host"]? || env.request.headers["Host"]?
scheme = env.request.headers["X-Forwarded-Proto"]? || "http"
ip_addr = env.request.headers["X-Real-IP"]? || env.request.remote_address.as?(Socket::IPAddress).try &.address
env.response.content_type = "application/json"
# You can modify this if you want to allow files smaller than 1MiB.
# This is generally a good way to check the filesize but there is a better way to do it
# which is inspecting the file directly (If I'm not wrong).
if CONFIG.size_limit > 0
if !env.request.headers["Content-Length"]?.try &.to_i == nil
if env.request.headers["Content-Length"].to_i > 1048576*CONFIG.size_limit
ee 413, "File is too big. The maximum size allowed is #{CONFIG.size_limit}MiB"
end
end
end
file = UFile.new
ip = UIP.new
HTTP::FormData.parse(env.request) do |upload|
upload_filename = upload.filename
if upload_filename
file.original_filename = upload_filename
else
LOGGER.debug "No file provided by the user"
ee 403, "No file provided"
end
file.filename = Utils.generate_filename
if file.original_filename == "control_v.png"
file.original_filename = file.filename
end
file.extension = File.extname("#{upload.filename}")
full_filename = file.filename + file.extension
file_path = "#{CONFIG.files}/#{full_filename}"
if CONFIG.blocked_extensions.includes?(file.extension.split(".")[1])
ee 401, "Extension '#{file.extension}' is not allowed"
end
File.open(file_path, "w") do |output|
IO.copy(upload.body, output)
end
file.uploaded_at = Time.utc.to_unix
if CONFIG.enable_checksums
file.checksum = Utils::Hashing.hash_file(file_path)
end
end
file.ip = ip_addr.to_s
ip.ip = file.ip
ip.date = file.uploaded_at
if CONFIG.delete_key_length > 0
file.delete_key = Random.base58(CONFIG.delete_key_length)
end
begin
spawn { Utils.generate_thumbnail(file.filename, file.extension) }
rescue ex
LOGGER.error "An error ocurred when trying to generate a thumbnail: #{ex.message}"
end
begin
Database::Files.insert(file)
exists = Database::IP.insert(ip).rows_affected == 0
Database::IP.increase_count(ip) if exists
rescue ex
LOGGER.error "An error ocurred when trying to insert the data into the DB: #{ex.message}"
ee 500, "An error ocurred when trying to insert the data into the DB"
end
res = Response.new(file, scheme, host)
res.to_json
end
end

18
src/routes/views.cr Normal file
View file

@ -0,0 +1,18 @@
module Routes::Views
extend self
def root(env)
host = env.request.headers["X-Forwarded-Host"]? || env.request.headers["Host"]?
scheme = env.request.headers["X-Forwarded-Proto"]? || "http"
files_hosted = Database::Files.file_count
render "src/views/index.ecr"
end
def chatterino(env)
host = env.request.headers["X-Forwarded-Host"]? || env.request.headers["Host"]?
scheme = env.request.headers["X-Forwarded-Proto"]? || "http"
render "src/views/chatterino.ecr"
end
end

View file

@ -1,106 +1,114 @@
require "./http-errors"
require "./macros"
require "./routes/**"
module Routing
extend self
@@exit_nodes = Array(String).new
def reload_exit_nodes
LOGGER.debug "Updating Tor exit nodes array"
@@exit_nodes = Utils.load_tor_exit_nodes
LOGGER.debug "IPs inside the exit nodes array: #{@@exit_nodes.size}"
end
{% for http_method in {"get", "post", "delete", "options", "patch", "put"} %}
before_post "/api/admin/*" do |env|
if env.request.headers.try &.["X-Api-Key"]? != CONFIG.adminApiKey || nil
halt env, status_code: 401, response: error401("Wrong API Key")
macro {{http_method.id}}(path, controller, method = :handle)
unless Kemal::Utils.path_starts_with_slash?(\{{path}})
raise Kemal::Exceptions::InvalidPathStartException.new({{http_method}}, \{{path}})
end
Kemal::RouteHandler::INSTANCE.add_route({{http_method.upcase}}, \{{path}}) do |env|
\{{ controller }}.\{{ method.id }}(env)
end
end
end
{% end %}
# before_post "/api/admin/*" do |env|
# env.response.content_type = "application/json"
# if env.request.headers.try &.["X-Api-Key"]? != CONFIG.admin_api_key || nil
# halt env, status_code: 401, response: "Wrong API Key"
# end
# end
before_post do |env|
if env.request.headers.try &.["X-Api-Key"]? == CONFIG.adminApiKey
# Skips Tor and Rate limits if the API key matches
tor_exit_nodes = Utils::Tor.exit_nodes
api_key = env.request.headers["X-Api-Key"]?
# Skips Tor blocking and Rate limits if the API key matches
if api_key == CONFIG.admin_api_key
next
end
if CONFIG.blockTorAddresses && @@exit_nodes.includes?(Utils.ip_address(env))
halt env, status_code: 401, response: error401(CONFIG.torMessage)
if CONFIG.block_tor_addresses && tor_exit_nodes.includes?(Headers.ip_addr)
halt env, status_code: 401, response: CONFIG.tor_message
end
# There is a better way to do this
if env.request.resource == "/upload"
begin
ip_info = SQL.query_all("SELECT ip, count, date FROM #{CONFIG.ipTableName} WHERE ip = ?", Utils.ip_address(env), as: {ip: String, count: Int32, date: Int32})[0]
time_since_first_upload = Time.utc.to_unix - ip_info[:date]
time_until_unban = ip_info[:date] - Time.utc.to_unix + CONFIG.rateLimitPeriod
if time_since_first_upload > CONFIG.rateLimitPeriod
SQL.exec "DELETE FROM #{CONFIG.ipTableName} WHERE ip = ?", ip_info[:ip]
end
if ip_info[:count] >= CONFIG.filesPerIP && time_since_first_upload < CONFIG.rateLimitPeriod
halt env, status_code: 401, response: error401("Rate limited! Try again in #{time_until_unban} seconds")
end
rescue ex
LOGGER.error "Error when trying to enforce rate limits: #{ex.message}"
next
end
before_post "/upload" do |env|
ip = Headers.ip_addr
if !ip
halt env, status_code: 401, response: "X-Real-IP header not present. Contact the admin to fix this!"
end
ip_info = Database::IP.select(ip)
if ip_info.nil?
next
end
if CONFIG.files_per_ip > 0
time_since_first_upload = Time.utc.to_unix - ip_info.date
time_until_unban = ip_info.date - Time.utc.to_unix + CONFIG.rate_limit_period
if time_since_first_upload > CONFIG.rate_limit_period
Database::IP.delete(ip_info.ip)
end
if ip_info.count >= CONFIG.files_per_ip && time_since_first_upload < CONFIG.rate_limit_period
halt env, status_code: 401, response: "Rate limited! Try again in #{time_until_unban} seconds"
end
end
end
def register_all
get "/" do |env|
host = Utils.host(env)
files_hosted = SQL.query_one "SELECT COUNT (filename) FROM #{CONFIG.dbTableName}", as: Int32
render "src/views/index.ecr"
end
get "/", Routes::Views, :root
get "/info/chatterino", Routes::Views, :chatterino
post "/upload" do |env|
Handling.upload(env)
end
post "/upload", Routes::Upload, :upload
post "/api/uploadurl" do |env|
Handling.upload_url(env)
end
get "/:filename", Routes::Retrieve, :retrieve_file
get "/thumbnail/:thumbnail", Routes::Retrieve, :retrieve_thumbnail
get "/:filename" do |env|
Handling.retrieve_file(env)
end
get "/delete", Routes::Deletion, :delete_file
get "/thumbnail/:thumbnail" do |env|
Handling.retrieve_thumbnail(env)
end
get "/api/stats", Routing::Misc, :stats
get "/info/sharex.sxcu", Routing::Misc, :sharex_config
get "/info/chatterinoconfig", Routing::Misc, :chatterino_config
get "/delete" do |env|
Handling.delete_file(env)
end
get "/api/stats" do |env|
Handling.stats(env)
end
get "/sharex.sxcu" do |env|
Handling.sharex_config(env)
end
if CONFIG.adminEnabled
self.register_admin
end
# if CONFIG.admin_enabled
# self.register_admin
# end
end
def register_admin
# post "/api/admin/upload" do |env|
# Handling::Admin.delete_ip_limit(env)
# end
post "/api/admin/delete" do |env|
Handling::Admin.delete_file(env)
end
end
# def register_admin
# # post "/api/admin/upload" do |env|
# # Routes::Admin.delete_ip_limit(env)
# # end
# post "/api/admin/delete" do |env|
# Routes::Admin.delete_file(env)
# end
# end
post "/api/admin/deleteiplimit" do |env|
Handling::Admin.delete_ip_limit(env)
end
# post "/api/admin/deleteiplimit" do |env|
# Routes::Admin.delete_ip_limit(env)
# end
post "/api/admin/fileinfo" do |env|
Handling::Admin.retrieve_file_info(env)
end
# post "/api/admin/fileinfo" do |env|
# Routes::Admin.retrieve_file_info(env)
# end
get "/api/admin/torexitnodes" do |env|
Handling::Admin.retrieve_tor_exit_nodes(env, @@exit_nodes)
# get "/api/admin/torexitnodes" do |env|
# Routes::Admin.retrieve_tor_exit_nodes(env, @@exit_nodes)
# end
error 404 do |env|
env.response.content_type = "text/plain"
"File not found.\nArchivo no encontrado."
end
end

24
src/types/ip.cr Normal file
View file

@ -0,0 +1,24 @@
struct UIP
# Without this, this class will not be able to be used as `as: IP` on
# SQL queries
include DB::Serializable
property ip : String
property count : Int32
property date : Int64
def initialize(
@ip = "",
@count = 1,
@date = 0,
)
end
def to_tuple
{% begin %}
{
{{@type.instance_vars.map(&.name).splat}}
}
{% end %}
end
end

34
src/types/ufile.cr Normal file
View file

@ -0,0 +1,34 @@
struct UFile
# Without this, this class will not be able to be used as `as: UFile` on
# SQL queries
include DB::Serializable
property original_filename : String
property filename : String
property extension : String
property uploaded_at : Int64
property checksum : String?
property ip : String
property delete_key : String
property thumbnail : String?
def initialize(
@original_filename = "",
@filename = "",
@extension = "",
@uploaded_at = 0,
@checksum = nil,
@ip = "",
@delete_key = "",
@thumbnail = nil,
)
end
def to_tuple
{% begin %}
{
{{@type.instance_vars.map(&.name).splat}}
}
{% end %}
end
end

View file

@ -1,270 +0,0 @@
module Utils
extend self
def create_db
if !SQL.query_one "SELECT EXISTS (SELECT 1 FROM sqlite_schema WHERE type='table' AND name='#{CONFIG.dbTableName}')
AND EXISTS (SELECT 1 FROM sqlite_schema WHERE type='table' AND name='#{CONFIG.ipTableName}');", as: Bool
LOGGER.info "Creating sqlite3 database at '#{CONFIG.db}'"
begin
SQL.exec "CREATE TABLE IF NOT EXISTS #{CONFIG.dbTableName}
(original_filename text, filename text, extension text, uploaded_at text, checksum text, ip text, delete_key text, thumbnail text)"
SQL.exec "CREATE TABLE IF NOT EXISTS #{CONFIG.ipTableName}
(ip text UNIQUE, count integer DEFAULT 0, date integer)"
rescue ex
LOGGER.fatal "#{ex.message}"
exit(1)
end
end
end
def create_files_dir
if !Dir.exists?("#{CONFIG.files}")
LOGGER.info "Creating files folder under '#{CONFIG.files}'"
begin
Dir.mkdir("#{CONFIG.files}")
rescue ex
LOGGER.fatal "#{ex.message}"
exit(1)
end
end
end
def create_thumbnails_dir
if !CONFIG.thumbnails
if !Dir.exists?("#{CONFIG.thumbnails}")
LOGGER.info "Creating thumbnails folder under '#{CONFIG.thumbnails}'"
begin
Dir.mkdir("#{CONFIG.thumbnails}")
rescue ex
LOGGER.fatal "#{ex.message}"
exit(1)
end
end
end
end
def check_old_files
LOGGER.info "Deleting old files"
dir = Dir.new("#{CONFIG.files}")
# Delete entries from DB
SQL.exec "DELETE FROM #{CONFIG.dbTableName} WHERE uploaded_at < date('now', '-#{CONFIG.deleteFilesAfter} days');"
# Delete files
dir.each_child do |file|
if (Time.utc - File.info("#{CONFIG.files}/#{file}").modification_time).days >= CONFIG.deleteFilesAfter
LOGGER.debug "Deleting file '#{file}'"
begin
File.delete("#{CONFIG.files}/#{file}")
rescue ex
LOGGER.error "#{ex.message}"
end
end
end
# Close directory to prevent `Too many open files (File::Error)` error.
# This is because the directory class is still saved on memory for some reason.
dir.close
end
def check_dependencies
dependencies = ["ffmpeg"]
dependencies.each do |dep|
next if !CONFIG.generateThumbnails
if !Process.find_executable(dep)
LOGGER.fatal("'#{dep}' was not found")
exit(1)
end
end
end
# TODO:
# def check_duplicate(upload)
# file_checksum = SQL.query_all("SELECT checksum FROM #{CONFIG.dbTableName} WHERE original_filename = ?", upload.filename, as:String).try &.[0]?
# if file_checksum.nil?
# return
# else
# uploaded_file_checksum = hash_io(upload.body)
# pp file_checksum
# pp uploaded_file_checksum
# if file_checksum == uploaded_file_checksum
# puts "Dupl"
# end
# end
# end
def hash_file(file_path : String) : String
Digest::SHA1.hexdigest &.file(file_path)
end
def hash_io(file_path : IO) : String
Digest::SHA1.hexdigest &.update(file_path)
end
# TODO: Check if there are no other possibilities to get a random filename and exit
def generate_filename
filename = Random.base58(CONFIG.fileameLength)
loop do
if SQL.query_one("SELECT COUNT(filename) FROM #{CONFIG.dbTableName} WHERE filename = ?", filename, as: Int32) == 0
return filename
else
LOGGER.debug "Filename collision! Generating a new filename"
filename = Random.base58(CONFIG.fileameLength)
end
end
end
def generate_thumbnail(filename, extension)
# Disable generation if false
return if !CONFIG.generateThumbnails
LOGGER.debug "Generating thumbnail for #{filename + extension} in background"
process = Process.run("ffmpeg",
[
"-hide_banner",
"-i",
"#{CONFIG.files}/#{filename + extension}",
"-movflags", "faststart",
"-f", "mjpeg",
"-q:v", "2",
"-vf", "scale='min(350,iw)':'min(350,ih)':force_original_aspect_ratio=decrease, thumbnail=100",
"-frames:v", "1",
"-update", "1",
"#{CONFIG.thumbnails}/#{filename}.jpg",
])
if process.normal_exit?
LOGGER.debug "Thumbnail for #{filename + extension} generated successfully"
SQL.exec "UPDATE #{CONFIG.dbTableName} SET thumbnail = ? WHERE filename = ?", filename + ".jpg", filename
else
end
end
# Delete socket if the server has not been previously cleaned by the server
# (Due to unclean exits, crashes, etc.)
def delete_socket
if File.exists?("#{CONFIG.unix_socket}")
LOGGER.info "Deleting old unix socket"
begin
File.delete("#{CONFIG.unix_socket}")
rescue ex
LOGGER.fatal "#{ex.message}"
exit(1)
end
end
end
def delete_file(env)
fileinfo = SQL.query_all("SELECT filename, extension, thumbnail
FROM #{CONFIG.dbTableName}
WHERE delete_key = ?",
env.params.query["key"],
as: {filename: String, extension: String, thumbnail: String | Nil})[0]
# Delete file
File.delete("#{CONFIG.files}/#{fileinfo[:filename]}#{fileinfo[:extension]}")
if fileinfo[:thumbnail]
# Delete thumbnail
File.delete("#{CONFIG.thumbnails}/#{fileinfo[:thumbnail]}")
end
# Delete entry from db
SQL.exec "DELETE FROM #{CONFIG.dbTableName} WHERE delete_key = ?", env.params.query["key"]
LOGGER.debug "File '#{fileinfo[:filename]}' was deleted using key '#{env.params.query["key"]}'}"
msg("File '#{fileinfo[:filename]}' deleted successfully")
end
MAGIC_BYTES = {
# Images
".png" => "89504e470d0a1a0a",
".heic" => "6674797068656963",
".jpg" => "ffd8ff",
".gif" => "474946383",
# Videos
".mp4" => "66747970",
".webm" => "1a45dfa3",
".mov" => "6d6f6f76",
".wmv" => "󠀀3026b2758e66cf11",
".flv" => "󠀀464c5601",
".mpeg" => "000001bx",
# Audio
".mp3" => "󠀀494433",
".aac" => "󠀀fff1",
".wav" => "󠀀57415645666d7420",
".flac" => "󠀀664c614300000022",
".ogg" => "󠀀4f67675300020000000000000000",
".wma" => "󠀀3026b2758e66cf11a6d900aa0062ce6c",
".aiff" => "󠀀464f524d00",
# Whatever
".7z" => "377abcaf271c",
".gz" => "1f8b",
".iso" => "󠀀4344303031",
# Documents
"pdf" => "󠀀25504446",
"html" => "<!DOCTYPE html>",
}
def detect_extension(file) : String
file = File.open(file)
slice = Bytes.new(16)
hex = IO::Hexdump.new(file)
# Reads the first 16 bytes of the file in Heap
hex.read(slice)
MAGIC_BYTES.each do |ext, mb|
if slice.hexstring.includes?(mb)
return ext
end
end
""
end
def retrieve_tor_exit_nodes
LOGGER.debug "Retrieving Tor exit nodes list"
HTTP::Client.get(CONFIG.torExitNodesUrl) do |res|
begin
if res.success? && res.status_code == 200
begin
File.open(CONFIG.torExitNodesFile, "w") { |output| IO.copy(res.body_io, output) }
rescue ex
LOGGER.error "Failed to write to file: #{ex.message}"
end
else
LOGGER.error "Failed to retrieve exit nodes list. Status Code: #{res.status_code}"
end
rescue ex : Socket::ConnectError
LOGGER.error "Failed to connect to #{CONFIG.torExitNodesUrl}: #{ex.message}"
rescue ex
LOGGER.error "Unknown error: #{ex.message}"
end
end
end
def load_tor_exit_nodes
exit_nodes = File.read_lines(CONFIG.torExitNodesFile)
ips = [] of String
exit_nodes.each do |line|
if line.includes?("ExitAddress")
ips << line.split(" ")[1]
end
end
return ips
end
def ip_address(env) : String
begin
return env.request.headers.try &.["X-Forwarded-For"]
rescue
return env.request.remote_address.to_s.split(":").first
end
end
def protocol(env) : String
begin
return env.request.headers.try &.["X-Forwarded-Proto"]
rescue
return "http"
end
end
def host(env) : String
begin
return env.request.headers.try &.["X-Forwarded-Host"]
rescue
return env.request.headers["Host"]
end
end
end

11
src/utils/hashing.cr Normal file
View file

@ -0,0 +1,11 @@
module Utils::Hashing
extend self
def hash_file(file_path : String) : String
Digest::SHA1.hexdigest &.file(file_path)
end
def hash_io(file : IO) : String
Digest::SHA1.hexdigest &.update(file)
end
end

38
src/utils/tor.cr Normal file
View file

@ -0,0 +1,38 @@
module Utils::Tor
extend self
@@exit_nodes : Array(String) = [] of String
def refresh_exit_nodes
LOGGER.debug "reload_exit_nodes: Updating Tor exit nodes list"
retrieve_tor_exit_nodes
LOGGER.debug "reload_exit_nodes: IPs inside the Tor exit nodes list: #{@@exit_nodes.size}"
end
def retrieve_tor_exit_nodes
LOGGER.debug "retrieve_tor_exit_nodes: Retrieving Tor exit nodes list"
ips = [] of String
HTTP::Client.get(CONFIG.tor_exit_nodes_url) do |res|
begin
if res.success? && res.status_code == 200
res.body_io.each_line do |line|
if line.includes?("ExitAddress")
ips << line.split(" ")[1]
end
end
@@exit_nodes = ips
else
LOGGER.error "retrieve_tor_exit_nodes: Failed to retrieve exit nodes list. Status Code: #{res.status_code}"
end
rescue ex : Socket::ConnectError
LOGGER.error "retrieve_tor_exit_nodes: Failed to connect to #{CONFIG.tor_exit_nodes_url}: #{ex.message}"
rescue ex
LOGGER.error "retrieve_tor_exit_nodes: Unknown error: #{ex.message}"
end
end
end
def exit_nodes : Array(String)
return @@exit_nodes
end
end

274
src/utils/utils.cr Normal file
View file

@ -0,0 +1,274 @@
module Utils
extend self
def create_db
files_table = <<-SQL
CREATE TABLE
IF NOT EXISTS files
(
original_filename text not null,
filename text not null,
extension text not null,
uploaded_at integer not null,
checksum text,
ip text not null,
delete_key text not null,
thumbnail text,
PRIMARY KEY(filename)
)
SQL
ip_table = <<-SQL
CREATE TABLE
IF NOT EXISTS ips
(
ip text,
count integer DEFAULT 0,
date integer,
PRIMARY KEY(ip)
)
SQL
files_table_check = <<-SQL
SELECT EXISTS
(
SELECT 1 FROM
sqlite_schema
WHERE type='table'
AND name='files'
)
SQL
ip_table_check = <<-SQL
SELECT EXISTS
(
SELECT 1 FROM
sqlite_schema
WHERE type='table'
AND name='ips'
)
SQL
files_table_exists = SQL.query_one(files_table_check, as: Bool)
ip_table_exists = SQL.query_one(ip_table_check, as: Bool)
if (!files_table_exists)
LOGGER.info "create_db: Creating table 'files'"
begin
SQL.exec(files_table)
rescue ex
LOGGER.fatal "#{ex.message}"
exit(1)
end
end
if (!ip_table_exists)
LOGGER.info "create_db: Creating table 'ips'"
begin
SQL.exec(ip_table)
rescue ex
LOGGER.fatal "#{ex.message}"
exit(1)
end
end
end
def create_files_dir
if !Dir.exists?("#{CONFIG.files}")
LOGGER.info "Creating files folder under '#{CONFIG.files}'"
begin
Dir.mkdir("#{CONFIG.files}")
rescue ex
LOGGER.fatal "#{ex.message}"
exit(1)
end
end
end
def create_thumbnails_dir
if CONFIG.thumbnails
if !Dir.exists?("#{CONFIG.thumbnails}")
LOGGER.info "Creating thumbnails folder under '#{CONFIG.thumbnails}'"
begin
Dir.mkdir("#{CONFIG.thumbnails}")
rescue ex
LOGGER.fatal "#{ex.message}"
exit(1)
end
end
end
end
def delete_file(env)
key = env.params.query["key"]
file = SQL.select_with_key(key)
full_filename = file.filename + file.extension
thumbnail = file.thumbnail
# Delete file
File.delete("#{CONFIG.files}/#{full_filename}")
if file.thumbnail
File.delete("#{CONFIG.thumbnails}/#{thumbnail}")
end
# Delete entry from db
Database::Files.delete_with_key(key)
LOGGER.debug "File '#{full_filename}' was deleted using key '#{key}'}"
msg("File '#{full_filename}' deleted successfully")
end
# TODO: Spawn a fiber and add each file to an array to bulk delete files from
# the database using a single SQL query.
# In the end, all old files should be not accessible, even if they are on the
# drive.
def check_old_files
LOGGER.info "check_old_files: Deleting old files"
files = Database::Files.old_files
files.each do |f|
full_filename = f.filename + f.extension
thumbnail = f.thumbnail
# TODO: Check if it's able to bypass the path using a filename with a `/` in their name
LOGGER.debug "check_old_files: Deleting file '#{full_filename}'"
begin
File.delete("#{CONFIG.files}/#{full_filename}")
if thumbnail
File.delete("#{CONFIG.thumbnails}/#{thumbnail}")
end
Database::Files.delete(f.filename)
rescue File::NotFoundError
LOGGER.error "check_old_files: File '#{full_filename}' doesn't seem to exist on the '#{CONFIG.files}', folder, deleting it from the database"
Database::Files.delete(f.filename)
rescue ex : File::AccessDeniedError
LOGGER.error "check_old_files: File '#{full_filename}' failed to be deleted due to bad permissions, deleting it from the database: #{ex.message}"
Database::Files.delete(f.filename)
rescue ex
LOGGER.error "check_old_files: File '#{full_filename}' failed to be deleted, deleting it from the database: #{ex.message}"
Database::Files.delete(f.filename)
end
end
end
def check_dependencies
dependencies = ["ffmpeg"]
dependencies.each do |dep|
next if !CONFIG.generate_thumbnails
if !Process.find_executable(dep)
LOGGER.fatal("'#{dep}' was not found.")
exit(1)
end
end
end
# TODO: Check if there are no other possibilities to get a random filename and exit
def generate_filename
filename = Random.base58(CONFIG.filename_length)
loop do
file = Database::Files.select(filename)
if !file
return filename
else
LOGGER.trace "Filename collision! Generating a new filename"
filename = Random.base58(CONFIG.filename_length)
end
end
end
def generate_thumbnail(filename, extension)
exts = [".jpg", ".jpeg", ".png", ".gif", ".bmp", ".tiff", ".webp", ".heic", ".jxl", ".avif", ".crw", ".dng",
".mp4", ".mkv", ".webm", ".avi", ".wmv", ".flv", "m4v", ".mov", ".amv", ".3gp", ".mpg", ".mpeg", ".yuv"]
# To prevent thumbnail generation on non image extensions
return if exts.none? { |ext| extension.downcase.includes?(ext) }
# Disable generation if false
return if !CONFIG.generate_thumbnails || !CONFIG.thumbnails
LOGGER.debug "Generating thumbnail for #{filename + extension} in background"
process = Process.run("ffmpeg",
[
"-hide_banner",
"-i",
"#{CONFIG.files}/#{filename + extension}",
"-movflags", "faststart",
"-f", "mjpeg",
"-q:v", "2",
"-vf", "scale='min(350,iw)':'min(350,ih)':force_original_aspect_ratio=decrease, thumbnail=100",
"-frames:v", "1",
"-update", "1",
"#{CONFIG.thumbnails}/#{filename}.jpg",
])
if process.normal_exit?
LOGGER.debug "Thumbnail for '#{filename + extension}' generated successfully"
SQL.exec "UPDATE files SET thumbnail = ? WHERE filename = ?", filename + ".jpg", filename
else
LOGGER.debug "Failed to generate thumbnail for '#{filename + extension}'. Exit code of ffmpeg: #{process.exit_code}"
end
end
# Delete socket if the server has not been previously cleaned by the server
# (Due to unclean exits, crashes, etc.)
def delete_socket
if File.exists?("#{CONFIG.unix_socket}")
LOGGER.info "Deleting old unix socket"
begin
File.delete("#{CONFIG.unix_socket}")
rescue ex
LOGGER.fatal "#{ex.message}"
exit(1)
end
end
end
MAGIC_BYTES = {
# Images
".png" => "89504e470d0a1a0a",
".heic" => "6674797068656963",
".jpg" => "ffd8ff",
".gif" => "474946383",
# Videos
".mp4" => "66747970",
".webm" => "1a45dfa3",
".mov" => "6d6f6f76",
".wmv" => "󠀀3026b2758e66cf11",
".flv" => "󠀀464c5601",
".mpeg" => "000001bx",
# Audio
".mp3" => "󠀀494433",
".aac" => "󠀀fff1",
".wav" => "󠀀57415645666d7420",
".flac" => "󠀀664c614300000022",
".ogg" => "󠀀4f67675300020000000000000000",
".wma" => "󠀀3026b2758e66cf11a6d900aa0062ce6c",
".aiff" => "󠀀464f524d00",
# Whatever
".7z" => "377abcaf271c",
".gz" => "1f8b",
".iso" => "󠀀4344303031",
# Documents
"pdf" => "󠀀25504446",
"html" => "<!DOCTYPE html>",
}
def detect_extension(file) : String
file = File.open(file)
slice = Bytes.new(16)
hex = IO::Hexdump.new(file)
# Reads the first 16 bytes of the file in Heap
hex.read(slice)
MAGIC_BYTES.each do |ext, mb|
if slice.hexstring.includes?(mb)
return ext
end
end
""
end
end

21
src/views/chatterino.ecr Normal file
View file

@ -0,0 +1,21 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title> <%= host %> </title>
<link rel="stylesheet" href="/styles.css">
<link rel="icon" href="/favicon.png" type="image/png" />
<script src="script.js"></script>
</head>
<body>
<div class="container">
<h1 style="font-size: 68px; text-align: center; margin: 20px;">Chatterino Config</h1>
<p>Request URL: <a style="color: #2cca00"><%= scheme %>://<%= host %>/upload</a></p>
<p>Form field: <a style="color: #2cca00">data</a></p>
<p>Image link: <a style="color: #2cca00">link</a></p>
<p>Delete link: <a style="color: #2cca00">deleteLink</a></p>
<img src="/chatterino.png">
</div>
</body>
</html>

View file

@ -4,14 +4,14 @@
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title> <%= host %> </title>
<link rel="stylesheet" href="styles.css">
<link rel="icon" href="./favicon.gif" type="image/gif" />
<link rel="stylesheet" href="/styles.css">
<link rel="icon" href="./favicon.png" type="image/gif" />
<script src="script.js"></script>
</head>
<body>
<div class="container">
<h1 style="font-size: 68px; text-align: center; margin: 20px;"><%= host %></h1>
<p style="text-align: center; font-size: 22px;"><%= CONFIG.siteInfo %></p>
<p style="text-align: center; font-size: 22px;"><%= CONFIG.site_info %></p>
<div id="drop-area">
<p style='padding: 0;margin: 0; color: #123718bf;'>Arrastra, Pega o Selecciona archivos.</p>
<input type="file" id="fileElem" accept="*/*" style="display: none;">
@ -21,24 +21,22 @@
</div>
<div>
<div style="text-align:center;">
<p>
<a href='./chatterino.png'>Chatterino Config</a> |
<a href='./sharex.sxcu'>ShareX Config</a> |
<a href='https://codeberg.org/Fijxu/file-uploader-crystal'>
file-uploader-crystal (BETA <%= CURRENT_TAG %> - <%= CURRENT_VERSION %> @ <%= CURRENT_BRANCH %>)
</a>
</p>
<p>Archivos alojados: <%= files_hosted %></p>
<% if CONFIG.blockTorAddresses %>
<p style="color: red"><%= CONFIG.torMessage %></p>
<% end %>
<% if !CONFIG.alternativeDomains.empty? %>
<p>
<% CONFIG.alternativeDomains.each do | domain | %>
<a href="https://<%= domain %>"><%= domain %></a>
<% end %>
</p>
<% end %>
<p>
<a href='/info/chatterino'>Chatterino Config</a> |
<a href='/info/sharex.sxcu'>ShareX Config</a> |
<a href='https://codeberg.org/Fijxu/file-uploader-crystal'>
file-uploader-crystal (BETA <%= CURRENT_TAG %> - <%= CURRENT_VERSION %> @ <%= CURRENT_BRANCH %>)
</a>
</p>
<p>Archivos alojados: <%= files_hosted %></p>
<% if !CONFIG.alternative_domains.empty? %>
<p>
<% CONFIG.alternative_domains.each do | domain | %>
<a href="https://<%= domain %>"><%= domain %></a>
<% end %>
</p>
<% end %>
</div>
</div>
</body>
</html>