feat: option to increase the number of videos shown on the popular page

Only works with cookies for now. Support for numbered backends will be
added later since it requires some black magic.

.forgejo/workflows/ci.yml

@@ -52,5 +52,5 @@ jobs:
           tags: ${{ steps.meta.outputs.tags }}
           platforms: linux/amd64
           push: true
-          build-args: |
+          cache-from: type=gha
-            "release=1"
+          cache-to: type=gha,mode=max

CHANGELOG.md

@@ -2,7 +2,102 @@
 
 ## vX.Y.0 (future)
 
+## v2.20250314.0
 
+### Wrap-up
+
+This release brings the long awaited feature of supporting multiple audio tracks in a video, some bug fixes and UX improvements, and many other things primarily oriented to self-hosting instances, and developers using the API.
+
+The `Community` channel tab has been replaced by `Posts` in light of YouTube changes, but the URL remains the same.
+
+Tamil is now available as an interface language
+
+Automatic instance redirects will no longer have the chance to annoyingly redirect to the same instance you're on.
+
+Due to their requirements for video playback, Invidious will log warning messages when either inv-sig-helper, `po_token` or `visitor_data` is not configured
+
+Invidious is now able to listen through a UNIX socket
+
+User notifications are now batched for each channel
+
+**The minimum Crystal version supported by Invidious now `1.12.0`**
+
+### New features & important changes
+
+#### For users
+
+* Invidious now supports videos with multiple audio tracks allowing you to select which one you want to hear with!
+* Channel pages now have a proper previous page button
+* RSS feeds for channels will no longer contain the channel's profile picture
+* Support for channel `courses` page has been added
+* `Community` tabs has been replaced with `Posts` to comply with YouTube changes
+* Tamil is now an available interface language.
+
+#### For instance owners
+* Invidious is now able to listen on a UNIX socket
+* User notifications are now batched by channels, significantly reducing database load.
+* **`1.12.0` is now the oldest Crystal version that Invidious supports**
+* The example config will no longer force an http proxy to be configured
+* Invidious will now warn when any top-level config option must be set to a custom value, instead of just `HMAC_KEY`
+* Due to their requirements for video playback, Invidious will log warning messages when either inv-sig-helper, `po_token` or `visitor_data` is not configured
+
+#### For developers
+* Invidious is now compliant to Crystal 1.15 formatting rules, which are incompatible with earlier versions.
+* `/api/v1/transcripts/{id}` has been added to the API to allow for fetching the transcripts for a video. The arguments are the same as the captions endpoint.
+* `author_thumbnail` field has been added to videos in the various paged api endpoints
+* `published` field has been added to the API response for a video's related videos.
+* Docker builds now uses the Crystal compiler cache, reducing build times on repeated builds significantly.
+* Invidious ajax action handlers has undergone a clean up and may face compatibility issues with code that depends on these endpoints.
+* The versions of Crystal that we test in CI/CD are now: `1.12.1`, `1.13.2`, `1.14.0`, `1.15.0`
+
+### Bugs fixed
+
+#### User-side
+* Local video listen mode is now preserved when clicking on a video in the sidebar playlist widget
+* Automatic instance redirects will no longer redirect to the same instance the user is on
+* Fix some thumbnails responses returning 404
+* Videos: Fix missing host parameter on playback URLs when `local=true`
+* Fix HLS being used for non-livestream videos
+* Fix timeupdate event errors when required elements are missing
+* User: Ensure IO is properly closed when importing NewPipe subscriptions
+
+#### For instance owners
+* Fix http proxy configuration being forced by the standard example config
+
+#### API
+* `/api/v1/videos/{id}` will no longer return an occasional empty JSON response
+
+### Full list of pull requests merged since the last release (newest first)
+* Make Invidious compliant to Crystal 1.15 formatting rules (https://github.com/iv-org/invidious/pull/5014, by @syeopite)
+* Remove formatter check on container workflows (https://github.com/iv-org/invidious/pull/5153, by @syeopite)
+* Videos: Fix missing host parameter on playback URLs when `local=true` (https://github.com/iv-org/invidious/pull/4992, by @SamantazFox)
+* Remove stdlib override for proxy initialization (https://github.com/iv-org/invidious/pull/5065, by @syeopite)
+* Add support for author thumbnails in search api for videos (https://github.com/iv-org/invidious/pull/5072, thanks @ChunkyProgrammer)
+* Skip route if resp got closed by before handlers (https://github.com/iv-org/invidious/pull/5073, by @syeopite)
+* Fix video thumbnails in mixes (https://github.com/iv-org/invidious/pull/5116, thanks @iBicha)
+* CI: Drop support for versions prior to 1.12 and add 1.15.0 (https://github.com/iv-org/invidious/pull/5148, by @syeopite)
+* [Continuing #5094] Set language info for dash audio streams and sort (https://github.com/iv-org/invidious/pull/5149, thanks @giuliano-macedo)
+* Warn when any top-level config is "CHANGE_ME!!" (https://github.com/iv-org/invidious/pull/5150, by @syeopite)
+* Comment out http_proxy in example config (https://github.com/iv-org/invidious/pull/5151, by @syeopite)
+* API: Add a 'published' video parameter for related videos (https://github.com/iv-org/invidious/pull/4149, thanks @RadoslavL)
+* Ensure IO is properly closed when importing NewPipe subscriptions (https://github.com/iv-org/invidious/pull/4346, thanks @ChunkyProgrammer)
+* Carry over audio-only mode in playlist links (https://github.com/iv-org/invidious/pull/4784, thanks @krystof1119)
+* Routes: Clean ajax actions handlers (https://github.com/iv-org/invidious/pull/5036, by @SamantazFox)
+* Frontend: Add a first page and previous page buttons for channel navigation (https://github.com/iv-org/invidious/pull/4123, thanks @RadoslavL)
+* RSS: Channel + Playlist improvements (https://github.com/iv-org/invidious/pull/4298, thanks @ChunkyProgrammer)
+* Batch user notifications together (https://github.com/iv-org/invidious/pull/4486, thanks @999eagle)
+* JS: Update timeupdate event making it more defensive to prevent errors (https://github.com/iv-org/invidious/pull/4782, thanks @PMK)
+* Add API endpoint for fetching transcripts from YouTube by (https://github.com/iv-org/invidious/pull/4788, by @syeopite)
+* Translations update from Hosted Weblate by (https://github.com/iv-org/invidious/pull/4989, thanks to our many translators)
+* Add the ability to listen on UNIX sockets (https://github.com/iv-org/invidious/pull/5112, thanks @Caian)
+* Pick a different instance upon redirect (https://github.com/iv-org/invidious/pull/5154, thanks @epicsam123)
+* Add Courses to channel page and channel API (https://github.com/iv-org/invidious/pull/5158, thanks @ChunkyProgrammer)
+* fix /api/v1/videos/:id returns 200 with no content (https://github.com/iv-org/invidious/pull/5162, thanks @Drikanis)
+* Use Crystal compiler cache in docker builds (https://github.com/iv-org/invidious/pull/5163, by @syeopite)
+* Channels: Fix community tab by (https://github.com/iv-org/invidious/pull/5183, thanks @Fijxu)
+* Fix typo in `src/invidious/routes/images.cr` (https://github.com/iv-org/invidious/pull/5184, by @syeopite)
+* Fix an issue with the HLS manifest check for livestream videos (https://github.com/iv-org/invidious/pull/5189, thanks @alexmaras)
+* Warn when `po_token`, `visitor_data` and/or `inv-sig-helper` is not configured (https://github.com/iv-org/invidious/pull/5202, by @syeopite)
 ## v2.20241110.0
 
 ### Wrap-up

README.md

@@ -13,33 +13,49 @@ https://git.nadeko.net/Fijxu/-/packages/container/invidious/latest
 
 ## Features and changes of this fork:
 
-- [Use a Redis compatible DB for video cache instead of just PostgreSQL](https://git.nadeko.net/Fijxu/invidious/commit/bbc5913b8dacaed4d466bcc466a0782d5e3f5edc): Invidious by default caches the video information for some hours in PostgreSQL. Since the data is accessed a lot, it is better off using an in memory database instead, it's faster and it will not wear out your SSD (due to constant writes to the database).
+- ~~[Use a Redis compatible DB for video cache instead of just PostgreSQL](https://git.nadeko.net/Fijxu/invidious/commit/bbc5913b8dacaed4d466bcc466a0782d5e3f5edc): Invidious by default caches the video information for some hours in PostgreSQL. Since the data is accessed a lot, it is better off using an in memory database instead, it's faster and it will not wear out your SSD (due to constant writes to the database).~~
 
-It can be set using this on `config.yml`:
+	~~It can be set using this on `config.yml`:~~
-```yaml
+	```yaml
-redis_url: tcp://127.0.0.1:6379
+	redis_url: tcp://127.0.0.1:6379
-```
+	```
+
+- [Ability to use different video caching backends](https://git.nadeko.net/Fijxu/invidious/commit/e76867aaba022d64ebab73648a37a0c63b788e0f): If you want, you can the PostgreSQL video cache the Redis one or the built-in in memory one that uses the LRU algorithm. Redis and LRU are recommended for public instances, but since Invidious has memory leaks, the LRU cache is lost if Invidious crashes or it's restarted, so because of this, redis is the default option.
+
+	```yaml
+	video_cache:
+	enabled: true
+	backend: 1 # 0 is PSQL, 1 Redis, 2 Built-in LRU
+	lru_max_size: 18000 # ~500MB (ignored if backend is 0 or 1)
+	```
+
+	If you choose to use Redis, make sure to set the `redis_url` config property:
+
+	```yaml
+	redis_url: tcp://127.0.0.1:6379
+	```
 
 - [Removal of materialized views on PostgreSQL](github.com/iv-org/invidious/pull/2469): If you don't have this on your Invidious public instance, your SSD will suffer and it will catch on fire https://github.com/iv-org/invidious/pull/2469#issuecomment-2012623454
 
-- External video playback proxy: Let's you use an external video playback proxy like https://git.nadeko.net/Fijxu/http3-ytproxy or https://github.com/TeamPiped/piped-proxy instead of the one that is bundled with Invidious. It's useful if you are proxying video and your throughput is not low. I did this to distribute the traffic across different servers. If you are selfhosting only for a few amount of people, this is not really useful for you.
-
-It can be set using this on `config.yml`:
-```yaml
-external_videoplayback_proxy: "https://inv-proxy.example.com"
-```
-
-> [!NOTE]
-> If you setup this, Invidious will check if the proxy is alive doing a request to `https://inv-proxy.example.com/health`, and if it doesn't get a response code of 200, Invidious will fallback to the local videoplayback proxy! This is only currently supported by https://git.nadeko.net/Fijxu/http3-ytproxy
-
 - Limit the DASH resolution sent to the clients: It can be set using `max_dash_resolution` on the config. Example: `max_dash_resolution: 1080`
 
 - [Limit requests made to Youtube API when pulling subscriptions (feeds)](https://git.nadeko.net/Fijxu/invidious/commit/df94f1c0b82d95846574487231ea251530838ef0): Due to the recent changes of Youtube ("This helps protect out community", "Sign in to confirm you are not a bot"), subscriptions now have limited information, this is because Invidious by default, makes a video request to youtube to be able to get more information about the video, like `length_seconds`, `live_now`, `premiere_timestamp`, and `views`. If you have a lot of users with a ton of subscriptions, Invidious will basically spam youtube API all the time, resulting in a block from youtube.
 
-It can be set using this on `config.yml`:
+	It can be set using this on `config.yml`:
-```yaml
+	```yaml
-use_innertube_for_feeds: false
+	use_innertube_for_feeds: false
-```
+	```
+
+
+- Autoreload configuration: If you are hosting Invidious on Linux without docker, this may be useful for you if you want to change the banner without restarting Invidious.
+
+	```yaml
+	reload_config_automatically: true
+	```
+
+## Development features
+
+- Option to disable CSP: Useful for local development, set `csp: false` on the config and done
 
 ---
 

assets/js/.gitignore

@@ -0,0 +1 @@
+minified

config/config.example.yml

@@ -1072,8 +1072,7 @@ default_user_preferences:
   ##
   #extend_desc: false
 
-# redis_url: 127.0.0.1:6379
+# redis_url: redis://127.0.0.1:6379/0?initial_pool_size=1&max_pool_size=10&checkout_timeout=10&retry_attempts=2&retry_delay=0.5&max_idle_pool_size=50
-# redis_socket: /var/run/valkey/valkey.sock
 # donation_url: "https://example.com/donate"
 # contact_url: "https://example.com/contact"
 # home_domain: "https://example.com/

docker/Dockerfile

@@ -1,4 +1,4 @@
-FROM mirror.gcr.io/crystallang/crystal:1.15.1-alpine AS builder
+FROM mirror.gcr.io/crystallang/crystal:1.16.0-alpine AS builder
 
 RUN apk add --no-cache sqlite-static yaml-static
 
@@ -7,6 +7,7 @@ ARG release
 WORKDIR /invidious
 COPY ./shard.yml ./shard.yml
 COPY ./shard.lock ./shard.lock
+
 RUN shards install --production
 
 COPY ./src/ ./src/
@@ -19,18 +20,11 @@ COPY ./scripts/ ./scripts/
 COPY ./assets/ ./assets/
 COPY ./videojs-dependencies.yml ./videojs-dependencies.yml
 
-RUN crystal spec --warnings all \
+RUN --mount=type=cache,target=/root/.cache/crystal \
-    --link-flags "-lxml2 -llzma"
-RUN --mount=type=cache,target=/root/.cache/crystal if [[ "${release}" == 1 ]] ; then \
         crystal build ./src/invidious.cr \
         --release --mcpu=x86-64-v2 \
         --static --warnings all \
-        --link-flags "-lxml2 -llzma"; \
+        --link-flags "-lxml2 -llzma";
-    else \
-        crystal build ./src/invidious.cr \
-        --static --warnings all \
-        --link-flags "-lxml2 -llzma"; \
-    fi
 
 FROM mirror.gcr.io/alpine:3.20
 RUN apk add --no-cache rsvg-convert ttf-opensans tini tzdata

locales/en-US.json

@@ -522,6 +522,12 @@
     "carousel_slide": "Slide {{current}} of {{total}}",
     "carousel_skip": "Skip the Carousel",
     "carousel_go_to": "Go to slide `x`",
-		"footer_contact_url": "Contact the Administrator"
+    "footer_contact_url": "Contact the Administrator",
-
+    "new_username": "New username",
+    "change_username": "Change username",
+    "username_required_field": "Username is a required field",
+    "username_empty": "Username cannot be empty",
+    "username_is_the_same": "This is your username, use another one",
+    "username_taken": "Username is already taken, use another one",
+    "backend_unavailable": "The backend you selected is unavailable. You have been redirected to the next one"
 }

locales/es.json

@@ -518,5 +518,12 @@
     "carousel_go_to": "Ir a la diapositiva `x`",
 	"footer_contact_url": "Contactar al Administrador",
     "preferences_preload_label": "Precargar datos del vídeo: ",
-    "Filipino (auto-generated)": "Filipino (generado automáticamente)"
+    "Filipino (auto-generated)": "Filipino (generado automáticamente)",
+    "new_username": "Nuevo nombre de usuario",
+    "change_username": "Cambiar nombre de usuario",
+    "username_required_field": "El nombre de usuario es un campo obligatorio",
+    "username_empty": "El nombre de usuario no puede estar vacío",
+    "username_is_the_same": "Este es tu nombre de usuario, usa otro",
+    "username_taken": "El nombre de usuario ya está en uso, usa otro",
+    "backend_unavailable": "El backend seleccionado no está disponible. Has sido redireccionado al siguiente"
 }

scripts/minify-js.cr

@@ -0,0 +1,97 @@
+require "http"
+require "colorize"
+
+ESBUILD_VERSION = "0.25.0"
+
+esbuild_os = ""
+esbuild_arch = ""
+
+# https://esbuild.github.io/getting-started/#other-ways-to-install
+{% if flag?(:linux) %}
+  esbuild_os = "linux"
+{% elsif flag?(:windows) %}
+  esbuild_os = "win32"
+{% elsif flag?(:darwin) %}
+  esbuild_os = "darwin"
+{% elsif flag?(:freebsd) %}
+  esbuild_os = "freebsd"
+{% elsif flag?(:openbsd) %}
+  esbuild_os = "openbsd"
+{% elsif flag?(:netbsd) %}
+  esbuild_os = "netbsd"
+{% elsif flag?(:solaris) %}
+  esbuild_os = "sunos"
+{% else %}
+  esbuild_os = "linux"
+{% end %}
+
+{% if flag?(:x86_64) %}
+  esbuild_arch = "x64"
+{% elsif flag?(:arm64) %}
+  esbuild_arch = "arm64"
+{% else %}
+  esbuild_arch = "x64"
+{% end %}
+
+tmp_dir_path = "#{Dir.tempdir}/invidious-esbuild-binary"
+esbuild_tar_location = "#{tmp_dir_path}/esbuild-#{esbuild_os}-#{esbuild_os}-#{ESBUILD_VERSION}.tgz"
+esbuild_binary_location = "#{tmp_dir_path}/package/bin/esbuild"
+Dir.mkdir(tmp_dir_path) if !Dir.exists? tmp_dir_path
+
+esbuild_url = "https://registry.npmjs.org/@esbuild/#{esbuild_os}-#{esbuild_arch}/-/#{esbuild_os}-#{esbuild_arch}-#{ESBUILD_VERSION}.tgz"
+
+# Download esbuild binary
+HTTP::Client.get(esbuild_url) do |response|
+  puts "Downloading esbuild from #{esbuild_url}"
+  data = response.body_io.gets_to_end
+  File.write(esbuild_tar_location, data)
+
+  `tar -vzxf '#{esbuild_tar_location}' -C '#{tmp_dir_path}'`
+  raise "Extraction for #{esbuild_tar_location} failed" if !$?.success?
+  puts "esbuild downloaded successfully"
+end
+
+files_to_minify = [
+  "_helpers.js",
+  "comments.js",
+  "embed.js",
+  "handlers.js",
+  "notifications.js",
+  "pagination.js",
+  "playlist_widget.js",
+  "post.js",
+  "sse.js",
+  "subscribe_widget.js",
+  "themes.js",
+  "watch.js",
+  "player.js",
+  "watched_indicator.js",
+  "watched_widget.js",
+]
+
+files_to_minify.each do |file|
+  file_path = "assets/js/#{file}"
+  outdir = "assets/js/minified"
+  process_output = IO::Memory.new
+
+  process = Process.run("#{esbuild_binary_location}", error: process_output, args: [
+    file_path,
+    "--color=false",
+    "--sourcemap",
+    "--minify",
+    "--outdir=#{outdir}",
+  ]
+  )
+
+  if process.success?
+    puts "Minified #{file}".colorize(:green)
+  elsif !process.success?
+    puts "Failed to minify #{file}, esbuild exited with exit code #{process.exit_code}: #{process_output.to_s.split("\n").first}".colorize(:red)
+    raise Exception.new("All files have to be minified sucessfully in order to compile!")
+  end
+end
+
+puts "Minify done!"
+
+# Cleanup
+`rm -rf #{tmp_dir_path}`

shard.lock

@@ -2,15 +2,15 @@ version: 2.0
 shards:
   ameba:
     git: https://github.com/crystal-ameba/ameba.git
-    version: 1.6.1
+    version: 1.6.4
 
   athena-negotiation:
     git: https://github.com/athena-framework/negotiation.git
-    version: 0.1.1
+    version: 0.1.5
 
   backtracer:
     git: https://github.com/sija/backtracer.cr.git
-    version: 1.2.2
+    version: 1.2.4
 
   db:
     git: https://github.com/crystal-lang/crystal-db.git
@@ -18,31 +18,24 @@ shards:
 
   exception_page:
     git: https://github.com/crystal-loot/exception_page.git
-    version: 0.2.2
+    version: 0.4.1
 
-  inotify:
-    git: https://github.com/petoem/inotify.cr.git
-    version: 1.0.3
   http_proxy:
     git: https://github.com/mamantoha/http_proxy.git
     version: 0.10.3
 
+  inotify:
+    git: https://github.com/petoem/inotify.cr.git
+    version: 1.0.3
+
   kemal:
     git: https://github.com/kemalcr/kemal.git
-    version: 1.1.2
+    version: 1.6.0
-
-  kilt:
-    git: https://github.com/jeromegn/kilt.git
-    version: 0.6.1
 
   pg:
     git: https://github.com/will/crystal-pg.git
     version: 0.28.0
 
-  pool:
-    git: https://github.com/ysbaddaden/pool.git
-    version: 0.2.4
-
   protodec:
     git: https://github.com/iv-org/protodec.git
     version: 0.1.5
@@ -52,8 +45,8 @@ shards:
     version: 0.4.1
 
   redis:
-    git: https://github.com/stefanwille/crystal-redis.git
+    git: https://github.com/jgaskins/redis.git
-    version: 2.9.1
+    version: 0.12.0
 
   spectator:
     git: https://github.com/icy-arctic-fox/spectator.git

shard.yml

@@ -1,5 +1,5 @@
 name: invidious
-version: 2.20241110.0-dev
+version: 2.20250314.0-dev
 
 authors:
   - Invidious team <contact@invidious.io>
@@ -17,10 +17,7 @@ dependencies:
     version: ~> 0.21.0
   kemal:
     github: kemalcr/kemal
-    version: ~> 1.1.2
+    version: ~> 1.6.0
-  kilt:
-    github: jeromegn/kilt
-    version: ~> 0.6.1
   protodec:
     github: iv-org/protodec
     version: ~> 0.1.5
@@ -28,7 +25,7 @@ dependencies:
     github: athena-framework/negotiation
     version: ~> 0.1.1
   redis:
-    github: stefanwille/crystal-redis
+    github: jgaskins/redis
   inotify:
     github: petoem/inotify.cr
     version: 1.0.3

src/ext/kemal_content_for.cr

@@ -1,16 +0,0 @@
-# Overrides for Kemal's `content_for` macro in order to keep using
-# kilt as it was before Kemal v1.1.1 (Kemal PR #618).
-
-require "kemal"
-require "kilt"
-
-macro content_for(key, file = __FILE__)
-  %proc = ->() {
-    __kilt_io__ = IO::Memory.new
-    {{ yield }}
-    __kilt_io__.to_s
-  }
-
-  CONTENT_FOR_BLOCKS[{{key}}] = Tuple.new {{file}}, %proc
-  nil
-end

src/ext/kemal_static_file_handler.cr

@@ -71,7 +71,7 @@ def send_file(env : HTTP::Server::Context, file_path : String, data : Slice(UInt
   filesize = data.bytesize
   attachment(env, filename, disposition)
 
-  Kemal.config.static_headers.try(&.call(env.response, file_path, filestat))
+  Kemal.config.static_headers.try(&.call(env, file_path, filestat))
 
   file = IO::Memory.new(data)
   if env.request.method == "GET" && env.request.headers.has_key?("Range")

src/invidious.cr

@@ -17,10 +17,8 @@
 require "digest/md5"
 require "file_utils"
 
-# Require kemal, kilt, then our own overrides
+# Require kemal, then our own overrides
 require "kemal"
-require "kilt"
-require "./ext/kemal_content_for.cr"
 require "./ext/kemal_static_file_handler.cr"
 
 require "http_proxy"
@@ -51,7 +49,8 @@ require "./invidious/channels/*"
 require "./invidious/user/*"
 require "./invidious/search/*"
 require "./invidious/routes/**"
-require "./invidious/jobs/**"
+require "./invidious/jobs/base_job"
+require "./invidious/jobs/*"
 
 # Declare the base namespace for invidious
 module Invidious
@@ -114,9 +113,11 @@ YT_POOL = YoutubeConnectionPool.new(YT_URL, capacity: CONFIG.pool_size)
 
 GGPHT_POOL = YoutubeConnectionPool.new(URI.parse("https://yt3.ggpht.com"), capacity: CONFIG.pool_size)
 
-COMPANION_POOL = CompanionConnectionPool.new(
+COMPANION_POOL = [] of CompanionConnectionPool
-  capacity: CONFIG.pool_size
+
-)
+CONFIG.invidious_companion.each do |companion|
+  COMPANION_POOL << CompanionConnectionPool.new(companion, capacity: CONFIG.pool_size)
+end
 
 # CLI
 Kemal.config.extra_options do |parser|
@@ -159,6 +160,15 @@ LOGGER = Invidious::LogHandler.new(OUTPUT, CONFIG.log_level, CONFIG.colorize_log
 # Check table integrity
 Invidious::Database.check_integrity(CONFIG)
 
+# Minifies Invidious Javascript
+{% if flag?(:minify_debug) || (flag?(:release) || flag?(:production)) && !flag?(:skip_minified_js) %}
+  {% puts "\nMinifying Invidious JavaScript\n" %}
+  {% puts run("../scripts/minify-js.cr").stringify %}
+  JS_PATH="js/minified"
+{% else %}
+  JS_PATH="js"
+{% end %}
+
 {% if !flag?(:skip_videojs_download) %}
   # Resolve player dependencies. This is done at compile time.
   #
@@ -209,20 +219,18 @@ Invidious::Jobs.register Invidious::Jobs::ClearExpiredItemsJob.new
 
 Invidious::Jobs.register Invidious::Jobs::InstanceListRefreshJob.new
 
-if !CONFIG.external_videoplayback_proxy.empty?
-  Invidious::Jobs.register Invidious::Jobs::CheckExternalProxy.new
-else
-  # Invidious will it's own videoplayback proxy unless the admin decides to rewrite
-  # the /videoplayback location in the reverse proxy configuration (NGINX, Caddy, etc)
-  LOGGER.info("jobs: Disabling CheckExternalProxy job. Invidious will it's own videoplayback proxy")
-end
-
 if !CONFIG.tokens_server.empty?
   Invidious::Jobs.register Invidious::Jobs::RefreshSessionTokens.new
 else
   LOGGER.info("jobs: Disabling RefreshSessionTokens job. Invidious will use the tokens that are on the configuration file")
 end
 
+if CONFIG.invidious_companion.present?
+  Invidious::Jobs.register Invidious::Jobs::CheckBackend.new
+else
+  LOGGER.info("jobs: Disabling CheckBackend job. invidious-companion and their respective external video playback proxies (if set on invidious-companion) will not be checked")
+end
+
 Invidious::Jobs.start_all
 
 def popular_videos
@@ -245,8 +253,8 @@ error 500 do |env, ex|
   error_template(500, ex)
 end
 
-static_headers do |response|
+static_headers do |env|
-  response.headers.add("Cache-Control", "max-age=2629800")
+  env.response.headers.add("Cache-Control", "max-age=2629800")
 end
 
 # Init Kemal

src/invidious/config.cr

@@ -52,6 +52,7 @@ struct ConfigPreferences
   property vr_mode : Bool = true
   property show_nick : Bool = true
   property save_player_pos : Bool = false
+  property enable_dearrow : Bool = false
 
   def to_tuple
     {% begin %}
@@ -82,6 +83,12 @@ class Config
 
     @[YAML::Field(converter: Preferences::URIConverter)]
     property public_url : URI = URI.parse("")
+
+    @[YAML::Field(converter: Preferences::URIConverter)]
+    property i2p_public_url : URI = URI.parse("")
+
+    property note : String = ""
+    property domain : Array(String) = [] of String
   end
 
   # Number of threads to use for crawling videos from channels (for updating subscriptions)
@@ -101,8 +108,8 @@ class Config
   # Database configuration using 12-Factor "Database URL" syntax
   @[YAML::Field(converter: Preferences::URIConverter)]
   property database_url : URI = URI.parse("")
-  property redis_url : String?
+  @[YAML::Field(converter: Preferences::URIConverter)]
-  property redis_socket : String?
+  property redis_url : URI = URI.parse("")
   # Use polling to keep decryption function up to date
   property decrypt_polling : Bool = false
   # Used for crawling channels: threads should check all videos uploaded by a channel
@@ -121,10 +128,6 @@ class Config
   property domain : String?
   # Materialious redirects
   property materialious_domain : String?
-  # Alternative domains. You can add other domains, like TOR and I2P addresses
-  property alternative_domains : Array(String) = [] of String
-  # Backend domains. Domains for numbered backends
-  property backend_domains : Array(String) = [] of String
 
   # Subscribe to channels using PubSubHubbub (requires domain, hmac_key)
   property use_pubsub_feeds : Bool | Int32 = false
@@ -216,13 +219,9 @@ class Config
   # of the backend
   property backends_delimiter : String = "|"
 
-  # External videoplayback proxies list. They should include `https://`
-  # at the start of the URI
-  property external_videoplayback_proxy : Array(String) = [] of String
-
   property pubsub_domain : String = ""
 
-  property server_id_cookie_name : String = "INVIDIOUS_SERVER_ID"
+  property server_id_cookie_name : String = "COMPANION_ID"
 
   property tokens_server : String = ""
 
@@ -237,6 +236,14 @@ class Config
     property lru_max_size : Int32 = 18432 # ~512MB
   end
 
+  property check_backends_interval : Int32 = 30
+
+  property force_local : Bool = true
+
+  property disable_livestreams : Bool = true
+
+  property max_popuplar_results : Int32 = 40
+
   {% if flag?(:linux) %}
     property reload_config_automatically : Bool = true
   {% end %}
@@ -383,10 +390,14 @@ class Config
       elsif config.invidious_companion_key == "CHANGE_ME!!"
         puts "Config: The value of 'invidious_companion_key' needs to be changed!!"
         exit(1)
-      elsif config.invidious_companion_key.size < 16
+      elsif config.invidious_companion_key.size != 16
-        puts "Config: The value of 'invidious_companion_key' needs to be a size of 16 or more."
+        puts "Config: The value of 'invidious_companion_key' needs to be a size of 16 characters."
         exit(1)
       end
+    elsif config.signature_server
+      puts("WARNING: inv-sig-helper is deprecated. Please switch to Invidious companion: https://docs.invidious.io/companion-installation/")
+    else
+      puts("WARNING: Invidious companion is required to view and playback videos. For more information see https://docs.invidious.io/companion-installation/")
     end
 
     # HMAC_key is mandatory

src/invidious/database/channels.cr

@@ -149,7 +149,7 @@ module Invidious::Database::ChannelVideos
       SELECT DISTINCT ON (ucid) *
       FROM channel_videos
       WHERE ucid IN (SELECT channel FROM (SELECT UNNEST(subscriptions) AS channel FROM users) AS d
-      GROUP BY channel ORDER BY COUNT(channel) DESC LIMIT 40)
+      GROUP BY channel ORDER BY COUNT(channel) DESC LIMIT #{CONFIG.max_popuplar_results})
       ORDER BY ucid, published DESC
     SQL
 

src/invidious/database/users.cr

@@ -184,6 +184,36 @@ module Invidious::Database::Users
     PG_DB.exec(request, pass, user.email)
   end
 
+  def update_username(user : User, username : String)
+    request = <<-SQL
+      UPDATE users
+      SET email = $1
+      WHERE email = $2
+    SQL
+
+    PG_DB.exec(request, username, user.email)
+  end
+
+  def update_user_session_id(user : User, username : String)
+    request = <<-SQL
+      UPDATE session_ids
+      SET email = $1
+      WHERE email = $2
+    SQL
+
+    PG_DB.exec(request, username, user.email)
+  end
+
+  def update_user_playlists_author(user : User, username : String)
+    request = <<-SQL
+      UPDATE playlists
+      SET author = $1
+      WHERE author = $2
+    SQL
+
+    PG_DB.exec(request, username, user.email)
+  end
+
   # -------------------
   #  Select
   # -------------------

src/invidious/database/videos.cr

@@ -97,21 +97,20 @@ module Invidious::Database::Videos
     end
 
     class Redis_
-      @redis : Redis::PooledClient
+      @redis : Redis::Client
 
       def initialize
-        @redis = Redis::PooledClient.new(unixsocket: CONFIG.redis_socket || nil, url: CONFIG.redis_url || nil)
+        @redis = Redis::Client.new(CONFIG.redis_url)
         LOGGER.info "Video Cache: Using Redis compatible DB to store video cache"
         LOGGER.info "Connecting to Redis compatible DB"
         if @redis.ping
-          LOGGER.info "Connected to Redis compatible DB via unix domain socket at '#{CONFIG.redis_socket}'" if CONFIG.redis_socket
+          LOGGER.info "Connected to Redis compatible DB at '#{CONFIG.redis_url}'" if CONFIG.redis_url
-          LOGGER.info "Connected to Redis compatible DB via TCP socket at '#{CONFIG.redis_url}'" if CONFIG.redis_url
         end
       end
 
       def set(video : Video, expire_time)
-        @redis.set(video.id, video.info.to_json, expire_time)
+        @redis.set(video.id, video.info.to_json, ex: expire_time)
-        @redis.set(video.id + ":time", video.updated, expire_time)
+        @redis.set(video.id + ":time", video.updated.to_s, ex: expire_time)
       end
 
       def del(id : String)

src/invidious/helpers/backend_info.cr

@@ -0,0 +1,85 @@
+module BackendInfo
+  extend self
+  @@exvpp_url : Array(String) = Array.new(CONFIG.invidious_companion.size, "")
+  @@status : Array(Int32) = Array.new(CONFIG.invidious_companion.size, 0)
+  @@csp : Array(String) = Array.new(CONFIG.invidious_companion.size, "")
+  @@mutex : Mutex = Mutex.new
+
+  def check_backends
+    check_companion()
+  end
+
+  private def check_companion
+    CONFIG.invidious_companion.each_with_index do |companion, index|
+      spawn do
+        begin
+          response = HTTP::Client.get "#{companion.private_url}/healthz"
+          if response.status_code == 200
+            check_videoplayback_proxy(companion, index)
+            generate_csp([companion.public_url.to_s, companion.i2p_public_url.to_s], @@exvpp_url[index], index)
+          else
+            @@status[index] = 0
+          end
+        rescue
+          @@status[index] = 0
+        end
+      end
+    end
+  end
+
+  private def check_videoplayback_proxy(companion : Config::CompanionConfig, index : Int32)
+    begin
+      info = HTTP::Client.get "#{companion.private_url}/info"
+      exvpp_url = JSON.parse(info.body)["external_videoplayback_proxy"]?.try &.to_s
+    rescue JSON::ParseException
+      @@status[index] = 2
+      return
+    end
+
+    exvpp_url = "" if exvpp_url.nil?
+    @@exvpp_url[index] = exvpp_url
+    if exvpp_url.empty?
+      @@status[index] = 2
+      return
+    else
+      begin
+        exvpp_health = HTTP::Client.get "#{exvpp_url}/health"
+        if exvpp_health.status_code == 200
+          @@status[index] = 2
+          return exvpp_url
+        else
+          @@status[index] = 1
+        end
+      rescue
+        @@status[index] = 1
+      end
+    end
+  end
+
+  private def generate_csp(companion_url : Array(String), exvpp_url : String? = nil, index : Int32? = nil)
+    @@mutex.synchronize do
+      @@csp[index] = ""
+      companion_url.each do |url|
+        @@csp[index] += " #{url}"
+      end
+      @@csp[index] += " #{exvpp_url}"
+    end
+  end
+
+  def get_status
+    return @@status
+  end
+
+  def get_exvpp
+    return @@exvpp_url
+  end
+
+  def get_csp(index : Int32)
+    # A little mutex to prevent sending a partial CSP header
+    # Not sure if this is necessary. But if the @@csp[index] is being assigned
+    # at the same time when it's being accessed, a data race will appear
+    @@mutex.synchronize do
+      return @@csp[index], @@csp[index]
+    end
+  end
+end

src/invidious/helpers/errors.cr

@@ -183,6 +183,8 @@ def error_redirect_helper(env : HTTP::Server::Context)
     go_to_youtube_embed = translate(locale, "videoinfo_youTube_embed_link")
     switch_instance = translate(locale, "Switch Invidious Instance")
 
+    show_embed_link = "(<a rel=\"noreferrer noopener\" href=\"https://youtube.com/embed/#{env.params.query["v"]}\">#{go_to_youtube_embed}</a>)" if env.params.query["v"]?
+
     return <<-END_HTML
       <p style="margin-bottom: 4px;">#{next_steps_text}</p>
       <ul>
@@ -194,7 +196,7 @@ def error_redirect_helper(env : HTTP::Server::Context)
         </li>
         <li>
           <a rel="noreferrer noopener" href="https://youtube.com#{env.request.resource}">#{go_to_youtube}</a>
-          (<a rel="noreferrer noopener" href="https://youtube.com/embed/#{env.params.query["v"]}">#{go_to_youtube_embed}</a>)
+          #{show_embed_link}
         </li>
       </ul>
     END_HTML

src/invidious/helpers/macros.cr

@@ -56,12 +56,11 @@ macro templated(_filename, template = "template", navbar_search = true, buffer_f
   {{ layout = "src/invidious/views/" + template + ".ecr" }}
 
   __content_filename__ = {{filename}}
-  content = Kilt.render({{filename}})
+  render {{filename}}, {{layout}}
-  Kilt.render({{layout}})
 end
 
 macro rendered(filename)
-  Kilt.render("src/invidious/views/#{{{filename}}}.ecr")
+  render("src/invidious/views/#{{{filename}}}.ecr")
 end
 
 # Similar to Kemals halt method but works in a

src/invidious/helpers/utils.cr

@@ -384,6 +384,29 @@ def parse_link_endpoint(endpoint : JSON::Any, text : String, video_id : String)
   return text
 end
 
+def decrypt_ecb_without_salt(data, key)
+  cipher = OpenSSL::Cipher.new("aes-128-ecb")
+  cipher.decrypt
+  cipher.key = key
+  cipher.padding = false
+
+  io = IO::Memory.new
+  io.write(cipher.update(data))
+  io.write(cipher.final)
+  io.rewind
+
+  data_ = io.to_s
+  padding = data_[-1].ord
+
+  return data_[0...(data_.bytesize - padding)]
+end
+
+def video_playback_decrypt(data)
+  data = Base64.decode(data)
+  decrypted_query = decrypt_ecb_without_salt(data, CONFIG.invidious_companion_key)
+  return decrypted_query
+end
+
 def encrypt_ecb_without_salt(data, key)
   cipher = OpenSSL::Cipher.new("aes-128-ecb")
   cipher.encrypt

src/invidious/http_server/utils.cr

@@ -4,30 +4,6 @@ module Invidious::HttpServer
   module Utils
     extend self
 
-    @@proxy_alive : String = ""
-
-    def check_external_proxy
-      CONFIG.external_videoplayback_proxy.each do |proxy|
-        begin
-          response = HTTP::Client.get("#{proxy}/health")
-          if response.status_code == 200
-            @@proxy_alive = proxy
-            LOGGER.debug("CheckExternalProxy: Proxy set to: '#{proxy}'")
-            break
-          end
-        rescue
-          LOGGER.debug("CheckExternalProxy: Proxy '#{proxy}' is not available")
-        end
-      end
-      if @@proxy_alive.empty?
-        LOGGER.warn("CheckExternalProxy: No proxies alive! Using own server proxy")
-      end
-    end
-
-    def get_external_proxy
-      return @@proxy_alive
-    end
-
     def proxy_video_url(raw_url : String, *, region : String? = nil, absolute : Bool = false)
       url = URI.parse(raw_url)
 
@@ -38,11 +14,7 @@ module Invidious::HttpServer
       url.query_params = params
 
       if absolute
-        if !@@proxy_alive.empty?
-          return "#{@@proxy_alive}#{url.request_target}"
-        else
         return "#{HOST_URL}#{url.request_target}"
-        end
       else
         return url.request_target
       end

src/invidious/jobs/backend_checker.cr

@@ -0,0 +1,13 @@
+class Invidious::Jobs::CheckBackend < Invidious::Jobs::BaseJob
+  def initialize
+  end
+
+  def begin
+    loop do
+      BackendInfo.check_backends
+      LOGGER.info("Backend Checker: Done, sleeping for #{CONFIG.check_backends_interval} seconds")
+      sleep CONFIG.check_backends_interval.seconds
+      Fiber.yield
+    end
+  end
+end

src/invidious/jobs/check_external_proxy.cr

@@ -1,13 +0,0 @@
-class Invidious::Jobs::CheckExternalProxy < Invidious::Jobs::BaseJob
-  def initialize
-  end
-
-  def begin
-    loop do
-      HttpServer::Utils.check_external_proxy
-      LOGGER.info("CheckExternalProxy: Done, sleeping for 10 seconds")
-      sleep 10.seconds
-      Fiber.yield
-    end
-  end
-end

src/invidious/routes/account.cr

@@ -78,6 +78,75 @@ module Invidious::Routes::Account
     env.redirect referer
   end
 
+  # -------------------
+  #  Username update
+  # -------------------
+
+  # Show the username change interface (GET request)
+  def get_change_username(env)
+    locale = env.get("preferences").as(Preferences).locale
+
+    user = env.get? "user"
+    sid = env.get? "sid"
+    referer = get_referer(env)
+
+    if !user
+      return env.redirect referer
+    end
+
+    user = user.as(User)
+    sid = sid.as(String)
+    csrf_token = generate_response(sid, {":change_username"}, HMAC_KEY)
+
+    templated "user/change_username"
+  end
+
+  # Handle the username change (POST request)
+  def post_change_username(env)
+    locale = env.get("preferences").as(Preferences).locale
+
+    user = env.get? "user"
+    sid = env.get? "sid"
+    referer = get_referer(env)
+
+    if !user
+      return env.redirect referer
+    end
+
+    user = user.as(User)
+    sid = sid.as(String)
+    token = env.params.body["csrf_token"]?
+
+    begin
+      validate_request(token, sid, env.request, HMAC_KEY, locale)
+    rescue ex
+      return error_template(400, ex)
+    end
+
+    new_username = env.params.body["new_username"]?.try &.downcase.byte_slice(0, 254)
+    if new_username.nil?
+      return error_template(401, "username_required_field")
+    end
+
+    if new_username.empty?
+      return error_template(401, "username_empty")
+    end
+
+    if new_username == user.email
+      return error_template(401, "username_is_the_same")
+    end
+
+    if Invidious::Database::Users.select(email: new_username)
+      return error_template(401, "username_taken")
+    end
+
+    Invidious::Database::Users.update_username(user, new_username.to_s)
+    Invidious::Database::Users.update_user_session_id(user, new_username.to_s)
+    Invidious::Database::Users.update_user_playlists_author(user, new_username.to_s)
+
+    env.redirect referer
+  end
+
   # -------------------
   #  Account deletion
   # -------------------

src/invidious/routes/api/manifest.cr

@@ -9,8 +9,8 @@ module Invidious::Routes::API::Manifest
     region = env.params.query["region"]?
 
     if CONFIG.invidious_companion.present?
-      invidious_companion = CONFIG.invidious_companion.sample
+      companion_public_url = env.get("companion_public_url").as(String)
-      return env.redirect "#{invidious_companion.public_url}/api/manifest/dash/id/#{id}?#{env.params.query}"
+      return env.redirect "#{companion_public_url}/api/manifest/dash/id/#{id}?#{env.params.query}"
     end
 
     # Since some implementations create playlists based on resolution regardless of different codecs,
@@ -221,21 +221,15 @@ module Invidious::Routes::API::Manifest
 
         raw_params["host"] = uri.host.not_nil!
 
-        proxy = Invidious::HttpServer::Utils.get_external_proxy
-
         if CONFIG.https_only
           scheme = "https://"
         else
           scheme = "http://"
         end
 
-        if !proxy.empty?
-          "#{proxy}/videoplayback?#{raw_params}"
-        else
         "#{scheme}#{env.request.headers["Host"]}/videoplayback?#{raw_params}"
       end
     end
-    end
 
     manifest
   end

src/invidious/routes/backend_switcher.cr

@@ -0,0 +1,16 @@
+{% skip_file if flag?(:api_only) %}
+
+module Invidious::Routes::BackendSwitcher
+  def self.switch(env)
+    referer = get_referer(env)
+    backend_id = env.params.query["backend_id"]?.try &.to_i
+
+    if backend_id.nil?
+      return error_template(400, "Backend ID is required")
+    end
+
+    env.response.cookies[CONFIG.server_id_cookie_name] = Invidious::User::Cookies.server_id(env.request.headers["Host"], backend_id)
+
+    env.redirect referer
+  end
+end

src/invidious/routes/before_all.cr

@@ -1,6 +1,7 @@
 module Invidious::Routes::BeforeAll
   def self.handle(env)
     preferences = Preferences.from_json("{}")
+    host = env.request.headers["Host"]
 
     begin
       if prefs_cookie = env.request.cookies["PREFS"]?
@@ -24,15 +25,50 @@ module Invidious::Routes::BeforeAll
     extra_connect_csp = ""
 
     if CONFIG.invidious_companion.present?
-      extra_media_csp = " #{CONFIG.invidious_companion.sample.public_url}"
+      current_companion_d = host.split(".")[0].scan(/(\d+)$/).last?.try &.[0].to_i
-      extra_connect_csp = " #{CONFIG.invidious_companion.sample.public_url}"
+
+      if current_companion_d
+        current_companion_d = current_companion_d - 1
+        env.set "using_domain", true
+        env.set "current_companion", current_companion_d
+        env.set "companion_public_url", CONFIG.invidious_companion[current_companion_d].public_url.to_s
+      else
+        if !env.request.cookies[CONFIG.server_id_cookie_name]?
+          env.response.cookies[CONFIG.server_id_cookie_name] = Invidious::User::Cookies.server_id(host)
         end
 
-    if !CONFIG.external_videoplayback_proxy.empty?
+        begin
-      CONFIG.external_videoplayback_proxy.each do |proxy|
+          current_companion = env.request.cookies[CONFIG.server_id_cookie_name].value.try &.to_i
-        extra_media_csp += " #{proxy}"
+        rescue
-        extra_connect_csp += " #{proxy}"
+          current_companion = rand(CONFIG.invidious_companion.size)
         end
+
+        if current_companion > CONFIG.invidious_companion.size
+          current_companion = current_companion % CONFIG.invidious_companion.size
+          env.response.cookies[CONFIG.server_id_cookie_name] = Invidious::User::Cookies.server_id(host, current_companion)
+        end
+
+        companion_status = BackendInfo.get_status
+
+        if companion_status[current_companion] != 2
+          alive_companion = companion_status.index(2)
+          if alive_companion
+            env.set "companion_switched", true
+            current_companion = alive_companion
+            env.response.cookies[CONFIG.server_id_cookie_name] = Invidious::User::Cookies.server_id(host, current_companion)
+          end
+        end
+
+        env.set "current_companion", current_companion
+
+        if host.split(".").last == "i2p"
+          env.set "companion_public_url", CONFIG.invidious_companion[current_companion].i2p_public_url.to_s
+        else
+          env.set "companion_public_url", CONFIG.invidious_companion[current_companion].public_url.to_s
+        end
+      end
+
+      extra_media_csp, extra_connect_csp = BackendInfo.get_csp(env.get("current_companion").as(Int32))
     end
 
     # Allow media resources to be loaded from google servers
@@ -48,13 +84,16 @@ module Invidious::Routes::BeforeAll
       frame_ancestors = "'none'"
     end
 
+    scheme = env.request.headers["X-Forwarded-Proto"]? || ("https" if CONFIG.https_only) || "http"
+    env.set "scheme", scheme
+
     # TODO: Remove style-src's 'unsafe-inline', requires to remove all
     # inline styles (<style> [..] </style>, style=" [..] ")
     env.response.headers["Content-Security-Policy"] = {
       "default-src 'none'",
       "script-src 'self'",
       "style-src 'self' 'unsafe-inline'",
-      "img-src 'self' data: " + HOST_URL,
+      "img-src 'self' data: " + "#{scheme}://#{env.request.headers["Host"]?}",
       "font-src 'self' data:",
       "connect-src 'self'" + extra_connect_csp,
       "manifest-src 'self'",

src/invidious/routes/embed.cr

@@ -203,6 +203,11 @@ module Invidious::Routes::Embed
       return env.redirect url
     end
 
+    if CONFIG.invidious_companion.present?
+      current_companion = env.get("current_companion").as(Int32)
+      invidious_companion = CONFIG.invidious_companion[current_companion]
+    end
+
     rendered "embed"
   end
 end

src/invidious/routes/login.cr

@@ -60,15 +60,7 @@ module Invidious::Routes::Login
           sid = Base64.urlsafe_encode(Random::Secure.random_bytes(32))
           Invidious::Database::SessionIDs.insert(sid, email)
 
-          # Checks if there is any alternative domain, like a second domain name,
+          env.response.cookies["SID"] = Invidious::User::Cookies.sid(env.request.headers["Host"], sid)
-          # TOR or I2P address
-          if alt = CONFIG.alternative_domains.index(env.request.headers["Host"])
-            env.response.cookies["SID"] = Invidious::User::Cookies.sid(CONFIG.alternative_domains[alt], sid)
-          elsif alt = CONFIG.backend_domains.index(env.request.headers["Host"])
-            env.response.cookies["SID"] = Invidious::User::Cookies.sid(CONFIG.backend_domains[alt], sid)
-          else
-            env.response.cookies["SID"] = Invidious::User::Cookies.sid(CONFIG.domain, sid)
-          end
         else
           return error_template(401, "Wrong username or password")
         end
@@ -168,15 +160,7 @@ module Invidious::Routes::Login
         Invidious::Database::Users.insert(user)
         Invidious::Database::SessionIDs.insert(sid, email)
 
-        # Checks if there is any alternative domain, like a second domain name,
+        env.response.cookies["SID"] = Invidious::User::Cookies.sid(env.request.headers["Host"], sid)
-        # TOR or I2P address
-        if alt = CONFIG.alternative_domains.index(env.request.headers["Host"])
-          env.response.cookies["SID"] = Invidious::User::Cookies.sid(CONFIG.alternative_domains[alt], sid)
-        elsif alt = CONFIG.backend_domains.index(env.request.headers["Host"])
-          env.response.cookies["SID"] = Invidious::User::Cookies.sid(CONFIG.backend_domains[alt], sid)
-        else
-          env.response.cookies["SID"] = Invidious::User::Cookies.sid(CONFIG.domain, sid)
-        end
 
         if env.request.cookies["PREFS"]?
           user.preferences = env.get("preferences").as(Preferences)

src/invidious/routes/preferences.cr

@@ -224,15 +224,7 @@ module Invidious::Routes::PreferencesRoute
         File.write("config/config.yml", CONFIG.to_yaml)
       end
     else
-      # Checks if there is any alternative domain, like a second domain name,
+      env.response.cookies["PREFS"] = Invidious::User::Cookies.prefs(env.request.headers["Host"], preferences)
-      # TOR or I2P address
-      if alt = CONFIG.alternative_domains.index(env.request.headers["Host"])
-        env.response.cookies["PREFS"] = Invidious::User::Cookies.prefs(CONFIG.alternative_domains[alt], preferences)
-      elsif alt = CONFIG.backend_domains.index(env.request.headers["Host"])
-        env.response.cookies["PREFS"] = Invidious::User::Cookies.prefs(CONFIG.backend_domains[alt], preferences)
-      else
-        env.response.cookies["PREFS"] = Invidious::User::Cookies.prefs(CONFIG.domain, preferences)
-      end
     end
 
     env.redirect referer
@@ -267,15 +259,7 @@ module Invidious::Routes::PreferencesRoute
         preferences.dark_mode = "dark"
       end
 
-      # Checks if there is any alternative domain, like a second domain name,
+      env.response.cookies["PREFS"] = Invidious::User::Cookies.prefs(env.request.headers["Host"], preferences)
-      # TOR or I2P address
-      if alt = CONFIG.alternative_domains.index(env.request.headers["Host"])
-        env.response.cookies["PREFS"] = Invidious::User::Cookies.prefs(CONFIG.alternative_domains[alt], preferences)
-      elsif alt = CONFIG.backend_domains.index(env.request.headers["Host"])
-        env.response.cookies["PREFS"] = Invidious::User::Cookies.prefs(CONFIG.backend_domains[alt], preferences)
-      else
-        env.response.cookies["PREFS"] = Invidious::User::Cookies.prefs(CONFIG.domain, preferences)
-      end
     end
 
     if redirect

src/invidious/routes/video_playback.cr

@@ -3,6 +3,11 @@ module Invidious::Routes::VideoPlayback
   def self.get_video_playback(env)
     locale = env.get("preferences").as(Preferences).locale
     query_params = env.params.query
+
+    if query_params["enc"]? == "yes"
+      query_params = URI::Params.parse(video_playback_decrypt(query_params["data"]))
+    end
+
     array = UInt8[0x78, 0]
     protobuf = Bytes.new(array.size)
     array.each_with_index do |byte, index|
@@ -26,7 +31,7 @@ module Invidious::Routes::VideoPlayback
     end
 
     # Sanity check, to avoid being used as an open proxy
-    if !host.matches?(/[\w-]+.googlevideo.com/)
+    if !host.matches?(/[\w-]+.googlevideo.com/) && !host.matches?(/[\w-]+.c.youtube.com/)
       return error_template(400, "Invalid \"host\" parameter.")
     end
 
@@ -262,8 +267,8 @@ module Invidious::Routes::VideoPlayback
   # so we have a mechanism here to redirect to the latest version
   def self.latest_version(env)
     if CONFIG.invidious_companion.present?
-      invidious_companion = CONFIG.invidious_companion.sample
+      companion_public_url = env.get("companion_public_url").as(String)
-      return env.redirect "#{invidious_companion.public_url}/latest_version?#{env.params.query}"
+      return env.redirect "#{companion_public_url}/latest_version?#{env.params.query}"
     end
 
     id = env.params.query["id"]?
@@ -307,16 +312,7 @@ module Invidious::Routes::VideoPlayback
     end
 
     if local
-      external_proxy = Invidious::HttpServer::Utils.get_external_proxy
-      if !external_proxy.empty?
-        url = URI.parse(url)
-        external_proxy = URI.parse(external_proxy)
-        url.host = external_proxy.host
-        url.port = external_proxy.port
-        url = url.to_s
-      else
       url = URI.parse(url).request_target.not_nil!
-      end
       url += "&title=#{URI.encode_www_form(title, space_to_plus: false)}" if title
     end
 

src/invidious/routes/watch.cr

@@ -52,7 +52,7 @@ module Invidious::Routes::Watch
     env.params.query.delete_all("listen")
 
     begin
-      video = get_video(id, region: params.region)
+      video = get_video(id, region: params.region, env: env)
     rescue ex : NotFoundException
       LOGGER.error("get_video not found: #{id} : #{ex.message}")
       return error_template(404, ex)
@@ -61,6 +61,10 @@ module Invidious::Routes::Watch
       return error_template(500, ex)
     end
 
+    if video.live_now && CONFIG.disable_livestreams
+      return error_template(403, "Livestreams are disabled as they are not working with invidious-companion right now. Please wait until an update comes out!")
+    end
+
     if preferences.annotations_subscribed &&
        subscriptions.includes?(video.ucid) &&
        (env.params.query["iv_load_policy"]? || "1") == "1"
@@ -209,6 +213,11 @@ module Invidious::Routes::Watch
       video_url = nil
     end
 
+    if CONFIG.invidious_companion.present?
+      current_companion = env.get("current_companion").as(Int32)
+      invidious_companion = CONFIG.invidious_companion[current_companion]
+    end
+
     templated "watch"
   end
 
@@ -338,8 +347,9 @@ module Invidious::Routes::Watch
       env.params.query["local"] = "true"
 
       if (CONFIG.invidious_companion.present?)
-        video = get_video(video_id)
+        video = get_video(video_id, env: env)
-        return env.redirect "#{video.invidious_companion["baseUrl"].as_s}/latest_version?#{env.params.query}"
+        companion_public_url = env.get("companion_public_url").as(String)
+        return env.redirect "#{companion_public_url}/latest_version?#{env.params.query}"
       else
         return Invidious::Routes::VideoPlayback.latest_version(env)
       end

src/invidious/routing.cr

@@ -21,6 +21,7 @@ module Invidious::Routing
       get "/privacy", Routes::Misc, :privacy
       get "/licenses", Routes::Misc, :licenses
       get "/redirect", Routes::Misc, :cross_instance_redirect
+      get "/switchbackend", Routes::BackendSwitcher, :switch
 
       self.register_channel_routes
       self.register_watch_routes
@@ -68,6 +69,8 @@ module Invidious::Routing
     # User account management
     get "/change_password", Routes::Account, :get_change_password
     post "/change_password", Routes::Account, :post_change_password
+    get "/change_username", Routes::Account, :get_change_username
+    post "/change_username", Routes::Account, :post_change_username
     get "/delete_account", Routes::Account, :get_delete
     post "/delete_account", Routes::Account, :post_delete
     get "/clear_watch_history", Routes::Account, :get_clear_history

src/invidious/user/cookies.cr

@@ -11,6 +11,10 @@ struct Invidious::User
     # Session ID (SID) cookie
     # Parameter "domain" comes from the global config
     def sid(domain : String?, sid) : HTTP::Cookie
+      # Strip the port from the domain if it's being accessed from another port
+      # Browsers will reject the cookie if it contains the port number. This is
+      # because `example.com:3000` is not the same as `example.com` on a cookie.
+      domain = domain.split(":")[0]
       # Not secure if it's being accessed from I2P
       # Browsers expect the domain to include https. On I2P there is no HTTPS
       if domain.not_nil!.split(".").last == "i2p"
@@ -30,6 +34,10 @@ struct Invidious::User
     # Preferences (PREFS) cookie
     # Parameter "domain" comes from the global config
     def prefs(domain : String?, preferences : Preferences) : HTTP::Cookie
+      # Strip the port from the domain if it's being accessed from another port
+      # Browsers will reject the cookie if it contains the port number. This is
+      # because `example.com:3000` is not the same as `example.com` on a cookie.
+      domain = domain.split(":")[0]
       # Not secure if it's being accessed from I2P
       # Browsers expect the domain to include https. On I2P there is no HTTPS
       if domain.not_nil!.split(".").last == "i2p"
@@ -45,5 +53,31 @@ struct Invidious::User
         samesite: HTTP::Cookie::SameSite::Lax
       )
     end
+
+    # Backend (CONFIG.server_id_cookie_name) cookie
+    # Parameter "domain" comes from the global config
+    def server_id(domain : String?, server_id : Int32? = nil) : HTTP::Cookie
+      if server_id.nil?
+        server_id = rand(CONFIG.invidious_companion.size)
+      end
+      # Strip the port from the domain if it's being accessed from another port
+      # Browsers will reject the cookie if it contains the port number. This is
+      # because `example.com:3000` is not the same as `example.com` on a cookie.
+      domain = domain.split(":")[0]
+      # Not secure if it's being accessed from I2P
+      # Browsers expect the domain to include https. On I2P there is no HTTPS
+      if domain.not_nil!.split(".").last == "i2p"
+        @@secure = false
+      end
+      return HTTP::Cookie.new(
+        name: CONFIG.server_id_cookie_name,
+        domain: domain,
+        path: "/",
+        value: server_id.to_s,
+        secure: @@secure,
+        http_only: true,
+        samesite: HTTP::Cookie::SameSite::Lax
+      )
+    end
   end
 end

src/invidious/videos.cr

@@ -298,7 +298,7 @@ struct Video
   predicate_bool upcoming, isUpcoming
 end
 
-def get_video(id, refresh = true, region = nil, force_refresh = false)
+def get_video(id, refresh = true, region = nil, force_refresh = false, env : HTTP::Server::Context | Nil = nil)
   if (video = Invidious::Database::Videos.select(id)) && !region
     # If record was last updated over 10 minutes ago, or video has since premiered,
     # refresh (expire param in response lasts for 6 hours)
@@ -308,7 +308,7 @@ def get_video(id, refresh = true, region = nil, force_refresh = false)
        force_refresh ||
        video.schema_version != Video::SCHEMA_VERSION # cache control
       begin
-        video = fetch_video(id, region)
+        video = fetch_video(id, region, env)
         Invidious::Database::Videos.insert(video)
       rescue ex
         Invidious::Database::Videos.delete(id)
@@ -316,7 +316,7 @@ def get_video(id, refresh = true, region = nil, force_refresh = false)
       end
     end
   else
-    video = fetch_video(id, region)
+    video = fetch_video(id, region, env)
     Invidious::Database::Videos.insert(video) if !region
   end
 
@@ -324,11 +324,11 @@ def get_video(id, refresh = true, region = nil, force_refresh = false)
 rescue DB::Error
   # Avoid common `DB::PoolRetryAttemptsExceeded` error and friends
   # Note: All DB errors inherit from `DB::Error`
-  return fetch_video(id, region)
+  return fetch_video(id, region, env)
 end
 
-def fetch_video(id, region)
+def fetch_video(id, region, env)
-  info = extract_video_info(video_id: id)
+  info = extract_video_info(video_id: id, env: env)
 
   if reason = info["reason"]?
     if reason == "Video unavailable"

src/invidious/videos/parser.cr

@@ -58,12 +58,12 @@ def parse_related_video(related : JSON::Any) : Hash(String, JSON::Any)?
   }
 end
 
-def extract_video_info(video_id : String)
+def extract_video_info(video_id : String, env : HTTP::Server::Context | Nil = nil)
   # Init client config for the API
   client_config = YoutubeAPI::ClientConfig.new
 
   # Fetch data from the player endpoint
-  player_response = YoutubeAPI.player(video_id: video_id, params: "2AMB", client_config: client_config)
+  player_response = YoutubeAPI.player(env: env, video_id: video_id, params: "2AMB", client_config: client_config)
 
   playability_status = player_response.dig?("playabilityStatus", "status").try &.as_s
 
@@ -108,7 +108,7 @@ def extract_video_info(video_id : String)
   params = parse_video_info(video_id, player_response)
   params["reason"] = JSON::Any.new(reason) if reason
 
-  if CONFIG.invidious_companion.present?
+  if !CONFIG.invidious_companion.present?
     new_player_response = nil
 
     # Don't use Android test suite client if po_token is passed because po_token doesn't
@@ -119,7 +119,7 @@ def extract_video_info(video_id : String)
       # following issue for an explanation about decrypted URLs:
       # https://github.com/TeamNewPipe/NewPipeExtractor/issues/562
       client_config.client_type = YoutubeAPI::ClientType::AndroidTestSuite
-      new_player_response = try_fetch_streaming_data(video_id, client_config)
+      new_player_response = try_fetch_streaming_data(video_id, client_config, env)
     end
 
     # Replace player response and reset reason
@@ -133,7 +133,7 @@ def extract_video_info(video_id : String)
     end
   end
 
-  {"captions", "playabilityStatus", "playerConfig", "storyboards", "invidiousCompanion"}.each do |f|
+  {"captions", "playabilityStatus", "playerConfig", "storyboards"}.each do |f|
     params[f] = player_response[f] if player_response[f]?
   end
 
@@ -154,9 +154,9 @@ def extract_video_info(video_id : String)
   return params
 end
 
-def try_fetch_streaming_data(id : String, client_config : YoutubeAPI::ClientConfig) : Hash(String, JSON::Any)?
+def try_fetch_streaming_data(id : String, client_config : YoutubeAPI::ClientConfig, env : HTTP::Server::Context | Nil = nil) : Hash(String, JSON::Any)?
   LOGGER.debug("try_fetch_streaming_data: [#{id}] Using #{client_config.client_type} client.")
-  response = YoutubeAPI.player(video_id: id, params: "2AMB", client_config: client_config)
+  response = YoutubeAPI.player(video_id: id, params: "2AMB", client_config: client_config, env: env)
 
   playability_status = response["playabilityStatus"]["status"]
   LOGGER.debug("try_fetch_streaming_data: [#{id}] Got playabilityStatus == #{playability_status}.")
@@ -200,7 +200,6 @@ def parse_video_info(video_id : String, player_response : Hash(String, JSON::Any
   microformat = player_response.dig?("microformat", "playerMicroformatRenderer")
 
   raise BrokenTubeException.new("videoDetails") if !video_details
-  raise BrokenTubeException.new("microformat") if !microformat
 
   # Basic video infos
 
@@ -216,13 +215,13 @@ def parse_video_info(video_id : String, player_response : Hash(String, JSON::Any
   views_txt ||= video_details["viewCount"]?.try &.as_s || ""
   views = views_txt.gsub(/\D/, "").to_i64?
 
-  length_txt = (microformat["lengthSeconds"]? || video_details["lengthSeconds"])
+  length_txt = (microformat.try &.["lengthSeconds"]? || video_details["lengthSeconds"])
     .try &.as_s.to_i64
 
-  published = microformat["publishDate"]?
+  published = microformat.try &.["publishDate"]?
     .try { |t| Time.parse(t.as_s, "%Y-%m-%d", Time::Location::UTC) } || Time.utc
 
-  premiere_timestamp = microformat.dig?("liveBroadcastDetails", "startTimestamp")
+  premiere_timestamp = microformat.try &.dig?("liveBroadcastDetails", "startTimestamp")
     .try { |t| Time.parse_rfc3339(t.as_s) }
 
   premiere_timestamp ||= player_response.dig?(
@@ -233,7 +232,7 @@ def parse_video_info(video_id : String, player_response : Hash(String, JSON::Any
     .try &.as_s.to_i64
       .try { |t| Time.unix(t) }
 
-  live_now = microformat.dig?("liveBroadcastDetails", "isLiveNow")
+  live_now = microformat.try &.dig?("liveBroadcastDetails", "isLiveNow")
     .try &.as_bool
   live_now ||= video_details.dig?("isLive").try &.as_bool || false
 
@@ -242,11 +241,11 @@ def parse_video_info(video_id : String, player_response : Hash(String, JSON::Any
 
   # Extra video infos
 
-  allowed_regions = microformat["availableCountries"]?
+  allowed_regions = microformat.try &.["availableCountries"]?
     .try &.as_a.map &.as_s || [] of String
 
   allow_ratings = video_details["allowRatings"]?.try &.as_bool
-  family_friendly = microformat["isFamilySafe"].try &.as_bool
+  family_friendly = microformat.try &.["isFamilySafe"].try &.as_bool
   is_listed = video_details["isCrawlable"]?.try &.as_bool
   is_upcoming = video_details["isUpcoming"]?.try &.as_bool
 
@@ -329,8 +328,8 @@ def parse_video_info(video_id : String, player_response : Hash(String, JSON::Any
 
   # Description
 
-  description = microformat.dig?("description", "simpleText").try &.as_s || ""
   short_description = player_response.dig?("videoDetails", "shortDescription")
+  description = microformat.try &.dig?("description", "simpleText").try &.as_s || short_description.try &.as_s || ""
 
   # description_html = video_secondary_renderer.try &.dig?("description", "runs")
   #  .try &.as_a.try { |t| content_to_comment_html(t, video_id) }
@@ -343,7 +342,7 @@ def parse_video_info(video_id : String, player_response : Hash(String, JSON::Any
     .try &.dig?("metadataRowContainer", "metadataRowContainerRenderer", "rows")
       .try &.as_a
 
-  genre = microformat["category"]?
+  genre = microformat.try &.["category"]?
   genre_ucid = nil
   license = nil
 

src/invidious/videos/video_preferences.cr

@@ -109,7 +109,9 @@ def process_video_params(query, preferences)
     quality = "high"
   end
 
-  if CONFIG.disabled?("local") && local
+  if CONFIG.force_local
+    local = true
+  elsif CONFIG.disabled?("local") && local
     local = false
   end
 

src/invidious/views/add_playlist_items.ecr

@@ -29,7 +29,7 @@
 }.to_pretty_json
 %>
 </script>
-<script src="/js/playlist_widget.js?v=<%= ASSET_COMMIT %>"></script>
+<script src="/<%= JS_PATH %>/playlist_widget.js?v=<%= ASSET_COMMIT %>"></script>
 
 
 <%= rendered "components/items_paginated" %>

src/invidious/views/channel.ecr

@@ -3,6 +3,7 @@
   author = HTML.escape(channel.author)
   channel_profile_pic = URI.parse(channel.author_thumbnail).request_target
   host = env.request.headers["Host"]
+  scheme = env.get("scheme")
 
   relative_url =
     case selected_tab
@@ -32,19 +33,19 @@
 <%- if selected_tab.videos? -%>
 <meta name="description" content="<%= channel.description %>">
 <meta property="og:site_name" content="Invidious">
-<meta property="og:url" content="<%= host %>/channel/<%= ucid %>">
+<meta property="og:url" content="<%= scheme %>://<%= host %>/channel/<%= ucid %>">
 <meta property="og:title" content="<%= author %>">
-<meta property="og:image" content="<%= host %>/ggpht<%= channel_profile_pic %>">
+<meta property="og:image" content="<%= scheme %>://<%= host %>/ggpht<%= channel_profile_pic %>">
 <meta property="og:description" content="<%= channel.description %>">
 <meta name="twitter:card" content="summary">
-<meta name="twitter:url" content="<%= host %>/channel/<%= ucid %>">
+<meta name="twitter:url" content="<%= scheme %>://<%= host %>/channel/<%= ucid %>">
 <meta name="twitter:title" content="<%= author %>">
 <meta name="twitter:description" content="<%= channel.description %>">
-<meta name="twitter:image" content="<%= host %>/ggpht<%= channel_profile_pic %>">
+<meta name="twitter:image" content="<%= scheme %>://<%= host %>/ggpht<%= channel_profile_pic %>">
 <link rel="alternate" type="application/rss+xml" title="RSS" href="/feed/channel/<%= ucid %>" />
 <%- end -%>
 
-<script src="/js/pagination.js?v=<%= ASSET_COMMIT %>"></script>
+<script src="/<%= JS_PATH %>/pagination.js?v=<%= ASSET_COMMIT %>"></script>
 
 <link rel="alternate" href="<%= youtube_url %>">
 <title><%= author %> - Invidious</title>

src/invidious/views/community.ecr

@@ -43,4 +43,4 @@
 }.to_pretty_json
 %>
 </script>
-<script src="/js/community.js?v=<%= ASSET_COMMIT %>"></script>
+<script src="/<%= JS_PATH %>/community.js?v=<%= ASSET_COMMIT %>"></script>

src/invidious/views/components/items_paginated.ecr

@@ -18,4 +18,4 @@
 %>
 </script>
 
-<script src="/js/watched_indicator.js"></script>
+<script src="/<%= JS_PATH %>/watched_indicator.js"></script>

src/invidious/views/components/player.ecr

@@ -22,8 +22,9 @@
                audio_streams.each_with_index do |fmt, i|
                 src_url  = "/latest_version?id=#{video.id}&itag=#{fmt["itag"]}"
                 src_url += "&local=true" if params.local
-                src_url = video.invidious_companion["baseUrl"].as_s + src_url + 
+                companion_public_url = env.get("companion_public_url").as(String)
-                            "&check=#{invidious_companion_encrypt(video.id)}" if (CONFIG.invidious_companion.present?)
+                src_url = companion_public_url + src_url + 
+                            "&check=#{invidious_companion_encrypt(video.id)}" if (invidious_companion)
 
                 bitrate = fmt["bitrate"]
                 mimetype = HTML.escape(fmt["mimeType"].as_s)
@@ -38,8 +39,9 @@
         <% else %>
             <% if params.quality == "dash"
                src_url = "/api/manifest/dash/id/" + video.id + "?local=true&unique_res=1"
-               src_url = video.invidious_companion["baseUrl"].as_s + src_url +
+               companion_public_url = env.get("companion_public_url").as(String)
-                            "&check=#{invidious_companion_encrypt(video.id)}" if (CONFIG.invidious_companion.present?)
+               src_url = companion_public_url + src_url +
+                            "&check=#{invidious_companion_encrypt(video.id)}" if (invidious_companion)
             %>
                 <source src="<%= src_url %>" type='application/dash+xml' label="dash">
             <% end %>
@@ -50,8 +52,9 @@
             fmt_stream.each_with_index do |fmt, i|
                 src_url  = "/latest_version?id=#{video.id}&itag=#{fmt["itag"]}"
                 src_url += "&local=true" if params.local
-                src_url = video.invidious_companion["baseUrl"].as_s + src_url +
+                companion_public_url = env.get("companion_public_url").as(String)
-                            "&check=#{invidious_companion_encrypt(video.id)}" if (CONFIG.invidious_companion.present?)
+                src_url = companion_public_url + src_url +
+                            "&check=#{invidious_companion_encrypt(video.id)}" if (invidious_companion)
 
                 quality = fmt["quality"]
                 mimetype = HTML.escape(fmt["mimeType"].as_s)
@@ -86,4 +89,4 @@
 }.to_pretty_json
 %>
 </script>
-<script src="/js/player.js?v=<%= ASSET_COMMIT %>"></script>
+<script src="/<%= JS_PATH %>/player.js?v=<%= ASSET_COMMIT %>"></script>

src/invidious/views/components/subscribe_widget.ecr

@@ -27,7 +27,7 @@
     }.to_pretty_json
     %>
     </script>
-    <script src="/js/subscribe_widget.js?v=<%= ASSET_COMMIT %>"></script>
+    <script src="/<%= JS_PATH %>/subscribe_widget.js?v=<%= ASSET_COMMIT %>"></script>
 <% else %>
         <a id="subscribe" class="pure-button pure-button-primary"
             href="/login?referer=<%= env.get("current_page") %>">

src/invidious/views/embed.ecr

@@ -11,7 +11,7 @@
     <link rel="stylesheet" href="/css/default.css?v=<%= ASSET_COMMIT %>">
     <link rel="stylesheet" href="/css/embed.css?v=<%= ASSET_COMMIT %>">
     <title><%= HTML.escape(video.title) %> - Invidious</title>
-    <script src="/js/_helpers.js?v=<%= ASSET_COMMIT %>"></script>
+    <script src="/<%= JS_PATH %>/_helpers.js?v=<%= ASSET_COMMIT %>"></script>
 </head>
 
 <body class="dark-theme">
@@ -32,6 +32,6 @@
 </script>
 
 <%= rendered "components/player" %>
-<script src="/js/embed.js?v=<%= ASSET_COMMIT %>"></script>
+<script src="/<%= JS_PATH %>/embed.js?v=<%= ASSET_COMMIT %>"></script>
 </body>
 </html>

src/invidious/views/feeds/history.ecr

@@ -25,7 +25,7 @@
 }.to_pretty_json
 %>
 </script>
-<script src="/js/watched_widget.js"></script>
+<script src="/<%= JS_PATH %>/watched_widget.js"></script>
 
 <div class="pure-g">
     <% watched.each do |item| %>

src/invidious/views/feeds/playlists.ecr

@@ -40,4 +40,4 @@
 <% end %>
 </div>
 
-<script src="/js/watched_indicator.js"></script>
+<script src="/<%= JS_PATH %>/watched_indicator.js"></script>

src/invidious/views/feeds/popular.ecr

@@ -17,4 +17,4 @@
 <% end %>
 </div>
 
-<script src="/js/watched_indicator.js"></script>
+<script src="/<%= JS_PATH %>/watched_indicator.js"></script>

src/invidious/views/feeds/subscriptions.ecr

@@ -53,7 +53,7 @@
 }.to_pretty_json
 %>
 </script>
-<script src="/js/watched_widget.js"></script>
+<script src="/<%= JS_PATH %>/watched_widget.js"></script>
 
 
 <div class="pure-g">
@@ -62,7 +62,7 @@
 <% end %>
 </div>
 
-<script src="/js/watched_indicator.js"></script>
+<script src="/<%= JS_PATH %>/watched_indicator.js"></script>
 
 <%=
   IV::Frontend::Pagination.nav_numeric(locale,

src/invidious/views/feeds/trending.ecr

@@ -46,4 +46,4 @@
 <% end %>
 </div>
 
-<script src="/js/watched_indicator.js"></script>
+<script src="/<%= JS_PATH %>/watched_indicator.js"></script>

src/invidious/views/playlist.ecr

@@ -118,7 +118,7 @@
 }.to_pretty_json
 %>
 </script>
-<script src="/js/playlist_widget.js?v=<%= ASSET_COMMIT %>"></script>
+<script src="/<%= JS_PATH %>/playlist_widget.js?v=<%= ASSET_COMMIT %>"></script>
 <% end %>
 
 

src/invidious/views/post.ecr

@@ -44,5 +44,5 @@
 }.to_pretty_json
 %>
 </script>
-<script src="/js/comments.js?v=<%= ASSET_COMMIT %>"></script>
+<script src="/<%= JS_PATH %>/comments.js?v=<%= ASSET_COMMIT %>"></script>
-<script src="/js/post.js?v=<%= ASSET_COMMIT %>"></script>
+<script src="/<%= JS_PATH %>/post.js?v=<%= ASSET_COMMIT %>"></script>

src/invidious/views/template.ecr

@@ -1,8 +1,6 @@
 <%
 	locale = env.get("preferences").as(Preferences).locale
 	dark_mode = env.get("preferences").as(Preferences).dark_mode
-	current_backend = env.request.cookies[CONFIG.server_id_cookie_name]?.try &.value || env.request.headers["Host"]
-    current_external_videoplayback_proxy = Invidious::HttpServer::Utils.get_external_proxy()
 %>
 <!DOCTYPE html>
 <html lang="<%= locale %>">
@@ -24,7 +22,7 @@
 		<link rel="stylesheet" href="/css/ionicons.min.css?v=<%= ASSET_COMMIT %>">
 		<link rel="stylesheet" href="/css/default.css?v=<%= ASSET_COMMIT %>">
 		<link rel="stylesheet" href="/css/carousel.css?v=<%= ASSET_COMMIT %>">
-		<script src="/js/_helpers.js?v=<%= ASSET_COMMIT %>"></script>
+		<script src="/<%= JS_PATH %>/_helpers.js?v=<%= ASSET_COMMIT %>"></script>
 	</head>
 
 	<body class="<%= dark_mode.blank? ? "no" : dark_mode %>-theme">
@@ -34,7 +32,9 @@
 				<div class="pure-g navbar h-box">
 					<% if navbar_search %>
 						<div class="pure-u-1 pure-u-md-4-24">
-							<a href="/" class="index-link pure-menu-heading">Invidious</a>
+							<a href="/" class="index-link pure-menu-heading">
+							Invidious
+							</a>
 						</div>
 						<div class="pure-u-1 pure-u-md-12-24 searchbar">
 							<% autofocus = false %><%= rendered "components/search_box" %>
@@ -106,30 +106,54 @@
 					</div>
 				</div>
 
-				<% if !CONFIG.backends.empty? %>
+				<%
-                <% if !CONFIG.backend_domains.includes?(env.request.headers["Host"]) %>
+					if CONFIG.invidious_companion.present?
+					current_backend = env.get?("current_companion").try &.as(Int32)
+					domain = env.get?("using_domain")
+					scheme = env.get("scheme")
+					status = BackendInfo.get_status
+					companion_switched = env.get?("companion_switched")
+				%>
 				<div class="h-box" style="margin-bottom: 10px;">
 					<b>Switch Backend:</b>
-					<% CONFIG.backends.each do | backend | %>
+					<% if domain %>
-					<% backend = backend.split(CONFIG.backends_delimiter) %>
+						<% CONFIG.invidious_companion.each_with_index do | companion, index | %>
-					<% if current_backend == backend[0] %>
+						<% host_backend = env.request.headers["Host"].sub(/([^.]+)(\d+)/, "\\1#{index+1}") %>
-						<a href="/switchbackend?backend_id=<%= backend[0] %>" style="text-decoration-line: underline; display: inline-block;">
+						<% is_current_backend_host = host_backend == env.request.headers["Host"] %>
-							Backend<%= HTML.escape(backend[0]) %>
+							<a href="<%= scheme %>://<%= host_backend %><%= env.request.resource %>" style="<%= is_current_backend_host ? "text-decoration-line: underline;" : "" %> display: inline-block;">
-					<% if backend.size == 2 %>
+								Backend<%= HTML.escape((index + 1).to_s) %> <%= HTML.escape(companion.note) %>
-							<%= HTML.escape(backend[1]) %>
+								<span style="color:
+									<% if status[index] == 0 %> #fd4848; <% end %>
+									<% if status[index] == 1 %> #d06925; <% end %>
+									<% if status[index] == 2 %> #42ae3c; <% end %>
+								">•</span>
+							</a>
+							<% if !(index == CONFIG.invidious_companion.size-1) %>
+							<span> | </span>
+							<% end %>
 						<% end %>
-						</a> <span> | </span>
 					<% else %>
-						<a href="/switchbackend?backend_id=<%= backend[0] %>" style="display: inline-block;">
+						<% CONFIG.invidious_companion.each_with_index do | companion, index | %>
-							Backend<%= HTML.escape(backend[0]) %>
+							<a href="/switchbackend?backend_id=<%= index.to_s %>" style="<%= current_backend == index ? "text-decoration-line: underline;" : "" %> display: inline-block;">
-					<% if backend.size == 2 %>
+								Backend<%= HTML.escape((index + 1).to_s) %> <%= HTML.escape(companion.note) %>
-							<%= HTML.escape(backend[1]) %>
+								<span style="color:
+									<% if status[index] == 0 %> #fd4848; <% end %>
+									<% if status[index] == 1 %> #d06925; <% end %>
+									<% if status[index] == 2 %> #42ae3c; <% end %>
+								">•</span>
+							</a>
+							<% if !(index == CONFIG.invidious_companion.size-1) %>
+							<span> | </span>
 							<% end %>
-						</a> <span> | </span>
 						<% end %>
 					<% end %>
 				</div>
 				<% end %>
+
+				<% if companion_switched %>
+					<div class="h-box">
+						<p><%= translate(locale, "backend_unavailable") %></p>
+					</div>
 				<% end %>
 
 				<% if CONFIG.banner %>
@@ -146,10 +170,10 @@
 
 		</div>
 	</div>
-    <script src="/js/handlers.js?v=<%= ASSET_COMMIT %>"></script>
+	<script src="/<%= JS_PATH %>/handlers.js?v=<%= ASSET_COMMIT %>"></script>
-    <script src="/js/themes.js?v=<%= ASSET_COMMIT %>"></script>
+	<script src="/<%= JS_PATH %>/themes.js?v=<%= ASSET_COMMIT %>"></script>
 	<% if env.get? "user" %>
-        <script src="/js/sse.js?v=<%= ASSET_COMMIT %>"></script>
+		<script src="/<%= JS_PATH %>/sse.js?v=<%= ASSET_COMMIT %>"></script>
 		<script id="notification_data" type="application/json">
 		<%=
 		{
@@ -159,7 +183,7 @@
 		%>
 		</script>
 		<% if CONFIG.enable_user_notifications %>
-        <script src="/js/notifications.js?v=<%= ASSET_COMMIT %>"></script>
+		<script src="/<%= JS_PATH %>/notifications.js?v=<%= ASSET_COMMIT %>"></script>
 		<% end %>
 	<% end %>
 
@@ -313,9 +337,11 @@
 			</div>
 			<hr/>
 			<div class="footer-footer">
-			<div class="box">You are currently using Backend: <%= current_backend %></div>
+			<%
-            <% if !current_external_videoplayback_proxy.empty? %>
+				if CONFIG.invidious_companion.present?
-            <div class="box">External Videoplayback Proxy: <%= current_external_videoplayback_proxy %></div>
+				companion_public_url = env.get("companion_public_url").as(String)
+			%>
+			<div class="box">You are currently using Backend: <%= current_backend ? companion_public_url : "Unable to get backend, this is bug, please report it!" %></div>
 			<% end %>
 				<span class="left">
 					<% if CONFIG.modified_source_code_url %>

src/invidious/views/user/change_username.ecr

@@ -0,0 +1,26 @@
+<% content_for "header" do %>
+<title><%= translate(locale, "change_username") %> - Invidious</title>
+<% end %>
+
+<div class="pure-g">
+    <div class="pure-u-1 pure-u-lg-1-5"></div>
+    <div class="pure-u-1 pure-u-lg-3-5">
+        <div class="h-box">
+            <form class="pure-form pure-form-aligned" action="/change_username?referer=<%= URI.encode_www_form(referer) %>" method="post">
+                <legend><%= translate(locale, "") %></legend>
+
+                <fieldset>
+                    <label for="new_username"><%= translate(locale, "new_username") %> :</label>
+                    <input required class="pure-input-1" name="new_username" type="text" placeholder="<%= translate(locale, "new_username") %>">
+
+                    <button type="submit" name="action" value="change_username" class="pure-button pure-button-primary">
+                        <%= translate(locale, "change_username") %>
+                    </button>
+
+                    <input type="hidden" name="csrf_token" value="<%= HTML.escape(csrf_token) %>">
+                </fieldset>
+            </form>
+        </div>
+    </div>
+    <div class="pure-u-1 pure-u-lg-1-5"></div>
+</div>

src/invidious/views/user/preferences.ecr

@@ -34,7 +34,10 @@
 
             <div class="pure-control-group">
                 <label for="local"><%= translate(locale, "preferences_local_label") %></label>
-                <input name="local" id="local" type="checkbox" <% if preferences.local && !CONFIG.disabled?("local") %>checked<% end %> <% if CONFIG.disabled?("local") %>disabled<% end %>>
+                <input name="local" id="local" type="checkbox" <% if CONFIG.force_local %>disabled="" <% end %><% if preferences.local && !CONFIG.disabled?("local") %>checked<% end %> <% if CONFIG.disabled?("local") %>disabled<% end %>>
+                <% if CONFIG.force_local %>
+                    <label for="local">(All videos are proxied by default)</label>
+                <% end %>
             </div>
 
             <div class="pure-control-group">
@@ -330,6 +333,10 @@
                     <a href="/change_password?referer=<%= URI.encode_www_form(referer) %>"><%= translate(locale, "Change password") %></a>
                 </div>
 
+                <div class="pure-control-group">
+                    <a href="/change_username?referer=<%= URI.encode_www_form(referer) %>"><%= translate(locale, "change_username") %></a>
+                </div>
+
                 <div class="pure-control-group">
                     <a href="/data_control?referer=<%= URI.encode_www_form(referer) %>"><%= translate(locale, "Import/export data") %></a>
                 </div>

src/invidious/views/watch.ecr

@@ -2,16 +2,17 @@
 <% title = HTML.escape(video.title) %>
 <% author = HTML.escape(video.author) %>
 <% host = env.request.headers["Host"] %>
+<% scheme = env.get("scheme") %>
 
 
 <% content_for "header" do %>
-<meta name="thumbnail" content="<%= thumbnail %>">
+<meta name="thumbnail" content="<%= scheme %>://<%= host %><%= thumbnail %>">
 <meta name="description" content="<%= HTML.escape(video.short_description) %>">
 <meta name="keywords" content="<%= video.keywords.join(",") %>">
 <meta property="og:site_name" content="<%= author %> | Invidious">
-<meta property="og:url" content="<%= host %>/watch?v=<%= video.id %>">
+<meta property="og:url" content="<%= scheme %>://<%= host %>/watch?v=<%= video.id %>">
 <meta property="og:title" content="<%= title %>">
-<meta property="og:image" content="<%= host %>/vi/<%= video.id %>/maxres.jpg">
+<meta property="og:image" content="<%= scheme %>://<%= host %>/vi/<%= video.id %>/maxres.jpg">
 <meta property="og:description" content="<%= HTML.escape(video.short_description) %>">
 <meta property="og:type" content="video.other">
 <!-- This shouldn't be empty, ever. -->
@@ -22,11 +23,11 @@
 <meta property="og:video:width" content="640">
 <meta property="og:video:height" content="360">
 <meta name="twitter:card" content="player">
-<meta name="twitter:url" content="<%= host %>/watch?v=<%= video.id %>">
+<meta name="twitter:url" content="<%= scheme %>://<%= host %>/watch?v=<%= video.id %>">
 <meta name="twitter:title" content="<%= title %>">
 <meta name="twitter:description" content="<%= HTML.escape(video.short_description) %>">
-<meta name="twitter:image" content="<%= host %>/vi/<%= video.id %>/maxres.jpg">
+<meta name="twitter:image" content="<%= scheme %>://<%= host %>/vi/<%= video.id %>/maxres.jpg">
-<meta name="twitter:player" content="<%= host %>/embed/<%= video.id %>">
+<meta name="twitter:player" content="<%= scheme %>://<%= host %>/embed/<%= video.id %>">
 <meta name="twitter:player:width" content="1280">
 <meta name="twitter:player:height" content="720">
 <link rel="alternate" href="https://www.youtube.com/watch?v=<%= video.id %>">
@@ -198,7 +199,7 @@ we're going to need to do it here in order to allow for translations.
                     }.to_pretty_json
                     %>
                     </script>
-                    <script src="/js/playlist_widget.js?v=<%= Time.utc.to_unix_ms %>"></script>
+                    <script src="/<%= JS_PATH %>/playlist_widget.js?v=<%= Time.utc.to_unix_ms %>"></script>
                 <% end %>
             <% end %>
 
@@ -207,6 +208,7 @@ we're going to need to do it here in order to allow for translations.
             <p id="views"><i class="icon ion-ios-eye"></i> <%= number_with_separator(video.views) %></p>
             <p id="likes"><i class="icon ion-ios-thumbs-up"></i> <%= number_with_separator(video.likes) %></p>
             <p id="dislikes" style="display: none; visibility: hidden;"></p>
+            <% if !video.genre.nil? && !video.genre.empty? %>
                 <p id="genre"><%= translate(locale, "Genre: ") %>
                     <% if !video.genre_url %>
                         <%= video.genre %>
@@ -214,6 +216,7 @@ we're going to need to do it here in order to allow for translations.
                         <a href="<%= video.genre_url %>"><%= video.genre %></a>
                     <% end %>
                 </p>
+            <% end %>
             <% if video.license %>
                 <% if video.license.empty? %>
                     <p id="license"><%= translate(locale, "License: ") %><%= translate(locale, "Standard YouTube license") %></p>
@@ -225,6 +228,7 @@ we're going to need to do it here in order to allow for translations.
             <p id="wilson" style="display: none; visibility: hidden;"></p>
             <p id="rating" style="display: none; visibility: hidden;"></p>
             <p id="engagement" style="display: none; visibility: hidden;"></p>
+            <% if !video.allowed_regions.nil? && !video.allowed_regions.empty? %>
                 <% if video.allowed_regions.size != REGIONS.size %>
                     <p id="allowed_regions">
                         <% if video.allowed_regions.size < REGIONS.size // 2 %>
@@ -234,6 +238,7 @@ we're going to need to do it here in order to allow for translations.
                         <% end %>
                     </p>
                 <% end %>
+            <% end %>
         </div>
     </div>
 
@@ -386,5 +391,5 @@ we're going to need to do it here in order to allow for translations.
         </div>
     <% end %>
 </div>
-<script src="/js/comments.js?v=<%= ASSET_COMMIT %>"></script>
+<script src="/<%= JS_PATH %>/comments.js?v=<%= ASSET_COMMIT %>"></script>
-<script src="/js/watch.js?v=<%= ASSET_COMMIT %>"></script>
+<script src="/<%= JS_PATH %>/watch.js?v=<%= ASSET_COMMIT %>"></script>

src/invidious/yt_backend/connection_pool.cr

@@ -49,7 +49,7 @@ end
 struct CompanionConnectionPool
   property pool : DB::Pool(HTTP::Client)
 
-  def initialize(capacity = 5, timeout = 5.0)
+  def initialize(companion, capacity = 5, timeout = 5.0)
     options = DB::Pool::Options.new(
       initial_pool_size: 0,
       max_pool_size: capacity,
@@ -58,23 +58,22 @@ struct CompanionConnectionPool
     )
 
     @pool = DB::Pool(HTTP::Client).new(options) do
-      companion = CONFIG.invidious_companion.sample
+      next make_client(companion.private_url, use_http_proxy: false)
-      next make_client(companion.private_url, force_resolve: true)
     end
   end
 
   def client(&)
     conn = pool.checkout
-    # Proxy needs to be reinstated every time we get a client from the pool
-    conn.proxy = make_configured_http_proxy_client() if CONFIG.http_proxy
 
     begin
       response = yield conn
     rescue ex
       conn.close
 
-      companion = CONFIG.invidious_companion.sample
+      scheme = "https" if conn.tls || "http"
-      conn = make_client(companion.private_url, force_resolve: true)
+      same_companion = "#{scheme}://#{conn.host}:#{conn.port}"
+
+      conn = make_client(URI.parse(same_companion), use_http_proxy: false)
 
       response = yield conn
     ensure

src/invidious/yt_backend/youtube_api.cr

@@ -456,6 +456,7 @@ module YoutubeAPI
     *, # Force the following parameters to be passed by name
     params : String,
     client_config : ClientConfig | Nil = nil,
+    env : HTTP::Server::Context | Nil = nil,
   )
     # Playback context, separate because it can be different between clients
     playback_ctx = {
@@ -492,7 +493,7 @@ module YoutubeAPI
     end
 
     if CONFIG.invidious_companion.present?
-      return self._post_invidious_companion("/youtubei/v1/player", data)
+      return self._post_invidious_companion("/youtubei/v1/player", data, env)
     else
       return self._post_json("/youtubei/v1/player", data, client_config)
     end
@@ -672,7 +673,8 @@ module YoutubeAPI
   #
   def _post_invidious_companion(
     endpoint : String,
-    data : Hash
+    data : Hash,
+    env : HTTP::Server::Context | Nil,
   ) : Hash(String, JSON::Any)
     headers = HTTP::Headers{
       "Content-Type"  => "application/json; charset=UTF-8",
@@ -686,7 +688,12 @@ module YoutubeAPI
     # Send the POST request
 
     begin
-      response = COMPANION_POOL.client &.post(endpoint, headers: headers, body: data.to_json)
+      if env.nil?
+        current_companion = rand(CONFIG.invidious_companion.size)
+      else
+        current_companion = env.get("current_companion").as(Int32)
+      end
+      response = COMPANION_POOL[current_companion].client &.post(endpoint, headers: headers, body: data.to_json)
       body = response.body
       if (response.status_code != 200)
         raise Exception.new(