Merge remote-tracking branch 'upstream/master'

CHANGELOG.md

@@ -2,7 +2,102 @@
 
 ## vX.Y.0 (future)
 
+## v2.20250314.0
 
+### Wrap-up
+
+This release brings the long awaited feature of supporting multiple audio tracks in a video, some bug fixes and UX improvements, and many other things primarily oriented to self-hosting instances, and developers using the API.
+
+The `Community` channel tab has been replaced by `Posts` in light of YouTube changes, but the URL remains the same.
+
+Tamil is now available as an interface language
+
+Automatic instance redirects will no longer have the chance to annoyingly redirect to the same instance you're on.
+
+Due to their requirements for video playback, Invidious will log warning messages when either inv-sig-helper, `po_token` or `visitor_data` is not configured
+
+Invidious is now able to listen through a UNIX socket
+
+User notifications are now batched for each channel
+
+**The minimum Crystal version supported by Invidious now `1.12.0`**
+
+### New features & important changes
+
+#### For users
+
+* Invidious now supports videos with multiple audio tracks allowing you to select which one you want to hear with!
+* Channel pages now have a proper previous page button
+* RSS feeds for channels will no longer contain the channel's profile picture
+* Support for channel `courses` page has been added
+* `Community` tabs has been replaced with `Posts` to comply with YouTube changes
+* Tamil is now an available interface language.
+
+#### For instance owners
+* Invidious is now able to listen on a UNIX socket
+* User notifications are now batched by channels, significantly reducing database load.
+* **`1.12.0` is now the oldest Crystal version that Invidious supports**
+* The example config will no longer force an http proxy to be configured
+* Invidious will now warn when any top-level config option must be set to a custom value, instead of just `HMAC_KEY`
+* Due to their requirements for video playback, Invidious will log warning messages when either inv-sig-helper, `po_token` or `visitor_data` is not configured
+
+#### For developers
+* Invidious is now compliant to Crystal 1.15 formatting rules, which are incompatible with earlier versions.
+* `/api/v1/transcripts/{id}` has been added to the API to allow for fetching the transcripts for a video. The arguments are the same as the captions endpoint.
+* `author_thumbnail` field has been added to videos in the various paged api endpoints
+* `published` field has been added to the API response for a video's related videos.
+* Docker builds now uses the Crystal compiler cache, reducing build times on repeated builds significantly.
+* Invidious ajax action handlers has undergone a clean up and may face compatibility issues with code that depends on these endpoints.
+* The versions of Crystal that we test in CI/CD are now: `1.12.1`, `1.13.2`, `1.14.0`, `1.15.0`
+
+### Bugs fixed
+
+#### User-side
+* Local video listen mode is now preserved when clicking on a video in the sidebar playlist widget
+* Automatic instance redirects will no longer redirect to the same instance the user is on
+* Fix some thumbnails responses returning 404
+* Videos: Fix missing host parameter on playback URLs when `local=true`
+* Fix HLS being used for non-livestream videos
+* Fix timeupdate event errors when required elements are missing
+* User: Ensure IO is properly closed when importing NewPipe subscriptions
+
+#### For instance owners
+* Fix http proxy configuration being forced by the standard example config
+
+#### API
+* `/api/v1/videos/{id}` will no longer return an occasional empty JSON response
+
+### Full list of pull requests merged since the last release (newest first)
+* Make Invidious compliant to Crystal 1.15 formatting rules (https://github.com/iv-org/invidious/pull/5014, by @syeopite)
+* Remove formatter check on container workflows (https://github.com/iv-org/invidious/pull/5153, by @syeopite)
+* Videos: Fix missing host parameter on playback URLs when `local=true` (https://github.com/iv-org/invidious/pull/4992, by @SamantazFox)
+* Remove stdlib override for proxy initialization (https://github.com/iv-org/invidious/pull/5065, by @syeopite)
+* Add support for author thumbnails in search api for videos (https://github.com/iv-org/invidious/pull/5072, thanks @ChunkyProgrammer)
+* Skip route if resp got closed by before handlers (https://github.com/iv-org/invidious/pull/5073, by @syeopite)
+* Fix video thumbnails in mixes (https://github.com/iv-org/invidious/pull/5116, thanks @iBicha)
+* CI: Drop support for versions prior to 1.12 and add 1.15.0 (https://github.com/iv-org/invidious/pull/5148, by @syeopite)
+* [Continuing #5094] Set language info for dash audio streams and sort (https://github.com/iv-org/invidious/pull/5149, thanks @giuliano-macedo)
+* Warn when any top-level config is "CHANGE_ME!!" (https://github.com/iv-org/invidious/pull/5150, by @syeopite)
+* Comment out http_proxy in example config (https://github.com/iv-org/invidious/pull/5151, by @syeopite)
+* API: Add a 'published' video parameter for related videos (https://github.com/iv-org/invidious/pull/4149, thanks @RadoslavL)
+* Ensure IO is properly closed when importing NewPipe subscriptions (https://github.com/iv-org/invidious/pull/4346, thanks @ChunkyProgrammer)
+* Carry over audio-only mode in playlist links (https://github.com/iv-org/invidious/pull/4784, thanks @krystof1119)
+* Routes: Clean ajax actions handlers (https://github.com/iv-org/invidious/pull/5036, by @SamantazFox)
+* Frontend: Add a first page and previous page buttons for channel navigation (https://github.com/iv-org/invidious/pull/4123, thanks @RadoslavL)
+* RSS: Channel + Playlist improvements (https://github.com/iv-org/invidious/pull/4298, thanks @ChunkyProgrammer)
+* Batch user notifications together (https://github.com/iv-org/invidious/pull/4486, thanks @999eagle)
+* JS: Update timeupdate event making it more defensive to prevent errors (https://github.com/iv-org/invidious/pull/4782, thanks @PMK)
+* Add API endpoint for fetching transcripts from YouTube by (https://github.com/iv-org/invidious/pull/4788, by @syeopite)
+* Translations update from Hosted Weblate by (https://github.com/iv-org/invidious/pull/4989, thanks to our many translators)
+* Add the ability to listen on UNIX sockets (https://github.com/iv-org/invidious/pull/5112, thanks @Caian)
+* Pick a different instance upon redirect (https://github.com/iv-org/invidious/pull/5154, thanks @epicsam123)
+* Add Courses to channel page and channel API (https://github.com/iv-org/invidious/pull/5158, thanks @ChunkyProgrammer)
+* fix /api/v1/videos/:id returns 200 with no content (https://github.com/iv-org/invidious/pull/5162, thanks @Drikanis)
+* Use Crystal compiler cache in docker builds (https://github.com/iv-org/invidious/pull/5163, by @syeopite)
+* Channels: Fix community tab by (https://github.com/iv-org/invidious/pull/5183, thanks @Fijxu)
+* Fix typo in `src/invidious/routes/images.cr` (https://github.com/iv-org/invidious/pull/5184, by @syeopite)
+* Fix an issue with the HLS manifest check for livestream videos (https://github.com/iv-org/invidious/pull/5189, thanks @alexmaras)
+* Warn when `po_token`, `visitor_data` and/or `inv-sig-helper` is not configured (https://github.com/iv-org/invidious/pull/5202, by @syeopite)
 ## v2.20241110.0
 
 ### Wrap-up

shard.yml

@@ -1,5 +1,5 @@
 name: invidious
-version: 2.20241110.0-dev
+version: 2.20250314.0
 
 authors:
   - Invidious team <contact@invidious.io>

src/invidious/config.cr

@@ -387,6 +387,10 @@ class Config
         puts "Config: The value of 'invidious_companion_key' needs to be a size of 16 or more."
         exit(1)
       end
+    elsif config.signature_server
+      puts("WARNING: inv-sig-helper is deprecated. Please switch to Invidious companion: https://docs.invidious.io/companion-installation/")
+    else
+      puts("WARNING: Invidious companion is required to view and playback videos. For more information see https://docs.invidious.io/companion-installation/")
     end
 
     # HMAC_key is mandatory

src/invidious/routes/embed.cr

@@ -203,6 +203,14 @@ module Invidious::Routes::Embed
       return env.redirect url
     end
 
+    if CONFIG.invidious_companion.present?
+      invidious_companion = CONFIG.invidious_companion.sample
+      env.response.headers["Content-Security-Policy"] =
+        env.response.headers["Content-Security-Policy"]
+          .gsub("media-src", "media-src #{invidious_companion.public_url}")
+          .gsub("connect-src", "connect-src #{invidious_companion.public_url}")
+    end
+
     rendered "embed"
   end
 end

src/invidious/routes/watch.cr

@@ -209,6 +209,14 @@ module Invidious::Routes::Watch
       video_url = nil
     end
 
+    if CONFIG.invidious_companion.present?
+      invidious_companion = CONFIG.invidious_companion.sample
+      env.response.headers["Content-Security-Policy"] =
+        env.response.headers["Content-Security-Policy"]
+          .gsub("media-src", "media-src #{invidious_companion.public_url}")
+          .gsub("connect-src", "connect-src #{invidious_companion.public_url}")
+    end
+
     templated "watch"
   end
 
@@ -339,7 +347,8 @@ module Invidious::Routes::Watch
 
       if (CONFIG.invidious_companion.present?)
         video = get_video(video_id)
-        return env.redirect "#{video.invidious_companion["baseUrl"].as_s}/latest_version?#{env.params.query}"
+        invidious_companion = CONFIG.invidious_companion.sample
+        return env.redirect "#{invidious_companion.public_url}/latest_version?#{env.params.query}"
       else
         return Invidious::Routes::VideoPlayback.latest_version(env)
       end

src/invidious/videos/parser.cr

@@ -108,7 +108,7 @@ def extract_video_info(video_id : String)
   params = parse_video_info(video_id, player_response)
   params["reason"] = JSON::Any.new(reason) if reason
 
-  if CONFIG.invidious_companion.present?
+  if !CONFIG.invidious_companion.present?
     new_player_response = nil
 
     # Don't use Android test suite client if po_token is passed because po_token doesn't
@@ -133,7 +133,7 @@ def extract_video_info(video_id : String)
     end
   end
 
-  {"captions", "playabilityStatus", "playerConfig", "storyboards", "invidiousCompanion"}.each do |f|
+  {"captions", "playabilityStatus", "playerConfig", "storyboards"}.each do |f|
     params[f] = player_response[f] if player_response[f]?
   end
 

src/invidious/views/components/player.ecr

@@ -22,8 +22,8 @@
                audio_streams.each_with_index do |fmt, i|
                 src_url  = "/latest_version?id=#{video.id}&itag=#{fmt["itag"]}"
                 src_url += "&local=true" if params.local
-                src_url = video.invidious_companion["baseUrl"].as_s + src_url + 
-                            "&check=#{invidious_companion_encrypt(video.id)}" if (CONFIG.invidious_companion.present?)
+                src_url = invidious_companion.public_url.to_s + src_url + 
+                            "&check=#{invidious_companion_encrypt(video.id)}" if (invidious_companion)
 
                 bitrate = fmt["bitrate"]
                 mimetype = HTML.escape(fmt["mimeType"].as_s)
@@ -38,8 +38,8 @@
         <% else %>
             <% if params.quality == "dash"
                src_url = "/api/manifest/dash/id/" + video.id + "?local=true&unique_res=1"
-               src_url = video.invidious_companion["baseUrl"].as_s + src_url +
-                            "&check=#{invidious_companion_encrypt(video.id)}" if (CONFIG.invidious_companion.present?)
+               src_url = invidious_companion.public_url.to_s + src_url +
+                            "&check=#{invidious_companion_encrypt(video.id)}" if (invidious_companion)
             %>
                 <source src="<%= src_url %>" type='application/dash+xml' label="dash">
             <% end %>
@@ -50,8 +50,8 @@
             fmt_stream.each_with_index do |fmt, i|
                 src_url  = "/latest_version?id=#{video.id}&itag=#{fmt["itag"]}"
                 src_url += "&local=true" if params.local
-                src_url = video.invidious_companion["baseUrl"].as_s + src_url +
-                            "&check=#{invidious_companion_encrypt(video.id)}" if (CONFIG.invidious_companion.present?)
+                src_url = invidious_companion.public_url.to_s + src_url +
+                            "&check=#{invidious_companion_encrypt(video.id)}" if (invidious_companion)
 
                 quality = fmt["quality"]
                 mimetype = HTML.escape(fmt["mimeType"].as_s)

src/invidious/yt_backend/connection_pool.cr

@@ -59,14 +59,12 @@ struct CompanionConnectionPool
 
     @pool = DB::Pool(HTTP::Client).new(options) do
       companion = CONFIG.invidious_companion.sample
-      next make_client(companion.private_url, force_resolve: true)
+      next make_client(companion.private_url, use_http_proxy: false)
     end
   end
 
   def client(&)
     conn = pool.checkout
-    # Proxy needs to be reinstated every time we get a client from the pool
-    conn.proxy = make_configured_http_proxy_client() if CONFIG.http_proxy
 
     begin
       response = yield conn
@@ -74,7 +72,7 @@ struct CompanionConnectionPool
       conn.close
 
       companion = CONFIG.invidious_companion.sample
-      conn = make_client(companion.private_url, force_resolve: true)
+      conn = make_client(companion.private_url, use_http_proxy: false)
 
       response = yield conn
     ensure

src/invidious/yt_backend/youtube_api.cr

@@ -672,7 +672,7 @@ module YoutubeAPI
   #
   def _post_invidious_companion(
     endpoint : String,
-    data : Hash
+    data : Hash,
   ) : Hash(String, JSON::Any)
     headers = HTTP::Headers{
       "Content-Type"  => "application/json; charset=UTF-8",