[BUG] BROKEN FEED (Button to mass un-re-subscribe) #21
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
by crosschecking the feed with other instances or youtube directly, the feed is for some reason broken! due to this, there are like 10 videos daily! (from 200 subscriptions) that are not entering nadeko feed properly.
right now, "the fix" is to flash JSON data file 2-3 times, so the lazy videos would enter the nadeko feed properly
suggestions:
About this. I'm already fixing it. Right now I'm running a feed fetcher on a completely different domain and it's only purpose is to receive data and send data to youtube, for subscriptions.
Right now It's subscribing to all channels (~140K) sooo this can take up to 4 hours to complete.
9878a3d4d6All feeds are updated. Wait a few days and if there is no missing videos from your subscriptions, then all good.
sadly, feed is still broken :( situation is better than it used to be but some videos still dont find its way into nadeko feed
still loosing from 1 to 3 videos daily from the feed:
[Enhancement] Button to mass un-re-subscribeto [BUG] BROKEN FEED (Button to mass un-re-subscribe)yesterday:
so, here are results of testing. after 3 days, 31 videos were missing in the feed.
steps to reproduce:
now questions are:
I just skipped one important thing while deploying the feed fetcher.
This line inside invidious: https://git.nadeko.net/Fijxu/invidious/src/branch/master/src/invidious/jobs/subscribe_to_feeds_job.cr#L18
only runs one time when invidious starts and the query never ends (which IMO, this shouldn't be like this) making Invidious not subscribe to the channels again (Because it needs to resubscribe every ~4 days). I added a cron job to restart the feed fetcher automatically every ~15min.
Thanks for your investigation! Tell me if you don't get videos in the subscription feed again in the next few days.
alright, the situation is much better now, but the feed is still broken. I made you video of how I need to fix it every time with JSON flashing.
as you can see I emptied the whole feed on all backends and after JSON re-injection, stuff magically appears.
3 days ago I was missing 3 videos from feed and today as you can see - 2 videos, so in total, 5 videos shorter since your last fix.
3 days went by since last report.
the result: an additional 3 videos are missing from feed
Huh. I might change the logic that invidious uses to subscribe to channels.
But that is significantly less than last time right?
yes, since this fix of yours: #21 (comment) it's much better than before, but still not 100% perfect.
I pretty much don't care if it would be delayed for up to one day (compared to yt) but the issue is that stuff is missing for days/weeks even longer, until JSON is flashed. without flashing it, it's unreliable.
I discovered the problem. There was 2 channels that for some reason Invidious has been unable to subscribe hanging the Subscribe Job. This job needs to run every 1 minute, but because of two channels hanging the job, the Job never completed and never ran again after 1 minute.
This job needs some sort of handling for those channels that Invidious cannot subscribe because of PubSubHubBub errors or restrictions.
alright, so I just flashed JSON to reset stuff (5 videos appeared that were missing from feed for past two days) and will leave it for few days and crosscheck with YT if all is good. then report back here my findings.
not sure if this is related to your latest fix but visiting https://inv.nadeko.net/channel/UCx3t9Tn-iebofzahMM1g-Ow (or any other channel) yields this error:
Could not extract JSON. Youtube API returned error 400 with message:
"Request contains an invalid argument."
strangely, the last uploaded video from this channel did not enter my feed. is your latest feed fix retrospective? or the feed should work only since the fix onward?
#36
sadly, the issue of broken feed persists. 3 days, another 2 videos missing :(
another 3 days - another 3 videos missing
At how many channels are you subscribed? To fix this, I would need to add error checking to Invidious SubcribeToFeeds job.
atm to 147
another 5 days - another 7 videos
also, it's never from same channels - if that matters
didnt do this in a while so here we go.
some are as fresh as 1h ago and oldest is 4 weeks old!
could it be that there is again some channel that hanging the subscribe job again or somethin' ??
still feels like adding button to mass un-re-subscribe would patch it best lol
No, is not the best solution since youtube has rate limits.
What do you mean with emptied whole feed?
emptied - like I watched all videos from all my subscriptions up to the point that there is nothing to be watched
I have this enabled:
so if I watch everything, this page (https://inv.nadeko.net/feed/subscriptions) is literally empty, eg. I "emptied it"
then I do JSON export > JSON import and all missing videos enter the https://inv.nadeko.net/feed/subscriptions finally
it is same as I would cross-check it with YT 1-by-1 and some stuff that was released on YT, nadeko's https://inv.nadeko.net/feed/subscriptions did not pick up, hence they are missing from the feed.