Tool "forgets" login status (v2.9.1.366)

[stuwm]

When extracting files from hidden tumblrs, the tool is "forgetting" that I have already logged into Tumblr. When clicking on "Authenticate", the tool is not asking for credentials, but switches to "logged in" status immediately. This typically happens after 5-10 mins.
Along with this behaviour the tool is recognizing "latest post" days from years back, although newer posts exist. Then the tool is starting a rescan for that tumblr.

[MSWallack]

The temporary fix suggested in response to #392 works; however, like stuwm, I'm also experiencing the problem of the app immediately (or almost immediately) forgetting the authentication status and giving an error message that I need to authenticate to download.

[thomas694]

I'm not sure what's going wrong with your downloads. Please test with the newest version.

Generally, a rescan isn't started automatically. Of course, if a blog hasn't been downloaded for a long time, it may seem like a rescan on the next update.
Are these blogs old ones which you try to update now?
The value "latest post" is updated once a full or update scan finishes without errors.

[stuwm]

I have tested it with v2.9.2.368, but it is still happening.

[MSWallack]

I also updated to the newest version (v2.9.2) and the app still fails to retain the authentication status. In fact, just for fun, I tried clicking on the Privacy Consent button (after authenticating). A window opened but there was no privacy notice (or anything else to do). When I closed that window, the app had already changed its status back to not being authenticated.

[thomas694]

I think you aren't really logged out when you get this error (while downloading a blog). The next time you open the settings dialog, wait a few seconds until the status has been fetched from the server.
And the next time you get the error, also hover over the error bar. There should pop up a tooltip with an additional message that maybe gives us a better reason for that behavior. The message should appear in the log file too.

[MSWallack]

I played around with this over the long weekend. The best that I could get was to close the program entirely. Restart. Authenticate. Wait. If the Authenticate button remained grayed out, then I'd save and start downloading. Usually, after about 20-30 minutes, I'd start getting the errors telling me that I needed to login and that the server returned a 401 error. Trying to authenticate again usually didn't work. I'd have to close the program and try again.

[thomas694]

Thanks for the update. The error doesn't appear here. I'll look more the next days.

[stuwm]

I think I got it solved by quitting the application, deleting the cookies.json and running the app again.

[MSWallack]

I tried removing the cookies.json file, restarting the app, authenticating, and then downloading. Same issue. After a while, the app starts giving 401 errors and thinks that I'm not authenticated.

[thomas694]

You were indeed logged out, because Tumblr replaced the cookies way before the expiration date.

[stuwm]

Open item though is the "quality" of the "Latest Post" date for some hidden blogs:

1. It gives me back some date in 2014 although the latest post was a week ago.
2. It re-scans the whole blog as if "force rescan" had been turned on, although it had already been scanned.

[thomas694]

Does this work now with the test version?

[thomas694]

Could it be that this blog has a pinned post from that time?
I think in the column "Type" you see "Tumblr" and not "Tumblr Hidden", right? Change the crawler to "Tumblr SVC" in the blog settings. Then it should work. The other one has currently a problem downloading such blogs.

[thomas694]

Please try the test version and tell us, if it solves both.

[MSWallack]

I tried the test version. The app still loses its authentication status (not immediately, but usually after 20-30 minutes).

[thomas694]

At least it's a bit better now.
But actually I hoped that the change would solve the problem, because as of now I don't know what else could be the problem.

[MSWallack]

Here is a thought for a temporary ... well, not a fix I suppose, but a workflow option. Can the app recognize when it gets an authentication error and move to the next blog in the queue without marking the blog with the error as having been scanned (or marking the last scan date as of the last post before the error was triggered)? That would allow a user to let the app scan the blogs and then then keep scanning blogs that don't require authentication after the authentication status is lost? Maybe even color blogs that can't download because of an authentication error in light blue (or something) so that they can be readily found later.

[thomas694]

I'm not sure, but I think that can already be done. Failing blogs are removed from the queue and not marked as scanned. They have an empty column "newly downloaded".
I'd like rather to invest time to solve it, then doing a workaround.

[MSWallack]

Failing blogs are removed from the queue and the newly downloaded column remains empty. However, the date of the last full scan is updated.

And while I agree that a solution is better than a workaround, it looks like the solution may be difficult. I wonder if we could add a simple workaround (that might also be a worthwhile option generally) to allow a user to select types of errors that should cause the app to stop running the queue (e.g., don't stop on a blog that can't be found but do stop for a login error).

[thomas694]

I have the same 'forgot login status' issue for the last 3 days now. Is there an update somewhere? Update check says that i'm up to date. version 2.9.2.368

Originally posted by @jjdnl in #404 (comment)

[thomas694]

@jjdnl You can try the test version and tell us, if it solves it.

[jjdnl]

I've tried the test version, unfortunately it does not solve the problem.

[jjdnl]

Not sure what i did or didn't do, but for me it is working like it should, so no more forgetting. Even in the old version 2.9.2.368

[jjdnl]

Well it was fun while it lasted, but the logging out is back again. Version 2.10.0.384
Also getting a lot of limit exceeded messages, but tumblr api connections is already limited to 1 per 90 secs.

[thomas694]

I've never changed the default connection limits. They work quite well. It affects the "download speed" and counts towards their speed limit.
If you downloaded too much resp. made too many API requests you are hit by the daily limit. Once hit you cannot do much, except changing connection or waiting a few hours.

[MSWallack]

I installed v2.9.3 but the app still forgot the authentication status. I closed the app and deleted cookies.json. Same result (except that the authentication window stayed white; I tried again and this time the window showed my tumblr blog). I closed the app and rebooted. Same result. Thank you for your efforts, but it must be something else going on.

[stuwm]

TumblThree.zip
This is my log file which throws an error. Although I'm logged in tumblr and can see my blogs, it always hangs when trying to open my account settings.

[thomas694]

The log file tells that
"The remote server returned an error: (500) Internal Server Error".
In general that means that there's a (hopefully) temporary error at their server. Try it again at a later time.

I can log in without errors (and stay logged in).

It's kind of strange. For me it worked and works without issues. For some it worked again after a few days and/or deleting the cookies. Others are still logged out after a couple of minutes.
Is there a dev who has this problem and can support?

[Someonemustnothavethis]

I use T3 at the beginning of each month to catch up with my many blogs. I always allow it to update. Last month (1st Dec) it worked flawlessly. Today it's logging out every 20-30 minutes as others have described. Here's something else that may point to a problem or change at Tumblr's end...
I use Tumbex (like many other's here) to have a quick look at a new blog to see if it's worth adding to my download collection. Since Tumblr removed the archive view, it's the only way to quickly browse a blog. Over the past few weeks I've been getting this error when trying to open a blog and am wondering if this is causing the log out problem.

[thomas694]

As it is opened in your browser it cannot interfere with the app and shouldn't be a problem. But every hint could help.

[Someonemustnothavethis]

Here's another odd one to add to the list,
I've had to babysit T3 over that past three days to keep authenticating every 20 to 30 mins as it went through my pile of blogs. This morning I logged in (again, even though it showed as being logged in) and added 26 new blogs to the list, set them to download, then headed out for the morning. When I got home, ALL those new blogs had downloaded (90 some GB's of pictures) with no errors. T3 has been running for the entire time with no restarts. Gotta wonder if Tumblr changed something overnight?

[thomas694]

That's interesting. But currently some users still have the problem.

[Someonemustnothavethis]

Interestingly, Tumbex now (again) works fine and presents Tumblr blogs in an archive view with no errors. I wonder if this is a fix on Tumbex's part or if Tumblr undid whatever they'd done to screw it up.

[MSWallack]

I'm curious... For those who are not having authentication issues, what happens when you click the Authenticate button? If I recall, clicking that button used to bring up a Tumblr login screen. However, since the above-problem began, when I click Authenticate button, I'm taken to my Tumblr home page as if I've already logged in; I'm never actually asked to login or authenticate. I wonder if that might have something to do with the problem.

[thomas694]

When I click on the button, it redirects to my Tumblr dashboard/home page as I'm currently logged in. However, if I click logout first, the login page is shown.

You aren't shown the login page, because the embedded browser's session is still valid. It stores its settings in the folder "EBWebView" beside the app's settings folder.

Maybe you can try to delete this browser session folder together with the cookies.json. It shouldn't make much difference, but who knows.

[MSWallack]

Sounded like a promising solution. I deleted cookies.json and the EBWebView folder. When I restarted the app, the authentication screen worked as it should (i.e., it asked for username and password). Unfortunately, after about 30 minutes, it lost the login status again (and when I click the authentication button it takes me to my Tumblr blog as if I was already logged in).

[MSWallack]

Given that no solution has been found, can we think about workarounds? For example, if clicking the Authenticate button doesn't actually require new authentication, would it be possible (say, a user option) for the app to automatically "re-authenticate" whenever it encounters a 501 error? Or, at minimum, stop processing when it gets a 501 error?

[thomas694]

I'll take a look in the next days.

[MSWallack]

I tried the newest version. The good news is that I'm not logged out quite as quickly; the bad news is that I'm still being logged out.

[Someonemustnothavethis]

I did my once-a-month update of all my 219 blogs, and updated to 2.10.0 first thing this morning.
T3 ran like a champ all day without skipping a beat or asking to be logged in.
It left 37.9 GB in 117,067 files on my drive, then sat there wagging it's tail like a good dog. I gave it some treats, a scratch on the head, then sent it off to bed for a well deserved snooze till next month.
Good work!

[Someonemustnothavethis]

This month on the first of March I did my once-a-month update and it was back to requiring regular logins. I kept track and it was every 40 minutes exactly. What I also noticed was that it downloaded a LOT faster. It's normally an entire days worth (18+ hours) to grab everything, while this time it only took around 7 hours to get about the same number of images. All settings are the default.

[Linktydraa]

I'm having a similar issue as well. It was working fine for months, and now all of a sudden, it keeps "forgetting" I'm signed in. It maybe stays signed in for 15 minutes max, and I now have to babysit the program. Twitters downloads work just fine, but that's because we don't sign in for those. I have tried the fixes suggested, from deleting the web folder, to deleting the cookies json, and nothing works. I would really like to know why the program is suddenly glitching.

[thomas694]

That's an interesting finding. Our performance improvement was a few versions ago. But maybe they also made some improvements and upgrades.
The strange thing is that the same users experience different behavior with the same version. One time it works, the other time not. So I expect the app isn't solely responsible for that.

I would really like to know why the program is suddenly glitching.

We'd like to find the reason too.

[Linktydraa]

With this new update, tumblrthree is doing that thing again where it is constantly and regularly "forgetting" that I'm signed in and I keep having the reauthenticate......I thought updates were supposed to fix problems.

[thomas694]

Other things, we didn't say something was fixed for this one (authentication problem).

[thomas694]

Please try the new test version and tell me the result.

[thomas694]

Has anyone already tried the new test version?

@stuwm @MSWallack @Someonemustnothavethis @jjdnl

[MSWallack]

Yes. I downloaded that version. Here is what I posted yesterday on the Issues page:

I had to postpone downloading any additional files via the app for a few weeks while I attended to some other matters. Anyway, when I came back to the app it was once again logging me out regularly. I updated to the new test version mentioned in the most recent Discussion post. Same problem. However, I did note two things that may be meaningful: First, the program now shows a 401 error where it previously showed a 501 error. Second, in the dialog with the Authenticate button, even after clicking Authenticate and then closing the popup Tumblr window, a logout button does not appear and the "not logged in" message continues to be displayed.

I played with the app further last night. Oddly, now the button to allow me to logout does appear (usually after 5-10 seconds, but not always). Further, I now stay logged in for a much, much shorter period of time (from 20-30 minutes down to 5-10 minutes). I've even seen the program generate a 401 error while downloading a particular blog.

[stuwm]

I can confirm the behaviour described by @MSWallack. Following observations:

1. When the authentication window opens, my account is always shown as "logged in". I never had to login again (using 2FA).
2. The tool is sometimes showing the status as logged in, sometimes it remains as logged out. However, the blogs can be downloaded.
3. With the "fake" logged in status the error "500" is shown in the log file, otherwise an error "401".

I'll be sharing my log file along with screenshots as the last posting date is not always accurate and re-downloading is finding "new" files over and over again, updating the total blog number in strange ways, even if no new files were downloaded.

[MSWallack]

If we can't figure out why the app keeps being logged out of Tumblr, would it be possible to have an option that a user could select that has the app go through the steps to authenticate before scanning each new blog (essentially, an internal script that opens the settings dialog, goes to the right tab, clicks on authenticate, close the browser window, and the click save, but all invisibly in the background)?

[stuwm]

It seems that the cookies behave differently: even after a couple of days I don't need to login again, clicking on "authenticate" always brings me in logged in status. However, when downloading the blogs the time out comes in play and I do need to "login" again. I guess it is 30 mins, so the tool would only need to login automatically if 29mins or more have passed since the last login.What me currently bothers me more is the wrong latest post date on about 50+% of all blogs I'm checking. The tool spends more time identifying already downloaded files as the date is several months or even years wrong (shows older date although more recent entries exist in blog).

[MSWallack]

I was really, really bored the other day, so I tried timing how long before the app would (at least apparently) log me out. My results were ... .um ... wholly inconclusive. In a few instances, it seemed to work for longer than 30 minutes. But I also had instances where it would download a blog with very few images and be logged out by the end (2-3 minutes). I'd say that on average, it would stay logged in for at least 10-15 minutes, but I'm not sure that I'd building any reauthentication functionality solely around time.

[thomas694]

These are two different things here. The authenticate window is a chromium-based browser and we "borrow" the cookies to be able to download the things with the app. Something has changed so that cookies become invalid and the app gets logged out after a while. And it also seems to depend on the region the user is using the app.
We'll search a workaround and create an option which can be turned on if needed.

If you want you can open a ticket for the "latest post date" problem and give us more details about it. Currently I'm not aware of a such a problem, but maybe they changed something again.

[thomas694]

Please try the new test version and tell me if it works better.

[MSWallack]

Good news and bad news. The good news is that the app seems to stay authenticated for longer ... usually. I was able to download some very large blogs without the app stopping with a 401 error midway and I was able to download a group of very small blogs. However, when downloading some medium-sized blogs, sometimes the app will work just fine and stay authenticated throughout the process and allow me to continue downloading another blog or three, but other times, it will throw a 401 error at random times and quit. One that I tried last night, immediately after authenticating, generated a 401 error (but kept downloading) after about 2 minutes.

[Someonemustnothavethis]

I fired up the latest T3 yesterday for my once-a-month download. It ran flawlessly all day (16 hours?) and left 68 GB and 127,000 pictures on my drive without asking for a login.
I had a 'serialization' error but I've had that several months already and will ask in another thread.

[Someonemustnothavethis]

I did a mid-month crawl of some new blogs yesterday and T3 ran like an eager little puppy all afternoon downloading pictures. Not once did it ask for a log-in. Speed seemed to be good too.
I wonder if Tumblr changed something.

[Someonemustnothavethis]

Can we consider this solved/fixed?
I did my once-a-month update a couple of days ago and the latest version again ran like a champ. It started logged in, downloaded 83 GB (143,000 files) of kittens, puppies and fluffy bunny pictures and videos, without skipping a beat or asking to be logged in.

[MSWallack]

Not for me. I still get logged out regularly (or, I suppose, irregularly as I have yet to figure out any sort or correlation between activity and being logged out; over the weekend, I was logged out twice within a minute of authenticating).

[jjdnl]

It did work like a charm for a few months, but then on July 1st out of the blue the issue was back. First time i actually had to logon again, after that just clicking on authenticate is enough.

[Hrxn]

[..], downloaded 83 GB (143,000 files) of kittens, puppies and fluffy bunny pictures and videos, [..].

Sure 😄

[MSWallack]

The error is ongoing and somewhat random. Sometimes, the app will work for an extended period. Other times, it will think I'm logged out within minutes. I guess that I've come to live with it... However, it still creates a bit of a headache in that the app continues to try to process blogs (unsuccessfully because I'm "logged out") and then marks them as complete. Would it be possible to change the functionality of the app to stop processing immediately upon receiving the error?

[thomas694]

It wasn't ready on the last release, but now.
Please use the newest version and let us know if you see any improvements.

[MSWallack]

Downloaded and installed v2.11.2, deleted .json and the online folder (sorry, can't recall the folder name offhand...), and restarted the computer.
Good news: The app does seem to stop marking blogs as completed once it starts throwing authentication errors.
Bad news: No change (was any expected?) on the frequency of the app losing the authentication status (and I haven't seen any success of the app being able to re-authenticate on its own).

[thomas694]

There should be a change, not necessarily on the frequency, as we only apply a workaround after the error occurred, but the possibility to automatically continue downloading.
I've seen this error myself, but very rarely and when I want to investigate, it already disappeared. On the other hand, you seem to have it quite often and regularly.

[MSWallack]

Lucky me...

[Someonemustnothavethis]

The latest version ran perfectly yesterday (Oct 1, 2023) for me. It took 9 hours to scour the 26-something blogs and ran fine without asking for a log-in. Either way, requiring log-ins or not, this is still a brilliant tool that beats the heck out of trying to keep up with lots of blogs manually or via Tumblr's ridiculous interface.
By the way, I read an interview with the CEO of Tumblr and he says that they lose 30 million a year. I'm not counting on it being around for the long term.
https://techcrunch.com/2023/07/12/tumblr-is-losing-30m-each-year-ceo-says/

[MSWallack]

Sadly, for me, the most that I can expect is that I won't be logged out until the second blog is complete. Usually, I have to authenticate again after each blog is completed (unless the blog has very, very few posts).