As part of my ongoing initiative to always show up to internet arguments with receipts, I built a bespoke Rube Goldberg machine that can automatically download and archive tweets I like, turning them into Markdown files, complete with attached Twitter images and video downloaded.
Once I really get all the kinks worked out I promise I’ll write about it in full, but needless to say, it’s a fun system. I’m also in the habit of meticulously putting tags on the files for every liked tweet for easier lookup later, and it’s making me realize that I like a ton of tweets (over 500 tweets just this month alone, and the month is barely half over!).
Recently, I started getting the suspicion that some of my favorited tweets weren’t ending up in the archive at all. Eventually I caught my system in the act. I found a tweet from Andrew Cuomo that was hitting differently more recently (this one, if you’re wondering). I liked the tweet (well, I didn’t like it, but you know what I mean). But a Markdown file of the tweet never showed up.
As I mentioned, this whole system is a Rube Goldberg machine. A liked tweet starts its journey with an IFTTT recipe. That recipe is set up so that when I like a tweet, IFTTT takes that tweet’s URL and appends it to a text file in my Dropbox. I have a Hazel script that watches for changes to that file which will then pass the file into a Rake task that grabs the tweet URLs in that file, imports them, then deletes the file and marks the URLs as processed.
I checked my IFTTT logs, and there’s no sign of the recipe having fired for this particular tweet.
I contacted support, and they were baffled themselves, because they could like that tweet and there were no problems for them.
I noticed the age of this particular tweet, and started testing this out with other older tweets. None of them worked, but relatively new tweets were fine.
I asked IFTTT support if they had logs of any webhooks or other events coming in from Twitter. They responded that they aren’t doing anything event-based, but rather they are just polling Twitter.
And that was my “aha” moment.
You see, I have more Twitter likes than your average bear. I’ve been liking tweets for over a decade now, and I have accumulated tens of thousands of them.
When IFTTT polls Twitter, they’re just sending a request to Twitter saying “Twitter, give me Aaron’s likes.” Twitter, in their infinite wisdom designing this particular API, will give IFTTT back those liked tweets in reverse chronological order. But the tweets aren’t sorted by when I liked them; they’re sorted by the tweets’ own timestamps. In other words, if I have thousands of liked tweets and I like a tweet from five years ago and I ask Twitter for my most recent likes, Twitter will give me the most recently tweeted tweets that I liked, leaving the older tweet buried in the list.
IFTTT probably never anticipated people liking as many tweets as I do, and I’m sure they never ran into this issue when they were testing this feature. For a normal user who hasn’t liked a ton of tweets, the trigger works correctly; aka it will always fire when the user likes a tweet, because the number of liked tweets is reasonable and Twitter can send the whole list back. But when they poll my list of Twitter likes, they’re just getting the ones for the most recent tweets (probably the last 100 or so). So if I like a tweet from five years ago, even though I just liked it, IFTTT’s polling will never see it because they’re not paging through the hundreds of pages of my favorites every time they poll Twitter.
In other words, for me, this trigger is incapable of doing what it says it does, and I don’t think there’s an easy way for IFTTT to fix it because I don’t think Twitter actually offers an API that will notify IFTTT on newly liked tweets as they come in; it’s just a polling system.
However, not all hope is lost!
My little bespoke Ruby script doesn’t care whether I liked a tweet or not, it just takes a tweet’s URL and it dutifully downloads it. So, I built two simple solutions for myself:
For iOS, I made a Shortcuts app shortcut that is accessible from the iOS share sheet. When I pass it a URL, it will write a file to the same Dropbox folder as my IFTTT action and the file will contain the URL of the tweet. My Hazel script takes it from there.
For my Mac, I don’t have the Shortcuts app yet, but never fear; I built a simple Alfred workflow that runs a Ruby script that does the same thing.
The iOS shortcut required no code but interestingly enough it was more cumbersome to put together and required more trial and error; the Ruby script was kind of a one-and-done situation. Of course, I write Ruby for my day job, so it damn well better be easy for me to do that.
That’s all well and good from here on out, but what about all those previous old tweets I liked that are missing? Well, not to worry; one of these days I’ll download another archive of my Twitter account and run through a script to re-import what’s missing.
IFTTT’s trigger behavior here is an example of what we call a leaky abstraction. I shouldn’t care about how it’s built; if it worked perfectly I would have been able to count on it to always do what it says: fire when I like a new tweet. But because of how it was built, it doesn’t do that, and when I saw this bad behavior, the reality leaked through.
Leave a Reply