How not to Monetize RSS and Atom Feeds

buy_more_tech.jpg

An RSS or Atom feed is a live document of recent events or items. For example, there are feeds of Twitter tweets, Facebook posts, and YouTube videos. Usually, other programs track the feeds, and then let you know when there's something new.

You can do other things with feeds, too. I've collated a few of my personal feeds into a Lifestream that updates automatically. It's a very convenient hands-off diary.

Companies like Twitter and Facebook have been trying to figure out if they should monetize feeds. They already inject ads into their main user interfaces (into Facebook's users' walls, and onto the Twitter timeline, for example), but what should they do about the feeds for the same information? So far, they've left feeds alone.

This week, Delicious tried injecting ads into its users' bookmarks feeds. The bookmarks feeds are activity feeds, because they reflect actions that the users have taken. On Delicious, users bookmark sites that they find noteworthy. So these new ads are clearly marked as Sponsored items, because otherwise it would look like everyone suddenly explicitly bookmarked the ad's product. That'd be straight up deception.

Still, there are problems with the way that Delicious is embedding ads into their users' activity feeds. To ensure that the ads are at the top of the list, they're always coded with a pubDate ("Published Date") of the current time. This goes against the original intent of these feeds, where each item's pubDate doesn't change if there's no actual change to the item itself. Old ads should get old, too.

Instead what happens is that every time the feed is fetched, since there's always a "new" item (the same old ad, but with a new, just-now pubDate), is that Delicious can no longer return HTTP code 304, which means, "I'm not providing the feed again, it hasn't changed since the last time you got it."

This is causing Delicious to be slammed with having to deliver full feeds for every request, and for all the clients to have to process what look like brand-new feeds. Beyond that, people who auto-tweet what they bookmark are all inadvertently tweeting ads now.

How did I discover this in the first place? My Lifestream started journalling that I've been repeatedly bookmarking the same ad over and over again.

That's not the way to monetize activity feeds.

February 20, 2016 | 0 Comments

Reports of my Death Greatly Exaggerated

dilapidated_gears.jpg

Last week, there was an insinuation that I had died. In a way, I made the insinuation myself. It was produced by a five year old script that runs continuously on its own. ("But wait, that linked blog post isn't five years old!" you might say. Actually, the script had been cooking for a couple of years before I wrote the post.)

So what happened? The script couldn't detect any online activity from me for three days, and so sent out preliminary notification that it was worried about me. This worked perfectly. … Well, perfectly for technology that'd been hibernating for five years (which is more like fifty years in Internet time) and tried to talk with entities in the new world into which it emerged.

What happened to my script, waking up to a changed world, reminds me of the home automation in There Will Come Soft Rains. (It's a great little story. Go ahead and find it and read it. I'll wait.)

The travails of my little Rip Van Winkle Dead Man's Switch presents us with an opportunity. Let's recount what really happens when you write a web app that's supposed to survive on its own for an indefinite number of years. There are a lot of moving parts, let's see how they tie together.

In 2008, WordPress's dashboard had taken a turn for the worse. Michael Heilemann published a handful of screencasts perfectly illustrating the bad UX decisions WordPress had made and had become an Emeritus Member of the Habari Project Management Committee. Habari was an ascendant 2nd generation blogging platform that had learned from its predecessors. Its native remote API was the similarly ascendant AtomPub. WordPress also supported AtomPub.

In 2010, I wrote my Dead Man's Switch (DMS), and used AtomPub to have it publish to my two blogs (one Habari, one WordPress). This was just one of many foundational decisions I made based on the trajectory of technologies at the time. I ensured the DMS worked, and then left it to do its work: watch over me.

…

In 2015, it triggered. And couldn't do its job..

While it was idling, the world was changing. Here are some things that had changed:

Here are some things that remained the same, for better or for worse:

  • Amazon's acquired Shelfari site never added support for RSS. Once upon a time, Amazon was going to web-API everything. How would my DMS know when I've read a book?
  • Amazon's acquired IMDB never completed its official API, either.
  • Google+ created a read-only API. There's still no API to post messages, so my DMS wouldn't be able to post on my behalf..
  • DreamHost was still there, running my DMS, and it successfully woke up.
  • Sites like Facebook and Twitter were still there to watch and post to.
  • Python 2 still has some support and incremental updates did not break the DMS.

I can't say that watching these breaking changes occur is unexpected. But I think there's value in documenting a real use case of web software that's meant to last on its own for so long. With any luck, I'll be here in another five years to give another status update. If not, hopefully my DMS can fill you in for me.

Photo by Darkday/Drainrat / CC BY 2.0 alt alt

September 6, 2015 | 0 Comments

FollowMee along OpenPaths

axiraa-path.jpg

"Give. Me. The data."

That's what many Quantified Self adherents said back in 2011 when Apple admitted that they accidentally had their phone save a database of geopositional data over recent past months. Apple promised to delete most of the database in the next update.

That bug didn't sound bad to me, it sounded awesome! I grabbed that database as quickly as I could, and looked for a service to continue the geoposition scraping so I could implement a location predictor.

The service OpenPaths appeared quickly on the scene. It's a project from NYTLabs. It's free, and a major benefit is that:

Users can securely store and manage their personal location data, and grant researchers access to portions of that data as they choose.

That's quite the magnanimous service! Neither the users who run the app nor the researchers who gather the data are paying anything for the service. We're users, but we're not paying customers. The NYTLabs must be running it out of the goodness of their own hearts, and the users and researched are indebted to them.

The downside became apparent pretty quickly. As there are no paying customers, the service doesn't get a lot of monitoring or attention. API calls can take a minute or more. Sometimes they fail, and sometimes the service simply goes down for days. The only appeal we can make as users are emotional appeals. We have no financial leverage.

Given that I'm not the customer of OpenPaths, but my data is the product, I went searching for a redundant service where I am the customer. I came across FollowMee, another app with the same feature set. It's a paid service, and the developer is responsive on their bulletin board. I've installed it as a backup and potential replacement for OpenPaths.

There's a saying that generally holds true of online services:

If you're not the customer, you're the product.

That's why I'm running both the free service OpenPaths, to share my geopositional data with researchers, and the paid service FollowMee, for some assurance the service will work while I pay for it. I'll continue to run FollowMee along OpenPaths while they both do their respective jobs.

Photo by Axiraa / CC BY-NC-ND 2.0

June 16, 2015 | Filed under | 0 Comments

Ways to Break a Dollar into Change

ian_britton_coins.jpg

I'm going to retire a technical interview question that I've had ready to ask, but turns out to probably be too difficult. It's more a wizard-level interview question, or a casual technical discussion with peers who don't feel like they're under the gun.

How many ways are there to break a U.S. dollar into coins?

It seemed appropriate to ask this to some candidates, because once you know what you're trying to do, the algorithm in Python naturally reduces down to five or six lines of code.

def ways_to_break(amount, coins):
    this_coin = coins.pop(0)

    # If that was the last coin, only one way to break it.
    if not coins:
        return 1

    # Sum the ways to break with the smaller coins.
    return sum([ways_to_break(amount - v, coins[:])
                for v in range(0, amount+1, this_coin)])

print ways_to_break(100, [100, 50, 25, 10, 5, 1])

You could click on the gist with more comments and better variable names if you want to try to better understand that. It's not meant to be production code, it's a whiteboard snippet made runnable.

Here's what I hoped to look for, from the candidate: After maybe clarifying what I wanted, ("what about the silver dollar or half-dollar coins? Did you want permutations or combinations?"), the problem should seem well-defined, but hard. Then, I'd hope the candidate would break it down to the most trivial cases: "How many ways are there to break 5¢ if you have pennies and nickels? How about 10¢ with pennies, nickels and dimes?" Can these trivial cases be used to compose the bigger problem?

At that point, different areas of expertise would appear. Googlers might start jumping towards MapReduce, and already figure that Reduce is simply the summation function because the question is "how many", and have the thing worked out at scale.

Imperative programmers, having thought about the simplest cases, probably clue in that there's an iterative approach and a recursive approach.

Pythonistas who prefer functional programming probably have reduce(lambda x, y: x+y [x, y in ...]) ready to go.

That got me thinking. I didn't see the need for reduce and operator.add and enum. My solution above isn't above criticism, but I think it's easier to read than some of the alternatives. In truth, it really depends on who that reader is, and how their experience has conditioned them.

While writing the algorithm, I searched the question online, and was pleasantly surprised to see Raymond Hettinger mentioned for solving the puzzle. He's a distinguished Python core developer. Too bad he probably didn't have Python then, in 2001, to help him solve the problem in a scant few lines.

Taking it up a Notch

Adam Nevraumont proposed the following challenge:

Drop the penny, and try to break $1.01. The Canadian penny has been retired, after all.

He provided the answer: Check that the remaining coin can break the amount. Don't assume it can. This also relieves the restriction that the container of denominations be ordered!

def ways_to_break(amount, coins):
    this_coin = coins.pop()

    # If that was the only coin, can it can break amount?
    if not coins:
        if amount % this_coin:
            return 0
        else:
            return 1

    # Sum the ways to break with the other coins.
    return sum([ways_to_break(amount - v, coins[:])
                for v in range(0, amount+1, this_coin)])

print ways_to_break(100, [1, 100, 10, 50, 25, 5])

(It's actually more efficient to pop the largest denominations first. But I wanted to show that you can use an unordered container now.)

Photo by Ian Britton / CC BY-NC 2.0

June 7, 2015 | Filed under , and | 0 Comments

The Smallest GitHub Fork

Not gonna bury the lede: I forked a project in GitHub to change a color in a colorscheme. That's basically a tiny one-word change.

I changed jobs a few months ago, and I have the longest commute yet. I far prefer working in the office, but I don't want to have to be in the office to resolve any little emergency. So I set up my environment to be able to work from home.

First, there's access to work's VPN. Absolutely no problem there. That's the first thing everybody needs to work from home.

Then, just in case I need to see my work computer's desktop, there's VNC. I got close, but no cigar.

vnc.jpg

I set up a server, and can connect via multiple clients, but they all have the same blitting problem when it comes to launching applications. No big deal, there's another way to skin that cat.

If I can't see my work computer's desktop, I can still run its GUI applications via X11 Forwarding. That's pretty good, but those apps always look a little weird on the client side.

Then there's the last resort: good old ssh with a text-based editor. And in my case, that's vim. Low-bandwidth, and gets most of the jobs done.

There was a problem even with that last resort. In the color scheme I use, sometimes words would become invisible when running vimdiff. It depends on the context coloring of the source files. In the following screenshot, there are words in the fuschia line that we can't see.

vim-desert-before.png

It's an easy-enough problem to fix. The normal thing to do is to find the configuration file, desert.vim, and change the offending color to not be fuschia. Then you're done, your problem is solved.

But I'm a developer, and to my mind, this stinks of a bug. And if it affects me, it could affect others, too. So the right thing to do is to fix the bug in a public fork, and possibly share the fix with a pull request.

I found the original repository, and made my fork. I fixed the bug. Text on the changed lines would now be visible. While I was in there, a couple of other changes wanted to be made. I made deletions muted red, and additions green. But without a screenshot, visitors wouldn't know how the change looks. So I made the screenshots.

vim-desert-after.png

(Both screenshots here look a bit garish. The screenshots don't reflect what a real vimdiff would look like. In practice the improved scheme works much better for me.)

Since I now had the screenshots, I updated the README file to show the effect of the change in place.

I committed my changes, and thought to myself, "isn't it interesting the lengths developers will go to, to fix little problems and document those fixes? And how a tiny fix can grow into something larger?" I oughtta blog about it.

February 7, 2015 | Filed under and | 2 Comments

Newer Entries » « Previous Entries