Friday, January 4, 2008

Screen scraping and DRM

I've been reading with great interest about the screen scraping incident with Robert Scoble and Facebook. Of most interest was this note from Facebook telling him that they had decided to let him back in.

Our standard process for handling cases when an account is disabled for security violations is to allow a user to appeal and remedy the situation. This is the process we have followed here. Since you contacted us and have agreed not to run the script again, we have reactivated your account. You should now be able to log in with your normal email and password. In the future, please refrain from running these types of scripts again

That's an interesting policy, as I'm thinking screen scraping could be big in the future. I spent a good bit of my holiday playing with Dapper and Yahoo Pipes scraping sites, and creating Atom feeds where the sites didn't have one. Did I ask permission before scraping the sites? Heck no! Did I violate someone's copyright? Got me, I never bothered to look at the terms of use.

I don't believe for a minute that Facebook didn't know what Robert was running against them. I also think that Facebook is smart enough to know the difference between a white-hat and black-hat bot. Facebook was sending an RIAA type of message through Robert, and that has to make you wonder about their DNA. (Not that we didn't already know.)

Personal data gathering bots are a part of our future. Technologies like Dapper and Pipes make this possible. The scripts will be shared like baseball cards, and we all will be running them. Scoble was just a little ahead of the masses, but the rest of us are not far behind. Web sites that resist people's ability to run these scripts will be going the same way as the record companies. Facebook just can't seem to get much right when it comes to respecting their users.


Anonymous said...

Screen scraping is remarkably old news, just like feeds. All of this first hit the light of day in the late 90's, early 2000's, promptly to be crushed for a time by content owners' desires to limit access.

Fast forward to 2008. Screen scraping is now easier than ever, and content owners seem to be rushing to monetize their content in open advertising driven models. But, the old tensions remain. Content owners want you to see content in venues where they can monetize it.

Facebook is an interesting case. They don't really own the data. Imagine if you could not use your birth date because you entered it into Facebook. They do seem to want to own access to a particular web representation of your data.

I don't think Facebook is going to change just because the trends are running against them. All businesses must monetize, and they are locked into indirectly monetizing access through advertising. Why not just charge Scoble directly for this kind of access?

Anonymous said...

Great comments Bud! Thank you.

Monetize scraping? That's an interesting idea. I'll have to give that some more thought.

Jim Russell said...

I'm glad to see someone else make the connection between the screen-scraping issue and DRM. I wrote my own post on the topic here.

Bud's ideas have merit. Just like with music and video, you have to build your business model with the understanding that technology is not going to save you.

Anonymous said...

Oooh, that is good stuff Jim. Thank you for sharing. I'm going to elevate it to a post so that more people will see it.


franklin said...

I have perused your article, it is exceptionally enlightening and helpful for me.I respect the profitable information you offer in your articles. Thanks for posting it. Black hat forum