Adam Loving's Blog — Seattle Social Web Development

Changes to FriendFeed Python Library for use with App Engine

by adam on July 22, 2008

UPDATE: you should definitely go with Benjamin Golub’s Python FriendFeed library for App Engine (fftogo). It is cleaner than mine and includes some undocumented API calls (like fetch_entry).

I had to make some minor modifications to the FriendFeed python library to get it to work with Google App Engine. App Engine does not support out urllib2, and has its own urlfetch library.

Here is the cut and paste-able code (see also pastie, I’m having trouble with code formatting in Wordpress)

# original (FriendFeed.py 0.9)
def _fetch(self, uri, post_args, **url_args):
    url_args["format"] = "json"
    args = urllib.urlencode(url_args)
    url = "http://friendfeed.com" + uri + "?" + args
    if post_args is not None:
        request = urllib2.Request(url, urllib.urlencode(post_args))
    else:
        request = urllib2.Request(url)
    if self.auth_nickname and self.auth_key:
        pair = "%s:%s" % (self.auth_nickname, self.auth_key)
        token = base64.b64encode(pair)
        request.add_header("Authorization", "Basic %s" % token)
    stream = urllib2.urlopen(request)
    data = stream.read()
    stream.close()
    return parse_json(data)

# modified
def _fetch(self, uri, post_args, **url_args):
    url_args["format"] = "json"
    args = urllib.urlencode(url_args)
    url = "http://friendfeed.com" + uri + "?" + args
    headers = {}
    if post_args is not None:
        post_data = urllib.urlencode(post_args)
        method = "POST"
        headers={'Content-Type': 'application/x-www-form-urlencoded'}
    else:
        post_data = None
        method = "GET"

    if self.auth_nickname and self.auth_key:
        pair = "%s:%s" % (self.auth_nickname, self.auth_key)
        token = base64.b64encode(pair)
        headers["Authorization"] = "Basic %s" % token

    result = urlfetch.fetch(url=url, payload=post_data, method=method, headers=headers)
    data = result.content

    self.result = result
    self.feed_json = data

    return parse_json(data)

{ 0 comments }

Special character hacks for Gmail addresses

by adam on July 22, 2008

Image via Wikipedia

Yesterday, my co-worker Matt told me about some Gmail tricks I hadn’t heard of. Well, one is a trick, and one is just “nice to know”.

Gmail will ignore (some, all?) punctuation in email addresses. So bob.smith@gmail.com is the same as bobsmith@gmail.com. (That’s the nice to know part).

Furthermore, text after a plus sign is ingored. So, when you sign up for a new service, you can use a unique email address. You could sign up for amazon.com using bobsmith+amazon.com. All your mail will be routed to the same place, but you can block certain services and track who gives your email to spammers.

{ 0 comments }

Addicted to FriendFeed? Try StumbleRead.

by adam on July 22, 2008

Image via Wikipedia

I got hooked on FriendFeed and wanted to create a different Web based interface. I wanted to be able to scan through several posts, open their links automatically, comment and “like” quickly, and have the list auto-update to show new posts. For some reason, I haven’t found a desktop client that I like (I tried Twhirl and AlertThingy). The clients are a pretty efficient way to comment (if you can find the right button), but I find notifications too disruptive, and neither client auto-opens links.

So, I created StumbleRead. Give it a try, and let me know what you think by commenting on this post (or below). Also, make sure you catch the hotkeys.

Disclaimer: Works best in Firefox, lightly tested in IE 7 and Safari.

{ 1 comment }

StumbleRead hotkeys

by adam on July 22, 2008

Here are the shortcut keys for StumbleRead, any suggestions for others?

P, ←, ↑	Previous
N, →, ↓	Next
T	Top
B	Bottom (also loads next page)
S	Share
C	Comment
L	Like

I was thinking I may need to create left hand versions of Next and Previous (so righties don’t have to take their hand off the mouse).

{ 3 comments }

Ruby boolean operator (or ruby parsing) bug

by adam on July 9, 2008

This has bitten me a couple times in the last few days. The Ruby “or” (or equals) operator appears to have a bug where, when used in an assignment, the first value is assigned rather than the results of the entire expression.

irb(main):001:0> x = false || true
=> true
irb(main):002:0> x
=> true
irb(main):003:0> z = false or true
=> true
irb(main):004:0> z
=> false

ruby version: ruby 1.8.6 (2007-03-13 patchlevel 0) [i686-darwin8.10.1]

I would expect z to be true in that last example.

{ 1 comment }

Google App Engine Evaluation

by adam on July 9, 2008

I spent about 4 or 5 days re-implementing one of my projects on Google App Engine. A basic version of the app is now live, but I’m waiting for the DNS to go through, so I won’t link to it just yet. Overall, I love it. This was my first exposure to Python and Django too – and I really like them.

Some quick pros and cons:

+ Python is really learn-able. I found it to be a easier learning curve than Ruby. Although, Ruby’s elegance will probably still win in the long term.

+ Local development server for App Engine is good. Behaves pretty much the same as the real thing.

+ Deployment easy (still need to figure out how best to do automated testing for Django/Python)

- Bulk load slow both locally and live. Only gripe is the speed of the local simulated datastore (I think it is file system based).

- Had some trouble with Python paths. Just beginner issues.

- Tried both XCode, TextMate, and Aptana (with Pydev) IDEs. Aptana wins because it requires less clicks to get to your file, and also has Javascript and HTML formatting. Pydev auto-complete is so-so, don’t know why you can’t navigate to a library method (maybe I don’t know the hot-key?)

- Bulk upload samples from Google were good learning material, but it probably would have been more time effective to just upload data by doing HTTP Posts to my app.

- No support for bulk delete. Work around is to write a page that deletes as many rows as it can without hitting your CPU/page response time quotas. Even better work around is to modify the “limit” parameter in the query string in the web based data editor so you can manually delete a couple hundred entity rows at a time.

- Not clear what the steps should be for performing a schema change. If you make one to your models, code just runs until you try and load data that doesn’t fit the new types. No way to transform existing rows without loading them all up into a page and saving them. This is going to make future revisions of an app difficult.

- Apparently no option for enforcing referential integrity (understandable given the goal of scale)

The restrictions on GQL and the App Engine data store count as both a benefit and a limitation. Even in my simple app, I was forced to denormalize and add more columns than I would’ve liked to satisfy the requirement for an index to match every query. My app is mostly read-only right now, adding more write logic will require updating these denormalized columns. As I gain more experience, and GQL gets richer, I don’t think this will be a problem compared with the huge scaling benefit. The App Engine “sandbox” just forces you to do your optimization work up front.

So what kind of apps will work well on App Engine?

Given its infancy, it is hard to see very data-intensive apps working on App Engine. It does look like the perfect environment for hosting mashups and Facebook/OpenSocial apps. Also, I expect a thriving ecosystem of simple open source App Engine components (a blog, a wiki, etc.) that can stiched together.

{ 1 comment }

It’s the Beginning of the End for the TV Biz, Says Analyst

by adam on July 8, 2008

It’s the Beginning of the End for the TV Biz, Says Analyst | Epicenter from Wired.com

“We believe the feature film and TV content businesses are on the verge of structural changes that appear to impact the core revenue and profits of entertainment business models,” wrote DiClemente.

Viva la revolution! I just discovered Strike.tv last week. Just show me a way to watch Hulu and YouTube on my TV and I’ll be happy. Oh, and better quality re-streamed Tour de France coverage.

{ 1 comment }

Web Design Reference

by adam on July 4, 2008

David over at Design Commission captured a sampling of 92 home pages. It is a great compendium of the state of the art in Web Design.

Start Screens – a set on Flickr

{ 1 comment }

The Long Tail is a Lonely Place

by adam on July 3, 2008

Image via Wikipedia

This is interesting, and certainly makes sense in the Facebook application world. Seems to me the Long Tail is already a prevalent concept in music though.

Prof. Elberse describes research showing that even in our cultural consumption we tend to be intensely social folks. We like experiencing the same things that other people are experiencing — and the mere fact that other people are experiencing and liking something makes us like it even more. Far from being cultural rugged individualists, most of us are only too happy to have others suggest to us what we’d like.

Portals – WSJ.com

Earlier this week I read about Zembly, a Sun “social” development project. One of the critiques against Zembly, is that it targets obscure app developers. I don’t see that as a problem for a development tool. Outfitters want to sell as many pick-axes as possible to gold searchers as possible?

{ 0 comments }

Gnip aggregates social services

by adam on July 2, 2008

The train shed of Grand Central Station.

Image via Wikipedia

Gnip: Grand Central Station for the Social Web – ReadWriteWeb

This is an interesting service making some really big claims architecturally. In fact the more I think about it, the more impossible it seems. Gnip is trying to become the hub for some really interactive sites. Piping all that data through their service and keeping up with all the changing APIs seems like a nightmare. Not to mention securing the requests and managing session keys…

I did have a related idea a few years ago about creating a unified (mostly read-only) interface to (non-personal) data provided by a multitude Web services. I will revisit that idea now that there the number of available datasources has multiplied 10 (or 100) fold.