Disqus - Latest Comments for btaylor

Re: Social Cookbook: an open source Open Graph app - Bret Taylor's blog

Bret Taylor — Wed, 28 Sep 2011 18:34:13 -0000

This is just because we are in the Developer Beta period of Timeline. All of this will work once Timeline rolls out more broadly in a couple of weeks.

Re: Social Cookbook: an open source Open Graph app - Bret Taylor's blog

Bret Taylor — Wed, 28 Sep 2011 18:34:04 -0000

This is just because we are in the Developer Beta period of Timeline. All of this will work once Timeline rolls out more broadly in a couple of weeks.

Re: Social Cookbook: an open source Open Graph app - Bret Taylor's blog

Bret Taylor — Wed, 28 Sep 2011 03:15:15 -0000

Yah, this is just because of the limited rollout.

Re: Web Sockets in Tornado - Bret Taylor's blog

Bret Taylor — Sun, 07 Feb 2010 15:54:59 -0000

I observed this issue as well, and it appears to be a bug in the kqueue implementation in ioloop.py that was recently submitted. For a quick workaround, delete lines 419 - 421 in tornado/ioloop.py. I am diagnosing the issue now and hope to have a "real" patch soon.

Re: crad on tumblr - Web Application Development with Tornado

Bret Taylor — Sun, 24 Jan 2010 17:17:15 -0000

Thanks for the great writeup!

Re: Web Sockets in Tornado - Bret Taylor's blog

Bret Taylor — Thu, 31 Dec 2009 16:53:30 -0000

What browser are you using? You may need the Developer Channel of Chrome for Web Socket support (http://dev.chromium.org/get....

Re: OAuth WRAP support in FriendFeed for feedback - Bret Taylor's blog

Bret Taylor — Tue, 22 Dec 2009 13:38:19 -0000

It is a custom blog written with Tornado (http://www.tornadoweb.org/) that runs on Google AppEngine. Source code is available at http://github.com/finiteloo....

Re: test - David Krisch's blog

Bret Taylor — Sat, 19 Sep 2009 15:59:58 -0000

Mind updating your disqus ID? I am getting emails for all these comments because they are still associated with the "bretttaylor" disqus ID from my blog template that you copied.

Re: Experimenting with Google App Engine - Bret Taylor's blog

Bret Taylor — Sat, 12 Sep 2009 17:41:10 -0000

The source code for this blog is now checked in at http://github.com/finiteloo... now that we have open sourced Tornado (http://bret.appspot.com/ent....

Re: The technology behind Tornado, FriendFeed's web server - Bret Taylor's blog

Bret Taylor — Sat, 12 Sep 2009 12:07:10 -0000

The asynchronous features are not available in Google AppEngine, but a subset of Tornado's features do work in AppEngine. See http://www.tornadoweb.org/d... and the included "appengine" example for a full-featured App Engine example app.

Re: The technology behind Tornado, FriendFeed's web server - Bret Taylor's blog

Bret Taylor — Thu, 10 Sep 2009 17:42:25 -0000

I can't imagine there is much of a performance difference. The bottom is not that complex in my opinion. See ioloop.py and iostream.py for the details.

Re: The technology behind Tornado, FriendFeed's web server - Bret Taylor's blog

Bret Taylor — Thu, 10 Sep 2009 13:50:36 -0000

When we started, we did use Twisted. In practice, I found Twisted tedious. The deferred abstraction works, but I didn't love it in practice. Likewise, the HTTP/web support in Twisted is very chaotic (see http://twistedmatrix.com/tr... - even they acknowledge this). In general, it seems like Twisted is full of demo-quality stuff, but most of the protocols have tons of bugs.

Given all those factors, it didn't seem to provide a lot of value. Our core I/O loop is actually pretty small and simple, and I think resulted in fewer bugs than would have come up if we had used Twisted.

Re: How FriendFeed uses MySQL to store schema-less data - Bret Taylor's blog

Bret Taylor — Sun, 16 Aug 2009 14:13:15 -0000

Replicate away - that's why we published it. FYI, the entire site is licensed under Creative Commons (http://creativecommons.org/..., and all the source code under Apache 2. I am not sure which type of license is appropriate for a description of a scheme like this, so if you want an official license, you can consider it licensed under both and choose whichever seems most appropriate.

Re: How FriendFeed uses MySQL to store schema-less data - Bret Taylor's blog

Bret Taylor — Mon, 25 May 2009 14:30:59 -0000

Indexes are stored as btrees on disk, so you can iterate over them both backwards and forwards equally efficiently.

Re: Why I Didn’t Get to See FriendFeed’s Redesign

Bret Taylor — Mon, 06 Apr 2009 13:32:20 -0000

So sorry. We were pretty swamped finishing things this weekend, and am extremely sorry we didn't connect. We will make ourselves more accessible in the future. We appreciate your being so candid, and sorry again for the miscommunication on our end.

Re: How FriendFeed uses MySQL to store schema-less data - Bret Taylor's blog

Bret Taylor — Sun, 08 Mar 2009 14:52:57 -0000

We do include multiple columns in the same table. We treat them just like we would treat keys on other tables. Here is one of our standard many-to-one indexes, which indexes two properties (stream_ids[] and sort_time):

CREATE TABLE index_stream_id_sort_time (
stream_id BINARY(16) NOT NULL,
sort_time INT NOT NULL,
entry_id BINARY(16) NOT NULL,
PRIMARY KEY (stream_id, sort_time, entry_id),
UNIQUE KEY (entry_id, stream_id)
) ENGINE=InnoDB;

We don't use any SQL joins in our current system.

Re: How FriendFeed uses MySQL to store schema-less data - Bret Taylor's blog

Bret Taylor — Sat, 07 Mar 2009 01:25:25 -0000

We do compression in the client as a part of the serialization process.

Re: How FriendFeed uses MySQL to store schema-less data - Bret Taylor's blog

Bret Taylor — Sat, 28 Feb 2009 18:36:30 -0000

We are definitely open to open sourcing it. It is not quite abstracted enough, and it is a bit intertwined with our code's FriendFeed-isms, but Ben Golub and I talked about it a bit recently, and I am committed to make it happen at some point.

Re: How FriendFeed uses MySQL to store schema-less data - Bret Taylor's blog

Bret Taylor — Sat, 28 Feb 2009 15:21:24 -0000

Yes, we explicitly were not trying to achieve ACID semantics. See http://bret.appspot.com/ent... for a more detailed explanation.

Re: How FriendFeed uses MySQL to store schema-less data - Bret Taylor's blog

Bret Taylor — Sat, 28 Feb 2009 15:19:24 -0000

1. We are actually ok with false negatives. As I said, the "Cleaner" runs continuously, so that false negatives are fixed within a couple of seconds. From a UI perspective, the user wouldn't see an entry in their feed for a couple of seconds if something crashes until the cleaner runs. For our system, that experience is acceptable (and the loosened constraints mean we can do a bunch of write optimizations that we wouldn't otherwise be able to do).

2. We have multiple databases, so the auto_increment can't work as a global ID. Furthermore, for practical reasons, we want to know the ID before we insert into the database. We use standard UUIDs for this purpose.

3. The choice of MEDIUMBLOB was simply based on our current needs.

4. We run many more than two instances of MySQL on a variety of different hosts. The example above is extremely simplified for the purposes of this blog post. We have run multiple instances on separate disks on the same machine. As long as they have enough memory so you are not going to disk for reads, this method works to a point. In particular, it helps if you want to, e.g., make more shards than you actually need right now so you can accommodate future growth. Put a couple of shards on each machine at first, then expand to 2X the number of machines when growth warrants it.

Re: How FriendFeed uses MySQL to store schema-less data - Bret Taylor's blog

Bret Taylor — Sat, 28 Feb 2009 00:46:16 -0000

250 million entries. We store many more entities (the biggest other chunk of entities are comments). We aren't giving out our user numbers at this point, sorry.

Re: How FriendFeed uses MySQL to store schema-less data - Bret Taylor's blog

Bret Taylor — Fri, 27 Feb 2009 20:03:27 -0000

We create new indexes in parallel by running one "cleaner" per shard. We don't want to parallelize much more than that because we don't want to disrupt other processes that are already using the DB. We don't use hadoop. It turns out it is so easy to parallelize, we haven't needed any specialized infrastructure to do so - we just parallelize by splitting into entry ID groups.

Re: How FriendFeed uses MySQL to store schema-less data - Bret Taylor's blog

Bret Taylor — Fri, 27 Feb 2009 15:09:23 -0000

We do shard our indexes. We query all the relevant index shards in parallel and over-fetch. The indexes are stored in sort order, so sorting is not an issue. To paginate, we fetch start + num and truncate in Python.

Re: How FriendFeed uses MySQL to store schema-less data - Bret Taylor's blog

Bret Taylor — Fri, 27 Feb 2009 13:23:08 -0000

Yes, we are definitely going to check that out. That was the one project that seemed most promising in our evaluation, and we still plan on doing a full evaluation.

Re: How FriendFeed uses MySQL to store schema-less data - Bret Taylor's blog

Bret Taylor — Fri, 27 Feb 2009 13:22:50 -0000

We use the Google Chart API: http://code.google.com/apis...