« JRS AWS | Main | Integer to int null pointer exception? »
Steve Williams from Digg pays a visit…
By Jeremy | February 25, 2009
Steve Williams, the second employee hired on at digg, came by our class today and gave a lecture. I really enjoyed the lecture. There were tons of new problems and thoughts that he introduced about running a big distributed system like digg. One thing I was really surprised about is that when your systems are as large as digg’s you run into Brewer’s Conjecture (PODC 2000).
What Brewer conjectured was that when you have a distributed system you want three main things from it: consistency, availability and partition-tolerance. The problem is that Brewer showed that it is impossible to achieve all three of these goals. Well, now what do you do? Basically, you throw consistency out the window…but not very far.
Gilbert and Lynch went on to show in their paper that if you didn’t mind waiting a little bit for consistency, you could achieve all three. That is what most high traffic websites are doing nowadays. They have to replicate their data from their main database so what choice do they have? Make the user wait? I don’t think so. You give them the best results you have knowing that eventually everything will become consistent.
One other lesson I learned that I felt was especially important was that we live in the real world. What I mean by this is that our applications don’t have to be exact unless it is specifically required. Does it really matter if we get an email 1 minute later than when it was sent or if we can’t see exactly how many pages of results a website has? No. Life isn’t perfect and we as people don’t really expect it to be. Why should we hold our computer programs to an unreachable standard? Mistakes need to be fixed when they are made but some things just don’t matter.
Topics: school | No Comments »
Comments
You must be logged in to post a comment.