A couple of days ago, I had lunch with Ben Laurie and Ben Hyde. Great discussion about identity infrastructures and standards bodies.
So I just read BenL’s post about the BenL+BenH idea of load-balancing a distributed hash table using logarithmic load hand-off… fantastic. The only downside I see is that this inherently increases the number of queries to the DHT, regardless of how popular the file is. Would it make more sense to have a short request to the main host first, which can respond with a 500-style error. Of course, this assumes the host is not so swamped that it can’t even respond to such pings. In the end, the right answer probably depends on the distribution curve of file popularity.
Benlog 