managing photos and videos

This holiday, I finally spent time digging into how I manage photos and videos. With 2 young kids and some remote family and friends, this requires a good bit of thinking and planning. I know I’m not the only one, so I figured documenting where I landed might be useful to others.

I started with Dave Liggat’s Robust Photo Workflow, and found much of it resonates with my needs. Here’s where I landed:

  1. I take photos with a DSLR and two phones. My wife takes photos with her phone. We both take videos with our phones. We use Dropbox/Carousel auto-upload, which works just fine on both iOS and Android. For the DSLR, I manually load photos over USB.
  2. All photos and videos are now available on my desktop Mac (via USB or Dropbox). When I’m ready to review/edit photos, I drag and drop the batch into an all-photos/ directory I keep within my Dropbox.
  3. Hazel automatically categorizes photos and videos into subdirectories of the form 2015/01/. It’s really kind of awesome.
  4. all-photos and all-videos are thus simple date-classified folders of all of my photos and videos. They’re backed up locally using Time Machine. They’re backed up to the network using Dropbox. I can imagine eventually snapshotting this to Amazon S3/Glacier, but right now that doesn’t feel too urgent.
  5. I use Lightroom5 as an editor only, so if I blow away my Lightroom proprietary catalog info, it’s not that big a deal. To do this, I tell Lightroom to look at photos in all-photos without moving/copying them. After I’ve added a bunch of photos to the all-photos directory by drag-and-drop, I tell Lightroom to synchronize its catalog with the source folder, which takes a few seconds and gives me a screen with my latest imported photos and videos. I can then edit photos, reject them if they’re bad, and write back JPG/XMP data to each photo’s originating directory using Lightroom export. Dropbox backs those up automatically. To remove bad photos (blurry, etc.), I flag them as “rejected” in Lightroom using the X key, and when I’m done I command-delete, which gives me the option of removing the files from disk, too. I do this only for clear rejects, and it makes my mild OCD happy since I know I am not keeping totally useless files around, and the overhead of deleting photos is low. I could also delete photos easily using the Dropbox UI, which is pretty good, and then re-synchronize in Lightroom.
  6. I can then use Carousel (or Dropbox) on any mobile device to browse through all of my photos. It’s surprisingly good at dealing with large photo libraries (I have 20K) and large photos (I have a bunch of 13MP photos). As in, really, really good, even on a puny phone. Better than anything else I’ve seen.
  7. I’ve been using Flickr for years for private photo sharing, and Lightroom is really good at exporting to Flickr. That said, at this point I’m thinking of moving to Dropbox/Carousel based sharing. I can easily bundle photos & videos into albums on Dropbox, whereas videos are still limited on Flickr. Carousel conversations around a few photos are great with family. The only bummer is that Carousel and Dropbox have some mutually exclusive features: albums on Dropbox, conversations on Carousel. I suspect Dropbox will fix that in the next year.
  8. What I’d love to see:
    • unified photo features in Dropbox and Carousel
    • export Dropbox albums as directories of symlinks in my Dropbox folder, and export Carousel conversations in some other file-based way, too.
    • Lightroom export compatibility with Dropbox/Carousel albums.

I’m super happy with this new process: one funnel, easy, low overhead, and a very solid long-term photo storage solution. I’m only relying on RAW/JPG files and directories of said files to be readable for the long term, and that seems pretty safe. Lightroom is awesome, but I could replace it with a different tool if I needed to.

One more thing: if you’re going to use Dropbox to store all of your photos, make sure you pick a strong password and set up 2-factor authentication.

Power & Accountability

So there’s this hot new app called Secret. The app is really clever: it prompts you to share secrets, and it sends those secrets to your social circle. It doesn’t identify you directly to your friends. Instead, it tells readers that this secret was written by one of their friends without identifying which one. The popularity of the app appears to be off the charts, with significant venture-capital investment in a short period of time. There are amazing stories of people seeking out emotional support on Secret, and awful stories of bullying that have caused significant uproar. Secret has recently released features aimed at curbing bullying.

My sense is that the commentary to date is missing the mark. There’s talk of the danger of anonymous speech. Even the founders of Secret talk about their app like it’s anonymous speech:

“Anonymity is a really powerful thing, and with that power comes great responsibility. Figuring out these issues is the key to our long-term success, but it’s a hard, hard problem and we are doing the best we can.”

And this is certainly true: we’ve known for a while that anonymous speech can reveal the worst in people. But that’s not what we’re dealing with here. Posts on Secret are not anonymous. Posts on Secret are guaranteed to be authored by one of your friends. That guarantee is enabled and relayed by the Secret platform. That’s a very different beast than anonymity.

In general, if you seek good behavior, Power and Accountability need to be connected: the more Power you give someone, the more you hold them Accountable. Anonymity can be dangerous because it removes Accountability. That said, anonymity also removes some Power: if you’re not signing your name to your statement, it carries less weight. With Secret, Accountability is absent, just like with anonymous speech, but the power of identified speech remains in full force. That leads to amazing positive experiences: people can share thoughts of suicide with friends who can help, all under the cloak of group-anonymity that is both protecting and empowering. And it leads to disastrous power granted to bullies attacking their victims with the full force of speaking with authority – the bully is one of their friends! – while carrying zero accountability. That kind of power is likely to produce more bullies, too.

This is so much more potent that anonymity. And if this fascinating experiment is to do more good than harm, it will need to seriously push the envelope on systems for Accountability that are on par with the power Secret grants.

Here’s a free idea, straight out of crypto land. In cryptographic protocols that combine a need for good behavior with privacy/anonymity protections, there is often a trigger where bad behavior removes the anonymity shield. What if Secret revealed the identity of those users found to be in repeated violation of a code of good behavior? Would the threat of potential shame keep people in line, leaving the good uses intact while disincentivizing the destructive ones?

getting web sites to adopt a new identity system

My team at Mozilla works on Persona, an easy and secure web login solution. Persona delivers to web sites and apps just the right information for a meaningful login: an email address of the user’s choice. Persona is one of Mozilla’s first forays “up the stack” into web services.

Typically, at Mozilla, we improve the Web by way of Firefox, our major lever with hundreds of millions of users. Take asm.js, Firefox’s new awesome JavaScript optimization technology which lets you run 60-frame-per-seconds games in your web browser. It’s such a great thing that Chrome is fast-following. Of course, Chrome also innovates by deploying features first, and Firefox often fast-follows. Standardization ensues. The Web wins.

With Identity, we’ve taken a different approach: out of the gate, Persona works on all modern browsers, desktop and mobile, and some not-so-modern browsers like IE8 and Android 2.2 stock. We’re not simply building open specs for others to build against: we are putting in the time and effort to make Persona work everywhere. We even have iOS and Android native SDKs in the works.

Why would we do such a thing? Aren’t we helping to improve our competitors’ platforms instead of improving our own? That reasoning, though tempting, is misguided. Here’s why.

working on all modern platforms is table-stakes

We talk about Persona to Web developers all the time. We almost always get the following two questions:

  1. does this work in other browsers?
  2. does this work on mobile?

These questions are actually all-or-nothing: either Persona works on other browsers and on mobile, or, developers tell us, they won’t adopt it. To date, we have not found a single web site that would deploy a Firefox-only authentication system. Some web sites have adopted Persona, only to back out once they built an iOS app and couldn’t use Persona effectively (we’re actively fixing that.) So, grand theories aside, we’re targeting all platforms because web sites simply won’t adopt Persona otherwise. After all, Facebook Connect works everywhere.

When you think about it, is that actually so different from the asm.js strategy? asm.js is much faster on Firefox, but it works on Chrome and any other JavaScript engine, too. Heck, even Google’s DART, a brand new language they want to see browsers adopt, comes with a DART-to-JavaScript-compiler so it works on all other browsers out of the gate. These are not after-thoughts. These are not small investments. asm.js was designed as a proper subset of JavaScript. The DART-to-JS compiler is a freaking compiler, built just so non-Chrome browsers can run DART.

When appealing to web developers to make a significant investment — rewriting code, building against a new authentication system, .. —, cross-browser and cross-device functionality from day 1 is table-stakes. The alternative is not reduced adoption, it’s zero adoption.

priming users is the winning hand

The similarities between Identity and purely functional improvements like asm.js stop when it comes to users. The reason web sites choose Facebook Connect is not just because it works, but because 1 Billion users are primed with accounts and ready to log in. Same goes for Google+ and Twitter logins.

Persona doesn’t have a gigantic userbase to start from. That sucks. The good news is that, unlike other identity systems, we don’t want to create a huge silo’ed userbase. What we want is a protocol and a user-experience that make Web logins better. We want users to choose their identity. We’re happy to bridge to existing userbases to help them do just that!

So, bridging is what we’re doing. You’ve seen it already with Yahoo Identity Bridging in Persona Beta 2. More is coming. With each bridge, hundreds of millions of additional users are primed to log in with Persona. That’s powerful. And it’s a major reason why sites are adopting Persona.

Working everywhere is table-stakes. Priming users so they’re ready to log in with just a couple of clicks, that’s the winning hand.

beautiful native user-agent implementations sweetens the pot

Meanwhile, the Persona protocol is specifically tailored to be mediated by the user’s browser. Long-term, we think this will be a fantastic asset for the Persona login experience. Beautiful, device-specific UIs. Universal logout buttons. Innovation in trusted UIs. And lots of other tricks we haven’t even thought of yet. We’re doing just that kind of innovation on Firefox OS with a built-in trusted UI for Persona.

But let’s be clear: that’s not an adoption strategy. An optimized Firefox UI for Persona will not affect web-site adoption because it does nothing to reduce login friction. In a while, once Persona is widespread with hundreds of thousands of web sites supporting it, and users are actively logging in with Persona on many devices and browsers, Firefox’s optimized Persona UI will be a competitive advantage that other browsers will feel pressure to match. Until then, web site adoption is the only thing that matters.

now you know our priorities

Wherever it makes sense, we’re implementing Firefox-specific Persona UIs. However, when it comes to an adoption strategy, we know from our customers that this won’t help. What will help is:

  1. Persona working everywhere
  2. As many users as possible primed to log in

Those are our priorities.

We know this is different for Mozilla. But it’s quite common for folks implementing Services. What you’re seeing here is Mozilla adapting as it applies its strongly held principle of user sovereignty up the stack and into the cloud.

Firefox is the unlocked browser

Anil Dash is a man after my own heart in his latest post, The Case for User Agent Extremism. Please go read this awesome post:

One of my favorite aspects of the infrastructure of the web is that the way we refer to web browsers in a technical context: User Agents. Divorced from its geeky context, the simple phrase seems to be laden with social, even political, implications.

The idea captured in the phrase “user agent” is a powerful one, that this software we run on our computers or our phones acts with agency on behalf of us as users, doing our bidding and following our wishes. But as the web evolves, we’re in fundamental tension with that history and legacy, because the powerful companies that today exert overwhelming control over the web are going to try to make web browsers less an agent of users and more a user-driven agent of those corporations. This is especially true for Google Chrome, Microsoft Internet Explorer and Apple Safari, though Mozilla’s Firefox may be headed down this path as well.

So so right… except for the misinformed inclusion of Firefox in that list. Anil: Firefox is the User Agent you’re looking for. Here’s why.

user agency

Two years ago, I joined Mozilla because Mozillians are constantly working to strengthen the User Agent:

In a few days, I’ll be joining Mozilla.

[..]

[I want] to work on making the browser a true user agent working on behalf of the user. Mozilla folks are not only strongly aligned with that point of view, they’ve already done quite a bit to make it happen.

browser extensions

Like Anil, I believe browser add-ons/extensions/user-scripts are critical for user freedom, as I wrote more than two years ago, before I even joined Mozilla:

Browser extensions, or add-ons, can help address this issue [of user freedom]. They can modify the behavior of specific web sites by making the browser defend user control and privacy more aggressively: they can block ads, block flash, block cookies for certain domains, add extra links for convenience (i.e. direct links to Flickr’s original resolution), etc.. Browser extensions empower users to actively defend their freedom and privacy, to push back on the more egregious actions of certain web publishers.

mobile

Again, like Anil, I saw, in that same blog post, the threat of mobile:

Except in the mobile space. Think about the iPhone browser. Apple disallows web browsers other than Safari, and there is no way to create browser extensions for Safari mobile. When you use Safari on an iPhone, you are using a browser that behaves exactly like all other iPhone Safaris, without exception. And that means that, as web publishers discover improved ways to track you, you continue to lose privacy and control over your data as you surf the Web.

This situation is getting worse: the iPad has the same limitations as the iPhone. Technically, other browsers can be installed on Android, but for all intents and purposes, it seems the built-in browser is the dominant one. Simplified computing is the norm, with single isolated applications, never applications that can modify the behavior of other applications. Thus, no browser extensions, and only one way to surf the web.

so Firefox?

To Anil’s concerns:

  • Firefox Sync, which lets you share bookmarks, passwords, tabs, etc. across devices, is entirely open-source, including the server infrastructure, and if you don’t want Mozilla involved, you can change your Firefox settings to point to a Sync server of your choosing, including one you run on your own using our open-source code. PICL (Profile in the Cloud), the next-generation Sync that my team is working on, will make it even easier for you to choose your own PICL server. We offer a sane default so things work out of the box, but no required centralization, unlike other vendors.
  • Mozilla Persona, our Web Identity solution, works today on any major browser (not just Firefox), and is fully decentralized: you can choose any identity provider you want today. This stands in stark contrast to competing solutions that tie browsers to vendor-specific accounts. Persona is the identity solution that respects users.
  • Firefox for Android is the only major mobile browser that supports add-ons. Anil, if you want “cloud-to-butt”, you can have it on Firefox for Android. You can also have AdBlock Plus. Try that on any other mobile browser.

the unlocked browser

Anil argues that we should talk about unlocked browsers. I love it. Let’s do that. Here’s my bet, Anil: write down your criteria for the ideal unlocked browser. I bet you’ll find that Firefox, on desktop, on mobile, and in all of the services Mozilla is offering as attachments, is exactly what you’re looking for.

the Web is the Platform, and the User is the User

Mid-2007, I wrote two blog posts — get over it, the web is the platform and the web is the platform [part 2] that turned out to be quite right on one front, and so incredibly wrong on another.

Let’s start with where I was right:

Apps will be written using HTML and JavaScript. […] The Web is the Platform. The Web is the Platform. It’s going to start to sink in fast.

[…]

Imagine if there’s a way to have your web application say: “please go pick a contact from your address book, then post that contact’s information back to this URL”. Or if the web application can actually prepare a complete email on your behalf, image attachments included (oh the security issues….), and have you just confirm that, yes, you really want to send that email (the web app definitely can’t do that without confirmation)?

[…]

[We could] begin to define some JavaScript APIs, much like Google Gears for offline data storage, that enables this kind of private-public mashup. It would be fantastically interesting, because the security issues are mind boggling, but the resulting features are worth it. And it would spearhead some standards body to look into this issue more closely.

Whatever happens, though, the web is the platform. If you’re not writing apps in cross-browser-compliant HTML+JavaScript, the clock is ticking.

And in my followup post:

Add incremental features in JavaScript. First an offline functionality package, like Google Gears, so applications can work offline. Then, an interface to access the user’s stored photos. Over time, a way for web applications to communicate with one another.

[…]

Then there’s one tweak that could make a huge difference. Let a web application add itself to the dashboard.

Where did I go wrong? I thought this innovation was going to be unleashed by Apple with their introduction of the iPhone.

In my defense, if you read between the lines of the iPhone announcements back in 2007, it’s possible that Apple actually meant to do this. But then they didn’t, and they released an Objective C API, and a single closed app store, and locked down payments, and disallowed competition with their own apps, … So much for the Web.

It’s only fitting that the organization that is making this happen is my employer, Mozilla, with Firefox OS. Don’t get me wrong, I’m not taking credit for Firefox OS: there is a whole team of amazing leaders, engineers, product managers, product marketers, and generally rockstars making that happen. But it’s nice to see that this vision from six years ago is now reality.

So, the Web is the platform. HTML and JavaScript are the engines.

What about data? What about services? It’s time we redesign those. They, too, need to be freed from central points of control and silos. Data & Services need to be re-architected around the user. I should get to choose which organization I want to trust and which of my existing accounts I want to use to log into a new site/app/service. I should be able to access my data, know who else is touching it, and move it around as I please. I should be able to like/share any piece of content from any publisher I read onto any social network I choose. Amazing new apps should have easy access to any data the user wishes to give them, so that new ideas can emerge, powered by consenting users’ data, at the speed of the Web.

That, by the way, is the mission of my team, Mozilla Identity, and those are the guiding principles of our Persona login service and our upcoming project codenamed PICL. And of course we’ll continue to build those principles and those technologies into the Firefox OS phone (Persona is already in there.)

The Web is the Platform. And the User is the User. I’m quite sure Mozilla is the organization made to deliver both.

connect on your terms

I want to talk about what we, the Identity Team at Mozilla, are working on.

Mozilla makes Firefox, the 2nd most popular browser in the world, and the only major browser built by a non-profit. Mozilla’s mission is to build a better Web that answers to no one but you, the user. It’s hard to overstate how important this is in 2012, when the Web answers less and less to individual users, more and more to powerful data silos whose interests are not always aligned with those of users.

To fulfill the Mozilla mission, the browser remains critical, but is no longer enough. Think of the Web’s hardware and software stack. The browser sits in the middle [1], hardware and operating system below it, cloud services above it. And the browser is getting squeezed: mobile devices, which outnumber desktop computers and are poised to dominate within a couple of years, run operating systems that limit, through technical means or bundling deals, which browser you can use and how you can customize their behavior. Meanwhile, browsers are fast becoming passive funnels of user data into cloud services that offer too little user control and too much lock-in.

Mozilla is moving quickly to address the first issue with Boot2Gecko, a free, open, and Web-based mobile operating system due to launch next year. This is an incredibly important project that aims to establish true user choice in the mobile stack and to power-charge the Open Web by giving HTML5 Apps new capabilities, including camera access, dialing, etc.

The Mozilla Identity Team is working on the top of the stack: we want users to control their transactions, whether using money or data, with cloud services. We want you to connect to the Web on your terms. To do that, we’re building services and corresponding browser features.

We’re starting with Persona, our simple distributed login system, which you can integrate into your web site in a couple of hours — a good bit more easily than our competitors. Persona is unique because it deeply respects users: the only data exchanged is that users wish to provide. For example, when you use Persona to sign into web sites, there is no central authority that learns about all of your activity.

From Persona, we’ll move to services connected to your identity. We’ll help you manage your data, connect the services that matter to you, all under your full control. We want to take user agency, a role typically reserved for the browser sitting on your device, into the cloud. And because we are Mozilla, and all of our code and protocols are open, you know the services will build will always be on your side.

All that said, we know that users pick products based on quality features, not grand visions. Our vision is our compass, but we work on products that fulfill specific user and developer needs today. We will work towards our vision one compelling and pragmatic product at a time.

The lines between client, server, operating system, browser, and apps are blurring. The Web, far more than a set of technologies, is now a rapidly evolving ecosystem of connections between people and services. The Mozilla Identity Team wants to make sure you, the user, are truly in control of your connections. We want to help you connect on your terms. Follow us, join us.


[1] David Ascher spoke about this in his post about the new Mozilla a few months ago.

cookies don’t track people. people track people.

The news shows are in a tizzy: Google violated your privacy again [CBS, CNN] by circumventing Safari’s built-in tracking protection mechanism. It’s great to see a renewed public focus on privacy, but, in this case, I think this is the wrong problem to focus on and the wrong message to send.

what happened exactly

(Want a more detailed technical explanation? Read Jonathan Mayer’s post. He’s the guy who discovered the shenanigans in question.)

Cookies are bits of data with which web sites tag users, so that when users return, the site can recognize them and provide continuity of service. This is mostly good for users, who don’t want to re-identify themselves every time they visit their favorite social network or e-commerce site. Cookies work mostly with strong compartmentalization: if cnn.com tags you, your browser sends that tag back only to cnn.com. This is important because users would be surprised (not the good kind of surprise) if one site could tag them once and then cause them to uniquely identify themselves with the same identifier to all other sites across the Web.

Things get complicated when web sites embed content served by third parties, for example ads within a news site. Should this third-party content also be able to tag your browser? Should the tag be sent back to that third party when its content is loaded?

Different browsers do different things. Firefox toyed with the idea of not sending the tag back to third parties, but in beta-testing realized that this would break some features that users have come to depend upon, for example Facebook sharing widgets. Safari chose a fairly unique approach: they mostly disallow third parties from tagging users, though they do allow existing tags to be read, so that things like Facebook widgets can still work.

For some reason (I won’t speculate why, Google claims it’s to enable the +1 button), Google used a known technique that tricks Safari into accepting a third-party tag from Google.

mechanism vs. intent

So the reason this whole controversy bugs me is that we’re discussing web privacy based on specific mechanisms, a bit like discussing home privacy by regulating infrared cameras. Sure, an infrared camera can be used to violate my home privacy, but it can be used for many good things, and there are many other ways to invade my home privacy. Cookies, like all technical mechanisms, have both good and evil uses. And browsers don’t all behave the same way with respect to cookies and other web features, so it’s typical for developers to find workarounds that effectively give them “standard behavior” from all browsers. Sometimes these workarounds are truly meant to help the user accomplish what they want. Sometimes these workarounds are used to evil ends, e.g. to track people without their consent.

Again, I don’t know what Google’s intentions were. All I know is that we’re prosecuting the wrong thing: a technical mechanism instead of the an intent to track. Cookies don’t track people. People track people. We should be focusing on empowering users to express their preferences on tracking and ensuring web sites are required to comply.

the tracking arms race

If we focus on technical mechanisms to protect user privacy, then we’re dooming users to an un-winnable arms race. There are dozens of ways of tracking users other than classic cookies. Google used a work-around for Safari third-party cookies, but let’s say they hadn’t. Let’s say instead they’d used Flash cookies, or cache cookies, or device fingerprinting, or a slew of other mechanisms that browsers do not defend against, in large part because it’s really hard to defend against these tracking mechanisms without also breaking key Web features. Would Google then be in the clear?

I fear that that’s exactly what we’re implying when we focus the privacy discussion on mechanisms of tracking. The trackers will move on to the next mechanism, and the browsers will scram to try to defend against these mechanisms without every being able to catch up. Blocking tracking at the technical level is, in my opinion, impossible.

the solution: Do Not Track and More

The beginning of a solution lies in the judo move that is Do Not Track, an idea that came out of a collaboration between Christopher Soghoian, Dan Kaminsky, and Sid Stamm (see the full history of DNT). Do Not Track was first implemented in Firefox last year, and soon thereafter in IE, Opera, and Safari. It’s being standardized now at the W3C. It simply lets the user express a preference for not being tracked. Is it a strong technical measure? No. It does nothing to directly prevent tracking. Instead, it lets the user express a preference. And, as support for it grows, it will become incredibly difficult for sites to justify tracking behavior, regardless of the mechanism, when the user has clearly expressed and communicated this choice.

We’ll need more than Do Not Track in the future. But it’s the right kind of battle. It doesn’t care about cookies or fingerprinting or who-knows-what.

If you want to get upset at Google, ask why they don’t provide Do Not Track support in Chrome. Ask why they don’t respect the Do Not Track flag on Google web properties when they see users waiving it. These are fights worth having. But fighting over cookies? That’s so last decade.

UPDATE: corrected origin credit for DNT header.

a simpler, webbier approach to Web Intents (or Activities)

A few months ago, Mike Hanson and I started meeting with James, Paul, Greg, and others on the Google Chrome team. We had a common goal: how might web developers build applications that talk to each other in a way that the user, not the site, decides which application to use? For example, how might a major news site provide a “share” button that connects to the user’s preferred sharing mechanism? Not everyone uses the same top-three social networks, yet users are constantly forced to search for their preferred service within a set of publisher-chosen buttons. That leads to undue centralization and significantly undercuts innovation and user choice. How incredibly inelegant!

We figured that, with a bit more browser smarts, we could do better.

to the design studio!

Mike and I proposed Web Activities, and put together a screencast.

The Google team proposed Web Intents, and put together a far more complete proposal.

Techcrunch covered our collaboration.

While all this was happening, the always amazing Tyler Close, of Web Introducer fame and also from Google, was whispering in our ears “Guys, I think you’re doing it wrong. It’s over-engineered. We can do simpler.” We all ignored him. I think that was a mistake. Tyler was right. Web Activities was over-engineered. And, I fear, Web Intents is too.

(Tantek also deserves credit for pointing out that we can do simpler.)

the glaring inconsistency

Web applications already have a mechanism for communicating with other web applications loaded within the same browser: postMessage. It isn’t perfect, but it works, and it is flexible enough that much innovation has been built on it. Google, Microsoft, Facebook all use it, oftentimes for embedding widgets within other pages, each in a very different way. At Mozilla, we use postMessage extensively for BrowserID, and we’ve built nice abstractions on top of it, like winchan to consistently build a message channel to a new popup window (including all IE workarounds).

postMessage is a very simple, very Webby, and very generative: it’s easy to build new ideas on top of it. It doesn’t care about mime types, dialogs, callbacks, etc. It’s just a simple, authenticated message channel. The only reason postMessage isn’t enough to do what we need is that the sender and receiver are, for the most part, tightly coupled. The sender has to specify its receiver, which means the user can’t easily step in and substitute the endpoint of her choice. postMessage tightly couples the sender and receiver of the channel. We’d like a loose coupling, where the user gets to mix and match senders and receivers.

So wait, if that’s the only gap, then why are we proposing a completely different approach to cross-application messaging? Why should tight and loose coupling of messaging channels be implemented in completely different ways? Given that the postMessage abstraction has been so successful and useful, the “right” way to move forward is to tweak it, minimally, not to redesign a different stack.

a minimalist way forward

A minimalist way forward is to use postMessage as is, and to provide only the bits necessary to enable loose coupling.

Here, Tyler comes to the rescue again. He proposed, in one of the last chats we had with Google, using custom protocol handlers as the target of postMessage channels. So, when a major news site wants to share an article, rather than postMessage’ing (or linking) to http://twitter.com/, it can use the one-indirection-away URL share://.... The browser can then jump in and substitute the user’s preferred implementation of a sharing provider at that custom protocol handler. Everything else, linking or communicating via postMessage, is then the same.The only difference is, there’s one level of indirection to give the user a chance to step in and say “that service, please.”

What’s even more interesting is that we already have basic mechanisms for sites to register themselves as custom protocol handlers: registerProtocolHandler. The current mechanisms aren’t quite good enough yet, but the tweaks we would need are far simpler than building a whole new messaging stack. Mozilla’s own Austin King has prototyped what some of these tweaks might look like using a JavaScript shim, and the results are surprisingly useful with only minor tweaks.

another minimalist approach

There’s also Ian Hickson’s proposal, which is a little bit different than using protocol handlers and has some nice properties. It’s quite similar to Tyler’s proposal in one key way: do the smallest amount of work to set up a message channel, and get out of the way. Mark Hammond has prototyped Ian’s proposal, and it looks like it can be nicely shimmed in pure JavaScript (with just one tweak to the API that’s probably worth considering even for the native implementation.) I like this proposal, too, and I wonder if it could be made to work with custom protocol handlers, which have a nice URL-based architecture.

so now what?

I propose that we stop for a second on the Web Intents discussion and ask ourselves: maybe we’ve been over-engineering this. Maybe we don’t need mime types and new HTML elements and new DOM properties, etc. Maybe there’s a much easier, good-enough solution, based on proven technology, with only minor tweaks to well-understood code paths. It won’t be perfect, we’ll probably need some JS libraries to make things more convenient for developers, but that’s okay. That’s better for the Web. Keep the platform simple, leave the real innovation to the edges.

I believe Web Intents, as currently proposed, are over-engineered. So are Web Activities. But it’s not too late to correct course. Let’s figure out the simplest way to involve the user in choosing an application, set up a message channel, and get out of the way.

encryption is (mostly) not magic

A few months ago, Sony’s Playstation Network got hacked. Millions of accounts were breached, leaking physical addresses and passwords. Sony admitted that their data was “not encrypted.”

Around the same time, researchers discovered that Dropbox stores user files “unencrypted.” Dozens (hundreds?) closed their accounts in protest. They’re my confidential files, they cried, why couldn’t you at least encrypt them?

Many, including some quite tech-savvy folks, were quick to indicate that it would have been so easy to encrypt the data. Not encrypting the data proved Sony and Dropbox’s incompetence, they said.

In my opinion, it’s not quite that simple.

Encryption is easy, it’s true. You can download code that implements military-grade encryption in any programming language in a matter of seconds. So why can’t companies just encrypt the data they host and protect us from hackers?

The core problem is that, to be consumable by human users, data has to be decrypted. So the decryption key has to live somewhere between the data-store and the user’s eyeballs. For security purposes, you’d like the decryption key to be very far from the data-store and very close to the user’s eyeballs. Heck you’d like the decryption key to be *inside* the user’s brain. That’s not (yet) possible. And, in fact, in most cases, it isn’t even practical to have the key all that far from the data-store.

encryption relocates the problem

Sony needs to be able to charge your credit card, which requires your billing address. They probably need to do that whether or not you’re online, since you’re not likely to appreciate being involved in your monthly renewal, each and every month. So, even if they encrypt your credit card number and address, they also need to store the decryption key somewhere on their servers. And since they probably want to serve you an “update your account” page with address pre-filled, that decryption key has to be available to decrypt the data as soon as you click “update my account.” So, if Sony’s web servers need to be able to decrypt your data, and hackers break into Sony’s servers, there’s only so much protection encryption provides.

Meanwhile, Dropbox wants to give you access to your files everywhere. Maybe they could keep your files encrypted on their servers, with encryption keys stored only on your desktop machine? Yes… until you want to access your files over the Web using a friend’s computer. And what if you want to share a file with a friend while they’re not online? Somehow you have to send them the decryption key. Dropbox must now ask its users to manage the sharing of these decryption keys (good luck explaining that to them), or must hold on to the decryption key and manage who gets the key…. which means storing the decryption keys on their servers. If you walk down the usability path far enough – in fact not all that far – it becomes clear that Dropbox probably needs to store the decryption key not too far from the encrypted files themselves. Encryption can’t protect you once you actually mean to decrypt the data.

The features users need often dictate where the decryption key is stored. The more useful the product, the closer the decryption key has to be to the encrypted data. Don’t think of encryption as a magic shield that miraculously distinguishes between good and bad guys. Instead, think of encryption as a mechanism for shrinking the size of the secret (one small encryption key can secure gigabytes of data), thus allowing the easy relocation of the secret to another location. That’s still quite useful, but it’s not nearly as magical as many imply it to be.

what about Firefox Sync, Apple TimeMachine, SpiderOak, Helios, etc.

But but but, you might be thinking, there are systems that store encrypted data and don’t store the decryption key. Firefox Sync. Apple’s TimeMachine backup system. The SpiderOak online backup system. Heck, even my own Helios Voting System encrypts user votes in the browser with no decryption keys stored anywhere except the trustees’ own machines.

It’s true, in some very specific cases, you can build systems where the decryption key is stored only on a user’s desktop machine. Sometimes, you can even build a system where the key is stored nowhere durably; instead it is derived on the fly from the user’s password, used to encrypt/decrypt, then forgotten.

But all of these systems have significant usability downsides (yes, even my voting system). If you only have one machine connected to Firefox Sync, and you lose it, you cannot get your bookmarks and web history back. If you forget your Time Machine or SpiderOak password, and your main hard drive crashes, you cannot recover your data from backup. If you lose your Helios Voting decryption key, you cannot tally your election.

And when I say “you cannot get your data back,” I mean you would need a mathematical breakthrough of significant proportions to get your data back. It’s not happening. Your data is lost. Keep in mind: that’s the whole point of not storing the decryption key. It’s not a bug, it’s a feature.

and then there’s sharing

I alluded to this issue in the Dropbox description above: what happens when users want to share data with others? If the servers don’t have the decryption key, that means users have to pass the decryption key to one another. Maybe you’re thinking you can use public-key encryption, where each user has a keypair, publishes the public encryption key, and keeps secret the private decryption key? Now we’re back to “you can’t get your data back” if the user loses their private key.

And what about features like Facebook’s newsfeed, where servers process, massage, aggregate, and filter data for users before they even see it? If the server can’t decrypt the data, then how can it help you process the data before you see it?

To be clear: if your web site has social features, it’s very unlikely you can successfully push the decryption keys down to the user. You’re going to need to read the data on your servers. And if your servers need to read the data, then a hacker who breaks into the servers can read the data, too.

so the cryptographer is telling me that encryption is useless?

No, far from it. I’m only saying that encryption with end-user-controlled keys has far fewer applications than most people think. Those applications need to be well-scoped, and they have to accompanied by big bad disclaimers about what happens when you lose your key.

That said, encryption as a means of partitioning power and access on the server-side remains a very powerful tool. If you have to store credit card numbers, it’s best if you build a subsystem whose entire role is to store credit-card numbers encrypted, and process transactions from other parts of your system. If your entire system is compromised, then you’re no better off than if you hadn’t taken those precautions. But, if only part of your system is compromised, encryption may well stop an attacker from gaining access to the most sensitive parts of the system.

You can take this encryption-as-access-control idea very far. An MIT team just published CryptDB, a modified relational database that uses interesting encryption techniques to strongly enforce access control. Note that, if you have the password to log into the database, this encryption isn’t going to hide the data from you: the decryption key is on the server. Still, it’s a very good defense-in-depth approach.

what about this fully homomorphic encryption thing?

OK, so I lied a little bit when I talked about pre-processing data. There is a kind of encryption, called homomorphic encryption, that lets you perform operations on data while it remains encrypted. The last few years have seen epic progress in this field, and it’s quite exciting…. for a cryptographer. These techniques remain extremely impractical for most use cases today, with an overhead factor in the trillions, both for storage and computation time. And, even when they do become more practical, the central decryption key problem remains: forcing users to manage decryption keys is, for the most part, a usability nightmare.

That said, I must admit: homomorphic encryption is actually almost like magic.

the special case of passwords

Passwords are special because, once stored, you never need to read them back out, you only need to check if a password typed by a user matches the one stored on the server. That’s very different than a credit-card number, which does need to be read after it’s stored so the card can be charged every month. So for passwords, we have special techniques. It’s not encryption, because encryption is reversible, and the whole point is that we’d like the system to strongly disallow extraction of user passwords from the data-store. The special tool is a one-way function, such as bcrypt. Take the password, process it using the one-way function, and store only the output. The one-way function is built to be difficult to reverse: you have to try a password to see if it matches. That’s pretty cool stuff, but really it only applies to passwords.

So, if you’re storing passwords, you should absolutely be passing them through a one-way function. You could say you’re “hashing” them, that’s close enough. In fact you probably want to say you’re salting and hashing them. But whatever you do, you’re not “encrypting” your passwords. That’s just silly.

encryption is not a magic bullet

For the most part, encryption isn’t magic. Encryption lets you manage secrets more securely, but if users are involved in the key management, that almost certainly comes at the expense of usability and features. Web services should strongly consider encryption where possible to more strictly manage their internal access controls. But think carefully before embarking on a design that forces users to manage their keys. In many cases, users simply don’t understand that losing the key means losing the data. As my colleague Umesh Shankar says, if you design a car lock so secure that locking yourself out means crushing the car and buying a new one, you’re probably doing it wrong.

BrowserID and me

A few weeks ago, I became Tech Lead on Identity and User Data at Mozilla. This is an awesome and challenging responsibility, and I’ve been busy. When I took on this new responsibility, BrowserID was already well under way, so we were able to launch it in my second week on the project (early July). It’s been a very fun ride.

Here’s the BrowserID demo at the Mozilla All-Hands last week:

Given my prior work on email-based authentication (EmID, Lightweight Email Signatures, BeamAuth), you might think BrowserID was my brainchild. In fact, it really wasn’t. And, in a testament to the shrinking impact of academic publication venues, none of the BrowserID team had ever heard of my work on email-based authentication before I arrived at Mozila, even though Mozilla folks are quite well versed in the state of the art. But who cares: when I found out about the ongoing work and how we agreed on just about every design principle, I was incredibly excited. And when I realized the fantastic work the team had already done on defining a scaffolding and adoption path for the technology, I was super impressed.

BrowserID started with the Verified Email Protocol, designed by Mike Hanson and Dan Mills, who came up with the approach after extensive exploration of web-based identity approaches over the last two years. It’s a simple idea: users can prove to web sites that they own a particular email address. That’s how they register, and that’s how they log back in the next time they visit the site. BrowserID, the code and site, was initially bootstrapped by Lloyd Hilaiel and Mike Hanson. Shane Tomlinson and I joined the team in June. We now also have an awesome UX design team (Bryan and Andy) and the team continues to grow (yay Austin!)

So, that’s what I’m working on these days: BrowserID and other Identity+UserData efforts at Mozilla. I’m excited to be leading this technical effort. The team is amazing, and we’ve got big aggressive plans to help you control your identity and data on the Web.