I’ll open with what I just said to some (perfectly reasonable) questions raised by Øivind Idsø on Twitter: most users make video with proprietary software and watch it in Flash. The idea is to change that. It turns out to be hard.

Open video advocates have now gotten some huge gifts from Google; I’m disinclined to look that gift horse in the mouth, as the saying goes. If you do, though, I don’t think you see anything too terribly unexpected.

Meanwhile… ah, Web commentary is adorable, isn’t it? The latest conspiracy theory is that Google dropping H.264 support from its browser is a far-reaching, evil plan to … um … make things … more or less exactly like they were before? I’m not sure, honestly.

The narrative runs like this: Google drops H.264, forcing people to use WebM + VP8. But H.264 has already won. That means that people will be forced to use Flash with Chrome, because Chrome will be incompatible with H.264. And then that will ensure that Flash is popular. But Flash is incompatible with Apple’s iPhone and iPad. That will leave Apple users frustrated. They’ll have no choice but to turn to Android – to run Flash! So Android will beat iOS! And, of course, all of this will really give Google more control over video. And that will mean more ads. So they just have to beat Flash to beat H.264 to help Flash to beat Apple to use Flash to beat Flash to use WebM to beat publishers to run ads.

It’s so … simple … I don’t know why I didn’t think of it myself.

In a widely-circulated (and Gruber-endorsed) comment from Slashdot’s znu, we’re treated to this weird scenario:
A really nasty trick

“…it advances a codec that’s de facto controlled by Google at the expense of a codec that is a legitimate open standard controlled by a multi-vendor governance process managed by reputable international standards bodies. (“Open source” != “open standard”.) And second, it will slow the transition to HTML5 and away from Flash by creating more confusion about which codec to use for HTML5 video, which benefits Google by hurting Apple (since Apple doesn’t want to support Flash), but also sucks for users.”

Well, that’d make this discussion a lot simpler, but it ignores some simple facts:

  • If this is a plan to hurt iOS, it’s a stupid plan. Google tells Wired today that the announcement covers Chrome, not YouTube or Android, which are likely to continue to offer H.264 support. So, this doesn’t really do much as a pro-Android, anti-iOS move. Android supports both codecs. Chrome and Android are different, sometimes frustratingly-independent teams at Google. And Apple chose to avoid Flash and WebM/VP8, so if this is an evil plan to destroy Apple, it’s Apple’s evil plan to destroy Apple. (Hint: Apple knows what they’re doing, too, so… no.)
  • This doesn’t particularly help or hurt Flash. It requires twisted logic to figure out how this advances Flash, because HTML5 video has had a codec stalemate since day one. Before the announcement: Firefox will only use Theora (and now VP8). Apple will only use H.264. After the announcement: same. So, uh… huh? All this does is move one player from one side to the other.
  • Members of the open source community did ask for this. Google was under pressure to open source VP8 from various open video advocates, which they did. So, if Google is pulling the wool over the open source community, they’re doing it by giving them … exactly what they want. Great evil, anti-open source plan, that one.
  • MPEG LA isn’t going to solve open source software’s problems. Why aren’t the open source community adopting the “open standard” that is H.264 and its (apparently heroic, in this narrative) multi-vendor governance? That might have something to do with the fact that members of said organization have routinely threatened lawsuits. And while MPEG LA has a patent pool, that didn’t stop, for instance, AT&T from threatening to sue MPEG LA members, as it did in the end of 2005.

Reality Check

Don’t get me wrong. Google’s move may prove to be a mistake. I think it may benefit free software, but that doesn’t necessarily mean it’ll benefit users, at least in the short term, partly because of the way patent laws are written. (I’m willing to give it a try – my next job is explaining a workflow that would allow publishers to be productive with WebM + VP8.) There are challenges on both sides.

I just can’t find any real evidence that it’s an evil plan. And if I’m wrong about the benefits of pursuing WebM/Vorbis/VP8, we still wind up with something too much like the status quo for anyone to argue it’s genuinely evil.

Let’s cut to the point: Web video, desktop and mobile, is a mess. If you want to make it better, get prepared for solutions that, however you slice it, aren’t neat and tidy. The HTML5 video tag is terrific, and opens up the potential for new functionality and finer-tuned performance, as it finally makes video stand on equal footing with other stuff we use in a Web page. The problem is the codec, and the simple fact of the matter is, you’re not really going to like either solution. Right now, there are two leading candidates:

1. HTML5 video tag + H.264 codec. (Note: this avenue doesn’t necessarily establish what the container is.)

2. HTML5 video tag + VP8 video codec, in WebM container. (Incidentally, technically WebM also uses the Vorbis audio codec, which unlike MP3 or AAC is free to use and has not been subject to the same patent liability or quality criticisms. Yay, audio – you’re less of a pain in the…)

Right now, I think popular support is lining up behind H.264, but that ignores some of its problems. It is not, according to a widely-used definition (one endorsed, among others, by the European Union), an open standard, because it is not royalty free. It is a standard, but not open. (See, for reference, um, the entire history of technology for more on how that can cause problems.)

MPEG-LA announced free royalties for end users, but that leaves out developers – think everything from your video editor to your compression tool to Firefox. Since as an end user you might want to be able to use Firefox and not see a huge broken video icon, that means the resulting cost to end users isn’t really free, either.

And that could lead to an ugly future. Take the example of Firefox, the free and open source browser that helped us get into the modern Web age. At least right now, you can install Flash and play a variety of codecs (soon to include VP8 as well as H.264). If the Web went to HTML5 video tag with H.264, you’d be out of luck, because legally Mozilla can’t ship that support in an open source browser. They’d have to buy a license, and the resulting browser couldn’t be distributed under a free license.

This isn’t, as some have argued, an ideological argument by open source advocates just because they have long beards or are secret Marxists. What they’re saying is that they are restricted by laws. If you use open source software (and if you’re reading this, you do), you’ll simply have to accept that that software is possible because it obeys the law of the land.

WebM support has a different set of problems, and to ignore them would be irresponsible, too. With a BSD license for the entire format and freely-licensed codecs VP8 and Vorbis, Mozilla can ship video support. So can a free and open source video editor or compressor tool. Of course, this brings a whole lot of other concerns – quality and hardware decoder availability (and resulting performance). It also brings potential patent liability, though I think it is possible to overstate this argument – because of the way patent law is written, so, too, would any video format. Ironically, nearly all of the big parties involved here – Microsoft, Apple, Google – have in the past been advocates of patent reform here in the US. That’s not to say you can’t point this out, only that in so doing, you’re pointing out the obvious.

Do you really want to have to choose between free software and usable Web video? The sincere hope right now is that you won’t have to. But that’s the challenge that lies ahead.

I think the simple way to put the problems with WebM is that it’s not ready yet – not for creators and not for publishers. But that assumes that HTML5 plus H.264 is ready – and I think that’s a big leap. Take this site – I wouldn’t for a second eliminate the third or so of our readers who access the site through Firefox.

Writers like John Gruber are right when they say H.264 is widely used. What they don’t tell you is that, apart from Apple’s iOS, most users (even in Apple’s own Safari) are likely viewing that H.264-encoded content through Flash.

All of these arguments have to be considered within the context of a world that still largely uses Flash. And that’s what makes this article a must-read:

Google and H.264 – Far From Hypocritical: “Not so fast calling Google hypocrites for not dropping Flash yet. This is a step on that journey” [Simon Phipps, for Computerworld UK]

Look, the bottom line is, standardization is generally about as much fun as a root canal with a toothpick. Its messy legacy is also bound up with the history of technology – including everything we love about it.

If you’re expecting a good-versus-evil battle with white and black flags, I suggest you watch Star Wars. I hear it just came out on Blu-Ray, which is a standard for delivering video … actually, never mind.

I’m going to lunch. The only tin foil I need will be the tin foil wrapped around my burrito.

  • RichardL

    I would not be terribly surprised if the cost of h.264 codec licensing for Android handset makers is a consideration for Google somewhere in this. The codec license costs become more pressing as Android picks up steam in the very cost sensitive low-end smartphone markets — think Asia and Africa. 

  • Wiley

    When having a heated discussion with someone, I think most diplomats suggest switching over from a two-sided model to a one-sided blog post, wherein you call the other party names in a big headline, characterize their concerns in the simplest and dumbest way possible, and then stomp off to get lunch.

  • Peter Kirn

    Well, I'm not a diplomat, I'm a pundit. I thought the points you raised were perfectly reasonable. But a lot of Web stories are now letting folks like the Slashdot commenter frame the discussion, and I think that's kind of silly.

    "WebM isn't practical right now" is a reasonable argument – I'm the first to agree with most of those points. "Google has an evil plan to take over the world and promote Flash by shooting their own technology in the foot" is not a reasonable argument.

  • victor

    I don't know much about the details of video codecs (I usually stick to H.264 since its small and looks good), but logic tells me that cutting support for anything is actually the opposite of openness. For me the best world would be where formats are invisible because everything is supported. I cannot stop thinking that google's move has a political/commercial reasoning behind it, whatever it might be. They are a for profit company after all, power is their main motivation; as it is apple's, microsoft's et al

  • victor

    Oh, there is an article in ars technica that talks about this topic, also stating that not supporting is the opposite of openness, just a heads up.

  • Peter Kirn

    @Victor: You can make the argument that removing H.264 isn't a good thing, as that Ars writer does. But my point is that the argument that Google has a nefarious plan to kill their own technology by promoting their own technology in order to promote a competing technology just doesn't make *any* rational sense.

    Google purchased On2 in order to acquire their video codec expertise. They then open sourced their VP8 codec and the WebM container they built around it.

    In doing so, they filled a gap that many – not just Google – wanted to see filled.

    And I think this is a very important point — Google's profit motive is not the same as Microsoft's or Apple's because they're in a different business. They see any Internet growth as benefiting their revenue streams. So, in some sense "openness" as they've defined it – the growth of Web use via standard Web pages – is akin to Standard Oil wanting to see GM sell more cars. The more people use, the more they benefit. That's not the position either Apple or Microsoft is in.

  • Peter Kirn

    This is the line in the Ars article I have to fault, which is the crux of the whole matter:

    "H.264 is unambiguously open."

    You can make the argument that H.264 should be the standard for Web video. You can argue that royalty-bearing standards, once ratified, should be called open standards.

    You cannot, however, say this is unambiguous. Many standards bodies refrain from calling non-free standards "open." H.264 is unambiguously a standard, both in its wide input and its adoption by standard bodies. But many definitions of open standards require as part of openness that anyone should be able to adopt the standard. That's not the case with H.264.

    In fact, if you loosen this definition this much, then even the previous complaint – that H.264 carried even a potential *end user* license fee once free rates expired – wouldn't have counted.

    I'll say it again: H.264 is *not free*. It requires royalties for publishers of non-free content, and it requires royalties for developers. Because it bears a significant and well-defended patent portfolio, it is the opposite of open source.

    I'm deeply disappointed that Ars of all outlets would publish a multiple-page article that streamrolls over these considerations in order to make the H.264 argument cleaner than it should be. There's a reason I say this is a mess. Yes, if you just wanted a ubiquitous video standard, H.264 would be ideal. But to ignore the contributions of open source developers like Firefox, to move the Web to something they legally can't use, I think is a tragedy.

    Looking around the Web, I'm much more alone in that than I ever would have imagined.

  • Wiley

    Peter, I don't think that Mozilla's motives are anything but pure, it's Mozilla. And I'm not really sure if Google would do something as foul as dumping YouTube's h.264 archive and replacing it with a WebM archive that is optimized for some future line of droid devices. Maybe they'd leave the archive, allow the iOS youtube app to keep working, and just bork the html5 pages for the good of the people. 

    I really don't know. The future tends to be less awful than we expect, and these are all big players who can afford to piss one another off, but generally try to avoid losing face in the public's eyes.

    I don't love H.264 (or any lossy format for the web) I hope something better comes along. I am grumpy that all the players, Apple included, have decided what the one and only element must be, and then refuse to give you a fallback to what's inside the tag (a flash embed or a 'screw you, you don't use my favorite browser' note). Apple pisses me off less, because they both use the current best looking codec (which allows for both their encoder and the vastly superior x264 encoder) and they allow reference movies that point to different videos based on bandwith and platform. Previously chrome had my FAVORITE behavior, in that it would play either an H.264 clip or a Theora clip. Firefox would ignore H.264 and Safari would ignore Ogg. Chrome kind of took the high road. 

    Chrome dropping H.264 altogether, which may come from the purest of "Let's help Mozilla out and stand with them against the man" places, does nothing for the user. It just means that I cannot use the html5 video element without doing seperate encodings of h.264 and WebM, which is non-trivial. I already do many many encodes of the same video just to make sure I get the best quality. WebM takes much, much longer to encode, and is lower quality. Right now tools are scarce, nobody is using it, and this will likely be a very different scenario when Google makes its next move and the format moves center stage. At the moment though, it feels like the first glimmerings of good, compatible online video have been yanked away and replaced with a smoking mess, and anyone who says it was for my good is going to get a funny look from me. 

    When compuserve started saber-rattling about how they wanted licensing money from gif's browsers didn't drop gif support and declare png the only possible image format that could be used in an img tag. I'd argue that mp4 is just as ubiquitous as gif now. This argument may be totally silly, but you have to see how this is a frustrating development to those who deploy online video, whether you see it as a philosophical victory or not, and it's inappropriate to tell us that we are wearing foil hats when we grumble that Google is likely using the Mozilla crowd as a cudgel against a competitor, like Republicans making donations to the green party.

  • Peter Kirn

    Well, I know the GIF/CompuServe debacle was foremost in the minds of people who pushed for an open format in HTML5, but it's all in how you interpret the significance of that episode.

    I think it's fair now to say this: no one is sure what Google's motives are. I believe it's possible to read too much into this stuff, in that not every decision made by a company as large as Google, or indeed any engineering decision anywhere, always has a coherent, grand strategy behind it. So people may be looking for a coherent motive that doesn't even exist.

    I think I'm also already prepared to say I was wrong. I thought Google jumping full force behind WebM would tilt the scales. I didn't anticipate what has happened, which is a widespread backlash from users and publishers who perceive this as Google forcing them to do something. That's a fair response. It's not over yet – those encoding tools are getting a *lot* better – but WebM still faces an uphill battle. And don't get me wrong – people uploading the content have every right to ask what it means to them. That's the whole point.

    I'd say, in fact, that Apple really won the PR battle on this by painting Flash as the enemy and H.264 as the solution. For all that H.264 does right, I think that's an oversimplification, but with the media ready to do color commentary on a Google/Apple rivalry, it fits their narrative.

    Make what you will of Mozilla's intentions, but I have to agree that as an open source project they can't legally or responsibly incorporate H.264 decoding code, period.

    Now, the ongoing mess suggests that we may all have to return to plug-ins – and, indeed, that the state of video IP is such that this may have been the reality all along.

  • Hob

         "Now, the ongoing mess suggests that we may all have to return to plug-ins – and, indeed, that the state of video IP is such that this may have been the reality all along."

    I believe we may have reached an understanding. Sad innit?

  • ex-fanboy

    as mr. kirn would say:

    "wait!". my suggestion is get rid of flash completely. yes, i agree with apple on this one. aaaand since adobe took over flash, in my opinion, catastrophe.

  • http://Wileywiggins.com Wiley

    I dont think we need to 'get rid of anything', and flash was never really part of the conversation anyway as far as html5 video goes, other than as a fallback, because it is ubiquitous. 

    Actionscript is great. It's been a catalyst for more great indie games than any other platform. Flash has it's place. It's a bad choice for site navigation or video delivery, and it's not something I want running on a mobile device. That doesn't mean I want it, or any other technology 'got rid of'.

  • Juno

    OK, Google should support both…

    … but then so should Apple.

  • mitch

    Isn't MP3 kind of the audio counterpart of H.264? Patent-encumbered, not supported by Mozilla etc. And OGG is much more, say, finalized and established than WebM is in comparison to H.264. I'd say let's drop that too.

  • http://noisepages.com/members/eatyone/ Guillaume LECTEZ

    And what about wmv for Windows phone?

    Sorry…

    I've got this problem of codec everyday in my job. What i think: today a lot of litlle company earn money with shareware converters…and i have to find them By…. GOOGLE!

  • Andy

    So what? F*ck Apple.

  • victor

    another article on the subject at appleinsider; which somehow seems more informed and neutral, also stating that googles move is to "save money" not to "advocate openness".

  • victor

    on second reading, maybe is not so neutral yet still very informative.

  • RichardL

    Google actually does a pretty good job clarifying what they are up to:

    http://blog.chromium.org/2011/01/more-about-chrom

    What I find fascinating is so many of the critics on the tech blogosphere rejecting the simplest explanations of Google's action in favor of far more complicated theories. 

  • http://leisuresonic.com/ Christopher Penrose

    I think it is probably a good plan in the long run — WebM has to make it to hardware for it to really catch on in the mobile sphere.  Google will lose some chrome users on the desktop in the short run.  But I the counter-argument that holds the most weight — and is free of conspiracy theory — is that the move may affect more fragmentation rather than unification.  But ultimately, free codecs have the greatest opportunity to be universal standards.  

  • Peter Kirn

    @Christopher: Aside from some angry pro-Apple fans I saw on comments switching back to Safari, why would Google lose Chrome users? I almost never see *any* H.264 video content without Flash, either as the default or the fallback.

    We're definitely going to see some fragmentation; I think the question is whether this makes it worse.

    I guess we're imagining you'd upload H.264 – so have just one codec – and have a Flash player fallback for everyone else (Firefox, Opera, etc.) I'm just trying to wrap my head around the scenario H.264 backers were envisioning here, but that seems to be it.

    I think what they're missing is that, for most publishers, Flash then wouldn't be a fallback. It'd be exactly what it is now – the de facto baseline. If you're trying to unseat Flash, you need both a significant incentive and one format that works. People can say it was going to be H.264 until they're blue in the face, but that only works if everyone gives up Firefox overnight (which seems to be what some of that crowd were arguing should happen, if necessary)

  • RichardL

    I guess the problem I see is that even if Google really just wants to break through the HTML5 VIDEO codec impasse that this move alone doesn't do it.

    Apple's iDevices seem unlikely to ever get an HTML5 VIDEO codec other than H.264 for political reasons. And, again for political reasons, Flash isn't even a fallback there. It's H.264 or a subset of Quicktime or whatever can be built into and published as an Apple-approved app. So it's easy to see why those with heavy bets on the iPhone and iPad platforms are extremely critical of Google's move, even though it's no net change from the status quo of HTML5 VIDEO stalemate. 

    Firefox isn't going to disappear overnight nor is Mozilla going rewrite their codebase to avoid the license conflicts with H.264. But to listen to the H.264 camp they seem to have been expecting one of those two eventualities to emerge in the near term.

    Ironically, rumors are emerging that next generation Apple iDevices may have a programmable GPU supported by Apple's OpenCL. Such an architecture would be very well suited to after-the-fact video codec acceleration. 
    http://www.macrumors.com/2011/01/14/ios-4-3-beta-

  • RichardL

    It should also be noted that the various iDevices only support very limited subsets of H.264 any way. Encode-once-deploy-everywhere is a myth. 

  • Peter Kirn

    To everyone – I do appreciate the level of discussion here; it's what I find lacking elsewhere online! Conversation is really to be an essential part of the process.

    @RichardL: Well, that's correct – you know, the one thing no one in the press has really said is this:

    There's an impasse on video log-jammed so badly as to render even something as significant as this move mostly irrelevant. Flash is no longer a baseline. "H.264" isn't, either. People can't wish either H.264 or WebM into being a solution when there are significant, potentially-insurmountable obstacles to adoption on both sides.

    John Gruber's ongoing self-congratulatory analysis continues to characterize his position (H.264 is the panacea that solves all your video problems) as "practical" thinking and everyone else as "idealistic." Not to pick on Gruber, but every time I see elsewhere, they return to his analysis. But Gruber continues to distort the reality of the situation for entities like Mozilla. Now he and Bott are arguing there's no "royalty trap" — ignoring the fact that, at this point, Mozilla et al have been very clear that it's the lack of open source encode and decode capability that is their primary argument with WebM.

    It's hard not to see this ongoing hostility from Web pundits in general as directed at open source software en masse. If only those idealistic eggheads would stop with their ideological arguments about open source software, we'd be free to enjoy Flash-free video on our Macs and iPhones in peace.

    To be completely fair, I think the fault lies in part with open video advocates for allowing the situation to deteriorate to the point that's quick becoming conventional wisdom. It's the result of two major communication failures:

    1. Critics of H.264, people who wanted OGG Theora and later WebM+VP8 as part of the HTML5 spec, conflated hypothetical arguments with real ones. That makes it all too easy for people to return to the "H.264 is royalty free" argument, even when it's not for developers. And they did a poor job adequately answering critics on the (admittedly) murky issues surrounding patent liability, and explaining what "open" means in clear, layperson-friendly terms.

    2. There has been an absence of explanation for content creators that says, here's what it'll look like and here's how to try this out now, from hardware to export encoding to software. The video producing public has experience with H.264 and they've got no clue on VP8.

    I don't think this is over yet, so it's time for people to get their facts straight on those two points, even if they ultimately conclude that H.264 is a better codec for Web distribution (which is a fair conclusion to reach).

    I can at least endeavor to do better, personally. It seems to me that journalism in all its forms – casual journalism included – involves being half an expert in things. That can either be dangerous or beneficial, largely dependent on whether or not you recognize the daylight between what you know and don't and use it as an opportunity. We've had a load of advocacy on this issue and seemingly far less research.