Adobe Flash And Macintosh

[rogerdee123]

ON FLASH AND MAC OS X APPLICATION CRASHES (by John Gruber at http://daringfireball.net

To my knowledge, Apple controls the entire source code to the iPhone OS. That’s not to say they wrote the whole thing from scratch. Many low-level OS components are open source. But they have the source. If there’s a bug, they can fix it. If something is slow, they can optimize or re-write it. That is not true for Mac OS X, and Flash is a prime example. The single leading source of application crashes on Mac OS X is a component that Apple can’t fix.

Several readers asked me for the source for my accusation contained in that last sentence, that Flash is the “leading source of application crashes on Mac OS X”. (And good for them for asking; I’m not sure what I was thinking including that without sourcing it.)

Here’s the deal. On stage at the WWDC 2009 keynote address last June, Apple senior vice president of software engineering Bertrand Serlet was explaining the new web content plugin mechanism for Safari in Snow Leopard. Rather than run within Safari’s application process, web content plugins now run in their own process, so if they crash, they (usually) don’t crash Safari itself. You get a broken little rectangle in the page where the plugin was executing, but the browser itself stays running.

Apple did this for two reasons. Serlet’s stated reason on stage was “crash resistance”, as mentioned above. As for why such crash resistance was worth implementing, Serlet explained that, based on data from the Crash Reporter application built into Mac OS X — the thing that asks if you’d like to send crash data to Apple after a crash — the most frequent cause of crashes across all of Mac OS X are (or at least were, pre-Snow Leopard) “plugins”.

Serlet didn’t name any specific guilty plugins. Just “plugins”. But during the week at WWDC, I confirmed with several sources at Apple who are familiar with the aggregate Crash Reporter data, and they confirmed that “plugins” was a euphemism for “Flash”.

In other words, in Apple’s giant pile of aggregate crash reports — from all app crashes on all Macs from all users who click the button to send these reports to Apple — Flash accounts for more of them than anything else. That doesn’t mean Flash somehow causes crashes in any various app. Presumably, most of the time it’s Safari or some other browser playing Flash content. And it’s worth noting that this doesn’t necessarily mean Flash is particularly crash-prone or poorly engineered. Think of it as a formula like this:

total crashes = (crashing bugs) × (actual use)

Flash’s number and severity of crashing bugs could well be somewhat low and it would still account for a large number of total crashes because it’s actually used all the time — by any Mac user with Flash content playing in a web page. And, if Flash Player for Mac OS X actually is poorly-engineered overly-buggy code, well, that’s even worse.

But there’s another reason why Apple created this new external process architecture for web content plugins in Snow Leopard: it was the only way they could ship Safari and the WebKit framework as 64-bit binaries. Flash Player is only available as a 32-bit binary. (This is true for other third-party web content plugins, like Silverlight, but Flash is the only one that ships as part of the system.) 64-bit apps cannot run 32-bit plugins. Apple doesn’t have the source code to Flash, so only Adobe can make Flash Player 64-bit compatible. They haven’t yet. So if Apple wanted Safari to be 64-bit in Snow Leopard (and they did), they needed to run 32-bit plugins like Flash in a separate process.

Maybe you don’t believe Apple that web content plugins are the most frequent source of crashes on Mac OS X. Maybe you don’t believe me and my unnamed sources at Apple that it’s Flash in particular that accounts for this. That’s cool, skepticism is good. So then in that case, maybe Bertrand Serlet blamed “plugin crash resistance” for political reasons, just to stick a knife in Adobe’s back, and the only reason Apple went with this external-process architecture was for the 64-bit/32-bit incompatibility.

But that just shines a light on the fact that Flash is still a 32-bit binary despite the fact that Apple wants to go 64-bit system-wide. Flash remains 32-bit and there’s nothing Apple can do about it. Instead of being able to make Flash 64-bit themselves, Apple had to engineer an entirely new plugin architecture.

This is why Apple wants to control the source code to the entire OS. If they want to go 64-bit with iPhone OS, it’s entirely in Apple’s own control to do so. And what happens if Apple goes to a new CPU architecture? For the components Apple controls the source code to, they can recompile for the new architecture. If the entire system doesn’t recompile cleanly for the new architecture, they can work on it until it does. For a component like Flash, where Adobe controls the source code, Apple instead has to wait.

Which situation do you think Apple is happier with? Mac OS X, where they had to create a new web content plugin architecture because Flash crashes frequently and isn’t 64-bit? Or iPhone OS, where they control the source code to every single component, and can do whatever they want, when they want?

Point is, even if you think Flash Player for Mac OS X is the greatest piece of software in the world and that a Flash Player for iPhone OS would run just fine, too — there’s no denying that Apple executives have said and continue to say anti-Flash things publicly. Apple doesn’t say much about Flash, but what they do say doesn’t sound like the sort of things they’d say if they were looking forward to supporting it more rather than less.

THE PROPRIETARY WEB

It’s probably pretty clear to regular DF readers that I don’t care for Flash, and that I’m hoping Apple never includes it in the iPhone OS. Might as well make my biases clear.

Why? At the core, because Flash is the only de facto web standard based on a proprietary technology. There are numerous proprietary web content plugins — including Apple’s QuickTime — but Flash is the only one that’s so ubiquitous that it’s a de facto standard. Flash is the way video is delivered over the web, and Adobe completely controls Flash. No other aspect of the web works like this. HTML, CSS, and JavaScript are all open standards, with numerous implementations, including several that are open source.

The simplest argument in favor of Flash support on the iPhone (and The Tablet, and everywhere) is that Flash is, by dint of its popularity and ubiquity, part of the web. But the best argument against Flash support is that it is harmful to the web as a whole to have something as important as video be in the hands of a single company, and the only way that’s going to change is if an open alternative becomes a compelling target for web publishers.

It’s a chicken-and-egg problem. Publishers use Flash for web video because Flash is installed on such a high percentage of clients; clients support Flash because so many publishers use Flash for web video. Apple, with the iPhone, is solving the chicken and egg problem. For the first time ever, there is a large and growing audience of demographically desirable users who don’t have Flash installed. If you want to show video to iPhone users, you need to use H.264.

Apple isn’t trying to replace Flash with its own proprietary thing. They’re replacing it with H.264 and HTML5. This is good for everyone but Adobe.

And yes, I know Flash does much more than just play video. But that’s the main thing everyone is talking about when they talk about Flash not working on the iPhone — video. And when you talk about other uses for Flash, you’re talking about serving as a software runtime, and whether you like it or not, Apple has a clearly stated opposition to third-party software runtimes for iPhone OS, and that policy seems to be working out pretty well for them.

Here’s an email I got from a DF reader:

I was in line waiting for a coffee on Christmas day. In front of me was a kid, about nine or ten, who had an iPhone. He clearly had gotten it that morning. He was pushing frantically at a white box on a web page with the broken plug-in symbol. He was squeezing it, swiping it. He was frustrated and on the verge of getting pissed with his new toy. It seemed like he was trying to hit an online game page, probably one he was used to playing on the family PC. Finally I couldn’t take it anymore. I leaned over and said, “It won’t load Flash. It won’t play your Flash games.” His mom, ignoring him up to that point, was triggered by a stranger talking to her kid. “That’s okay honey,” she said, “we’ll get you a game from the App Store.” His response to this? He started working that device even harder. He didn’t want an App Store game; he wanted his Flash game. And that iPhone suddenly took a huge dive in value to him.

Like it or not, Apple needs to come to terms with this. If only for the kids.

I think this anecdote, and this reader’s takeaway from it, accurately captures the feeling behind much of the “Apple has got to bend on this eventually” sentiment that’s out there.

But think about it from Apple’s perspective. How do you think this situation turned out in the long run? Do you think the kid told his mom to return the iPhone for a refund? Or, do you think they went home and started buying games from the App Store? That there was a period of initial frustration due to Flash games not playing doesn’t change the fact that they (a) bought an iPhone and ( were set to buy games from the App Store.

I’m not arguing that Apple’s apparent executive-level antipathy toward Flash is about anything other than Apple’s own interests. (I do think, though, that Apple’s WebKit team is genuinely idealistic about helping the web as a whole.)

But while Apple may be acting spitefully, they’re not spiting themselves. The iPhone’s lack of Flash has not hurt it one bit. Perhaps that will change in the future, if Flash someday proves popular on other mobile platforms, but don’t hold your breath.

FLASH PERFORMANCE ON MAC OS X

In addition to the principled concerns outlined above regarding Flash being proprietary, there are also practical issues. One, Flash’s aforementioned crashiness on Mac OS X. Second, crashiness aside, its performance on Mac OS X is not as good as it is on Windows. And for video playback specifically, Flash’s performance pales compared to H.264 played through QuickTime. This is not subjective. My machine is a two-year-old MacBook Pro. It plays full-screen H.264 video through QuickTime without problem. When I play full-screen Flash video, my fan kicks in within a few seconds, every time.

I’ve been hard on Flash Player for Mac OS X, but this performance situation is not entirely in Adobe’s hands. On Windows, Flash makes use of hardware decoding for H.264, if available. On Mac OS X, it does not. This is one reason why Flash video playback performs better on Windows than Mac OS X, and also why H.264 playback on Mac OS X is better through QuickTime (which does use hardware decoding).

According to Adobe, though, this is because they can’t. Here’s an entry from their Flash Player FAQ:

Q. Why is hardware decoding of H.264 only supported on the Windows platform?

A. In Flash Player 10.1, H.264 hardware acceleration is not supported under Linux and Mac OS. Linux currently lacks a developed standard API that supports H.264 hardware video decoding, and Mac OS X does not expose access to the required APIs. We will continue to evaluate when to support this feature on Mac and Linux platforms in future releases.

Adobe platform evangelist Lee Brimelow recently posted a weblog entry addressing this:

But let’s talk more about the Flash Player on the Mac. If it is not 100% on par with the Windows player people assume that it is all our fault. The facts show that this is simply not the case. Let’s take for example the question of hardware acceleration for H.264 video that we released with Flash Player 10.1. Here you can see some published results for how much the situation has improved on Windows. Unfortunately we could not add this acceleration to the Mac player because Apple does not provide a public API to make this happen. You can easily verify that by asking Apple. I’m happy to say that we still made some improvements for the Mac player when it comes to video playback, but we simply could not implement the hardware acceleration. This is but one example of stumbling blocks we face when it comes to Apple.

I’m aware of no reason to dispute this. Windows is more hospitable to a third-party runtime like Flash than Mac OS X. I think most would agree that Apple is an opinionated company (to say the least), and they make opinionated products. The runtimes Apple cares about are Cocoa and WebKit. The Apple way to play H.264 is through the QuickTime APIs (and really, as of Snow Leopard the new QuickTime X APIs), not to write your own H.264 playback code that seeks to directly access hardware accelerators.

You can argue about why Apple has taken this stance. One could argue that it’s pragmatic — that Apple doesn’t allow third-party software access to things like hardware H.264 acceleration because it seeks to maintain a layer of abstraction between third-party software and the underlying hardware. One could argue that it’s political — that Apple is happy to make Flash look bad performance-wise because Flash is competitive with Apple products in several different regards. (E.g. you may wish that Hulu, which is entirely Flash-based, worked on your iPhone and worked better on your Mac. Apple wishes that Hulu’s content was going through the iTunes Store.)

I would argue that it’s both — that Apple’s distaste for Flash Player is both a matter of engineering taste (that third-party software should only have access to high-level APIs) and politics. But objectively, regardless of what you personally wish Apple would do with regard to Flash, if Adobe needs Apple to grant them further access to the hardware to make the Mac version of Flash Player better, what are the odds that they’d get that sort of low-level hardware access on the iPhone OS? (Hint: zero.)

I’ll leave the last word to Apple COO Tim Cook, who a year ago said, “We believe in the simple, not the complex. We believe that we need to own and control the primary technologies behind the products we make, and participate only in markets where we can make a significant contribution.”

Flash is owned and controlled by Adobe.

[nikster]

I don't buy the Adobe excuse for H.264 hardware acceleration.

If they wanted, surely, like any other app, they could use those high level Quicktime APIs to hardware accelerate their video.

Just a point of note: H.264 apparently has some severe licensing issues. That's why HTML5 can't replace Flash video. This is from a Gizmodo article.

[rogerdee123]

Inside Apple's iPad: Adobe Flash

By Daniel Eran Dilger

Published: 12:00 AM EST

Apple's new iPad is being criticized for lacking the capacity to render interactive content built using Adobe's Flash platform, but the company shows no sign of reversing course.

Since the iPhone debuted in 2007 without any support for Flash, Adobe has begun a revitalized campaign to breathe interest in Flash. This includes the announcement of a new series of Flash 10.1 runtimes for Windows Mobile, Nokia S60/Symbian, Palm WebOS, and Android phones (but not RIM's Blackberry). This suggests not having Flash will be a problem for the iPad.

Adobe has also staged a regular conversation amplified by analysts and pundits that essentially claims Apple is unfairly restricting choice in the market by not supporting Flash on its iPhone platform.

Additionally, while Adobe says it is supporting open standards for the web related to HTML5, it still maintains that Flash is "critical to the web" while it also works to cement as much new content as possible into the proprietary mold of its Flash platform and the related Flex and AIR initiatives.

Will being Flash-free hurt the iPad?

Adobe's arguments for Flash are difficult to support in the mobile realm. The iPhone has been wildly popular since its debut despite its lack of support for Flash. Apple's smartphone dramatically raised the bar for what customers expected in a mobile web browser. By doing this without Flash, Apple essentially redefined what the web should look like, at least on a mobile device.

While a few mobile devices can render Flash content designed for desktop PCs, Adobe's original strategy for Flash on mobile devices prior to the iPhone was Flash Lite. This subset of the Flash runtime is based upon Adobe's old Flash 7 (MX 2004) and ActionScript 2.0 bytecode, which uses an entirely different ActionScript virtual machine than Adobe's newer Flash 9/10 (which use ActionScript 3.0 bytecode).

On the desktop, Adobe simply included two engines for running both old legacy Flash and more modern content. That's not really possible or desirable in a mobile environment where the Flash runtime is supposed to respect the device's limited processor and memory resources.

Adobe basically delivered Flash Lite as a way to say Flash was playable on mobile devices without actually doing the work of bringing a real Flash runtime to each mobile platform. The company has had a difficult enough time just supporting Flash on Windows PCs and Mac OS X at the same time, let alone Linux, the PlayStation 3's NetFront brewer, the Wii's Opera browser, and the top several mobile platforms.

Adobe hopes to roll out Flash 11 with support for ActionScript 3.0 bytecode across all desktop and mobile platforms (apart from the Blackberry and iPhone, iPod touch and iPad) soon, but the fact that it will be missing on the fastest growing mobile platforms and two of the most popular smartphone platforms will definitely be a problem, as developers creating content simply can't reach the top mobile platforms using Adobe's technology.

The iPhone's lack of support for Flash does not appear to have had any impact on its popularity, but clearly has played a significant role in devaluing the importance of Flash in mobile devices, even if other platforms are enthusiastically embracing Flash. At the same time, if developers on other platforms use Flash to reach those mobile audiences, they'll being doing that instead of creating native software for Android, Symbian, Windows Mobile, and so on. That will also benefit Apple, because it will keep its iPhone App Store well ahead of rivals.

[rogerdee123]

Inside Apple's iPad: Adobe Flash [Page 2]

By Daniel Eran Dilger

Published: 12:00 AM EST

Is Apple being unfair to Adobe and restricting choice?

A second issue being raised about the iPhone OS' intentional lack of support for Flash is whether platform vendors like Apple have the right to decide which software partners they want to support. Adobe's stance is that Apple should give its customers options, which means that the iPhone should include a Flash runtime just like nearly every other device that uses the web.

Interestingly, the history of Flash indicates that Apple isn't just persecuting it as a bully. If anything, Apple is just reclaiming its position in media delivery. After all, it was Apple that introduced video, animation, and multimedia on the desktop with QuickTime in 1991, back before Microsoft was even able to get reliable audio playback working across the spectrum of Windows PCs.

The Origins of Flash

Flash sprang from a tool called SmartSketch, originally conceived as a drawing app for handheld pen computing devices. It then moved to the Macintosh and PC to become a drawing tool competing against Adobe Illustrator and Aldus Freehand. In order to survive against those entrenched rivals, it morphed into an animation tool called FutureSplash Animator in 1996.

As the web started to emerge as a new platform of its own, FutureWave, the developers behind FutureSplash, first worked to create animations that could play on the web via Sun's Java, which was notoriously slow. When Netscape launched its own API for browser plugins, the company created its own native plugin for FutureSplash content. Apple had already delivered a similar plugin for Quicktime that enabled users to play videos and other content within web pages.

Microsoft's war on the open web

As the web gained in popularity, Microsoft began working hard to take over the medium to prevent it from competing for developer's attention in preference its own Windows platform. It worked to destroy Netscape via Internet Explorer, which tied the web to Windows; it partnered with Sun to sidetrack the company's Java and lace it with dependence on Windows; and it also attempted to clone Apple's QuickTime as a medium for delivering video.

Of its three primary foes related to the web, Microsoft was only unable to kill QuickTime. Part of the company's efforts to do so involved code theft from Apple related to the San Francisco Canyon scandal. This armed Apple with the legal leverage to aggressively bargain with Microsoft, and was a key element in getting Microsoft to reinitiate support for Office on the Mac after a long hiatus of focusing entirely on Windows apps.

At the same time, Microsoft also used its new power with Internet Explorer to invent compatibility issues with QuickTime and simply fail to load Apple's plugin when rendering web content designed for QuickTime. In 1996, Microsoft partnered with Disney, Macromedia and FutureWave to create animated content that replaced the open web with proprietary content. This resulted in Macromedia buying FutureWave and rolling the product into Flash.

Flash displaces QuickTime

Even as Apple's QuickTime was adopted as the container specification for the open MPEG-4 specification in 1998, Microsoft worked to use its monopoly position with Internet Explorer to widely distribute Flash as a proprietary way to deliver video and animations on the web.

Adobe, once a direct competitor to Macromedia and Flash and a key rival to the Microsoft-Macromedia alliance to oppose SVG as an open specification for web animation in preference to Flash, has since purchased Macromedia, primarily to obtain Flash.

After Flash became the dominant method for video playback, Microsoft started work on Silverlight, its own proprietary plugging for replacing open web standards with binaries dependent upon a web plugin. Apple, Google, and other companies supporting open web standards have worked to push HTML5 as an enhancement to the web to allow it to deliver multimedia without a plugin, with the browser itself rendering video via the MPEG-4 open specification.

In view of all this, Apple's opposition to Adobe's Flash isn't an attack on a popular plugin to limit choice, but really an effort to restore the use of open standards on the web, which creates a real marketplace for consumer choice. If Adobe were really interested in supporting open standards rather than being a gatekeeper wielding proprietary control over multimedia playback on the web, it could have opened up Flash just as it once did with PDF.

Instead, while making comments supporting HTML5 in general terms, Adobe CEO Shantanu Narayen has answered the question of how his company plans to deal with HTML5 by saying, "I think the challenge for HTLM 5 will continue to be how do you get a consistent display of HTML 5 across browsers. And when you think about when the rollout plans that are currently being talked about, they feel like it might be a decade before HTML 5 sees standardization across the number of browsers that are going to be out there."

[rogerdee123]

Inside Apple's iPad: Adobe Flash [Page 3]

By Daniel Eran Dilger

Published: 12:00 AM EST

Is Flash critical to the web?

Adobe would like to pretend that HTML5 is "a decade" away because this offers some window of opportunity for Flash to remain relevant. Apple has proven over the last three years that the iPhone and iPod touch could be wildly successful without Flash. That indicates no real problem for the iPad lacking support for Flash either.

Again, while Adobe claims vast licensing agreements and developer support for Flash, the only relevant content related to Flash is targeted to desktop users. Flash Lite doesn't even play that modern content. One side effect to Flash's desktop focus is that most Flash animations are not at all designed to scale down to mobile devices. Flash content is largely targeted toward a interface that assumes the use of a mouse pointer rather than a multitouch display, and the runtime is optimized for desktop-class computing resources, not the limited capacity of mobile devices that must remain idle as much as possible to preserve battery life.

Anyone who knows how to run Activity Monitor can observe that even the most trivial use of Flash within in a webpage eats up extraordinary resources. If Greenpeace were a legitimate environmental watchdog, it would target Flash as a bigger threat than PVC and BFRs combined, just by the composite amount of energy it consumes to do absolutely nothing of value.

Flash on the wane in video delivery

Flash is also losing its primary uses on the web. Most web videos used to be delivered encoded via On2 proprietary codecs within an FLV file (the proprietary native media container of Flash). Most future development is moving toward the open H.264 codec specification inside the MPEG-4 container (based on Apple's QuickTime container format).

Even Adobe has moved Flash to support H.264 video within its version of an MPEG-4 container, which it calls F4V. With that transition in progress, there's very little reason for anyone to need Flash just to deliver video, as Google is proving in its migration from Flash to H.264 in YouTube.

Flash and Rich Internet Applications

Adobe is also pushing Flash as a way to deliver interactive versions of traditional print content, the related Flex and AIR as a way to deliver Rich Internet Applications.

With the iPad, Apple is promoting to print developers and content companies the idea of using HTML5 instead, and simply avoiding paying a runtime tax to Adobe just to add interactivity to their content. Apple itself has been a big proponent of HTML5 and using JavaScript frameworks to create rich Internet apps, such as the MobileMe web apps it built using SproutCore and the interactive iTunes LP and iTunes Extras content it has launched in partnership with music labels and movie studios.

Google is also leading the development of rich web apps using HTML5, a strategy that is woven into its Chrome OS. Google employees have described Android's Java-like platform as a stop gap measure that will eventually be replaced by HTML5 web apps, rather than a long term platform in the sense that Apple describes its own Cocoa Touch iPhone OS platform.

Apple's preference for open over Adobe Flash

By not putting Flash on the iPhone, iPod touch and iPad, Apple is creating a significant installed base of affluent users who simply can't be reached via proprietary binaries like Flash and Silverlight. That has successfully shifted attention both to Apple's own App Store platform for mobile apps and to the open web, encouraging developers to embrace standards-based rich web apps and multimedia delivery based on open specifications.

In contrast, while Google is also staunchly supporting open standards on the web, it's also trying to support Flash playback as a feature of Android, something that can only make developing native Android apps less attractive. Microsoft is similarly trying to promote Flash and Silverlight while also supporting legacy Windows Mobile apps dependent upon a stylus. Nokia's Symbian and Linux platforms also embrace Flash at the expense of their own native development.

Apple appears to recognize that the more platforms its competitors support, the better it is positioned in its lead as the top App Store for mobile devices. All of which should leave users with zero hope for ever seeing a Flash runtime on the company's iPhone OS devices.

[rogerdee123]

Quoted from "The Dairy of an x264 Developer" .... http://x264dev.multimedia.cx/?p=292

02/22/2010 (3:05 pm)

Flash, Google, VP8, and the future of internet video

This is going to be a much longer post than usual, as it’s going to cover a lot of ground.

The internet has been filled for quite some time with an enormous number of blog posts complaining about how Flash sucks–so much that it’s sounding as if the entire internet is crying wolf. But, of course, despite the incessant complaining, they’re right: Flash has terrible performance on anything other than Windows x86 and Adobe doesn’t seem to care at all. But rather than repeat this ad nauseum, let’s be a bit more intellectual and try to figure out what happened.

Flash became popular because of its power and flexibility. At the time it was the only option for animated vector graphics and interactive content (stuff like VRML hardly counts). Furthermore, before Flash, the primary video options were Windows Media, Real, and Quicktime: all of which were proprietary, had no free software encoders or decoders, and (except for Windows Media) required the user to install a clunky external application, not merely a plugin. Given all this, it’s clear why Flash won: it supported open multimedia formats like H.263 and MP3, used an ultra-simple container format that anyone could write (FLV), and worked far more easily and reliably than any alternative.

Thus, Adobe (actually, at the time, Macromedia) got their 98% install base. And with that, they began to become complacent. Any suggestion of a competitor was immediately shrugged off; how could anyone possibly compete with Adobe, given their install base? It’d be insane, nobody would be able to do it. They committed the cardinal sin of software development: believing that a competitor being better is excusable. At x264, if we find a competitor that does something better, we immediately look into trying to put ourselves back on top. This is why x264 is the best video encoder in the world. But at Adobe, this attitude clearly faded after they became the monopoly. This is the true danger of monopolies: they stymie development because the monpolist has no incentive to improve their product.

In short, they drank their own Kool-aid. But they were wrong about a few critical points.

The first mistake was assuming that Linux and OS X didn’t matter. Linux is an operating system used by a tiny, tiny minority of end-users, yet those users make up a huge portion of the world’s software developers and web developers. Merely going by user count suggests that Linux isn’t worth optimizing for; accordingly, Adobe allocated just one developer, one single developer, to the entire Linux platform. And in terms of OS X, Macs have become much more popular in recent years–especially among that same group of developers. Furthermore, Apple is a huge company; Flash performing terribly on their platform is a very good incentive for Apple to want to position themselves in opposition. Thus, Adobe made enemies of Apple and developers alike.

The second mistake was attacking free software. Practically all the websites on the internet use free software solutions on their servers–not merely limited to LAMP-like stacks. Youtube, Facebook, Hulu, and Vimeo all use ffmpeg and x264. Adobe’s H.264 encoder in Flash Media Encoder is so utterly awful that it is far worse than ffmpeg’s H.263 or Theora; they’re practically assuming users will go use x264 instead. For actual server software, the free software Red5 is extraordinarily popular for RTMP-based systems. And yet, despite all this, Adobe served a Cease&Desist order to servers hosting RTMPdump, claiming (absurdly) that it violated the DMCA due to allowing users to save video streams to their hard disk. RTMPdump didn’t die, of course, and it was just one program, but this attack lingered in the minds of developers worldwide. It made clear to them that Adobe was no friend of free software.

The third mistake was not supporting a free software Flash implementation. The lack of a good free software Flash client is not really Adobe’s fault; it has become clear that the Gnash folks are completely incompetent and nobody else seems interested. Cody Brocious wrote his own Flash rendering code in a matter of days for purpose of a Flash->iPhone app converter; he only stopped because Adobe released their own mere days before he had intended to release his. The Flash spec is open, and there are existing free software implementations of every single codec in Flash: there’s really nothing stopping a good free implementation. But Adobe’s mistake is one of inaction: they didn’t push for it because it wasn’t important to them.

By comparison, look at Moonlight, the free software implementation of Silverlight. Microsoft has actively worked with the free software community to help produce Moonlight. Think about how absurd that sounds; Microsoft–the bane of free software, if one goes by Slashdot–has been actively supporting an LGPL free software project, while Adobe has not! The biggest problem this creates is one of monopoly: people feel insecure using Flash because there is only one implementation, leaving them at the mercy of Adobe. In any situation, once there are multiple popular implementations of a file format, it’s far more difficult for any one party to commit abuse. Of course, this is intentional by Adobe: they wanted to have that power of abuse, which is why they didn’t support an alternative implementation.

Now it becomes clear why Flash is so disliked. It’s nowhere near the most insecure of popular browser plugins; Java has had far more vulnerabilities according to Secunia. It’s certainly not the least reliable, nor is it completely proprietary; as previously mentioned, the spec is public. Yet because of the above three mistakes, Adobe has made enemies of developers worldwide.

So, what now? Flash is crap, we hate Flash, but how do we get rid of Flash, at least for purposes of internet video?

Let’s start with HTML5 <video>. It’s quite clear that, barring an act of God (or Google, more on that later), if Flash is replaced in the near future, this will be how. But at the moment there are many serious problems, most of which must be solved for it to even have a chance:

1. Missing features. Developers who haven’t worked with Flash often underestimate its capabilities and assume that displaying video is as simple as displaying images. But there are many things that are useful to control. Flash lets you tell the client how long to buffer before playing a stream (critical for reliable playback of any live video). It provides signalling back to the server of packet loss rates, so that the server can throttle bandwidth accordingly.

There are dozens more; these are just a few. But this is the core problem mentioned at the start of this article, the problem that hit Adobe so hard: “believing that a competitor being better is excusable”. Many free software advocates promote HTML5 while declaring that these missing features are not a big issue and that we can do without them. This is not excusable! If you want to outcompete Adobe, you need to provide a superset of all commonly used features.

2. Video/audio/container format issues. Theora is a seriously hard sell to most companies, given its compression is much worse than x264’s and H.264 has no royalties for web video; as before, it’s hard to market something with at most nebulous benefits. Being “patent-free” may sound nice to free software advocates like us, but most business types only care about the bottom line, and if being “patent-free” doesn’t benefit the bottom line, they’re probably not going to care. (NB: a commenter noted that H.264 is only royalty free for free content, not paid content. This probably is not a big issue, since the royalty % is very small for paid content, and if you’re charging for content, you can probably afford to pay the small fees. But it obviously is a slightly different situation.)

But even if you ignore the compression issue, most companies don’t like storing multiple versions of every video–they still need H.264 for iPhone support. As a side note, Dirac is a potential patent-free option as well, and may provide better compression, but is slower to decode than Theora. It’s definitely an option to consider though, and one which is way too often ignored when considering formats for HTML5 video.

Youtube, for example, has thrown away petabytes of bandwidth in the pursuit of fewer versions of each video: the default “low quality H.264″ format, which now uses x264, is Baseline Profile-only. Providing a High Profile alternative could save them 30-50% bandwidth for desktop users, which make up the vast majority of Youtube users. But it would require storing yet another copy of the video (since Baseline is needed for iPhones), which is too costly for them. Duplicating each video again would require there to be some serious benefit to doing so–and Google apparently believes that a 30-50% compression improvement is not sufficient, though there seems to be something weird going on with the 360p/480p madness they recently unrolled (neither is High Profile though).

Of course, despite having $50 million dumped on their doorstep each year by Google, Mozilla will never pay the H.264 license fees, nor will they probably ever support users installing their own codecs. Thus, we are at an impass. If Microsoft supports HTML5 <video> in IE9, which is quite possible, they will almost certainly support H.264 and probably not Theora. Thus, even ignoring the case of mobile devices like the iPhone, neither H.264 nor Theora will span the whole market. So which will web companies pick? Most likely, neither–they’ll see the split market as a reason to avoid HTML5 altogether, and drop back to Flash.

3. Ubiquity. Flash has the 98% market installation base on its side, a powerful force. Until Internet Explorer becomes a low-popularity browser (unlikely in the near term) or supports HTML5, Flash simply won’t be replaced. Furthermore, this effectively forces websites into using H.264: if they want to support both Flash and HTML5, using Theora would force them to store two redundant copies of the video, since Flash can’t play Theora.

4. Quality of implementations. Existing HTML5 implementations range from “bad” to “atrocious”; despite years of developers ragging on Flash, many of the existing implementations are still far slower than native media players, use terrible pixelated scaling (esp. Chrome), are outright buggy, or some combination of the above. Not only are the implementations often bad, but they’re inconsistently bad! Even if some work well, it does no good if many others don’t.

With all these problems, HTML 5 <video> looks to be in serious danger despite its promise. And this brings us to the main topic: what about Google, On2, and VP8? If, as the FSF frantically pleads, Google opens VP8, what problems does it solve and what problems does it create? And what benefits would this bring Google?

VP8 solves the compression problem: while still probably not as good as x264 (see the Addendum at the end for more details on this prediction), the gap is far smaller than with Theora, enough so that compression is far less of an issue. But it also brings up a host of new problems.

1. A few years ago, Microsoft re-released the proprietary WMV9 as the open VC-1, which they claimed to be royalty-free. Only months later, dozens of companies had come out of the woodwork claiming patents on VC-1. Within a year, a VC-1 licensing company was set up, and the “patent-free” was no more. Any assumption that VP8 is completely free of patents is likely a bit premature. Even if this does not immediately happen, many companies will not want to blindly include VP8 decoders in their software until they are confident that it isn’t infringing. Theora has been around for 6 years and there are still many companies (notably Nokia and Apple) who still refuse to include it! Of course this attitude may seem absurd, but one must understand who one is marketing to. One cannot get rid of businesspeople scared of patents by ignoring them.

2. VP8 is proprietary, and thus even if opened, would still have many of the problems of a proprietary format. There may be bugs in the format that were never uncovered because only one implementation was ever written (see RealVideo for an atrocious example of this). There will be only one implementation for quite some time; Theora has been around for 6 years now and there’s still only one encoder. Lack of competing implementations breeds complacency and stagnates progress. And given the quality of On2’s source releases in the past, I don’t have much hope for the actual source code of VP8; it will likely have to be completely rewritten to get a top-quality free software implementation.

3. It does nothing to solve the problems of hardware compatibility: most mobile devices uses ASICs for video decoding, most of which probably cannot be easily repurposed for VP8. This might be less of a problem if they’re targetting software implementations though; while it would eat more battery and be limited to mobile devices with powerful CPUs, it would not be unreasonable to play back VP8 on a fast ARM chip (see the Addendum for more on this).

The big advantage of VP8 is that it solves a problem that is unsolvable for Theora: Theora is forever crippled by its outdated technology and weak feature set. With state-of-the-art RD and psy optimization, as in x264, Theora can likely become competitive with Xvid or even maybe WMV9, but probably not x264. The only way to fix this would be a “Theora 2″, and attempting to ensure Theora’s “patent-free” status while adding new features would be extraordinarily difficult in today’s software patent environment. VP8, on the other hand, offers an immediate jump to what is hopefully an H.264-comparable level of compression.

But now for the big question: why would Google want to open VP8, and if they did, how would they do it? Google probably doesn’t pay a cent in license fees for Youtube; H.264 is free until at least 2016 for internet distribution and encoder fees only apply if you have more than 100,000 encoding servers. The cost of the license fees for Chrome are minimal (a few million dollars a year, capped). But despite that, there are actually some very good reasons.

1. Control. Google may view the control of other companies over H.264 as a threat: even though H.264 is licensed under RAND terms (Reasonable and Non-Discriminatory, they legally cannot be anti-competitive), there are many reasons for Google to want more control. If they push VP8, they not only give compete with Flash via HTML5, but they also prevent Flash from playing their video streams. As it is unlikely (for the reasons mentioned at the start of the article) that Adobe will immediately jump ship to VP8, this creates a window of opportunity for Google to steal control from Adobe.

2. Blitzkrieg. The most risky, but most powerful thing Google could do is switch Youtube over to exclusively VP8 and roll out a new browser plugin to play it (but support HTML5 if available). Given Youtube’s popularity, this would likely get them 80%+ install base in a matter of a month or two; effectively a “blitzkrieg” targeting Adobe’s market share. This would be powerful because it wouldn’t rely on waiting for existing browsers (especially Internet Explorer) to switch over to VP8.

3. Trump card. Google may be worried about the future; if H.264 does succeed in eliminating all competition in the web marketplace, it would be quite possible that MPEG-LA would attempt to abuse their position and start charging fees for web usage. Perhaps MPEG-LA needs a good “scare” to make sure they never consider such a thing. Software monoculture is dangerous.

These seem like good enough reasons, albeit somewhat insidious ones (especially 2), for Google to release this attack. Do we really want Google having this much control? I’m not sure, but it’s sure as hel_l a better option than Adobe. Will it actually happen? Quite possibly; the only other sane purpose of an acquisition for $100 million would be to acquire patents to use as leverage in patent lawsuits. Would it succeed? Depends on how they do it and what other companies they rope into their plan. It also depends on what their target is: would they try to push hardware support too?

Where does x264 fit in all this? H.264 is certainly not going away, not for quite a while. In most sane parts of the world, software patents are a non-issue. But in the end, none of it matters for x264: we will continue our quest to create the best video compression software on Earth. Unlike Adobe, we don’t sit complacent when we are the best; we keep trying to become better. We add new features, improve compression, support new platforms, improve performance, and there’s far more to come. We don’t care that many H.264 encoders are so bad that they can be beaten by Theora or Xvid. We don’t care if VP8 comes out; that’s just another encoder to beat. We are here to ensure that the best choice is always free software by constantly making free software better.

But, of course, we wholeheartedly support the quest for royalty-free, free-software multimedia formats. There are many use-cases in which being free of patents is more important than compression, quality, performance, or even features. Bink Video is a staggeringly popular example: used in tens of thousands of games despite having compression 10 times or more worse than modern video formats–almost entirely because of its royalty-free (albeit proprietary) nature. If the day comes when Bink is replaced by a free software alternative, we will know the quest for a widely-accepted, free software, patent-free video format has succeeded. Until then–I wish luck to those pursuing such a goal.

Addendum: VP8’s feature set and compression capabilities

Many people have been wondering what the reality behind VP8 is, behind the usual marketing bullshit that these sorts of companies put out. As there is no public specification and even the encoder itself still isn’t released, this is all an educated guess based on what information I do have.

VP8 has been marketed in press releases as basically an “improved VP7″, primarily with the intent of being faster to decode in software on mobile devices, especially those without SIMD (e.g. ARM11). Thus, it is likely reasonable to start approaching VP8 by commenting on VP7. VP7 was released in ~2003, around the same time as H.264. It made waves due to being dramatically better than practically all H.264 encoders at the time. The reason wasn’t that VP7 was better than H.264, but rather that On2 had a far more mature codebase: they had been developing their encoder for years, while most H.264 encoders were slapped together in the months following the finalization of the specification. However, VP7 never caught on because it was completely proprietary; nobody wants to rely on a proprietary codec anymore. Over the years, as far as I can tell On2 never updated VP7 and the best H.264 encoders, like x264, moved well ahead. VP7 was mostly forgotten except for a few apps like Skype that licensed it.

Now let’s look at VP7 technically. While I don’t know too much about the internals, VP7 is notable in relying very heavily on strong postprocessing filters. This is not unique; practically all On2 codecs have this. Even Theora has an optional postprocessing filter that it inherited from VP3 in addition to its in-loop deblocker. On2’s postprocessing filters usually fall into three categories: deblocking, sharpening, and dithering/grain. The dithering filter is useful for avoiding blocking in dark and flat areas, similar to the effects of the gradfun mplayer filter. The sharpening filter helps compensate for the natural blurring effect of the quantizer in encoders that are not very psychovisually optimized. The deblocking filter is also notorious for blurring out tremendous amounts of texture and detail (example: vp7, x264). But this also provides a significant advantage: by moving many features of the codec into postprocessing, the video format becomes scalable; a decoder can do “less work” while still playing back the file, albeit at a lower quality. This doesn’t work if all the steps are mandatory.

Since VP8 is marketed as less complex than VP7, it likely still does not contain arithmetic coding, B-frames, or other computationally intensive features. We know from marketing material that one of the big promotions was it allowing the encoder to pick between interpolation modes to allow faster decoding if necessary. Clearly, VP8 is big on speed–which means they likely have not added a lot of new compression-related features. If it improved greatly over VP7, it would most likely be due to psychovisual optimizations in the encoder. But given their last press releasing showing a “comparison with x264″, it’s clear that they haven’t done this. Their “VP8″ image is a blurry disaster with nearly no detail at all, as opposed to the artifacty-but-detailed x264 image, which actually looked better to many commentors at Doom9, despite the obviously staged test.

Overall, I expect VP8 to be surely better than MPEG-4 ASP (Xvid/DivX) and WMV9/VC-1. It will likely be nearly as good as Mainconcept’s H.264 encoder (one of the best non-x264 encoders), but assuming they still believe that blurring out the entire image is a good idea, probably still significantly inferior to x264.

Update: According to gmaxwell, a Theora dev, this seems quite likely: “What I’d heard from ex-on2 folks was that there is some philosophical disagreement about how to optimize [encoder] tuning, and the tune for PSNR camp mostly won out.“ Apparently around the time of VP6, On2 went the full-retard route and optimized purely for PSNR, completely ignoring visual considerations. This explains quite well why VP7 looked so blurry and ugly.

If there’s anything to take away from this, it’s that psy optimizations are the single most critical feature of a modern video encoder. They’re the reason that Vorbis beat MP3 for audio, and now they’re just as important for video.

[bubba]

I for one would be very happy if flash died and stopped being used in entry pages to websites. It's just a gimmick. I can't count how many times I have wanted to pick up some quick information from a site, only to be faced with a "loading" message and wait for some silly animated graphic to download on a slow connection when all I want is some information. Thai websites seem particularly bad about this, and ironically so since so many users are on slow connections.

Die flash, die!

[rogerdee123]

Quoted from Apple Insider 16 March 2010

"In addition to new App Store software, National Public Radio and The Wall Street Journal also plan to create specific versions of their Web sites completely devoid of Adobe Flash for iPad users.

...... .....The news comes weeks after Virgin America revealed it dropped Flash content from its new Web site in order to allow users with iPhones to check in for flights."

[rogerdee123]

CBS.com Prepping HTML5 Video Playback for iPad. ( macrumors.com dated 25 March)

[rogerdee123]

Quoted from MacRumors.com dated 9 April 2010

Daring Fireball notes a very specific change in the iPhone OS 4 SDK that will directly thwart Adobe's efforts to directly compile Flash applications onto the iPhone. The new terms dictate the following:

Applications must be originally written in Objective-C, C, C++, or JavaScript as executed by the iPhone OS WebKit engine, and only code written in C, C++, and Objective-C may compile and directly link against the Documented APIs (e.g., Applications that link to Documented APIs through an intermediary translation or compatibility layer or tool are prohibited).

This seems to go directly against Adobe's plans to release Flash Professional CS5 that would have allowed developers to export native iPhone applications from Flash. Adobe had hoped to provide this compatibility layer to allow their Flash developers to write once and then deploy to multiple platforms.

Adobe has acknowledged the change to the New York Times, but doesn't have any change in plans just yet.

"We are aware of Apples new SDK language and are looking into it. We continue to develop our Packager for iPhone OS technology, which we plan to debut in Flash CS5."

[britmaveric]

mactards not allowing flash is so annoying.

[negreanu]

Flash is a notorious resource hog, CPU and memory on both Windows and Mac. Not to mention it's instability and crashes.

The best thing that could happen is websites all change to HTML 5 which is coming thanks to Apple.

Windows users should be grateful

[rogerdee123]

Well .... as expected ..... it looks like Adobe is waving the white flag !

quote ........ Not only has Adobe launched its own HTML5 video player, added HTML5 export tools to Adobe Illustrator and Dreamweaver CS5, but it has now released an experimental Flash to HTML5 conversion tool, codenamed Wallaby, over on Adobe Labs. Wallaby, currently at Prerelease 1, is an AIR application that allows designers and developers to convert Adobe Flash Professional (FLA) files into HTML5 with a simple drag and drop.

Adobe says it is not yet able to commit to a roadmap for the experimental technology since Wallaby is still in the testing and validation phase. That being said, the company really wants you to try it out.

"Adobe invites customers to download this tool, try out the code it generates and provide feedback on how they are using it," an Adobe spokesperson said in a statement. "With more than 3 million Flash developers in the creative community, Adobe continues to look for new ways to help them build on their existing skills and to make their content available to the widest possible audiences. User response to the Wallaby technology preview will enable Adobe to better understand what types of innovations are needed in our long-term investments in both Flash and HTML5 technology."

Back in October 2010, Adobe showed off an impressive demo of the tool. In fact, Adobe says Wallaby is being released on Adobe Labs in response to the tremendous amount of customer interest after it was first demoed.

The tool allows Flash developers to easily reuse graphics, masks, and animations from their Flash projects in an HTML file. This process previously took hours to do by hand. The tool also warns you which elements can't be converted, like animated masks, filters, and ActionScript.

The tool doesn't generate the best markup, but the ability to export your animations out of Flash to HTML, even if the final code needs some clean up, will certainly be appreciated by many developers. Flash isn't going away anytime soon but many will want to move their Flash content to HTML5 so it can run on devices that don't support Adobe's plug-in, including iOS devices like the iPad, iPhone, and iPod touch. ........ unquote

NOTE: The above was taken from an article posted on "TechSpot.com"