/Word stream

In my day-to-day doing development at Firstborn, it’s pretty common that we run into questions that don’t have a very clear answer – for example, what would be the best technique to do something (in terms of performance).

When something like that happens, depending on the time constraints, we may either do some quick test, or just go with what we assume is best – creating a theory and sticking with it. At times, however, a certain issue becomes so important it requires a more extensive research – having the scientific method applied to it, so to speak.

I’ve ran into one of those recently with a website we’ve just created – Expedition Titanic. In that website, we have a few videos running in quasi-fullscreen (taking the whole website space, that is) and we were having trouble with performance in some computers – dropping frames, not playing smoothly, etc.

In previous projects, we had some assumptions of how full-site videos should be encoded – resolution, bitrate, codec, etc. Most of those were based on actual experience, but it was scattered information without any real data to back it up.

Video playback performance was becoming more of an issue in every website I’ve been working on – this is also true of the previous website I’ve developed here, for the 5 REACT campaign – and I decided to stop relying on myths, personal theories or word-of-mouth and do some real tests – rather, benchmarks – to actually come up with the best solutions for video playback in Flash in terms of performance.

My question was not about the video quality itself, but rather its playback quality – how smooth the video frames were being played, how many frames were being dropped; in sum, what was the impact of different encoding methods on playback performance. We generally assume F4V (H.264) videos have better quality, but which one requires more from the CPU to decode? What’s the impact of actual resolution and bitrate?

I have to confess I had some personal beliefs I wanted to put to the test with these benchmarks. My theories were as such:

• On2 VP6, (FLV) videos are faster to decode than H.264 (F4V) videos.
• H.264 decoding is faster on multi-core machines; On2 VP6, not as much.
• Overall, OS X has even worse decoding performance for H.264 videos.
• Videos with higher bitrates are slower to decode and this impacts the rendering speed of high-quality videos.
• Video playback performance in Linux is horrible.

Still, I approached the problem with a neutral stance and would be as happy proving as I would be with disproving any of those theories. The main reason why I wanted to do the test in the first place is because I wasn’t really sure any of them were really true, but I still used them as guidelines when creating content.

Measuring video playback quality

My initial plan to measure the playback quality was using the (Flash 10 only) droppedFrames property of the NetStreamInfo object and the (undocumented) decodedFrames property of the NetStream object. That way, I hopped to be able to measure the number of skipped frames and measure the time spent decoding in a more rendering-agnostic fashion.

After some initial tests, however, I decided against it because the values I hot were just not reliable enough: while the behavior of both properties was predictable for FLVs (the sum of decodedFrames and droppedFrames would be the number of total frames in the video), their values while playing an F4V video were quite undecipherable; there seemed to be a correlation between playback quality and the value of both properties, but their actual relationship is a mistery – for example, does droppedFrames contain the number of frames dropped during decoding, or during decodoing and playback? Does decodedFrames contain the number of frames decoded, or decoded and rendered? The numbers state different things at different times, giving that either property can have a higher value than the other, and the fact that they seem to behave differently depending on the choice of video format.

Due to this, I’ve decided to try and have a 30fps video running inside a 60fps SWF, and use the final SWF speed (measured by onEnterFrame events) indicating how much time was being spent with video decoding and rendering. Nevertheless, the number of frames decoded and dropped are still included in the final results.

Test rationale

The idea of the test was finding out the best combination of encoding variables for proper video playback, so I decided to use these parameters:

• Format: FLV (On2 VP6) and F4V (H.264)
• Bitrate: 500Kbps, 1000Kbps, 1500Kbps, 2000Kbps
• Resolution: 1920×1080, 960×540, 480×270

This gives a total of 24 different encoded files for the same video. They would all be rendered at the same size (taking the whole browser space); the idea is that it’d allow me to have a better understanding of how resolution, bitrate and format changes impacted the playback quality, with the same area being drawn.

The file used was a rendering of the intro to the Expedition Titanic’s “Explore” area, originally rendered at 1920×1080 and set at 30FPS.

The testing project was a small SWF file that would load a file, play it, unload it, and skip to the next file. The SWF was set at 60fps, using maximum quality; nothing else was present on the staging area (the idea was to test video decode speed without compositing tradeoffs), and the video was scaled to fit inside the full browser area (similar to the Stage scaleMode of StageScaleMode.NO_BORDER). The executing SWF was kept focused during the whole test.

I used On2 VP6 as the sole codec for the FLV videos. Sorenson Spark, while considered to have better decoding performance, is usually not a parameter on the video encoding decision equation anymore due to its sub-standard encoding quality.

For the FLV encoding, I used Adobe Flash Video Encoder (CS3) using a normal encoding (1-pass). For the F4V encoding, I used Adobe Media ENcoder (CS4) with a “High” profile, Level 4.1, and encoded with 2-pass VBR where both target and maximum bitrates where the bitrate being tested. And since the idea was testing raw playback quality, as opposed to accurate seeking ability or any other parameter, keyframe placement distance was kept as automatic on both.

The videos were encoded without an audio track.

Results

Here are the test results, plotted in line charts. The question these numbers try to answer is: when playing a 30FPS video on a 60FPS SWF, what’s the actual rendering framerate?

The test was ran on a number of different computers, from low-end to high-end, Windows PCs and Macs, multi-core and single-core machines. The idea was not measuring performance of each setup against each other – they had different specs, and were ran at different resolutions – but rather to see the differences between different videos being played on the same machine.

“Slow Windows” was an old Dell laptop running Windows XP SP2 at 2GHz (Intel single core) with 1gb memory. Tests used Firefox 3.6.8 and Flash Player 10.1.82.76.

“Medium Windows” was a desktop running Windows XP SP3 at 3GHz (Intel single core) with 4gb memory. Tests used Firefox 3.6.8 and Flash Player 10.1.82.0.

“Fast Windows” was a desktop machine running Windows 7 64bits at 3.33GHz (Intel Core 2 Duo, 2 cores) with 4gb memory. Tests used Chrome 6.0.472.51 (beta) and Flash Player 10.1.53.0.

“Fast Mac” was a 27″ iMac running OS X 10.6.4 at 3GHz (Intel Core 2 Duo, 2 cores) with 4gb memory. Tests used Safari 5.0.1 and Flash Player 10.1.53.64.

“Slow Linux” was a laptop running Ubuntu 10.04 64bits at 2GHz (AMD 3500+ single core) with 461MB memory. Tests used Firefox 3.6.8 and Flash Player 10.1.82.76.

Result analysis

Some points can be drawn from the results:

F4Vs have better playback quality

Simply put, H.264 decoding is faster than On2 VP6 decoding, so F4V videos easily give best results in terms of overall playback performance of a video on a Flash website . This holds true for both single-core and multi-core machines, and for every platform tested (Windows, Macintosh, Linux). On2 VP6 actually uses a new thread on multi-core systems for better performance, but apparently this is not enough to be faster than the (usually) hardware-accelerated H.264 decoding.

Adding to the best performance the (visibly) better quality of H.264 encoding, F4V becomes an obvious choice for video encoding in Flash. My recommendation is that F4V videos should always be used instead of FLV videos, except when a feature supported by FLVs is needed (transparency channel; legacy Flash versions support; cuepoints; odd video dimensions).

H.264 doesn’t have any special impact on OS X machines

F4V video performs just as well on a Macintosh, not being particularly taxed in relation to comparisons with any other system. Specially with the newly added hardware acceleration, there’s doesn’t seem to exist any valid concern with how the decoder performs in this platform.

Actual bitrate doesn’t have that much of an impact on performance

While technically not true – a higher bitrate/quality does mean a lot more data to process – the actual impact of video bitrate in overall performance was near negligible in my tests. In the case where one needs better performance for a video, it makes more sense to lower the resolution than to lower the bitrate.

Linux performance is not that bad

Simply put, my tests on a very low-end machine produced results that were easily watchable. They would drop considerably below the target framerate of 30fps, but considering the target machine, and a comparison to my low-end Windows machine, it does feel pretty good overall. Not fast, but not too terrible either.

Conclusion, or TL;DR

H.264 (F4V) videos are better performance-wise. Use them when possible.

Additional notes

You’ll find the full results (with data for every video tested) here. The actual benchmark is here if you want to test if yourself (not recommended at all, as it takes at least 31 minutes to complete, and downloads 240mb of videos – and you can only see the results once it’s done).

Scaleform has announced that Scaleform GFx, their Flash-based solution for graphical user interfaces for games, is set to be included free of charge with the Unreal engine.

They’ve made a nice video of the thing in action, going all the way from the Flash IDE to a new menu and interface for Unreal Tournament III. The video brought a smile to my face — try to find out why:

The funny thing? I don’t remember whether I’ve said this publicly before, but Tweener was initially planned as an UnrealScript extension – I needed some tweening extension for a project I was working at the time (the UI for the Defence Alliance 2 mod for Unreal Tournament 2004), and a MC Tween-like syntax wouldn’t work. In the end I dropped the idea since I would have not enough time to complete it for the mod, and instead took it over for AS2/AS3 a little bit later. It’s super nice seeing it going all the way around and finally being used in the game by way of Scaleform GFx – I don’t think game UI systems have anything nearly as practical as the tweening concepts Actionscript has had for a few years.

Of course, any other tweening extension will probably work within Scaleform GFx, so it’s not like Tweener is any kind of bundle; it was merely what they were using in their example.

Pretty bold title, uh?

Before I even begin, let me say two things. First, this is a pretty obvious use for the new Flash 10 drawing API, so I’m quite sure somebody has created something similar already, specially because it’s something I can see a lot of people in need of. Second, please read the entire post before commenting, because it may not be exactly what you think it is.

So getting straight to the point, I’ve created a drawPlane function that uses the new drawTriangle() method in Flash 10′s ActionScript 3 native APIs to draw a correctly distorted plane in 3d, based in four bidimensional points. Other people would call it distortImage or something similar. Click below to check an example:

By now, you’re probably thinking, “hey, I’ve seen this before”. There’s this, this, this, this, and probably many others. So why another one?

Well, there’s one special (and blunt) reason why: because they’re all wrong.

Here’s a shitty image I created a few years ago to illustrate my point:

Subdividing a plane into a bunch of triangles using their bi-dimensional distances doesn’t give you a projected plane. It gives you a good approximation, and I guess the calculation is easier, but try distorting a plane like that by any amount and you’ll see what kind of deformity it generates.

Secondary to that, I wanted to use the new Flash 10 capabilities to project planes without a lot of hassle (meaning, without having to rely on other libraries or actual 3d position) and, most importantly, by using two triangles only for the entire plane – with no subdivisions of any kind.

Now, don’t get me wrong. There were reasons for (correct) triangulation before Flash 10: Bitmap fills had to rely on a non-distorted fill, so adding more triangles created more accurate results. However, Flash 10 allows AS3 developers to set the distortion factor of a Bitmap fill by using the new drawTriangle() method and its UVT parameters, achieving just that – a perfectly projected plane using two triangles only.

But here lies another problem. While finding the projection is pretty trivial if you’re working with three dimensions (the T portion of the of UVT combination is just a function of the Z of each point), things get a bit more confusing when you’re dealing with points set in two dimensions – points defined by their X and Y only. And that’s exactly what I was going to need, since this is all supposed to be used in a project I’m creating at Firstborn where we’ll be combining video renders with Flash-drawn content over the video at specific positions, and I don’t have any kind of 3d information about the points, just anchor points matched to the video. To put it another way, I wanted to have the same ability to move corners as you have in Photoshop’s Free Transform tool.

I honestly could not find an answer to the problem online. I’m pretty sure there’s some complex way that makes a lot of sense to project 2d points into a 3d field, and therefore find the T of each point, but instead, I was lucky enough to get what I wanted working after a good amount of crazy trial and error (and beer). It actually made the solution very small and much simpler than I expected.

So without further ado, here’s the actual implementation. It’s a pretty simple method that draws a BitmapData to a Graphics instance and sets the perspective accordingly. The semi-secret is the weird diagonals-based ratio that’s calculated for each corner, giving the final plane render uncanny precision.

package com.zehfernando.display {
	import flash.display.BitmapData;
	import flash.display.Graphics;
	import flash.geom.Point;
	/**
	 * @author zeh
	 */
	public function drawPlane(graphics:Graphics, bitmap:BitmapData, p1:Point, p2:Point, p3:Point, p4:Point) : void {
		var pc:Point = getIntersection(p1, p4, p2, p3); // Central point

		// If no intersection between two diagonals, doesn't draw anything
		if (!Boolean(pc)) return;

		// Lengths of first diagonal
		var ll1:Number = Point.distance(p1, pc);
		var ll2:Number = Point.distance(pc, p4);

		// Lengths of second diagonal
		var lr1:Number = Point.distance(p2, pc);
		var lr2:Number = Point.distance(pc, p3);

		// Ratio between diagonals
		var f:Number = (ll1 + ll2) / (lr1 + lr2);

		// Draws the triangle
		graphics.clear();
		graphics.beginBitmapFill(bitmap, null, false, true);

		graphics.drawTriangles(
			Vector.<Number>([p1.x, p1.y, p2.x, p2.y, p3.x, p3.y, p4.x, p4.y]),
			Vector.<int>([0,1,2, 1,3,2]),
			Vector.<Number>([0,0,(1/ll2)*f, 1,0,(1/lr2), 0,1,(1/lr1), 1,1,(1/ll1)*f]) // Magic
		);
	}
}

import flash.geom.Point;
function getIntersection(p1:Point, p2:Point, p3:Point, p4:Point): Point {
	// Returns a point containing the intersection between two lines
	// http://keith-hair.net/blog/2008/08/04/find-intersection-point-of-two-lines-in-as3/
	// http://www.gamedev.pastebin.com/f49a054c1

	var a1:Number = p2.y - p1.y;
	var b1:Number = p1.x - p2.x;
	var a2:Number = p4.y - p3.y;
	var b2:Number = p3.x - p4.x;

	var denom:Number = a1 * b2 - a2 * b1;
	if (denom == 0) return null;

	var c1:Number = p2.x * p1.y - p1.x * p2.y;
	var c2:Number = p4.x * p3.y - p3.x * p4.y;

	var p:Point = new Point((b1 * c2 - b2 * c1)/denom, (a2 * c1 - a1 * c2)/denom);

	if (Point.distance(p, p2) > Point.distance(p1, p2)) return null;
	if (Point.distance(p, p1) > Point.distance(p1, p2)) return null;
	if (Point.distance(p, p4) > Point.distance(p3, p4)) return null;
	if (Point.distance(p, p3) > Point.distance(p3, p4)) return null;

	return p;
}

Inside a DisplayObject, use it like so to redraw a BitmapData instance using 4 given 2d points:

drawPlane(this.graphics, myBitmapData, topLeft, topRight, bottomLeft, bottomRight);

You can download the source for the example editor above from here: TriangleTest.zip. Warning: it’s a Flash 10, AS3 classes-only example. There’s no FLA file and no project settings file. The structure should be fairly simple to understand, however; the class roots are at /src/src (actual source) and /src/libs (a few additional files used). Just compile /src/src/documents/TriangleTest.as as your document class and you should be good to go.

Anyway, I like it. It can probably be a bit optimized (specially the getIntersection function, or the many Point.distance() calls) but it works well so far – don’t forget it’s only dealing with 4 points per plane, so there’s no excruciating loop that begs for optimization.

If you still think this was done before – Flash 10 drawTriangles() based image drawing, with two triangles, and correct perspective distortion – please post a link on the comments and I’ll correct the article. I still think the title and the premise of the post is bold, but so far, I’m sticking to it.

Finally, do notice that this is not a 3d engine. It’s just a quick way to draw a plane when you only know the 2d points of the corners. There are no 3d points, vertices, or angles of any kind involved here. If you want to rotate a plane you already have, it makes a lot more sense to just rotate it using the native rotation properties, or use any 3d engine of your preference.

PS. No, the cat is not mine. It’s Don Citarella‘s Floyd.

Roughly 17 years ago, I was in love with QuickBasic. After a while using GWBasic (not to mention years of using BASIC on other systems, such as the Brazilian Sinclair ZX81 clone TK-85, the MSX 1.0, and the Apple II), QuickBasic was a real milestone in my life. It was when I changed from using line numbers and a pretty mangled structure in my programs (using subroutines with GOSUBs and GOTOs) to a more organized one with functions and procedures. I also loved how easy it was to debug inside the Quickbasic 4.5 IDE – you could stop your program at any time, do changes to your code, and then continue running. Programs were interpreted, and while speed wasn’t the best compared to other platforms available for similar DOS machines, immediate execution, easy debugging and the ability to compile executables later made it the platform of choice for all the small programs I liked to create – database merging systems, small graphical or text-based games, and general tools I used on my daily work.

QuickBasic 4.5 (click for source)

But even though I used QuickBasic for most of my stuff, I also used other languages like C and Pascal at high school. For the latter, my IDE of choice was Turbo Pascal. And it’s within that development environment that I had my first run-in with Turbo Vision.

Turbo Vision was, in a nutshell, a framework to create user interface elements for Turbo Pascal programs. It allowed you to easily create menus, buttons, dialog windows and the alike – something that was still pretty rare at the time (considering DOS programs were never really about the user interface). Being able to quickly construct a program with such a high-level interface without doing everything from scratch was quite a shock to me.

Screenshot of "Quadra", showing what a Turbo Vision interface looked like (Click for source)

An even bigger shock was understanding that you could actually create common libraries or frameworks and reuse them on different projects. As obvious as it is today, my self-taught understanding of program structures at the time hadn’t grasped this concept yet. Turbo Vision itself was beautiful, but the idea of Turbo Vision was, for my little coder mind, revolutionary.

So after having my first experience with Turbo Vision, I set out to create a similar framework in QuickBasic that I could use for my own programs (while I enjoyed the speed and correctness of languages like C and Pascal, not to mention the speed, I enjoyed the agility I had with QuickBasic even more).

The result was a bunch of subroutines that could be called to draw and control everything. You could create screen elements like buttons, editable text fields, windows, check boxes, radio buttons, menus and the like, with full tabbing/focusing/clipboard support, and control them using a higher-level API. That allowed me to add great interface elements to my simple programs quite easily, without having to worry about rewriting it every time I needed to create something new. The implementation was somewhat ugly, since QuickBasic would run in a single thread and I had to move execution from one element to another and deal with the return result to know what to do, but it was clear enough in a way that allowed me to reuse elements with certain ease. I still have that code somewhere around my computer.

Fast forward to 2003, around a decade later – after I had abandoned QuickBasic, used a few other platforms and languages, until finally settling with development for the web. While I’m working with ActionScript and developing some visual components for a website, I had an epiphany: I was doing the same thing I had done 10 years before.

The operating system was different: Windows, instead of DOS; the language was not QuickBasic anymore, but ActionScript; I wasn’t building a text-based application, but rather one that used graphical user interface elements and the mouse; I wasn’t reading data from the disk, but from some asynchronous network resource. But still, I was doing the same thing: creating graphical elements that were meant to be reused.

A decade before, I had no idea I was going to be working with Flash (since that didn’t exist), or even be building those kinds of interfaces. Of course I had no idea I would be building something meant to be used over the Internet either. And I had even take some pretty odd routes along the way of my earlier career, focusing in graphic design and animation instead of programming, but still, there I was, full circle, back doing what could be said to be basically the same thing I had been doing at high school, only in an updated environment.

And while I’m not really that old, I like to see this as a great example that no technology lasts forever, and that programmers can’t have any idea of what they’ll be working 10 years later. And, perhaps more strongly now, that while the environment they’re using might be completely different, the concepts they’ve learned before will carry on. In a nutshell, your experience is not the language you use, but what you do with that language.

I guess it doesn’t take much to see how that applies to the issue I’m seeing people debating today – “Rich Internet” platforms like Flash, SilverLight, HTML 5, and a few others. I’ll be the douchebag who quotes himself and just paste what I had to say a short while ago in my FWA interview:

Q. Do you think Flash is here to stay?

A. In a way, most definitely. Flash was the first software platform to allow developers to easily create rich interactive animations for the everyday user, so whatever Flash has brought and is still bringing is something other platforms are playing catch up to adopt.

But on the other hand, you can never predict what the future will be like. When I started working with interface development, Flash didn’t exist and I couldn’t even predict I’d be working with something like it 15 years later.

I love Flash, but after having worked with it for more than a decade and seeing it radically transformed so many times, it’s as if the name doesn’t matter that much anymore. I think what we call RIA development is the thing that’s here to stay and that’ll just become bigger with time, but the name under the hood becomes almost irrelevant. Even if it’s Flash and ActionScript, it’s going to be a different Flash and a different language in a few years. With similarities, but then again there are similarities between ActionScript, Java, and C#. Being attached to that kind of brand isn’t very healthy I guess

Here’s something a lot of people forget – we’re not in this business because we’re using language XYZ. While a lot of people like to draw that kind of conclusion – because, after all, their experience is usually focused on a single language, not to mention the need to identify with a certain group – the market itself changes over time and it’s natural for one language to replace another as the language of choice for a specific task. Remember when Perl was the de-facto server-side language for the web? If you considered yourself a “Perl specialist”, instead of a server-side programmer, you’d be stuck when people started to migrate their systems to PHP and others.

I see the current state of rich interfaces development the same way. We’re doing Flash today – because it’s widespread, the tools available (official or not) are stable enough, it has an awesome community, and it’s generally easy to develop for. What are we going to use tomorrow? I don’t know, but the knowledge I’ve gathered over the years – developing the interface elements I’ve been using, learning about asynchronous data loading and user interface issues – will carry over, regardless of the new platform I’m using.

Now, there’s a lot that could be said against HTML5. Performance issues, compatibility, much slower user penetration, lack of features, how it’s only adding the same features and problems people criticized in Flash before. All has been said and, I’m sure, will be said as soon as HTML5 start to become a real contender and its flaws become more obvious in a real world scenario.

But you know what? It doesn’t matter. Because HTML5 will either work out or it won’t. And, as a developer, it’s not my job to root for one specific side. It’s my job to absorb whatever there’s to be absorbed and move on. Be it HTML and JavaScript, be it ActionScript, be it Silverlight, I’m pretty sure the higher-level knowledge I’ve gathered through the years that are outside the syntax realm will carry on.

I can’t be sure of what I’ll be working with in 10 years. And neither can you.

One of the interesting things about working at Firstborn is that from time to time employees will host “Group Therapy” sessions, which are nothing other than presentations on diverse topics (read more about the idea behind this in this great article by former Firstborn employee Jens Fischer).

A week or so ago I had the nerve to host a presentation of my own. It was called “Lessons learned maintaining Open Source ActionScript projects” and it was, quite obviously, all about things I learned while publishing Open Source projects – namely, MC Tween and Tweener.

Now, I’m not really an OS guru and much less a proper OS project manager, but I think there were some interesting experience points gained in those quests that I wanted to share so I decided to make it publicly available. Check it out – I promise it has some interesting links mentioned:

The presentation is available on Slideshare, or as a direct PDF download.

This was also presented at the (awesome) FlashCodersNY weekly meeting of October 14th.

One year ago, in October 2008, Flash Player 10 was publicly released. At the time, due to some major early adoption, I predicted it’d reach a 90% browser penetration after 6 months.

It turns out I was a bit too optimistic, but in a way, the prediction now doesn’t seem to be as crazy as it was when I first wrote about it. The reason? The new statistics are out, and it still managed to beat the adoption rate of all other major versions of the player quite handily.

At 11 months after release, it has a 93.5% penetration for mature markets, having reached the oh-so-important 90% penetration rate in about 9 or 10 months. Not bad; can we start developing for Flash 10 only now?

All historical data is collected on my shared spreadsheet.

Adobe Max 2009 will finally start in a few weeks. For the past few months, I’ve heard small bits of information about the future of the Flash platform, often complemented by someone from Adobe saying we’d have more information at MAX – making the event sound more and more important each time around.

Ted Patrick’s recent tweet corroborates with this:

I am suprized more folks haven’t dug into the MAX session scheduler. The future of Flash Platform is there. http://bit.ly/pFz34

So what is the future of the Flash Platform? Well, apart from the usual stablished/educational content, this is what caught my attention from the list:

• GPU acceleration for the next Flash Player: known, but may only apply to mobile devices.
  • GPU Acceleration in the next Flash Player: Learn what’s possible with GPU acceleration in the next version of Flash Player. NVIDIA and Adobe collaborated to develop GPU acceleration that enables Flash Player to run fluidly on battery-powered mobile devices that are changing the face of computing. See how GPUs will change your perceptions of what is possible on mobile and how you can create content that taps into this significant opportunity.

• The next version of AIR (AIR 2) will have different distribution methods
  • Explore Deployment and Distribution Options for Adobe AIR Applications: Learn how to get your AIR applications to your users and how to keep them up to date. We will discuss important considerations for distribution on the Internet or an intranet, including impacts on your auto-update mechanism. We will cover existing deployment options such as badge installation and IBM Tivoli support. Finally, we will explore the new deployment options that will be available in Adobe AIR 2, including the native installer support required to use some of the advanced new AIR 2 APIs.

• Flash 10 and AIR are coming to mobile platforms
  • Preview: Flex for Mobile Devices: Hear directly from the team involved in enabling Flex on mobile devices regarding the challenges faced and progress made in optimization, components, and tooling. In this session, you’ll see Flex running on smartphones and learn how to create your first mobile Flex application.
  • Building Mobile Applications with Adobe AIR: Learn how Adobe is working to bring Adobe AIR development out of the desktop and onto a mobile phone near you. We will cover how the AIR SDK and platform will evolve to add capabilities to help developers mobile enable, test, and publish their content. Mobile computing and mobile applications provide publishers and developers with exciting opportunities to get their products into the pockets of millions of people.
  • Designing Applications for Desktops and Mobile Devices with Adobe AIR: See how Adobe AIR has made it possible for web developers to build cross-platform desktop applications. Learn how the platform is expanding to enable the delivery of applications not just for desktop operating systems, but also for mobile devices.

• More information on Flash Player for TVs and other embedded devices
  • Flash for the Digital Home: Flash on TV!: Discover Adobe’s optimized implementation of the Flash runtime that enables the delivery of HD video and rich applications to Internet-connected TVs, set-top boxes, Blu-ray players, and other devices that are connected to the television. Learn about the platform and supported features, and view some sample TV applications that highlight these features. This session should inspire content providers, developers, and designers who want to take advantage of Flash in the TV space.

• Multi-touch: this sounds like it means using third-party multi-touch devices, something that’s already possible (instead of a native API, which would be real news), but should still be interesting
  • Multi-touch and the Flash Platform: Discover what you can do with multi-touch and the Flash Platform. Learn how you can build your own multi-touch table for a fraction of the cost of commercial products. We’ll also discuss an approach to building multi-touch applications for the Flash Platform on these tables that will open up an entire new world of possibilities not only for experimentation, but for your customers and clients as well.
  • Multitouch Development with Flex: Explore new techniques for interacting with software and devices with the latest capabilities of the Flash Platform.  Multitouch experiences are taking the software world by storm, from the 2008 election night coverage to the latest bars and clubs. This session will explore the multitouch capabilities of Flash Player and demonstrate how you can take advantage of them within your own applications.

And other presentations that should be of note and will hopefully be released into the wild post-MAX:

• Roadmap: Flash Platform Runtimes: Come see the roadmap for the Flash Platform runtimes: Flash Player and Adobe AIR. Learn about what the runtime teams are working on today for delivery soon, and hear how Adobe envisions the future direction of the runtimes.
• Top Secret Flash Player Session: Sorry, we can’t tell you what this one is about. If we told you we’d have to… But we will say that this is one session you won’t want to miss. More information will be available on-site at MAX. And now that you have read the description, the session will self-destruct.
• What’s Coming in Adobe AIR 2: Adobe AIR allows developers to build rich Internet applications (RIAs) that run outside the browser on multiple operating systems. In this session, you will learn about the planned capabilities of the upcoming release of Adobe AIR 2.

More on the Adobe Max Session Catalog. Should be really interesting.

Update: Ted Patrick himself has posted his pick of the most interesting sessions at his blog.

Nowadays it’s often that I find somebody talking about something or doing something that I wish I was doing myself – but that I can’t for a number of reasons (one of them being the fact that I’ve just moved and I don’t have a real computer and Internet connection at home). The latest installment in this pattern is Eric Decker’s great posts about creating a 3d engine of his own for educational purposes, where he goes through the steps he’s taking to create such an engine in Flash. A nice read and some interesting demos.

As a side note, working with such talented people – Eric is also at Firstborn – makes me feel like such a fraud.

Another post that has been a long time in the making.

From the start. Until a few years ago, whenever I needed some kind of animation added to the Flash-based work I developed, I used an ActionScript tweening extension I created called MC Tween. This extension was created strictly according to my own tastes; other great tweening extensions already existed at the time, but I didn’t particularly like any of them, as I considered the API for most of them too abstract for what I needed them for. MC Tween used a prototype-based approach, adding new functionality to already existing Object types such as MovieClips. Apparently a few other people agreed with my syntactical approach to programmatic tweening, so it ended up enjoying quite a popularity for a while.

In 2005, however, I started growing increasingly irritated with MC Tween. While it managed to do its job well, ActionScript syntax and paradigm changes (specially the ones introduced with AS2), coupled with my better understanding of object-oriented programming,  made me realize it didn’t quite fit my development workflow very well anymore. What’s more, MC Tween development had begun to grow increasingly stale, and due to its popularity, any sort of experimental feature change was too dangerous to even be considered. It was sort of stuck.

So in june of that year, I started an experiment, creating a new, class-based, tweening extension. Initially, it was instance-based (you created instances of tweens), but it was later changed to a static class that created all your tweenings through a single method call, addTween(), and handled additional features through a few additional methods like stopTween(), getTweens() and such.

The class went through a lot of changes, like having its own name changed (initially called Twina, then Tweener, then ZTweener, then Tweener again), its package moved around (from generic.*, to zeh.easing.*, to caurina.transitions.*) and its API changed a lot, not to mention changing from the instanced approach to a static approach, and other big internal modifications. I think the lesson here – and this is honestly the point of most of this post – is that when you create something for yourself, knowing that no one is going to look or judge the output, much less use it on their own mission-critical work, you allow yourself to be a lot more wrong simply because you’re the only one who has to accept the risk; but, in the process, you find better ways of doing something.

At the time I started this project, Fuse Kit was becoming the most popular tweening extension out there. This was something I did not want to compete with, and I specially enjoyed the freedom of creating something in an iterative fashion, for myself, and changing it to fit my needs on every new project. That’s why Tweener remained under wraps (used only by myself) for quite a while, even though I was using it on every project I had to build for Gringo (at the time).

Tweener was “made public” on January of 2007, almost two years after its first version was created. Nate Chatellier had just joined the team – and created the AS3 version of Tweener – and I thought it wasn’t fair to keep it under wraps forever. And at that point, the API wasn’t likely to change a lot, so input would be appreciated.

Tweener wasn’t something so out of this world, and its API wasn’t anything too brand new either, since the kind of loose approach it took with new tween definition could already be found on several other existing tweening extensions – even AS1-based extensions. Still, it was probably the right syntax at the right time – as it also managed to be quite popular for a while, being even ported to other languages.

That apparently was 4 years ago (judging from the logs, Tweener was started on June 15, 2005). Other tweening extensions exist today – maybe specially popular are Go, TweenLite/TweenMax, GTween, and the new BetweenAS3 – and you know what? It’s that time again. Even if it has a modern AS3 version available, many of the paradigms adopted Tweener are pretty old-fashioned now. I’ve been growing increasingly discontent with it, I’ve learned more, and it’s time to move on.

I could just move on, but I want to have a look back at what worked and what didn’t. Consider this a Tweener post-mortem – my analysis of what I’ve learned these past 4 years and how has ActionScript changed for libraries like Tweener.

So. When moving from AS2 to AS3, there’s one important thing we need to consider: errors, in AS2, are pretty silent. You can try changing properties of a null object and Flash Player will happily oblige. Sure, it won’t work since the object doesn’t even exist, but it won’t give you any sort of error message.

This creates a problem because when you expect something to work tightly, you need several levels of verification to see whether the data you’re getting is what you expect it to be, so you can warn the user otherwise. With Tweener, under AS2, I tried to play it safe for the user: more precisely, I’d check whether objects existed on every update, whether the property being tweened already had a default value.

It turns out that with AS3, this is an unneeded feature. If you try accessing objects that don’t exist, or properties that are not there, the execution of your code will break in all kinds of ways. And this is good - it helps the user understand he’s making something very, very wrong, and that he must fix it.

When ported to AS3, however, Tweener carried many of these AS2 paradigms, so it still has a lot of verifications of that kind. Try to tween the value of a property that doesn’t exist and you’ll see a custom Tweener error, not some Flash Player error.

if (rScopes[0][istr] == undefined) {
	printError("The property '" + istr + "' doesn't seem to be a normal object property of " + String(rScopes[0]) + " or a registered special property.");
}

The same thing happens on every iteration of the property update loop – it checks whether the object being updated exists, whether the property exists, etc. Which is to say, Tweener tries too hard. The side effects of this approach are decreased performance and increased file size.

Other compromise that probably has ties with this AS2 way of distrusting the API user is that, internally, Tweener tries to play as safe as possible, giving the user little room to break stuff. One of the biggest examples of this resolution is that, when you create new tweens, Tweener searches for similar tweens (tweens applied to the same object and same properties with overlapping time) so they can be deleted. While this may make sense, since it’s the expected behavior of an animation afterall, this creates two problems – first, it has never been an option, as the overwriting is always enforced. And second, it introduces a huge loss of performance when creating new tweens, specially if you already have too many tweens in place, because of all the tweens and objects it has to check for overlaps first.

I’ve long been fond of this safer approach, but with increasingly complexity of ActionScript, and increasing performance you can tap into as long as you allow yourself to walk on the edge, it becomes too much of a compromise. In the end, it’s better to make things moderately safe, but giving the user the ability to circumvent issues that may arise with default use.

That is all to say, nowadays I believe tweening overwriting should have been off by default, but offered as a feature (not surprisingly, this is the approach that classes like TweenMax take).

Comparing performance of tweening engines recently made this problem more obvious. While it still holds moderately well with actual tween updating time and pretty well with memory consumption, Tweener has a much harder time the higher the number of tweens it has to create; tween creation time gets exponentially higher, since it checks each new tween against the entire list, and with tween numbers in the thousands it’s quite often that just trying to create the new tweens will take more time than the animation itself, just hanging the player for a while.

Now, Tweener uses a single array for its tween list, and this could be resolved by clever AS3-native tricks, like using a Dictionary instead (again, not surprisingly, this is the solution TweenMax uses, and Jack Doyle was actually the first to mention the technique to me). While doing this might have served as a stopgap, the problem still remains – Tweener just tries too hard to play it safe; to guide the user by the hand, giving him no room to explore if he just wants performance and knows what he’s doing.

That’s why, with today’s state of Tweener, I think this is not a question of optimization anymore. Stopgaps could be made, like the removal of a few checks, the addition of some modern AS3 features like a Dictionary for the list, and heck, maybe even Vectors, but I honestly believe that, by design, Tweener is already past its prime, so those changes would be just that, stopgaps.

So it makes no sense to update Tweener anymore. Sure, it works (specially if you don’t need thousands of concurrent tweens), and will continue to work, but in terms of advancements, it has reached a dead-end. There are now better ways to do things – Tweener is close to that, I think, but because it doesn’t have a lot of room to maneuver, it’s best to just leave the project as it is and move on to something else.

But, here’s something.

While I won’t be changing Tweener anymore unless some new, hideous bug shows up, I’ve made a small update – a new version, 1.33.74, is available both at the Google Code project page and at the subversion server. This update keeps the default tweening overwriting, but adds a new parameter, overwrite (that, when set to true, enables the old behavior) and a new static property, autoOvewrite (that changes the default overwriting behavior). Why is that a good thing? Because it’s faster.

On the above graphic, the blue bar means no tween, the orange bar represents Tweener 1.31.74, and the yellow bar means Tweener 1.33.74 with overwriting set to false.

This is not much, but consider this my thank-you for the support, suggestion, lessons, and overall awesomeness I’ve received over the years due to this project. My most grateful thanks not just to people who helped the project more prominently, but to everybody who helped the project in one way or another.

And since this is some sort of closure, even though this doesn’t have much to do with the post, here’s an interesting graphic I’ve been assembling over the years since Tweener has moved to the Google Code website. It compares the percentages of download of stable versions of Tweener based on the version (see the full download count). It’s slow, but we can see how AS3 is slowly dominating the statistics. It doesn’t include the version released today.

Here’s the same graphic, but without normalization.

As a closing note, I must add that, personally, I’m using slightly different approaches to tweening than what was introduced by Tweener. While Tweener still works and will continue to work, my own workflow has changed slightly. I never use special shortcuts for color tweening and stuff anymore, preferring instead to create separate classes and specific functionality via getter/setters, decorators and the like. Should other people be doing that too? Heck if I know, but the reason why I work in things like Tweener in the first place is because I was happy to work on something that would fit my own development flow well, and making it available to other people later if they wanted it. You can even say it’s a bit of a naive thing to do, and that’s part of the reason why it took me so long to make it public, but I prefer to use something that’s tailored to the needs of a person (as long as I agree with that person) instead of something that’s tailored to the needs of everybody. That’s not to say any of the other libraries out there do this – I wouldn’t know – but rather to explain how is my approach to that kind of library and why I prefer to move on instead of keep hammering on something I don’t think has the best approach.

Nowadays, I’m using mixed solutions for tweening: Tweener for some stuff, an experimental tweening engine I’ve built which fits my workflow very well and that I can abuse when I want top performance (because it doesn’t try anything other that I want it to) for others, and a tweening engine built by the folks from Firstborn of which I’ll be contributing to (as soon as I get to the office – from next week on probably) and that I hope I’ll be able to talk more about the future.

To be clear, there won’t be a “Tweener 2″, or anything like that.

And thanks again!

I’ve just returned from EDTED (formerly EWD) São Paulo, where I gave the same presentation I gave last month at EWD Rio de Janeiro. Awesome event as usual, and I think my presentation was alright. Thanks to everybody who attended, it was a huge honor to be there. This post is more to let people know that the slideshow I’ve used can finally be found here (in Brazilian Portuguese). I also wrote some honest, personal impressions here (also in Brazilian Portuguese).