A mobile AIR application post-mortem

With Kana Mind released (even if still in beta form), I figured I’d talk a little bit about the experience of writing a mobile application using Flash (and Adobe AIR) as a platform (there’s also the weird timing of Adobe announcing they won’t be updating the Flash plugin for mobile devices anymore, focusing instead on AIR apps).

The application was written in “pure” AS3 (no components or frameworks used), using FDT 4, and compiled with Flex SDK 4.5.1.21328, and AIR 3.0.0.4080.  The project was mostly adapted from the built-in mobile templates provided by Powerflasher, but the iOS and Android project versions (and desktop) were merged together.

FDT 4 with the Kana Mind project opened

FDT 4 with the Kana Mind project opened

Even though the application is only available in the Android platform now, all versions were built together and use the same code. I just haven’t tried releasing an iOS version on Apple’s app store because I’m still adding features to it and fixing some bugs (as far as I know, Apple’s policy doesn’t allow beta applications on their store).

Interestingly, I’ve also been developing a native Android application at work for the past few weeks, so developing this small project allowed me some important insight on how using each platform compares. I’ll talk about it in parts.

Coding

The cool thing is that for an ActionScript developer like me, it’s pretty easy to just leverage your knowledge to another platform. You basically learn about the interface caveats and you’re good to go – continue building your mobile application as you would do with a normal website. It’s a pretty rapid development.

Kana Mind TO-DO notes

Kana Mind TO-DO notes

Comparing AIR to Android, things can be both good and bad. For example, it’s interesting how it is easy to create custom visual content in Flash – it’s all vector-based (you can resize anything), the display list is pretty easy to deal with. Other important point is that Flash is a pretty mature platform, and many of the most things you’d need in order to develop a proper, rich, user interface are already available in the platform, in the form of easy-to-use libraries, or built-in Flash Player capabilities (don’t get me started with using Events or XML loading in Android – Flash is heaven in comparison).

On the other hand, when developing for a native platform, you can take advantage of the built-in interface frameworks – buttons, scrollers, etc – that are already there and used by everyone. With Flash, you have to build them yourself, and given the differences in user interactivity patterns (e.g., I had to build a MobileButton class so I could have a LONG_PRESS event), you can’t just use anything that was developed for the desktop. I also ended up having to rebuild my editable textfield classes to take advantage of native features like StageText. Third-party libraries for all of that already exist, of course, but my point is that if you’re like me and you like to have complete control over your interface elements, expect to have to adapt them to that platform.

Developing the application probably took around 50 hours, on and off, split around my free time. No tablet version is available yet (I don’t have a tablet I can test on), but I believe doing so is trivial – just making sure things work and coming up with slightly different layouts based on screen size conditions (right now, the application works, but of course things are just resized and there’s no landscape mode).

Testing

One interesting thing is that testing is, in most ways, easier when you’re using AIR. For Kana Mind, I’d normally compile the application and run it as a desktop application – no emulator needed and, of course, very fast.

Kana Mind running as a desktop application

Kana Mind running as a desktop application (purple boxes are for debugging purposes)

Compiling and deploying to an Android mobile device was also pretty easy, since installation on a device can easily be done automatically after compilation.

The issue here is, of course, developing for iOS. Deployment isn’t automatic – you have to compile a new .ipa file and the sync it using iTunes. Luckily, though, I didn’t have to do that very often as the application simply worked the same in Android and iOS.

Some Kana Mind device testing

Some Kana Mind device testing

Finally, at least in my case (I wasn’t many mobile-exclusive features), testing on the devices wasn’t frequently needed; I’d usually spend a few hours coding and testing in the desktop, and then doing a round of tests on a device to make sure performance was OK, and that was it.

Adaptation

Flash developers are generally very well prepared for most fragmentation woes – they’re used to dealing with different screen sizes and CPUs. Doing mobile development introduces news challenges, however: you have to deal with screen density (rounded corners, text size, or element margins, for example, can’t just be pixel-based), and different input schemes (what to do if you want to make your application available to a device that doesn’t support touch screens?).

None of this is a showstopper, however. For screen density, I created a function that takes Capabilities.screenDPI (which is useless in desktops, but mostly correct on devices) into consideration and I use that instead of any actual pixel dimension.

Performance

Performance is one of those topics that are very difficult to discuss. People can easily have unrealistic expectations, especially from a platform that is being ported from the desktop to a mobile device. One can easily expect their code to work the same, forgetting it’s a completely different device – with a lot less processing power and memory.

With that being said, it’s easy to be critical of AIR (and Flash) performance in a mobile device. Obviously, it doesn’t work the same as in a PC. My general assessment, however, is that performance of a mobile AIR application is for the most part OK, but not great.

Overall, user interaction and screen updates work well. It is difficult for one to achieve, say, 60fps on an animation, but as long as you keep all your screen rendering in check (knowing when to use bitmap caches and when not to, how to avoid re-renderings of unnecessary elements, etc), it is easy to make a responsive application that works at around 30fps.

The negatives about the platform are the ones that pertain to points the developer has no control over. To me, these are especially true during the application initialization.

The problem for me is that regardless of what you have on your first frame, application initialization takes a long time – up to 6 seconds, on my experience – before any code is even ran or anything is displayed (this is probably especially true of the Android captive runtime, which I have used for Kana Mind). This is true even if you split the application in two frames, like you’d do with an online SWF, to offload code initialization and asset loading.

After the application is initialized, there’s also a few seconds where the application is unresponsive (doing some major memory swapping, I’m guessing), causing user interaction to be glitchy or not work at all.

A long startup time for applications is one of those things that can drive a user crazy. You click an icon and you expect feedback – when that doesn’t happen and you’re just starting at a black screen for too long, it’s easy to just assume the application has crashed or that it’s just some shitty code. It taints the user’s opinion of your application, regardless of what happens later.

Size

When it comes to mobile applications, size matters. Because of this, it’s important to remember the AIR captive runtime adds around 8mb (Android) or 5mb (iOS) to the application size, so it can be pretty bulky. Of course, this expands a little bit once it’s actually installed on a device – Kana Mind itself takes around 20mb of space once it’s installed in my Nexus S.

In Android’s case, if you choose to have a non-captive application instead, it works better – it’s basically your SWF size – but then it requires users to install the AIR runtime.

Conclusion

Regardless of the result, building an application using AIR is one of those things I really wanted to do, to get a chance to test the platform and find its pros and cons myself. As it turns out, there’s plenty of those, so it’s hard to get to a final verdict on whether someone should use it or not.

If you’re building something quickly or free and you want to use the same code on iOS and Android (including mobile devices and tablets) and potentially desktop, AIR is the way to go. You can have the same project (with maybe just some conditional layout or user interface elements) and, at least in my case, cross-platform testing is seldom necessary. It’s a very rapid development and testing workflow and, if you spend time building some user interface and application libraries to work around the caveats of the platform, you’ll be able to create cross-platform applications in the blink of an eye.

On the other hand, it looks like the platform has reached its limits in terms of performance and there’s not much that can be done (I’d love to be wrong, though). Mobile AIR applications are usable, but not exactly snappy, even if you take all the performance tricks out of the bag. The unavoidable truth is, if you want the best performance for your application, there’s no way around it: it has to be a native application.

I’m pretty proud of how Kana Mind turned out and I’ll probably do a few parallel applications for other languages, as I like how well the teaching algorithm works, and still use it as my excuse to play with AIR. But if I was developing a serious, bigger, commercial application, I’m not sure I’d have gone the AIR route.

TL;DR: AIR is good. But not for every case.

Update (27-February-2012): I just tested the same application on an Android 4.0.4 (Ice Cream Sandwich) phone and I have to say, it’s so much faster. My application, on this same phone (Nexus S 4G), used to get around 10-30fps when under Android 2.3.7, with frequent render stuttering. Although I don’t have a FPS counter to back me up, under the new version, it feels like it’s getting a fixed 60fps, with no stutters. It still takes a few seconds to start up the first time, but the fact that it’s much faster overall – with no tweaks like hardware acceleration – makes a strong case for AIR on Android phones. That’s not to say it’s a perfect solution; as of now, version 4 is still only available on 1% of the Android phones out there, and it’ll take 1 or 2 years until it’s a valid target version. It does, however, paint a brighter future than I expected for Adobe’s platform.

  • Bart

    Did you use AIR in GPU mode or CPU mode? It matters a lot, CPU mode is soooo slow, but in GPU mode you can do quite a lot of visual stuff if you keep it fully hardware accelerated (no scrollrects and so on). We do an iPad thing now and it’s amazing what you can do if you optimize, and just as amazing how a few simple mistakes can utterly destroy performance (scrollrect around whole game area.. better not 🙂

    One thing we found out that the ‘auto’ rendermode mode does nothing and sets CPU mode (yes, it’s stupid, auto seemed the smart choice right? Except for tiny line in a PDF somehwere that says it’s unsupported. 🙂

  • Zeh

    I’ve tested GPU mode and it was overall much, much slower. I’m using “auto” which, yes, means CPU mode in the cases I’ve tested.

    Speed for screen updates was ok, though. I pretty much only slide pages into view and game pieces in and out of view, and it’s ~30fps for those. Maybe it’d matter if I had a lot of different elements in screen that were rotating and whatnot. I’ll have to test that some time in the future.

    My issue was with initialization time and the freezes during the first few seconds. And for those, rendering mode didn’t matter.

  • Brent

    Awesome article – just the kind of AIR info I was looking for.

  • Gil Amran

    Hi,
    I just finished developing a game (Memory Cards) and I walked this exact path…

    Size -> Too big. Nothing you can do about it
    Load Time -> Too long, I might have a solution… (Investigating)
    Speed -> Just OK…
    GPU -> Worse than CPU. I hope Molehill for mobile will do the job.

    Coding, Debugging, Design, and Cross platform -> Pure joy!!!

    Verdict: Maybe in the near future… it will be THE BEST way to do TRUE cross platform rich application.

    Gil

  • Bart

    Have you tried the ‘GPU rendering diagnostics’? It’s a flag you can use to let the debugger give you information on how the GPU is rendering your content (it draws colored overlays over each element), it pretty usefull.

    Optimizing can be a bit (huge 😉 of a hassle (figuring a strategy, doing all the cacheAsBitmapMatrix stuff, ditching vector animations and so on), but it pays of big time. And if something is wrong it gets slower then CPU (like when it needs to upload cached bitmaps to GPU every frame, you’d not want to have that). And the limit on like shadows and so on are a bugger. But transforms on bitmaps that are allready on the GPU are BLAZING fast.

    On startup: agreed, that’s not very pretty.

    I heard Adobe is considering if it might be doing a Captive Runtime that excludes features you don’t use (less bytes in the download en faster start) but it’s not clear if/when they’d do it.

  • Zeh

    Thanks Bart, that’s important information. I didn’t know about the GPU rendering diagnostics. For games with a lot of moving parts it’d probably make more sense to use GPU and to optimize rendering by investigating piece by piece. But since Kana Mind was working ok on CPU mode, I decided not to go through that route after some brief tests with GPU and cacheAsBitmapMatrix (which was giving me mad glitches – maybe because dimensions were not power of two? not sure).

    There’s one small game I want to try developing, and I’ll probably use that as an excuse to try the GPU mode instead.

    @Gil: you summed up this whole article in one brief comment. Bravo. 🙂

  • You’re wrong zeh 🙂
    http://esdot.ca/site/2011/fast-rendering-in-air-3-0-ios-android

    GPU Mode is an absolute beast on Android and iOS if you know how to use it properly.

    My new game runs at 30FPS on everything, easily is 60FPS on iPAd 2. You could run 1/2 a dozen of those tile games at once, at 60fps if you do it right.

    Trailer here for context:
    http://snowbomber.ca/#!prettyPhoto%5Bgallery2%5D/0/

  • Also not sure about your startup times, but something is wrong… I get about 2 seconds on a moto zoom or nexus one, immediate launching on Playbook and iOS (with splash screens… about 3-4 seconds from there, but this is normal for native apps too).

    Make sure you’re doing nothing major in your document class, or app for a couple frames… I usually have a startup command with a shot setTimeout call to actually bootstrap the app.

  • Zeh

    Thanks for the link and article Shawn, I’ll investigate it and try it next time.

    Like I said earlier, I’m not doing anything crazy on startup. The ~6 seconds is before my code even starts, and there’s nothing crazy embedded. I could have an empty SWF and the result is the same (I’ve tested), and the player VM doesn’t even start either (e.g. a getTimer() executed first thing on my app will trace things like 164ms, even though it has been ~5-6 seconds since the application actually started).

    What I know about it is that the Nexus One is normally faster than the Nexus S (god knows why), and that being a captive version makes it take longer too.

  • No worries, it’s like the best kept secret out there. If you build from the groundUp with this in mind, it’s actually super easy to do, just takes a tiny bit of re-thinking.

    Weird on the startup times… in general I find them quite good. You’re not compiling from Flash Pro are ya?

  • Zeh

    No, pure SDK project with no additional libraries.

  • Hi Zeh!
    Thanks for this post, I’m going through basically the same issues and feelings myself.

    Regarding the loading time, at least on iOS you get the ‘default.png’ to display while the app loads. I wish something like this was available for Android too.

    I hadn’t heard about FDT until now, have been using FlashDevelop for years now and find it really great. If by any chance you tried it too I would be interested to have your opinion, like which one you’re more comfortable with…

    Thanks again anyway 🙂

  • Zeh

    iOS loading times have been better, so not too concerned about it.

    I have tried FB many years ago. Before Switching to FDT, it was my code writing editor of choice. Haven’t used in a while though; I know it has changed a lot so it’s probably pretty good.

  • Hi,

    You probably don’t remember me, but I created GTweener (then Actuate), and was a long-time fan of Tweener (and still consider the core principles of “set it and forget it” reliability/functionality key to library development).

    I saw another “Flash is dead” blog post, so I decided to go jumping around to see what everyone was up to. How do you think things would compare if you were using NME instead of AIR?

    I was thinking of making my next application (not a game) using NME. The thing you lose is that workflow from FLA -> SWC, but you gain a lot in performance. I’ve been thinking about doing some kind of interface editor, but I can’t decide if it would be better to have a standalone GUI editor or to have something which exports from Flash using a JSFL script.

    Anyway, I would be interested to hear your thoughts

  • Zeh

    Hey Joshua, I do remember you!

    I honestly have no idea how that would compare to NME. Frankly I didn’t know about NME until just before writing this comment, and it took me a while to find it. Sounds like a good concept, but I have no real way to judge. If anything, though, I was curious and installed “Ponon! Deluxe” on my Nexus S and it seemed to work pretty well.

    AIR on mobiles provide a good workflow, but the big negative point in my opinion is really the performance (plus bad captive support on Android as a minor point). I bet some of that can be optimized, but I can’t see the starting time shrinking without big engineering changes on the runtime side.

  • I was working for a Facebook game company, building the engine for a new isometric RPG game. The design and look was kind of similar to the first Diablo game — isometric perspective, without the boxy feel, and everything rendered from 3D Studio Max. It was a cowboy game, and turning pretty well.

    The CTO thought that supporting iOS as well as Flash might be a good idea, so we decided to use NME to build the engine, knowing we could switch back to AS3 (or just continue to publish to Flash through haxe, without mobile support) if it didn’t work out. I was really surprised how much I was able to throw at it.

    When day, a couple months into the project, I decided to give it a try on my phone (which was a Palm Pre at the time). Would it work? I was able to run the game using Flash and using NME on my phone, and I was blown away — while Flash ran at 6 FPS, NME was running a solid 30 FPS — the whole game. Obviously we would have wanted to create a custom UI to handle a screen size that small, but I was really impressed.

    Regarding performance, Iain Lobb made a Flash benchmark called “BunnyMark”, which Philippe Elsass tested in AIR and got 600 bunnies, 30 FPS, in AIR. We converted BunnyMark to haxe and tested with NME, and Philippe got *4000* bunnies at 30 FPS. Here’s the whole blog post, if you’re interested: http://philippe.elsass.me/2011/11/nme-ready-for-the-show/

    I think its great for games, but I’ve thinking more lately about what would make it easier to use for applications. I made an extension for SQLite support, but handling the workflow in creating the interface is the thing that keeps me pondering

  • Wait … I just read it wrong. I think he used an HTC Desire, not an HTC Desire S, so that would make it 1500 bunnies instead of 600.

    I also know, however, that the new test he’s running involves rotation, scale and transparency. If transparency had been involved with the original test on AIR, I’m sure that would have hurt the 600 bunny score a lot.

  • Pingback: Zeh Fernando » Blog Archive » You should use to-do lists()

  • Pingback: Zeh Fernando » Blog Archive » Using Adobe AIR to create OUYA games()