Seeing shades of the future

Other than fixes to broken features and other small things, there are two huge features I see in the future of Flash: hardware-accelerated 3D support, and pixel shader support. This dream became closer to reality yesterday, as Adobe announced some of the new features of Flash Player 10 (codenamed “Astro”) at Adobe Max 2007 (see a video here).

What caught my attention – other than the “real” 3d for planes – is the fact that now they’ll have their own language for pixel shaders.

Codenamed Hydra, and apparently created by these guys, the language is the basis of Adobe Image Foundation, Adobe’s new image processing platform, to be supported by their whole line of products. With its C-like syntax, it’s also analogous to other shader languages, such as OpenGL’s GLSL and DirectX’s HLSL, so shader programmers will feel right at home.

However, what’s cool about it is that Hydra will allow developers to write their own shaders and execute them inside the Flash Player, as they were executing normal filters such as Blur and Glow. By running Hydra code at the lowest level possible, developers will be able to use the available computer power at its best, so this opens a huge new era for graphic effects inside Flash movies, and one that has already been known for many people specially on the game development industry.

The big question, however, is whether there will be hardware acceleration on Hydra shaders. The presentation mentioned running those shaders with “the performance of native filters that are already in the Flash Player”, with no mention of GPU support for either; the LLVM documentation mentions Hydra can run both at CPU and GPU level, and while the AIF toolkit can only be executed on computers with a recent GPU (Pixel Shader 3 is required), it doesn’t mention whether specific products such as Astro will make use of the GPU acceleration path or not.

Regardless, even if it only means software support, a whole new field will open for Flash developers. Several people have already been experimenting with the toolkit, so I saved some time to test it out and port some of the shaders and C++ code I had been creating for some college work. Here’s a sample, before and after the shader is applied:

Adjustable threshold with Hydra

This employs an adjustable threshold – unlike a straight threshold filter, you can set the black and white threshold, and everything in-between those two values is featured as a grayscale point – that ends up allowing for a kind of anti-aliased threshold, or a very strong levels adjustement.

Here’s the source in Hydra:

kernel AdjustableThreshold {
    parameter float blackThreshold <
        minValue:float(0);
        maxValue:float(1);
        defaultValue:float(0.4);
    >;

    parameter float whiteThreshold <
        minValue:float(0);
        maxValue:float(1);
        defaultValue:float(0.5);
    >;

    void evaluatePixel(in image4 src, out pixel4 dst) {
        float4 inputColor = sampleNearest(src, outCoord());
        float brightness = (inputColor.r + inputColor.g + inputColor.b) / 3.0;

        dst = inputColor;

        if (brightness < blackThreshold) {
            // Below threshold
            dst.r = dst.g = dst.b = 0.0;
        } else if (brightness > whiteThreshold) {
            // Above threshold
            dst.r = dst.g = dst.b = 1.0;
        } else {
            // Between the threshold
            dst.r = dst.g = dst.b = (brightness - blackThreshold) / (whiteThreshold - blackThreshold);
        }

    }
}

The black threshold must always be lower than the white threshold.

Astro’s support for Hydra has some limitations, however, so it can’t have IF statements or loops. Some of it can be worked around with inline selections, so the code below do the same thing, but it seems to be Astro-compatible:

kernel AdjustableThreshold {
    parameter float blackThreshold <
        minValue:float(0);
        maxValue:float(1);
        defaultValue:float(0.4);
    >;

    parameter float whiteThreshold <
        minValue:float(0);
        maxValue:float(1);
        defaultValue:float(0.5);
    >;

    void evaluatePixel(in image4 src, out pixel4 dst) {
        float4 inputColor = sampleNearest(src, outCoord());
        float brightness = (inputColor.r + inputColor.g + inputColor.b) / 3.0;

        dst = inputColor;

        float grayscale = (brightness - blackThreshold) / (whiteThreshold - blackThreshold);
        float ifAboveBlack = brightness > whiteThreshold ? 1.0 : grayscale;
        dst.r = dst.g = dst.b = brightness < blackThreshold ? 0.0 : ifAboveBlack;
    }
}

Maybe this is not the best example of AIF's new capabilities, as it's not just about color manipulation, and there's a bit of unneeded overhead on how the colors are calculated (should use a lookup table instead of calculating the color for every pixel). However, what's cool about this is that I simply ported my code directly from a quick HLSL test I had, and it works pretty much the same, with just a few adjustments needed for syntax (and working around the IF/loop issue, which I admit was a let down).

Anyhow, max will have an AIF-specific presentation tomorrow (wednesday). Hopefully by then we'll be able to answer, with 100% certainty, whether Astro will use GPU-accelerated hydra processing or not.

  • Mac├írio

    there’s a hardware accelerated implementation of the flash engine done by scaleform. it’s mostly used by console game developers to make interfaces, ti’s also integrates with some high-end game engines like unreal3 and gamebryo, so it’s license prices must be very high. you can see more about it here: http://www.scaleform.com/products_gfx

    anyway, it’s great news for flash game developers that they will be able to natively use the GPU power. that will change a lot of things…

  • Speaking of CPU vs GPU – on the Labs site there’s a mention of what GPUs will be supporting Hydra, so I certainly hope that listing is there for a reason? http://labs.adobe.com/wiki/index.php/AIF_Toolkit#Video_Card_Support

    There’s also this note: “This version of the AIF Toolkit only supports filter execution on the GPU. However, support for software-based rendering is expected in a future update.” That certainly sounds promising, does it not?

    J

  • Zeh

    Jensa: yeah, but it’s only about the toolkit executable itself and not about Astro.

  • Great example, and could be used in a variety of ways. One implementation of your adjustable threshold shader could be to assist colorblind/partially-sighted viewers. They could adjust the image’s threshold to see detail that might get lost in a fully-colored image.

    Hyrda looks very promising – thanks for sharing this.

    jkc