Unity tidbits: changing the visibility of Android’s navigation and status bars, and implementing immersive mode

Posted by Zeh Fernando on 6/February/2015 at 9:05

I was recently working on a mobile Unity project at home and I wanted to take advantage of Android’s immersive mode, which allows you to hide elements of the OS chrome such as the navigation bar or the status (top) bar. Unity itself has a Screen.fullScreen property that tries to control the appearance of the game interface, but it falls short in this case: on Android, all it does is dim the system bars a little bit. The project settings have some kind of status bar control, but not only it doesn’t work for me, it’s also limited in what it can do. After all, the Android system itself allows you to control the visibility, color, and even scaling model of the bars, and that’s what I wanted to tap into.

That is somewhat easy in native Android code. However, I wanted to run my code from a pure Unity game since this current project is not wrapped by a native application, and I thought this was a good reason to investigate how well Unity allows you to call native Android code.

The answer is, pretty easily. Unity has two classes, AndroidJavaObject and AndroidJavaClass, that allows developers to create objects and interact with the Android side of your application by essentially injecting calls, all without never actually creating a native Java application. It has pitfalls of its own – for example, being forced to know the secret handshake so calls run from the UI thread – but all in all it does what it’s supposed to do in an efficient way.

This means that implementing calls to change how the status and navigation bars work from the Unity side was pretty straightforward, and this is the subject of this first “Unity tidbit” I’m sharing from the collection of random Unity classes I created to help me when developing Unity.

Without further ado, the class I came up with to solve this problem is called ApplicationChrome. It uses a simple interface to do all the underlying changes to the Android application Chrome from the Unity side:

#if UNITY_ANDROID && !UNITY_EDITOR
#define USE_ANDROID
#endif

using System;
using System.Collections.Generic;
using UnityEngine;

/**
 * @author zeh fernando
 */
class ApplicationChrome {

	/**
	 * Manipulates the system application Chrome to change the way the status bar and navigation bar work
	 *
	 * References:
	 * . http://developer.android.com/reference/android/view/View.html#setSystemUiVisibility(int)
	 * . http://forum.unity3d.com/threads/calling-setsystemuivisibility.139445/#post-952946
	 * . http://developer.android.com/reference/android/view/WindowManager.LayoutParams.html#FLAG_LAYOUT_IN_SCREEN
	 **/

	// Enums
	public enum States {
		Unknown,
		Visible,
		VisibleOverContent,
		TranslucentOverContent,
		Hidden
	}

	// Constants
	private const uint DEFAULT_BACKGROUND_COLOR = 0xff000000;

	#if USE_ANDROID
		// Original Android flags
		private const int VIEW_SYSTEM_UI_FLAG_VISIBLE = 0;					// Added in API 14 (Android 4.0.x): Status bar visible (the default)
		private const int VIEW_SYSTEM_UI_FLAG_LOW_PROFILE = 1;				// Added in API 14 (Android 4.0.x): Low profile for games, book readers, and video players; the status bar and/or navigation icons are dimmed out (if visible)
		private const int VIEW_SYSTEM_UI_FLAG_HIDE_NAVIGATION = 2;			// Added in API 14 (Android 4.0.x): Hides all navigation. Cleared when theres any user interaction.
		private const int VIEW_SYSTEM_UI_FLAG_FULLSCREEN = 4;				// Added in API 16 (Android 4.1.x): Hides status bar. Does nothing in Unity (already hidden if "status bar hidden" is checked)
		private const int VIEW_SYSTEM_UI_FLAG_LAYOUT_STABLE = 256;			// Added in API 16 (Android 4.1.x): ?
		private const int VIEW_SYSTEM_UI_FLAG_LAYOUT_HIDE_NAVIGATION = 512;	// Added in API 16 (Android 4.1.x): like HIDE_NAVIGATION, but for layouts? it causes the layout to be drawn like that, even if the whole view isn't (to avoid artifacts in animation)
		private const int VIEW_SYSTEM_UI_FLAG_LAYOUT_FULLSCREEN = 1024;		// Added in API 16 (Android 4.1.x): like FULLSCREEN, but for layouts? it causes the layout to be drawn like that, even if the whole view isn't (to avoid artifacts in animation)
		private const int VIEW_SYSTEM_UI_FLAG_IMMERSIVE = 2048;				// Added in API 19 (Android 4.4): like HIDE_NAVIGATION, but interactive (it's a modifier for HIDE_NAVIGATION, needs to be used with it)
		private const int VIEW_SYSTEM_UI_FLAG_IMMERSIVE_STICKY = 4096;		// Added in API 19 (Android 4.4): tells that HIDE_NAVIGATION and FULSCREEN are interactive (also just a modifier)

		private static int WINDOW_FLAG_FULLSCREEN = 0x00000400;
		private static int WINDOW_FLAG_FORCE_NOT_FULLSCREEN = 0x00000800;
		private static int WINDOW_FLAG_LAYOUT_IN_SCREEN = 0x00000100;
		private static int WINDOW_FLAG_TRANSLUCENT_STATUS = 0x04000000;
		private static int WINDOW_FLAG_TRANSLUCENT_NAVIGATION = 0x08000000;
		private static int WINDOW_FLAG_DRAWS_SYSTEM_BAR_BACKGROUNDS = -2147483648; // 0x80000000; // Added in API 21 (Android 5.0): tells the Window is responsible for drawing the background for the system bars. If set, the system bars are drawn with a transparent background and the corresponding areas in this window are filled with the colors specified in getStatusBarColor() and getNavigationBarColor()

		// Current values
		private static int systemUiVisibilityValue;
		private static int flagsValue;
	#endif

	// Properties
	private static States _statusBarState;
	private static States _navigationBarState;

	private static uint _statusBarColor = DEFAULT_BACKGROUND_COLOR;
	private static uint _navigationBarColor = DEFAULT_BACKGROUND_COLOR;

	private static bool _isStatusBarTranslucent; // Just so we know whether its translucent when hidden or not
	private static bool _isNavigationBarTranslucent;

	private static bool _dimmed;


	// ================================================================================================================
	// INTERNAL INTERFACE ---------------------------------------------------------------------------------------------

	static ApplicationChrome() {
		applyUIStates();
		applyUIColors();
	}

	private static void applyUIStates() {
		#if USE_ANDROID
			applyUIStatesAndroid();
		#endif
	}

	private static void applyUIColors() {
		#if USE_ANDROID
			applyUIColorsAndroid();
		#endif
	}

	#if USE_ANDROID

		private static void applyUIStatesAndroid() {
			int newFlagsValue = 0;
			int newSystemUiVisibilityValue = 0;

			// Apply dim values
			if (_dimmed) newSystemUiVisibilityValue |= VIEW_SYSTEM_UI_FLAG_LOW_PROFILE;

			// Apply color values
			if (_navigationBarColor != DEFAULT_BACKGROUND_COLOR || _statusBarColor != DEFAULT_BACKGROUND_COLOR) newFlagsValue |= WINDOW_FLAG_DRAWS_SYSTEM_BAR_BACKGROUNDS;

			// Apply status bar values
			switch (_statusBarState) {
				case States.Visible:
					_isStatusBarTranslucent = false;
					newFlagsValue |= WINDOW_FLAG_FORCE_NOT_FULLSCREEN;
					break;
				case States.VisibleOverContent:
					_isStatusBarTranslucent = false;
					newFlagsValue |= WINDOW_FLAG_FORCE_NOT_FULLSCREEN | WINDOW_FLAG_LAYOUT_IN_SCREEN;
					newSystemUiVisibilityValue |= VIEW_SYSTEM_UI_FLAG_LAYOUT_FULLSCREEN;
					break;
				case States.TranslucentOverContent:
					_isStatusBarTranslucent = true;
					newFlagsValue |= WINDOW_FLAG_FORCE_NOT_FULLSCREEN | WINDOW_FLAG_LAYOUT_IN_SCREEN | WINDOW_FLAG_TRANSLUCENT_STATUS;
					newSystemUiVisibilityValue |= VIEW_SYSTEM_UI_FLAG_LAYOUT_FULLSCREEN;
					break;
				case States.Hidden:
					newFlagsValue |= WINDOW_FLAG_FULLSCREEN | WINDOW_FLAG_LAYOUT_IN_SCREEN;
					if (_isStatusBarTranslucent) newFlagsValue |= WINDOW_FLAG_TRANSLUCENT_STATUS;
					break;
			}

			// Applies navigation values
			switch (_navigationBarState) {
				case States.Visible:
					_isNavigationBarTranslucent = false;
					newSystemUiVisibilityValue |= VIEW_SYSTEM_UI_FLAG_LAYOUT_STABLE;
					break;
				case States.VisibleOverContent:
					// TODO: Side effect: forces status bar over content if set to VISIBLE
					_isNavigationBarTranslucent = false;
					newSystemUiVisibilityValue |= VIEW_SYSTEM_UI_FLAG_LAYOUT_STABLE | VIEW_SYSTEM_UI_FLAG_LAYOUT_HIDE_NAVIGATION;
					break;
				case States.TranslucentOverContent:
					// TODO: Side effect: forces status bar over content if set to VISIBLE
					_isNavigationBarTranslucent = true;
					newFlagsValue |= WINDOW_FLAG_TRANSLUCENT_NAVIGATION;
					newSystemUiVisibilityValue |= VIEW_SYSTEM_UI_FLAG_LAYOUT_STABLE | VIEW_SYSTEM_UI_FLAG_LAYOUT_HIDE_NAVIGATION;
					break;
				case States.Hidden:
					newSystemUiVisibilityValue |= VIEW_SYSTEM_UI_FLAG_FULLSCREEN | VIEW_SYSTEM_UI_FLAG_HIDE_NAVIGATION | VIEW_SYSTEM_UI_FLAG_IMMERSIVE_STICKY;
					if (_isNavigationBarTranslucent) newFlagsValue |= WINDOW_FLAG_TRANSLUCENT_NAVIGATION;
					break;
			}

			if (Screen.fullScreen) Screen.fullScreen = false;

			// Applies everything natively
			setFlags(newFlagsValue);
			setSystemUiVisibility(newSystemUiVisibilityValue);
		}

		private static void runOnAndroidUiThread(Action target) {
			using (var unityPlayer = new AndroidJavaClass("com.unity3d.player.UnityPlayer")) {
				using (var activity = unityPlayer.GetStatic("currentActivity")) {
					activity.Call("runOnUiThread", new AndroidJavaRunnable(target));
				}
			}
		}

		private static void setSystemUiVisibility(int value) {
			if (systemUiVisibilityValue != value) {
				systemUiVisibilityValue = value;
				runOnAndroidUiThread(setSystemUiVisibilityInThread);
			}
		}

		private static void setSystemUiVisibilityInThread() {
			//Debug.Log("SYSTEM FLAGS: " + systemUiVisibilityValue);
			using (var unityPlayer = new AndroidJavaClass("com.unity3d.player.UnityPlayer")) {
				using (var activity = unityPlayer.GetStatic("currentActivity")) {
					using (var window = activity.Call("getWindow")) {
						using (var view = window.Call("getDecorView")) {
							view.Call("setSystemUiVisibility", systemUiVisibilityValue);
						}
					}
				}
			}
		}

		private static void setFlags(int value) {
			if (flagsValue != value) {
				flagsValue = value;
				runOnAndroidUiThread(setFlagsInThread);
			}
		}

		private static void setFlagsInThread() {
			//Debug.Log("FLAGS: " + flagsValue);
			using (var unityPlayer = new AndroidJavaClass("com.unity3d.player.UnityPlayer")) {
				using (var activity = unityPlayer.GetStatic("currentActivity")) {
					using (var window = activity.Call("getWindow")) {
						window.Call("setFlags", flagsValue, -1); // (int)0x7FFFFFFF
					}
				}
			}
		}

		private static void applyUIColorsAndroid() {
			runOnAndroidUiThread(applyUIColorsAndroidInThread);
		}

		private static void applyUIColorsAndroidInThread() {
			using (var unityPlayer = new AndroidJavaClass("com.unity3d.player.UnityPlayer")) {
				using (var activity = unityPlayer.GetStatic("currentActivity")) {
					using (var window = activity.Call("getWindow")) {
						//Debug.Log("Colors SET: " + _statusBarColor);
						window.Call("setStatusBarColor", unchecked((int)_statusBarColor));
						window.Call("setNavigationBarColor", unchecked((int)_navigationBarColor));
					}
				}
			}
		}

	#endif

	// ================================================================================================================
	// ACCESSOR INTERFACE ---------------------------------------------------------------------------------------------

	public static States navigationBarState {
		get { return _navigationBarState; }
		set {
			if (_navigationBarState != value) {
				_navigationBarState = value;
				applyUIStates();
			}
		}
	}

	public static States statusBarState {
		get { return _statusBarState; }
		set {
			if (_statusBarState != value) {
				_statusBarState = value;
				applyUIStates();
			}
		}
	}

	public static bool dimmed {
		get { return _dimmed; }
		set {
			if (_dimmed != value) {
				_dimmed = value;
				applyUIStates();
			}
		}
	}

	public static uint statusBarColor {
		get { return _statusBarColor; }
		set {
			if (_statusBarColor != value) {
				_statusBarColor = value;
				applyUIColors();
				applyUIStates();
			}
		}
	}

	public static uint navigationBarColor {
		get { return _navigationBarColor; }
		set {
			if (_navigationBarColor != value) {
				_navigationBarColor = value;
				applyUIColors();
				applyUIStates();
			}
		}
	}
}

And using it is, I hope, simple enough:

// Toggles the dimmed out state (where status/navigation content is darker)
ApplicationChrome.dimmed = !ApplicationChrome.dimmed;

// Set the status/navigation background color (set to 0xff000000 to disable)
ApplicationChrome.statusBarColor = ApplicationChrome.navigationBarColor = 0xffff3300;

// Makes the status bar and navigation bar visible (default)
ApplicationChrome.statusBarState = ApplicationChrome.navigationBarState = ApplicationChrome.States.Visible;

// Makes the status bar and navigation bar visible over the content (different content resize method) 
ApplicationChrome.statusBarState = ApplicationChrome.navigationBarState = ApplicationChrome.States.VisibleOverContent;

// Makes the status bar and navigation bar visible over the content, but a bit transparent
ApplicationChrome.statusBarState = ApplicationChrome.navigationBarState = ApplicationChrome.States.TranslucentOverContent;

// Makes the status bar and navigation bar invisible (animated)
ApplicationChrome.statusBarState = ApplicationChrome.navigationBarState = ApplicationChrome.States.Hidden;

This solution has caveats of its own: you can’t mix dimmed state with translucent state, for example, and using colored backgrounds only works well if the bars are fully opaque. I believe those are native Android problems more than anything; the APIs for screen control have grown little by little over time, and in different ways, causing some flags to be at odds with each other (I’ve love to be wrong, though, in case there’s some room for improvement in the implementation).

Finally, for reference, this is what each of those modes looks like:

Visible bars (default)

Visible bars (default)

Visible bars, dimmed

Visible bars, dimmed

Visible bars, colored background

Visible bars, colored background

Visible bars over content

Visible bars over content

Translucent bars over content

Translucent bars over content

Hidden bars

Hidden bars (“immersive”)

The images above were created with a test Unity application I compiled for Android. Feel free to download the installation APK and try for yourself on Android devices. And while the screenshots show the status and navigation bars having the same state for the sake of simplicity, they can be controlled separately.

The dawn of a new Microsoft

Posted by Zeh Fernando on 3/February/2015 at 18:57

In January 2009, I was at Campus Party Brasil. This was the second edition of this big LAN party in the country, and as was the norm with the event, they hosted a number of presentations, many of them geared towards developers.

One of the several presentations I watched was by Microsoft. They were basically introducing a new version of Silverlight (and its editors) to the audience.

The presentation left me insulted and outraged.

As a Flash and ActionScript developer at the time, I was specially interested – if somewhat unimpressed so far – in Microsoft’s purported Flash killer. I went into the presentation wanting to hear more about their platform, and while I was aware this presentation was directed towards the general audience of the event, I thought I could learn a thing or two about the product.

My hopes went down the drain quick enough, however, as it became clear right away that the presentation was full of lies and falsehoods. In an attempt to persuade the audience towards their product and against Flash, the entire presentation relied in misrepresenting Flash’s features, exaggerating Silverlight’s capabilities, and overall just misinforming the audience about either of the platforms in an attempt to prove the (nonexistent) superiority of their offering. Interestingly, it seemed to work well, and the audience, composed mostly of technology and design students, accepted the speaker’s very eloquent lies without much questioning.

To me, however, it sent a different, but just as clear, message: stay away from this company. When looking for facts, I very much dislike being lied to, and I was very concerned about a company that feels the need to do so when attempting to sell their product.

After the fact, I wrote a pretty long blog article about the rhetoric of falsehoods in tech. This tactic is not new, of course, and something many companies, including Microsoft, have been known to rely on from time to time.

Nevertheless, I never posted the article. In the end, I didn’t have much stake in the topic, I wasn’t interested in engaging with talking heads, and it was clear the technology wasn’t getting anywhere any time soon.

Still, I assumed that was the norm for Microsoft, and stayed away from them for a while. This bet more or less paid off, as Silverlight was left in limbo, XNA was abandoned in the wilderness, and overall Microsoft wasn’t able to commit to any given platform that I felt like using.

That was 5 years ago. How things have changed.

They acquired, and released for free, the most popular Unity development extension plugin. They just rolled out a free version of Visual Studio Professional. The .NET Runtime is now available as open source in GitHub. They have created one of the best ES6-like/JS-transpiler languages, TypeScript. I used to hate the typical .NET APIs, but my current favorite language is easily C#, including its libraries. Windows 10 will be a free upgrade for users of Windows 7. Even their browser engine is looking good, and a new wrapper won’t hurt. This all came with a change of tone I could not have predicted 5 years ago.

This is a new Microsoft.

I wouldn’t say it’s all because of a new CEO. More than a top-down decision to become better, maybe what’s happening here is the shedding of old habits, and the awaited abandonment of hostile tactics that used to work in software development 20 years ago; if anything, it’s the understanding that making developers (and users) happy matters, and the lack of lies and free access to tools help too.

In this day and age, I know better than to commit to any single given platform. But so far I’ve certainly been enjoying this new Microsoft.

ApplicationLogger: a free application usage logger for Windows

Posted by Zeh Fernando on 5/January/2015 at 17:27

I may like charts too much.

A while ago, I created a chart depicting all the programming platforms I’ve had used over the years:

Programming platforms chart

This was initially created mostly to illustrate the fact that the fall in popularity of Flash and ActionScript wasn’t a surprising event – technologies constantly come and go – but when I saw the result, I was still surprised at the number of platforms I’ve actually worked with in the past, and forgotten or abandoned.

Admittedly, however, the way I gathered the data for the chart wasn’t very scientific: I’d look at the projects I had worked in a given year, and fill the numbers with whatever percentage of my attention I thought I had given to each platform.

Starting with 2013, I decided to collect that data monthly, to try and make it a little bit more accurate. However, my approach for data collection didn’t change much; I was still mostly estimating the amount of time I had dedicated to each platform.

Every time I’d show my data to someone, people would ask me what I was using as a unit (was it time? lines of code? commits?). This led me to decide to make things a bit more accurate for 2014. Call it a New Year’s resolution, I suppose.

So for 2014, I wanted to have a way for measuring where I spent my time more accurately by measuring the application I’m running at any given time, and what kind of file I’m editing with it. Surprisingly, there are a few services that already do something like that – e.g., RescueTime – but none with the level of granularity and simplicity I wanted.

Long story short, to solve this problem and learn a bit more about desktop application development at the same time, I created a small application called Application Logger. This is a small Windows C#-based application that runs in the background and, well, simply logs details about applications being ran, saving it to a text file that can be analyzed later. It’s mostly a one-class application, with a very small footprint and not many features other than logging when the current user process changes and some customization options. Its output looks like this:

2014-12-11T12:55:12.3908117-05:00   app::focus  ZMACHINE    TweetDeck   C:\Program Files (x86)\Twitter\TweetDeck\TweetDeck.exe  TweetDeck
2014-12-11T13:00:39.6875381-05:00   app::focus  ZMACHINE    chrome  C:\Program Files (x86)\Google\Chrome\Application\chrome.exe Showtime Anytime app comes to Xbox One | Polygon - Google Chrome
2014-12-11T13:29:55.1320650-05:00   app::focus  ZMACHINE    firefox C:\Program Files (x86)\Mozilla Firefox\firefox.exe  C# snippets - Google Docs - Mozilla Firefox
2014-12-11T13:33:14.0059504-05:00   app::focus  ZMACHINE    notepad++   C:\Program Files (x86)\Notepad++\notepad++.exe  D:\apps\logger\ApplicationLogger.cfg - Notepad++ [Administrator]
2014-12-11T13:37:15.4130887-05:00   app::focus  ZMACHINE    explorer    C:\Windows\explorer.exe src
2014-12-11T11:37:23.3319560-05:00   app::focus  ZMACHINE    WDExpress   C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\WDExpress.exe   ApplicationLogger - Microsoft Visual Studio Express 2013 for Windows Desktop (Administrator)
2014-12-11T13:44:53.9509379-05:00   app::focus  ZMACHINE    studio64    C:\Users\zeh.fernando\AppData\Local\Android\android-studio-0.8.14\bin\studio64.exe  android - [D:\work\QWERTY_APPS2852\dev\android-app] - [app] - ...\app\src\main\java\com\querty\obfuscated\ApplicationConstants.java - Android Studio 1.0
2014-12-11T12:24:14.1160028-05:00   status::idle            
2014-12-11T13:49:01.4716875-05:00   status::stop  

It runs silently, in the background, as a tray application:

ApplicationLogger

I’ve been running it for a few weeks and now I consider it solid enough for daily use. I have it running on both my work and home computers, saving separate log files that can be analyzed later.

For illustration, after processing the data and using some RegEx to group them properly, this is what my programming time looked like for the last two weeks of December 2014 (mostly from working on a Unity game on my free time at home, and before I started running ApplicationLogger at work):

Application Logger chart

The application is available as an open source project on GitHub under the WTFPL license. A pre-compiled executable is also available, although it has no installation file (just copy it to where you want to run it from). And while it just generates a raw log that requires analyzing later, I’ve been working on a simple, standalone HTML page that does the trick (not very configurable yet, though).

An experiment: writing code without feedback

Posted by Zeh Fernando on 1/October/2014 at 12:51

Computing has come a long way in the past few decades. Originally, programmers used to write code by themselves, using punch cards, and only later be allowed to run them on machines. The end result was a process of writing code and debugging as completely separated steps.

I never used punch cards (I’m not that old), but I had similar experiences. In high school, I didn’t own a PC, but I had about 2 hours of lab time per week. I normally stretched that to 4 hours, but for a nerdy teenager with lots of programming ideas, that wasn’t enough time. This made me maximize the quality of the time I actually spent with the school’s computers: I would spend the whole week writing code on paper, and when I got to the computer lab, I could spend the whole time actually typing the code rather than thinking on what code to write, or debugging. I’d then run the program, print a list of the error messages I got, print the source for the whole program, take that home with me, and spend the next week debugging the code by hand, doing corrections, and writing new code for the next time I had access to the computers.

That is obviously a very alien concept today. We’re constantly writing code and running it to see if it works, and we get immediate feedback from our code writing tools so simple mistakes can be quickly fixed. In fact, if something takes more than a few seconds to compile it frustrates me to no end, and a slow environment – for example, API calls that time out – can drag my productivity to the ground just out of sheer impatience.

Recently, I started an experiment in doing something a little bit different.

I have a project that I’m doing at home using Unity, C#, and Visual Studio. Those are great tools, and the project is evolving well.

However, from time to time, I find myself having just a few minutes of free time (e.g. just before meetings), or conjuring some idea that I want to quickly implement in this project.

In these cases, rather than firing up Visual Studio and Unity to write the code out, I’ve started editing the project’s source files directly, with minimal feedback. That means opening the files in a simple text editor (Notepad++) and writing code with no regard for how (or even whether) it is running. Of course, I try to write something that I believe is correct, and that hopefully will run at once, but the interesting thing about this approach is that I don’t care about errors that much. I am not checking them, either via the (helpful) IDE checks or by running the code. I am writing code, not fixing it.

The result of this approach is a deeper concentration in what I’m writing, rather than how it’s running.

Of course, this approach requires a second step, which is getting home and actually testing the project. That means running my environment – Visual Studio and Unity – and fixing any errors that show up in what I tried to accomplish. These are typically plentiful; sometimes, really stupid syntax errors creep in too.

Still, it has been very interesting seeing how much can be accomplished with this system. By keeping my focus on writing code only, it’s easier to do it in bursts of a few minutes at a time, as if I’m writing the blueprint of something and delegating the responsibility of testing and getting it to actually run to someone else. It also allows me to spend time dedicated to testing and fixing errors later, so I don’t have to constantly switch between different frames of mind: writing new code and testing it.

I don’t have any valid data points here. This is something I just started. Still, it’s something I definitely recommend other people try, as it’s giving me a new perspective on code writing and debugging.

Creating a tweening library for Unity

Posted by Zeh Fernando on 29/September/2014 at 10:02

A week or so ago, I started toying with the idea of creating a tweening solution for Unity. Trying experimental tweening solutions are the kind of thing I find myself always spending time with: in the work I usually do, tweening is something I constantly have to deal with, and I strive to find the most efficient syntax that fits my coding style. That’s one of the reasons why I like reinventing the wheel even when mature solutions already exist, or at least the excuse I like giving myself for doing so.

Like for most visual platforms, the concept of tweening is not new to Unity. There are several different solutions available for animating UI elements in the platform, from LeanTween to iTween to the (more popular) HOTween and its successor DOTween. Still, it is something worth exploring, especially as I’m still learning the platform and the language.

Tweening values in Unity presents two interesting challenges when compared to the syntax one would use for tweening in, say, JavaScript or ActionScript.

The first challenge is the language itself. While Unity also supports for a JavaScript-based language normally called “UnityScript”, C# is a more popular choice for developers, and the design of this language make some things a little bit more tricky to write. To illustrate, consider this Tweener call:

Tweener.addTween(myMovie, {x:10, y:10, time:1, transition:"linear"});

In the above line, the second parameter is an Object (sometimes called a map) that can contain any number of properties, and, if desired, even other objects. Tweener uses that for parameters that are optional, and, indeed, for any number of properties that one wants to tween. The name of those propertie are not known beforehand; they’re not hard-coded in the library, but instead read at runtime.

That is very easy to read and understand, but that syntax doesn’t work in C#. While the language has many advanced features over ActionScript 3 (or JavaScript), it forces a certain strictness to the way methods and functions are called. The closest equivalent of an AS3 Object in C# would be a Hashtable, and its inline syntax would look something like this:

Tweener.addTween(myMovie, new Hashtable() { {"x":10}, {"y":10}, {"time":1}, {"transition":"linear"} });

While close in structure, the syntax is a little bit more verbose, harder to write and read, and more error-prone. While not a total disaster, it doesn’t make the act of writing the code particularly swift.

The second challenge for tweening engines in Unity the way the Unity platform’s works. In Unity, many objects are considered virtual getters and setters, and cannot be changed directly. Consider the scale of a GameObject/Transform object, controlled by its localScale member, which is a Vector3 instance. Using C#, you can set the new scale like so:

transform.localScale = new Vector3(2, 2, 2);

This scales your object up to 200% of its original size, in all dimensions. Now, imagine you want to change the scale of your object in the X axis only. This may look like a possible solution:

transform.localScale.x = 2;

This code, however, is invalid in Unity’s C# implementation. The reason is that the localScale object is created when the property is read, and changes to it wouldn’t be reapplied to the object. The correct solution would, then, look like this:

Vector3 newScale = transform.localScale;
newScale.x = 2;
transform.localScale = newScale;

Or if you want to be succinct, and can afford creating a new Vector3 instance:

transform.localScale = new Vector3(2, transform.localScale.y, transform.localScale.z);

What this means is that you cannot tween values of certain object properties directly. This is similar to the problem of tweening filters such as blur in ActionScript 3: you may change one of the filter values, but unless you re-apply it to the Sprite that holds it, you wouldn’t see any change. That’s why tweening engines in AS3 had to work around that limitation with additional functionality. In Unity’s case, sometimes you will even run into compilation errors when trying to modify some of these properties directly.

Put together, those two challenges mean that one cannot take the syntax solutions matured in JavaScript and ActionScript and apply them to Unity. The platform requires solutions of its own.

The aforementioned iTween adopts two different syntaxes that somewhat resemble the way classic ActionScript 3 libraries deal with tweening, with all the problems it entails:

// Scale to 2x in 1 second
iTween.ScaleTo(gameObject, new Vector3(2, 2, 2), 1);

// Scale to 2x in 1 second, with more options
iTween.ScaleTo(gameObject, new Hashtable() { {"scale": new Vector3(2, 2, 2)}, {"time": 1}, {"easetype":iTween.EaseType.easeOutBack}, {"delay": 1} });

// Same as above, using a helper function
iTween.ScaleTo(gameObject, iTween.Hash("scale", new Vector3(2, 2, 2), "time", 1, "easetype", iTween.EaseType.easeOutBack, "delay", 1));

LeanTween takes a somewhat different approach:

// Scale to 2x in 1 second
LeanTween.scale(gameObject, new Vector3(2, 2, 2), 1);

// Scale to 2x in 1 second, with more options
LeanTween.scale(gameObject, new Vector3(2, 2, 2), 1).setEase(LeanTweenType.easeOutBack).setDelay(1);

This is a very interesting syntax: it allows the library to use strongly-typed methods for all kinds of actions. While it demands a number of methods to be added to the library’s interface, errors due to mistyping are minimized and developers can rely on auto completion for all their needs. In my option, it’s a much more elegant solution both for writing and reading code. Also, while it requires specialized method calls to tween some of the properties of a GameObject, this is an inevitable side effect given the way Unity works.

DOTween adopts a similar syntax with an interesting twist: it uses a C# feature called extension methods to add methods to the objects themselves, creating a syntax that reminds me of MC Tween:

// Scale to 2x in 1 second
transform.DOScale(new Vector3(2, 2, 2), 1);

// Scale to 2x in 1 second, with more options
transform.DOScale(new Vector3(2, 2, 2), 1).SetEase(Ease.OutBack).SetDelay(1);

Indeed, this “chaining” of method calls has been popular among libraries for other platforms for a while (jQuery was the library I saw using it). Some tweening solutions use this approach not just to set optional parameters for animations, but to create complex sequences in a very elegant way. Take TweenJS’s approach:

// Wait 0.5s, change alpha to 0 in 1s, then call a function
Tween.get(target).wait(500).to({alpha:0, visible:false}, 1000).call(someFunction);

Which, to me, looks like a solution that is as elegant as it gets.

When deciding on a tweening solution for Unity, I wanted something that was easy to write, read, fully type safe, and with easy auto completion. The two latter points are a given based in the language of choice, the two former ones are a little bit subjective and may require some creativity. I also wanted to explore bigger tweening chains that operated as animation sequences, something that in a solution like Tweener would require several lines of code.

The solution I decided on looks like this:

// Scales a gameObject to 200% in the Z axis in 0.2 seconds using an EaseOutExpo equation
ZTween.use(gameObject).scaleTo(new Vector3(1, 1, 2), 0.2f, Easing.expoOut);

What the code does is generate an object instance (via the use method) that can then have commands issued to it. Each new method return the same object, so they can be chained and new methods can be added. For example:

// Scales a gameObject to 200% in the Z axis, then 200% in the Y axis, then back to 100%
ZTween.use(gameObject).scaleTo(new Vector3(1, 1, 2), 0.2f, Easing.expoOut).scaleTo(new Vector3(1, 2, 2), 0.2f, Easing.expoOut).scaleTo(new Vector3(1, 1, 1), 0.2f, Easing.expoOut);

And additional methods can exist to enforce initial values, if necessary:

// Scales a gameObject from 100% to 200% in the Z axis
ZTween.use(gameObject).scaleFrom(new Vector3(1, 1, 1).scaleTo(new Vector3(1, 1, 2), 0.2f, Easing.expoOut);

What’s interesting about all this chaining is that the concept of events is more or less ignored – you don’t have events for the beginning and ending of a particular tween, but instead you can add calls to the method chain itself. For example:

// Call functionA(), scales an object, then call functionB()
ZTween.use(gameObject).call(functionA).scaleTo(new Vector3(1, 1, 2), 0.2f, Easing.expoOut).call(functionB);

And given how C# works, parameters can be passed via lambdas, another C# feature:

// Scales an object, then writes to the console
ZTween.use(gameObject).scaleTo(new Vector3(1, 1, 2), 0.2f, Easing.expoOut).call(() => Debug.Log("Done animating");

And delays are another method, rather than a parameter:

// Wait 1s then scale an object
ZTween.use(gameObject).wait(1).scaleTo(new Vector3(1, 1, 2), 0.2f, Easing.expoOut);

Custom numeric properties get a tweening object of their own, via a reference (when a pure member):

// Transition "something" from the current value to 1
ZTween.use(ref something).valueTo(1, 0.2f, Easing.quadOut);

Or via lambdas created as a get and set parameters (for pure members, getter/setters, or get/set pairs):

// Transition "something" (a numeric property or a getter/setter) from the current value to 1
ZTween.use(() => something, val => something = val).valueTo(1, 0.2f, Easing.quadOut);

// Transition using getSomething() and setSomething() from the current value to 1
ZTween.use(getSomething, setSomething).valueTo(1, 0.2f, Easing.quadOut);

This is not a full solution by any stretch of the imagination; it currently only works for scaling, translating, and value transitions (for real world use, I’d suggest DOTween instead). However, this is a solution I’m enjoying building and what I’ll be using in future projects, including a little game test I’m building:

Again, this is just an experiment, but for the curious, source code is available on GitHub.

New usfxr release (1.3)

Posted by Zeh Fernando on 8/August/2014 at 17:02

I have just published release 1.3 of usfxr on its GitHub repository. This is a small update, but it provides some important fixes (especially when publishing on mobile platforms) and adds the option to export your audio as WAV files (similarly to other SFXR ports). I have to thank Michael Bailey and Owen Roberts for their help with bug detection and fixing in this version.

The Asset Store version should be live some time next week is also available now.

Presentation: Using Unit Testing

Posted by Zeh Fernando on 7/August/2014 at 14:35

Every Thursday morning at Firstborn is “Scrummy Thursday”. For one hour, Firstborn developers present random topics to each other. Those are usually little, 15-minute presentations on projects we’ve worked or things we discovered, and it is meant as a way to share knowledge among the development team. It’s a nice idea, and it has been working well for a while.

Today I had a little presentation on how I used Unit Testing to help me refactor a part of Pepsi Spire (my most recent project) with some confidence. I’ve made the slides available publicly, and you can check them below.

Click the slideshow and press “S” to show the presentation notes, which will help make sense of what’s being displayed.

The content of the presentation is not anything groundbreaking (there are better unit testing introductions out there), but I thought I’d share it nonetheless. For the longest time, I’ve looked at unit testing with some contempt; it normally wouldn’t work with the kind of UI-heavy, animation-heavy, short-lived code I had to create. Still, this was an instance where unit testing helped me avoid trouble and surprises, and save time in the long run. And while this particular project was made in ActionScript 3, this is a tale that can be repeated on any given platform.

usfxr now supports BFXR’s advanced audio synthesis features

Posted by Zeh Fernando on 14/July/2014 at 9:03

header_bfxr

In preparation for the next Ludum Dare, I have finished adding all advanced sound synthesis features first introduced by BFXR to usfxr, my own Unity port of the SFXR game audio synthesis engine. The new version is 1.2 and is available as a zip download on the GitHub usfxr repository (the asset store version will be updated later this week).

(If you don’t know what usfxr or SFXR is, this post is less cryptic)

Interface for usfxr 1.2

The slightly updated interface for usfxr 1.2

At first, I wasn’t so sure I’d like to add the BFXR features to my port; I have to confess I always saw BFXR as a rogue fork of SFXR, and the fact that parameter strings were incompatible between the two projects always rubbed me the wrong way. However, after testing BFXR for a while, I came to really like its original features, and decided to adopt them in usfxr. This is what this update is about.

The new features are as such (as described by BFXR’s interface):

  • New wave form types
    • Triangle: robust at all frequencies, stand out quite well in most situations, and have a clear, resonant quality
    • Breaker: a little bit more hi-fiwave type; like a smoother, slicker triangle wave
    • Tan: a potentially crazy wave, tends to produce plenty of distortion
    • Whistle: a sine wave with an additional sine wave overlayed at a lower amplitude and 20x the frequency; it can sound buzzy, hollow, resonant, or breathy.
    • Pink noise: random numbers with a filtered frequency spectrum to make it softer than white noise
  • New filters
    • Compression: pushes amplitudes together into a narrower range to make them stand out more; very good for sound effects when you want them to stick out against background music
    • Harmonics: overlays copies of the waveform with copies and multiples of its frequency;g ood for bulking out or otherwise enriching the texture of the sounds
    • Bit Crusher: resamples the audio at a lower frequency, for that extra retro feeling
  • Expanded pitch-jumping abilities; good for arpeggiation effects

On top of that, this new version is still compatible with previous versions, as well as SFXR itself; instead of starting anew and breaking compatibility, usfxr accepts both standard (SFXR/as3sfxr style) parameter strings, as well as the new BFXR parameter strings. This means old code will still work, but you can also copy & paste effect parameter strings directly between usfxr’s Unity interface and BFXR.

There are a few additional non-core BFXR properties that I will have to add support for in the future, specifically property locking for mutation and the UI. This should be added soon along with other UI updates.

Round-up of usfxr uses from around the web

Posted by Zeh Fernando on 6/June/2014 at 17:09

Now that usfxr has an in-editor window for audio generation right inside Unity, I’m considering it stable. I’ll probably do a few benchmarks in the future to improve any performance bottlenecks I’m able to identity, and maybe add a few more examples to the repository, but for all intents and purposes it is production-ready.

Given that, I figured I’d do a search to see if anyone was using the library, and how. I found quite a few, so I’d do like to share some picks here.

D-Lask has been posting a few interesting videos of Unity experiments into his Vine stream, including one using usfxr with the Playstation move (enable audio to hear it):

Love Connection is a Ludum Dare game by thecodermonkey using the “Minimalism” theme. It’s a great, great entry for Ludum Dare, and it’s not surprising it ranked very well (#8) in the overall Ludum Dare results.

1Fuel is an old Ludum Dare entry by unitycoder_com under the theme “You Only Get One”. It’s a simple game, but one with an interesting gameplay mechanic.

The same developer has a blog with a lot more small examples and tests that use usfxr.

Super Minimalistic Nuclear Space Potatoes! is a Unity game by Hatscat also for Ludum Dare under the “Minimalism” theme.

And finally, a while ago, Jorge Garcia tweeted a picture of the “SpaceGame” example running on a PS Vita, which makes me pretty giddy:

Awesome seeing other people trying it out!

Create 16-bit sound effects right inside Unity with usfxr

Posted by Zeh Fernando on 4/June/2014 at 15:49

It’s been a little more than one year since I introduced usfxr, a Unity/C# version of the well-known real-time procedural game audio generator sfxr. What it has been lacking for quite some time was the ability to generate sound effects right inside the Unity interface; developers were forced to visit an online source like as3sfxr to generate their audio parameters (as a string), and only then use usfxr to play the audio in Unity games.

Well, no more. I finally got around to improving the in-game editor window first created by Tiaan Geldenhuys, and now you can use a Unity tab/window to generate audio parameters, and then copy the parameters so you can use them in code. The sound generating window looks like this:

usfxr Sound Generator

It also plays audio automatically every time one of the parameters is changed, so it’s easy to mess around with it until you get the sound effect you want. The generator window also works whether you’re in play mode or not, so it should make it easier to create sounds on-the-go.

As a reminder, this is how a usfxr sound is played in Unity by using a generated parameter string:

SfxrSynth synth = new SfxrSynth();
synth.parameters.SetSettingsString("0,,0.032,0.4138,0.4365,0.834,,,,,,0.3117,0.6925,,,,,,1,,,,,0.5");
synth.Play();

The new version of usfxr can be found on GitHub.