If you’ve been developing for Android for some time now, you might have encountered a little thing called “The 65K limit”. You must have at least heard about it around the water cooler. There were times were Android developers dreaded it.

You see, the Android system put an almost arbitrary limit on the number of methods an Android application can have. A limit which may have sounded reasonable in the past but is near far-fetched in today’s terms. The limit itself, which stands roughly around 65,000 methods, may sound a bit excessive. “What are the odds I’ll ever reach that?” a novice Android developer may be thinking. However, applications are using more and more third-party libraries (otherwise known as SDKs – Software Development Kits). Just imagine writing your app and wanting a library to help you parse JSON objects, wanting to connect to a social network such as Facebook, or looking to make a profit for your effort by displaying ads in your app. All of these libraries add more methods to your app, and slowly that high limit doesn’t seem so unreachable.

Adding insult to injury, Google themselves contributed massively to the problem, with their Play Services SDK containing a whopping 29,000 methods! That number is almost inconceivable – that is nearly half of the limit (In an upcoming post, I will describe how you can reduce the number of methods you get from this SDK. Stay Tuned).

Today, however, things are different and solving the limit issue is quite simple. So simple, actually, that many apps found refuge in it. Today there are apps in the Play Store with well over 80K methods!

Into Every Generation a Dex File is Born: One File in all the App

So what is the 65K limit and why does it exist?

Android devices run a virtual machine called Dalvik. The compiled Java code of Android is called dex (“Dalvik Executable”). It’s one file that contains the entire compiled code. That file is known in the Android realm as “classes.dex”.

So far it’s all puppies and rainbows. However, the dex file comes with some fine print – every method inside this file receives an ID. The ID field is limited. To what? Well, you’ve guessed correctly – 65,536. Meaning, the size of the field is four hexadecimal digits (0xffff in hex is the limit).

Once that limit is exceeded, the dex file could not be created and a very obscure error message was received:

Conversion to Dalvik format failed:
Unable to execute dex: method ID not in [0, 0xffff]: 65536

Quite the helpful error message, isn’t it?

Breaking the Limit with Multidex

Acknowledging the increasing issue of this limit and the struggle app developers were running into, in late October 2014 a solution was introduced with the release of the multidex support library (since Android SDK 21 and higher, the support library is included as part of the SDK).

The introduction of the library changed the error message but did not make it much more readable:

Too many field references: 131000; max is 65536.
You may try using --multi-dex option

Android actually uses a jar file called dx.jar which compiles bytecode (compiled Java classes) to dex. With the introduction of the support library, the dx.jar file received a new parameter, the –multi-dex argument, which allowed the compiler to simply create a secondary dex file should the limit be reached. In early versions of Android Studio, you had to manually pass this argument to the dex Gradle task. And actually, the headache did not stop here (but more on that later).

multidexnow

Today, however, things are different and solving the limit issue is quite simple.

To understand the true overhead of the multidex solution, we must first understand what the solution really is.

The good news is that today making your app multidex is ridiculously easy. Once you have the multidex support library, you can choose one of three convenient ways to implement it:

  1. 1. If you’re not implementing an Application class yourself, you can simply define the library’s MultiDexApplication in your Android Manifest file under the application tag:
    android:name="android.support.multidex.MultiDexApplication"
  2. 2. If you are implementing your own Application class, you can either simply override the MultiDexApplication file:
    public class MyAwesomeApplication extends MultiDexApplication {
  3. 3. Or if your application class is already extending another class, you can just override attachBaseContext method and add the following call to it:
    @Override
    protected void attachBaseContext(Context base) {
       super.attachBaseContext(context);
       Multidex.install(this);
    }

All of these solutions basically do the same thing (MultiDexApplication simply overrides attachBaseContext method and makes the static call as above). And even if you have a complex application class hierarchy, you don’t have to worry – the Multidex class knows if you’ve already called install and will ignore any duplicate calls to it, so you can add the call anywhere you want in your hierarchy, even multiple times if you’re not sure if it’s already called, and still only pay the penalty as if the method was called once.

But what does it do? Simply put – When your app is launched, the install method creates a class loader that can search for methods not just in the classes.dex file, but also in any secondary files (such classes2.dex, classes3.dex and so on).

In newer version of Android Studio you don’t need the support library, you just need to raise the multiDexEnabled flag inside any of your build type configurations and it’s as if you have the support library:

android {
   ...
   myConfig {
     ...
     multiDexEnabled true
   }
}

The Consequences of Multidexing

Suppose for a minute you have the following scenario: Your app is exceeding the 65K limit and is therefore about to “spill” into a secondary dex file. However, your application class, where the Multidex.install call is made, is placed inside the secondary dex file. When your app loads, the app launcher can’t find your application class (which you have declared in your manifest), since it natively only knows how to look for classes in a single classes.dex file. Subsequently, your app crashes.

The same is true for any activities (especially the launcher activity), services and basically everything that the Dalvik virtual machine would expect to find in the primary dex file before reaching the Multidex.install call that would tell it there’s another hot new place in town where methods hang.

For that reason, whenever you compile your app as multidex, you receive a penalty in the form of a script called mainDexClasses. That script basically uses Proguard to strip your app to its birthday suit, its’ bare essentials, keeping only what must be present in the primary classes.dex file. The script then outputs all classes that were kept inside and passes that list to the dx.jar. The good news is that while in the past you had to manually tell Gradle to run this script, today it does it for you automatically (this is the “extra” headache in the past I was referring to before).

So which classes must always be present in the primary dex file? Instrumentation, Application (with constructor and attachBaseContext), Activity, Service, ContentProvider, BroadcastReceiver, BackupAgent and any Java Annotations (since Proguard minifies them by default).

To MultiDex or Not to MultiDex, That is the Question

The script described above is also the “greatest” disadvantage of making your app multidex – the script will run every single compilation, prolonging the overall compilation time.

I didn’t call it a penalty for nothing – the more classes your app has, the longer the script takes. And wait, it gets worse. You see, by default, the Gradle script for compiling Android (used by Android Studio, but you can also use it to compile your app via the command line), does a little thing called “pre-dex.”

What is pre-dex? Well, since large portions of your app aren’t likely to change (here are those third-party libraries again), Gradle computes a separate classes.dex file for each of your app dependencies as well as for your own code (meaning, if you have n libraries – in either jar or project form – you’ll have n+1 classes.dex files created). At the later stages of compilation, Gradle merges all these files together into one (or more) classes.dex file. That way, the compilation from bytecode to dex occurs each time only on your code, and not on the entire code (meaning, third-parties excluded), and that way you can compile your changes faster.

But here’s the problem – If you define your app to support multidex, even if you added the support “just in case”, but are not actually reaching the limit, the process of running the prolonged script to compute which files must be present in the primary dex file (even if you only end up with one!) will still run. And since the script only knows how to handle an input of a single jar file, you lose the precious pre-dex option. So you end up paying twice – both with the script and the loss of predexing.

At SafeDK, we’ve seen numerous single-dex apps that added the support library anyway (although, whether they were working with Gradle and feeling that consequence is unclear). Not only that, but in a survey we did of some 600+ apps (including the top 500 in the US Play Store), we found that roughly 10% of applications are in fact multidex.

More to come

At this point you might be thinking to yourself “problem solved. However…” and you’d be right. As we’ve seen, while the multidex solution offers a workaround the 65K limit, it’s not ideal. And when Google revolutionized Android in Lollipop (5.0), the multidex solution took on a whole other turn.

Stay tuned for part 2 of this post to find out the ins and outs of the Lollipop solution.