Sapan Diwakar

Software developer

Follow me on Twitter Check out my code on GitHub View some of my designs on Dribbble Take a look at my Linked In profile

How to Set Up Android App to Support Expansion Files

Google Play currently requires that your APK file be no more than 50MB. For most applications, this is plenty of space for all the application's code and assets. However, some apps need more space for high-fidelity graphics, media files, or other large assets. Previously, if your app exceeded 50MB, you had to host and download the additional resources yourself when the user opens the app. Hosting and serving the extra files can be costly, and the user experience is often less than ideal. To make this process easier for you and more pleasant for users, Google Play allows you to attach two large expansion files that supplement your APK.

If you are just looking for implementation details, jump right to Implementation.

Overview

Each time you upload an APK using the Google Play Developer Console, you have the option to add one or two expansion files to the APK. Each file can be up to 2GB and it can be any format you choose.

  • The main expansion file is the primary expansion file for additional resources required by your application.
  • The patch expansion file is optional and intended for small updates to the main expansion file.

File name format

You can upload any format (ZIP, PDF, MP4, etc) as an expansion file. Regardless of the file type you upload, Google Play considers them opaque binary blobs and renames the files using the following scheme:

[main|patch].<expansion-version>.<package-name>.obb 

Storage Location

When Google Play downloads your expansion files to a device, it saves them to the system's shared storage location. To ensure proper behavior, you must not delete, move, or rename the expansion files. In the event that your application must perform the download from Google Play itself, you must save the files to the exact same location.

The specific location for your expansion files is:

<shared-storage>/Android/obb/<package-name>/  
  • <shared-storage> is the path to the shared storage space, available from getExternalStorageDirectory().
  • <package-name> is your application's Java-style package name, available from getPackageName().

Download process

Most of the time, Google Play downloads and saves your expansion files at the same time it downloads the APK to the device. However, in some cases Google Play cannot download the expansion files or the user might have deleted previously downloaded expansion files. To handle these situations, your app must be able to download the files itself when the main activity starts, using a URL provided by Google Play.

Implementation

Download required packages

To use the Downloader Library, you need to download two packages from the SDK Manager and add the appropriate libraries to your application.

First, open the Android SDK Manager, expand Extras and download:

  • Google Play Licensing Library package
  • Google Play APK Expansion Library package

Android Manifest

Declare the following permissions in AndroidManifest.xml

<!-- Required to access Google Play Licensing -->  
<uses-permission android:name="com.android.vending.CHECK_LICENSE" />

<!-- Required to download files from Google Play -->  
<uses-permission android:name="android.permission.INTERNET" />

<!-- Required to keep CPU alive while downloading files (NOT to keep screen awake) -->  
<uses-permission android:name="android.permission.WAKE_LOCK" />

<!-- Required to poll the state of the network connection  
    and respond to changes -->
<uses-permission  
    android:name="android.permission.ACCESS_NETWORK_STATE" />

<!-- Required to check whether Wi-Fi is enabled -->  
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE"/>

<!-- Required to read and write the expansion files on shared storage -->  
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />  

While you are there, let's also add the service and broadcast receiver we are going to need to handle the downloads. We will add these classes later to the project. Add these somewhere inside your application tags.

<service android:name=".expansion.DownloaderService"/>  
<receiver android:name=".expansion.DownloaderServiceBroadcastReceiver" />  

Integrate Libraries

Downloader Library

To use APK expansion files and provide the best user experience with minimal effort, we will use the Downloader Library that's included in the Google Play APK Expansion Library package. This library downloads your expansion files in a background service, shows a user notification with the download status, handles network connectivity loss, resumes the download when possible, and more.

Copy the source for the Downloader Library from <sdk>/extras/google/play_apk_expansion/downloader_library/src to your project.

Licensing

In order to facilitate the expansion file functionality, the licensing service has been enhanced to provide a response to your application that includes the URL of your application's expansion files that are hosted on Google Play. So, even if your application is free for users, you need to include the License Verification Library (LVL) to use APK expansion files. Of course, if your application is free, you don't need to enforce license verification—you simply need the library to perform the request that returns the URL of your expansion files.

Copy the sources from <sdk>/extras/google/play_licensing/library/src to your project.

Zip File

The Google Market Apk Expansion package includes a library called the APK Expansion Zip Library (located in <sdk>/extras/google/google_market_apk_expansion/zip_file/). This is an optional library that helps you read your expansion files when they're saved as ZIP files. Using this library allows you to easily read resources from your ZIP expansion files as a virtual file system.

Copy the sources from <sdk>/extras/google/google_market_apk_expansion/zip_file/ to your project.

Downloader Service

Now, create a new package named expansion and create two files DownloaderService.java and DownloaderServiceBroadcastReceiver.java inside the package.

The DownloaderService handles the downloading of expansion files from the Play Store and informs about the the progress to subscribing activities. We will take a look at how to configure the activity to begin downloads in a moment.

Notice: You must update the BASE64_PUBLIC_KEY value to be the public key belonging to your publisher account. You can find the key in the Developer Console under your profile information. This is necessary even when testing your downloads.

// DownloaderService.java
public class DownloaderService extends com.google.android.vending.expansion.downloader.impl.DownloaderService {  
    public static final String BASE64_PUBLIC_KEY = "<<YOUR PUBLIC KEY HERE>>"; // TODO Add public key
    private static final byte[] SALT = new byte[]{1, 4, -1, -1, 14, 42, -79, -21, 13, 2, -8, -11, 62, 1, -10, -101, -19, 41, -12, 18}; // TODO Replace with random numbers of your choice

    @Override
    public String getPublicKey() {
        return BASE64_PUBLIC_KEY;
    }

    @Override
    public byte[] getSALT() {
        return SALT;
    }

    @Override
    public String getAlarmReceiverClassName() {
        return DownloaderServiceBroadcastReceiver.class.getName();
    }
}

The BoradcastReceiver will start the download service if the files need to downloaded.

// DownloaderServiceBroadcastReceiver.java
public class DownloaderServiceBroadcastReceiver extends android.content.BroadcastReceiver {  
    @Override
    public void onReceive(Context context, Intent intent) {
        try {
            DownloaderClientMarshaller.startDownloadServiceIfRequired(context, intent, DownloaderService.class);
        } catch (PackageManager.NameNotFoundException e) {
            e.printStackTrace();
        }
    }
}

Initiating Downloads

Now, let's see how we can initiate the downloads when the app starts. Add this somewhere in your main activity.

private IDownloaderService mRemoteService;  
private IStub mDownloaderClientStub;  
private int mState;  
private boolean mCancelValidation;

// region Expansion Downloader
private static class XAPKFile {  
    public final boolean mIsMain;
    public final int mFileVersion;
    public final long mFileSize;

    XAPKFile(boolean isMain, int fileVersion, long fileSize) {
        mIsMain = isMain;
        mFileVersion = fileVersion;
        mFileSize = fileSize;
    }
}

private static final XAPKFile[] xAPKS = {  
        new XAPKFile(
                true, // true signifies a main file
                2, // the version of the APK that the file was uploaded against
                47529382L // the length of the file in bytes
        )
};
static private final float SMOOTHING_FACTOR = 0.005f;

/**
 * Connect the stub to our service on start.
 */
@Override
protected void onStart() {  
    if (null != mDownloaderClientStub) {
        mDownloaderClientStub.connect(this);
    }
    super.onStart();
}

/**
 * Disconnect the stub from our service on stop
 */
@Override
protected void onStop() {  
    if (null != mDownloaderClientStub) {
        mDownloaderClientStub.disconnect(this);
    }
    super.onStop();
}

/**
 * Critical implementation detail. In onServiceConnected we create the
 * remote service and marshaler. This is how we pass the client information
 * back to the service so the client can be properly notified of changes. We
 * must do this every time we reconnect to the service.
 */
@Override
public void onServiceConnected(Messenger m) {  
    mRemoteService = DownloaderServiceMarshaller.CreateProxy(m);
    mRemoteService.onClientUpdated(mDownloaderClientStub.getMessenger());
}

/**
 * The download state should trigger changes in the UI --- it may be useful
 * to show the state as being indeterminate at times. This sample can be
 * considered a guideline.
 */
@Override
public void onDownloadStateChanged(int newState) {  
    setState(newState);
    boolean showDashboard = true;
    boolean showCellMessage = false;
    boolean paused;
    boolean indeterminate;
    switch (newState) {
        case IDownloaderClient.STATE_IDLE:
            // STATE_IDLE means the service is listening, so it's
            // safe to start making calls via mRemoteService.
            paused = false;
            indeterminate = true;
            break;
        case IDownloaderClient.STATE_CONNECTING:
        case IDownloaderClient.STATE_FETCHING_URL:
            showDashboard = true;
            paused = false;
            indeterminate = true;
            break;
        case IDownloaderClient.STATE_DOWNLOADING:
            paused = false;
            showDashboard = true;
            indeterminate = false;
            break;

        case IDownloaderClient.STATE_FAILED_CANCELED:
        case IDownloaderClient.STATE_FAILED:
        case IDownloaderClient.STATE_FAILED_FETCHING_URL:
        case IDownloaderClient.STATE_FAILED_UNLICENSED:
            paused = true;
            showDashboard = false;
            indeterminate = false;
            break;
        case IDownloaderClient.STATE_PAUSED_NEED_CELLULAR_PERMISSION:
        case IDownloaderClient.STATE_PAUSED_WIFI_DISABLED_NEED_CELLULAR_PERMISSION:
            showDashboard = false;
            paused = true;
            indeterminate = false;
            showCellMessage = true;
            break;

        case IDownloaderClient.STATE_PAUSED_BY_REQUEST:
            paused = true;
            indeterminate = false;
            break;
        case IDownloaderClient.STATE_PAUSED_ROAMING:
        case IDownloaderClient.STATE_PAUSED_SDCARD_UNAVAILABLE:
            paused = true;
            indeterminate = false;
            break;
        case IDownloaderClient.STATE_COMPLETED:
            showDashboard = false;
            paused = false;
            indeterminate = false;
            validateXAPKZipFiles();
            return;
        default:
            paused = true;
            indeterminate = true;
            showDashboard = true;
    }
    int newDashboardVisibility = showDashboard ? View.VISIBLE : View.GONE;
    if (mDownloadViewGroup.getVisibility() != newDashboardVisibility) {
        mDownloadViewGroup.setVisibility(newDashboardVisibility);
    }
    mDownloadProgressBar.setIndeterminate(indeterminate);
}

/**
 * Sets the state of the various controls based on the progressinfo object
 * sent from the downloader service.
 */
@Override
public void onDownloadProgress(DownloadProgressInfo progress) {  
    mDownloadProgressBar.setMax((int) (progress.mOverallTotal >> 8));
    mDownloadProgressBar.setProgress((int) (progress.mOverallProgress >> 8));
    mProgressPercentTextView.setText(Long.toString(progress.mOverallProgress * 100 / progress.mOverallTotal) + "%");
}

/**
 * Go through each of the Expansion APK files and open each as a zip file.
 * Calculate the CRC for each file and return false if any fail to match.
 *
 * @return true if XAPKZipFile is successful
 */
void validateXAPKZipFiles() {  
    AsyncTask<Object, DownloadProgressInfo, Boolean> validationTask = new AsyncTask<Object, DownloadProgressInfo, Boolean>() {

        @Override
        protected void onPreExecute() {
            mDownloadViewGroup.setVisibility(View.VISIBLE);
            super.onPreExecute();
        }

        @Override
        protected Boolean doInBackground(Object... params) {
            for (XAPKFile xf : xAPKS) {
                String fileName = Helpers.getExpansionAPKFileName(MainActivity.this, xf.mIsMain, xf.mFileVersion);
                if (!Helpers.doesFileExist(MainActivity.this, fileName, xf.mFileSize, false))
                    return false;
                fileName = Helpers.generateSaveFileName(MainActivity.this, fileName);
                ZipResourceFile zrf;
                byte[] buf = new byte[1024 * 256];
                try {
                    zrf = new ZipResourceFile(fileName);
                    ZipResourceFile.ZipEntryRO[] entries = zrf.getAllEntries();
                    /**
                     * First calculate the total compressed length
                     */
                    long totalCompressedLength = 0;
                    for (ZipResourceFile.ZipEntryRO entry : entries) {
                        totalCompressedLength += entry.mCompressedLength;
                    }
                    float averageVerifySpeed = 0;
                    long totalBytesRemaining = totalCompressedLength;
                    long timeRemaining;
                    /**
                     * Then calculate a CRC for every file in the Zip file,
                     * comparing it to what is stored in the Zip directory.
                     * Note that for compressed Zip files we must extract
                     * the contents to do this comparison.
                     */
                    for (ZipResourceFile.ZipEntryRO entry : entries) {
                        if (-1 != entry.mCRC32) {
                            long length = entry.mUncompressedLength;
                            CRC32 crc = new CRC32();
                            DataInputStream dis = null;
                            try {
                                dis = new DataInputStream(zrf.getInputStream(entry.mFileName));

                                long startTime = SystemClock.uptimeMillis();
                                while (length > 0) {
                                    int seek = (int) (length > buf.length ? buf.length : length);
                                    dis.readFully(buf, 0, seek);
                                    crc.update(buf, 0, seek);
                                    length -= seek;
                                    long currentTime = SystemClock.uptimeMillis();
                                    long timePassed = currentTime - startTime;
                                    if (timePassed > 0) {
                                        float currentSpeedSample = (float) seek / (float) timePassed;
                                        if (0 != averageVerifySpeed) {
                                            averageVerifySpeed = SMOOTHING_FACTOR * currentSpeedSample + (1 - SMOOTHING_FACTOR) * averageVerifySpeed;
                                        } else {
                                            averageVerifySpeed = currentSpeedSample;
                                        }
                                        totalBytesRemaining -= seek;
                                        timeRemaining = (long) (totalBytesRemaining / averageVerifySpeed);
                                        this.publishProgress(new DownloadProgressInfo(totalCompressedLength, totalCompressedLength - totalBytesRemaining, timeRemaining, averageVerifySpeed));
                                    }
                                    startTime = currentTime;
                                    if (mCancelValidation)
                                        return true;
                                }
                                if (crc.getValue() != entry.mCRC32) {
                                    Log.e(Constants.TAG, "CRC does not match for entry: " + entry.mFileName);
                                    Log.e(Constants.TAG, "In file: " + entry.getZipFileName());
                                    return false;
                                }
                            } finally {
                                if (null != dis) {
                                    dis.close();
                                }
                            }
                        }
                    }
                } catch (IOException e) {
                    e.printStackTrace();
                    return false;
                }
            }
            return true;
        }

        @Override
        protected void onProgressUpdate(DownloadProgressInfo... values) {
            onDownloadProgress(values[0]);
            super.onProgressUpdate(values);
        }

        @Override
        protected void onPostExecute(Boolean result) {
            if (result) {
                mDownloadViewGroup.setVisibility(View.GONE);
            } else {
                mDownloadViewGroup.setVisibility(View.VISIBLE);
            }
            super.onPostExecute(result);
        }

    };
    validationTask.execute(new Object());
}

boolean expansionFilesDelivered() {  
    for (XAPKFile xf : xAPKS) {
        String fileName = Helpers.getExpansionAPKFileName(this, xf.mIsMain, xf.mFileVersion);
        if (!Helpers.doesFileExist(this, fileName, xf.mFileSize, false))
            return false;
    }
    return true;
}

private void setState(int newState) {  
    if (mState != newState) {
        mState = newState;
    }
}


@Override
protected void onDestroy() {  
    this.mCancelValidation = true;
    super.onDestroy();
}
// endregion

Add this to the end of onCreate in your main activity to start the downloads.

mDownloaderClientStub = DownloaderClientMarshaller.CreateStub(this, DownloaderService.class);

/**
 * Before we do anything, are the files we expect already here and
 * delivered (presumably by Market) For free titles, this is probably
 * worth doing. (so no Market request is necessary)
 */
if (!expansionFilesDelivered()) {

    try {
        Intent launchIntent = MainActivity.this.getIntent();
        Intent intentToLaunchThisActivityFromNotification = new Intent(MainActivity.this, MainActivity.this.getClass());
        intentToLaunchThisActivityFromNotification.setFlags(Intent.FLAG_ACTIVITY_NEW_TASK | Intent.FLAG_ACTIVITY_CLEAR_TOP);
        intentToLaunchThisActivityFromNotification.setAction(launchIntent.getAction());

        if (launchIntent.getCategories() != null) {
            for (String category : launchIntent.getCategories()) {
                intentToLaunchThisActivityFromNotification.addCategory(category);
            }
        }

        // Build PendingIntent used to open this activity from
        // Notification
        PendingIntent pendingIntent = PendingIntent.getActivity(MainActivity.this, 0, intentToLaunchThisActivityFromNotification, PendingIntent.FLAG_UPDATE_CURRENT);
        // Request to start the download
        int startResult = DownloaderClientMarshaller.startDownloadServiceIfRequired(this, pendingIntent, DownloaderService.class);

        if (startResult != DownloaderClientMarshaller.NO_DOWNLOAD_REQUIRED) {
            // The DownloaderService has started downloading the files, show progress
            initializeDownloadUI();
            return;
        } // otherwise, download not needed so we fall through to the app
    } catch (PackageManager.NameNotFoundException e) {
        Log.e(TAG, "Cannot find package!", e);
    }
} else {
    validateXAPKZipFiles();
}

This assumes that your layout is something like this:

<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"  
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:orientation="vertical"
    tools:context=".MainActivity">

    <!-- DOWNLOAD PROGRESS -->
    <RelativeLayout
        android:id="@+id/downloadViewGroup"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:gravity="center"
        android:visibility="gone">

        <TextView
            android:id="@+id/downloadTextView"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:gravity="center"
            android:lines="2"
            android:padding="10dp"
            android:text="@string/wait_file_download" />

        <ProgressBar
            android:id="@+id/downloadProgressBar"
            style="@android:style/Widget.ProgressBar.Horizontal"
            android:layout_width="match_parent"
            android:layout_height="wrap_content"
            android:layout_below="@id/downloadTextView"
            android:padding="10dp" />

        <TextView
            android:id="@+id/downloadProgressPercentTextView"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:layout_below="@id/downloadProgressBar"
            android:layout_centerHorizontal="true"
            android:lines="2"
            android:padding="10dp"
            tools:text="10%" />
    </RelativeLayout>

    <!-- YOUR MAIN CONTENT HERE -->
    <LinearLayout
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:orientation="vertical">

    </LinearLayout>
</LinearLayout>  

If everything is all right, this should download the expansion files from Google Play if ever the user deletes them from their location or tampers with them.

Using the expansion files

Now, to the final part of this tutorial, how to use the expansion files that you just downloaded. Let's assume we have some music files inside the expansion file and we want to play them using a media player. We need to use the APKExpansionSupport class to obtain a reference to the expansion file and then open the music asset from the file.

// Get a ZipResourceFile representing a merger of both the main and patch files
try {  
    ZipResourceFile expansionFile = APKExpansionSupport.getAPKExpansionZipFile(this, 2, 0);
    AssetFileDescriptor afd = expansionFile.getAssetFileDescriptor("path-to-music-from-expansion.mp3");

    try {
        mMediaPlayer.setDataSource(afd.getFileDescriptor(), afd.getStartOffset(), afd.getDeclaredLength());
    } catch (IllegalArgumentException | IllegalStateException | IOException e) {
        Log.w(TAG, "Failed to update data source for media player", e);
    }

    try {
        mMediaPlayer.prepareAsync();
    } catch (IllegalStateException e) {
        Log.w(TAG, "Failed to prepare media player", e);
    }
    mState = State.Preparing;

    try {
        afd.close();
    } catch (IOException e) {
        Log.d(TAG, "Failed to close asset file descriptor", e);
    }
} catch (IOException e) {
    Log.w(TAG, "Failed to find expansion file", e);
}

Reading media files from a ZIP

If you're using your expansion files to store media files, a ZIP file still allows you to use Android media playback calls that provide offset and length controls (such as MediaPlayer.setDataSource() and SoundPool.load()). In order for this to work, you must not perform additional compression on the media files when creating the ZIP packages. For example, when using the zip tool, you should use the -n option to specify the file suffixes that should not be compressed:

zip -n .mp4;.ogg main_expansion media_files

Submitting to play store

When submitting to the play store, you need to upload your expansion files after uploading the apk. Currently Google Play doesn't allow submitting an expansion file with the first versino of your app. When submitting a new app with expansion files, you must submit an app with build version 1 without an expansion file and then another one with build version 2 along with the expansion file.

If you ever need help setting this up or have some questions, don't hesitate to write to me at [email protected] or leave a comment.