In the last issue, we looked at an exemplary implementation of data backup on the example of Apple iOS devices. How is it on other platforms? Today we will look at how to hack databack app iOS – the Google Android platform and find out how to save data with and without root, how to restore data from a backup, and how to unearth someone else’s local or cloud backup.
Let’s get straight to the terminology. In this article, we will write exclusively about the version of Android that comes with Google services. We are not particularly interested in third-party firmware: the number of their users is minimal while creating and restoring backup copies of data when flashing the next “custom” these users are excellent at. No, today we will talk about the other 99% of users who want to open the box, enter their login and password from the account and get something workable.
The article is based on a study during which we used about ten devices from ASUS, Google Nexus, LG, Motorola, Oppo and Sony. Both restoring data to the same device after a factory reset and migrating data to another device were tested.
Software from the manufacturer
Device manufacturers often release proprietary data backup utilities. Some (for example, Sony) offer to install software on a computer, while others (ASUS, LG) build the corresponding functionality into the firmware. Samsung provides the ability to create backups in its own cloud. In short, confusion and vacillation.
Combines solutions from manufacturers two things. Firstly, the created backup will be complete enough, which allows you to fully restore data after resetting the device, updating the firmware, or upgrading. Secondly, you will not be able to restore a backup from a Sony phone to an ASUS tablet (and vice versa): you need to restore it using the same software on a model from the same manufacturer.
However, if you plan to use the device for a long time, why not create a backup? Yes, it is not always convenient, and yes, it is not automated in any way, but there is a possibility. And if something happens to your phone and if you decide to replace it with a device from the same manufacturer, then you might even be able to restore data to it. Of course, there is no certainty: the manufacturer guarantees successful recovery only to a device of the same model from which the data was copied.
Backup: Google version
Android devices are a motley zoo of platforms, architectures, manufacturers, hardware, and software configurations. It is difficult to ensure that backups created from a phone from one manufacturer do not destabilize a smartphone on a completely different architecture. This is probably the main reason for the speed with which Google develops a backup mechanism.
Apart from the mechanisms for synchronizing contacts, calendar, and other Google applications with the cloud that existed since the first version of Android, a full-fledged mechanism for backing up applications and settings appeared only in Android 4.3. It was only available in development mode and only through the Android Debug Bridge. In other words, it didn’t exist for “ordinary” users.
At some point, Google started syncing some data with the cloud. When restoring the device, it was proposed to restore data (shortcuts, applications, and settings) from one of the previous devices. This functionality, strictly speaking, is not part of Android, but is implemented in Google’s proprietary services.
With Android 6.0, cloud backup of settings has officially become part of the operating system. Now it is enough for the application developer to include a checkbox in the application manifest that allows data backup, and the system will automatically copy them to the cloud. Of course, the cloud is from Google, and the data is tied to a Google Account, so users of AOSP builds without Google services are left out. Well, let’s look at these mechanisms in more detail. Breaking the chronology, let’s start with the most modern and interesting mechanism introduced in Android 6.0.
Android 6.0: we made it!
Starting with Android 6.0, the system automatically saves system and application settings to the user’s Google Drive. Decided to upgrade your device? Your new smartphone will automatically pick up the settings from the cloud, install the applications you used on your old device, and automatically configure them in the way you are used to. Almost like Apple! And so it all worked in the preliminary builds of Android M until the release.
And in the official version of Android 6.0, the developers have dramatically changed the plate. If in preliminary builds automatic backup worked for all applications whose authors did not explicitly block this feature (opt-out flag in the manifest), then in the official version of the system, backups are created only for applications whose authors explicitly requested the service ( opt-in via manifest) and added support for Android 6.0 (target API level 23).
How many developers do you think have taken advantage of this opportunity? In the excellent article Android 6.0 has a great auto backup system that no one is using (yet), Ars Technica journalists took a detailed look at which applications use and which do not use the built-in backup mechanism in Android 6.0.
The results were… unexpected. First of all, the built-in backup mechanism DOES NOT USE Google applications. Yes, yes, the developer of this magnificent system decided to do without it. Basic system settings, alarms, and quiet mode are restored, but Google app data is not. But clients of social networks, email clients, games and other popular applications are in no hurry to add support. Of course, the situation will change over time, but for now, this is how it is. After resetting an editorial Nexus 6 and restoring from the cloud, the following happened:
- All applications are restored. At the same time, they were installed from Google Play, that is, the latest versions were always restored;
- Some of the settings have been restored: built-in keyboard languages, “silent mode” settings, alarm clocks;
- The history of calls and SMS was not restored;
- Facebook settings not restored;
- The data of most applications was not restored (for example, gReader Pro had to be configured again).
In other words, the system installed all previously installed applications, but did not restore the data of the vast majority of them. However, contacts and email were picked up from the cloud, and access to photos too. And to set up a couple of dozen applications again is no stranger to us. You can read more about how the Android Backup Service works in Google Help.
Can data be retrieved from the cloud?
If Google can save data to the cloud, then we can retrieve it, right? Let’s see what can be done.
First, just like in the example of downloading data from iCloud described in the previous issue, you need a username and password for a Google account. That’s not all; if two-factor authentication is enabled (and it is being activated more and more often), a one-time code will also be required, which will be generated by the Google Authenticator, Microsoft Authenticator, FreeOTP or any of the many third-party applications (they work on the same principle, and only the cryptographic initialization code that is issued to the user differs in the form of a colored QR code).
We also need the appropriate software (you can do without it – more on that below). We used Elcomsoft Cloud eXplorer as software. We launch the application, log in to our Google account, and select the data to download:
The amount of information Google collects is, frankly, shocking. Yes, abstractly we know that we are being watched. We know that on every page that we visit, every bookmark in the browser is carefully saved (of course, for our convenience – synchronization!). Of course, every search query addressed to the Good Corporation is saved (have you already realized that searching the Internet for a recipe for creating a vigorous bond is not a good idea?):
A list of devices, applications installed on them, and the actual application data are available:
Of course, there is access to photos (hello, iCloud!):
For our convenience, a detailed history of movements is saved:
Traveled well! And here is the same in text form:
In general, just a lot of interesting things. To be honest, there’s a lot more to be found in a Google account than Apple’s solutions ever dared to keep.
Where and how is all this data retrieved? But this is perhaps the most interesting. “Corporation of Good” adheres to the policy of maximum information openness. You can always view or download all the information that the corporation has collected about you. You can delete any data, and this does not require you to delete your account. Finally, you can disable the collection of certain types of data (for example, set your phone so that your location information is not sent to Google).
In this context, we are interested in the clause according to which you have the right to download all the information collected about you by Google. The official way to do this is through the Google Takeout service. Here you can choose what types of data we want to download:
The selected data will be packed into a file and provided as an archive:
As you can see, nothing complicated. What’s the catch? Why do I need Elcomsoft Cloud Explorer? Wouldn’t it be easier to download data directly from Google Takeout?
There is no trick, as such. And the data can be downloaded. A small problem arises with the analysis of the received information. To store and export data, Google uses a lot of different formats (mostly open). For example, you will receive data about your movements in the form of a JSON file – do what you want with it, Google is not your assistant. He is not an assistant to the special services either: according to the official position of the company, Google obeys the law and transmits data in clear text and in a standard format … what will be done with them next is not the company’s concern in the least. But the very fact of issuing information to the special services Google will record, save and publish.
One more moment. When downloading through the Google Takeout service, the user will definitely receive a notification: such and such data was downloaded from such and such an IP. If you do not need it, turn to third-party tools: their use does not cause Google anxiety. And for a snack – the most interesting. Google Takeout for some reason does not allow downloading passwords synced to Chrome. And Elcomsoft Cloud eXplorer extracts them without any problems:
Magic? No, Google provides access to this information as well. All you need to do is access your synced Chrome browser data and download it as an XML file. A web interface for viewing synchronized passwords is available.
Backup via ADB
Starting with Android 4.3, the system has a regular way to create a backup through the Android Debug Bridge (ADB) interface. To create a backup, you need to use something like this command:
adb backup -apk -shared -system -all -f C:\backup.ab
Why “about”? Yes, by virtue of the same “zoo” of devices. We tested a large number of devices from different manufacturers running different versions of Android from 4.4 to 6.0.1 inclusive. On some devices, the command worked in this form, on others, specifying keys
-sharedled to the creation of an empty file, and still, others refused to accept the key
-all. We could not catch any logic in the behavior of the adb command; one thing can be said for sure: its behavior depends little on the version of Android. Rather, the dependence here on the settings specified by a particular manufacturer.
For example, on an editorial Nexus 6 running Android 6.0.1, the following command passed:
adb backup -all -f c:\nexus6.ab
But the option
-noapk“broke” the backup: an empty file was created. And
adb backupit may not work if data partition encryption is enabled. Recall that encryption is enabled by default on Nexus devices, as well as (at the request of Google) on all devices that come with Android 6 preinstalled and equipped with 64-bit processors.
One more moment.
Adb backupdesigned in such a way that a backup created on one device can be restored without problems on another. And the key word here is not “restore” at all, but “no problem”: the restored device should work and should not fail. Accordingly, only those data and settings are saved and restored that will definitely not harm stable operation even when data is transferred from a 32-bit smartphone with a MediaTek chipset to a 64-bit tablet with Intel Atom.
It will not be difficult to restore data from a backup using the command
What gets into such backups? Again, the answer depends on the device manufacturer. For example, in Sony smartphones, contacts, call logs, and SMS are not included in ADB backups, while Samsung phones save this data. The same applies to device settings (which are often manufacturer-specific) and system application data.
The list of installed applications is definitely included in the backup. APK files are extracted and saved (if the corresponding option was specified during the creation of the copy). But application data may or may not be saved: it depends on the developers who allow or do not allow backup in the application manifest file.
From a practical point of view, we have not been able to get much use from such backups. When restoring via adb restore, you still have to log in to Gmail, Facebook, and other mail and social network clients. The FBReader and Nova Launcher settings were not saved (which, by the way, has its own backup mechanism). What has been preserved? I hardly remember that on some devices it was possible to restore the call log and the SMS archive.
The built-in backup mechanism in Android is also used by some third-party applications. They are innumerable, so we will not consider all of them. The principle of operation of all such programs is similar, and they differ only in added features. The most popular program of this type is Helium AppSync and Backup from the well-known ClockworkMod development team (CWM custom recovery is their development).
What is inside?
Backups created via ADB are a fairly simple thing. The output is an archive containing application data (depending on the settings – and the actual .apk). They are saved in the form in which the application itself stores them. As a rule, applications use the SQLite format, less often – XML, even more rarely – binary data in a native format.
So many tools have been invented for SQLite analysis that a very brief overview would require a separate article. Let me just say that with the help of such tools, you can pull out deleted records. Example? Please. If you are lucky and the manufacturer of your phone has allowed you to copy the call log and SMS, then you can restore messages and calls that were deleted by the user.
Custom recovery and Nandroid backups
Talking about the Android backup system, one cannot fail to mention the phenomenon of Nandroid backups. The term was formed from the words NAND (a type of flash memory) and Android and is most often used in the context of creating a copy of the entire user (and often system) partition using custom recovery (most often CWM or TWRP). The portability of Nandroid backups is limited. It is recommended to restore them to the same device from which they were made, and preferably to the same firmware.
What is inside?
Nandroid backups are pretty abstract. Each recovery has its own format. Moreover, the formats may differ depending on the device (let me remind you: custom recovery is, in fact, a separate operating system with its own features for each supported model).
What do we get? Most often, the output is an image of the file system (together with the original file system that was used on a particular device). The analysis is simple: mount the image (you will need a driver for the appropriate file system) and roam the file system.
In some cases, we are given a set of ZIP archives with application data. Here, too, everything is simple: we go into the archive and look; the data format is the same as in the case of adb backup, but the data set itself is much more complete. Sometimes a single archive is created with a set of files inside. We came across both simple .zip and .tar.gz (the extension may not match).
A common feature of Nandroid backups is that not a single custom recovery we tested (and we tested dozens of options) creates a complete image of the data partition. By a “full” image, we mean an image that contains both the original file system and unallocated blocks – free space. Analysis of free blocks would allow scanning for deleted files. Unfortunately, it won’t work. If that’s what you want, you’ll have to use other methods. (In parentheses, I note that the image of the system partition is created entirely by most recovery, with all the “offal”.)
Standing apart is Titanium Backup, the most popular backup software available on Google Play. With its help, we were able to back up the data of all installed applications (including .apk), and then successfully restore them to a new device. Please note: Titanium copies binary data from app sandboxes, so we highly recommend not using it to migrate Android system app data. When restoring them to another device, the system may become unstable.
Today we have explored some of the backup mechanisms available on Android devices. Platform fragmentation does not allow considering all existing methods and applications designed to facilitate data backup and migration, but even those that have been described demonstrate rather severe limitations both in compatibility and in the completeness of the copied data.
In general, our conclusion is this. Are you using stock Android? Turn on cloud sync for contacts and photos. Cloud backup can partially restore previously installed applications, and if you are very lucky, some data may also be partially restored in individual applications. If the device from which the backup was created and to which it was restored is running Android 6.0 or higher, then more data will be restored from the cloud compared to older versions of Android.
The built-in mechanism
adb backupcan help recover some of the data for users of older versions of Android. Third-party applications are only effective when rooted. Using custom recovery and creating a Nandroid backup will solve most of the problems, but this mode is available to a meager number of users.
As a result, Android’s backup system gets a “better than nothing” rating from us. Nobody could surpass Android in terms of inconvenience: even in the old Windows Phone 8, backup works much better.
And what about backups for market outsiders – phones running the mobile version of Windows and BlackBerry 10? More on that in the next issue!