Offline first Android application development: challenges and lessons learned

Tips on database structure, data synchronisation and database management

Mobile app development

06 December 2024

LogiNet Mobile Development Team

Create a high performing mobile app with us!

Offline first Android app development guide

Today, it is rare to find an application that does not require an internet connection to function. While some level of data caching is common, it is usually only used to improve the user experience. However, it is unusual for all data to be stored primarily on the device, with the server used only as a backup. This contributes to the scarcity of online information on the subject, making the creation of such a system require significantly more planning and research. We found ourselves in this situation when one of our clients approached us with an exciting and unique requirement for Android application development. Read the article for details!

As highlighted in the first part of our series, internet coverage is now ubiquitous and it is not often that a device cannot connect to some form of Wi-Fi or cellular network.

However, one of our customers approached us with a unique requirement due to geographical circumstances: the mobile application needed to operate in an offline-first mode, with data uploads and downloads occurring infrequently, sometimes after several days.

Fortunately, database management has become easier since the introduction of Room. Developers who have been working with Android for a long time may recall the significant amount of boilerplate code required to implement database management before Room, as well as the risks involved in writing database queries.

What exactly is Room?

Room is a persistence library, part of the Android Jetpack, that simplifies the use of SQLite databases. It has three main components:

Database: The database class which manages the SQLite database.
Entity: A class that defines the data model that represents tables in the database.
DAO (Data Access Object): An interface that defines database operations such as insert, query, update and delete.

How do you integrate Room into an Android project?

First, add the Room dependencies to your build.gradle file:

dependencies {
    def room_version = "2.5.2" // Ellenőrizd a legújabb verziót!
    
    implementation "androidx.room:room-runtime:$room_version"
    kapt "androidx.room:room-compiler:$room_version"

    // Kotlin Coroutines támogatás, ha szükséges
    implementation "androidx.room:room-ktx:$room_version"
}

Creating a simple table with Room

1. Create an entity

A class annotated with @Entity represents a database table. For example, a simple user entity:

@Entity
data class User(
    @PrimaryKey(autoGenerate = true) val id: Int = 0,
    val name: String,
    val age: Int
)

2. Create a DAO

The DAO defines how to access data in the database. For example, a simple query to get users:

@Dao
interface UserDao {

    @Insert(onConflict = OnConflictStrategy.REPLACE)
    suspend fun insertUser(user: User)

    @Query("SELECT * FROM user WHERE id = :userId")
    suspend fun getUserById(userId: Int): User?
    
}

3. Creating the Database class

The Database class, derived from RoomDatabase, manages the Room database. This is where we connect the Entity and DAO classes:

<

@Database(entities = [User::class], version = 1)
abstract class AppDatabase : RoomDatabase() {
    abstract fun userDao(): UserDao
}

4. Initialising Room database

The database can be created using Room.databaseBuilder() in the Application or Activity class.

val db = Room.databaseBuilder(
    applicationContext,
    AppDatabase::class.java, "appDatabase"
).build()

val userDao = db.userDao()

What are the challenges?

At this point, if we want to collect additional information, we run into obstacles. We might find a very simple example of how to store a class with one or two primitive data types in a database, or how to initialise the database itself. But beyond that, the developer is on his own.

Many questions remain unanswered when it comes to solving real-world problems. For example, how do these sample codes fit into an architecture that uses dependency injection? Or, if we use coroutines - which is now considered standard - how do we incorporate database calls? What's more, there are no clear answers as to whether Room supports the use of LiveData or Flow, how to create relationships between tables, or how to store tables that contain more than just primitive data members.

Answering these questions requires extensive research, often involving experimentation with different solutions.

The rest of this article aims to speed up this process by presenting a working solution to a complex problem.

Offline first mode: what requirements can arise?

In the following example, we aim for simplicity: we show solutions to the most common requirements, such as database structure, relationships between objects, and embedded objects.

Database structure

We will stick with the User object, but extend it a little.

@Entity
@Parcelize
data class User(
    @PrimaryKey
    val userId: String = "",
    @Embedded(prefix = "personalInfo_")
    val personalInfo: PersonalInfo?,
    val dataSyncState: DataSyncState,
    val lastSyncDate: Date? = null,
) : Parcelable

@Parcelize
data class PersonalInfo(
    val firstName: String,
    val lastName: String,
    val email: String,
    val phoneNumber: String,
) : Parcelable

As you can see, we have added a PersonalInfo object to the User class, which we have specified using the @Embedded annotation. This causes Room to add the properties from the PersonalInfo object to the User table with a personalInfo_ prefix, such as personalInfo_firstName, personalInfo_lastName, etc. This approach is useful when we don't want to maintain a separate table to store our object because it is closely related to another object.

We also introduced a date type parameter and a new enum type. If we tried to build the project now, we would get an error. This error would indicate that Room can only store primitive types by default, but our table now contains more complex types.

To fix this, we need to tell Room how to store these data types. This is where type converters come in.

class DateTypeConverter {
    @TypeConverter
    fun fromDate(value: Long?): Date? {
        return value?.let { Date(it) }
    }

    @TypeConverter
    fun toDate(date: Date?): Long? {
        return date?.time
    }
}

class DataSyncStateTypeConverter {
    @TypeConverter
    fun fromStatus(enum: DataSyncState?): String? {
        return enum?.name
    }

    @TypeConverter
    fun toStatus(value: String?): DataSyncState? {
        return value?.let {
            enumValueOf<DataSyncState>(it)
        }
    }
}

As you can see from the example above, in the case of Date, we use the Time property available on the object, which returns a Long type.

As this is already a primitive type, Room can store it.

In the case of the enum value, we convert the enum to a string.

These type converters need to be registered in our database class, similar to entities.

@Database(
    entities = [
        User::class,
        Task::class,
    ], version = 1, exportSchema = false
)
@TypeConverters(
    DateTypeConverter::class,
    DataSyncStateTypeConverter::class,
)
abstract class AppDatabase : RoomDatabase() {

    abstract fun userDao(): UserDao

    abstract fun taskDao(): TaskDao

}

As you can see from the example code, registering type converters is easily done using the @TypeConverters annotation.

There may be cases where we want to handle a complex object or a list of objects by creating a type converter.

In such cases, we can use a JSON converter library (Gson, Moshi) which allows us to create a JSON string and store it in the database.

However, this is not the most elegant solution and we recommend using it only as a last resort.

class WeatherListTypeConverter {

    private val gson = GsonBuilder()
        .create()

    @TypeConverter
    fun listToString(list: List<DemoObject>): String {
        return gson.toJson(list)
    }

    @TypeConverter
    fun stringToList(data: String): List<DemoObject> {
        val type = object : TypeToken<List<DemoObject>>() {}.type
        return try {
            gson.fromJson(data, type)
        } catch (e: JsonParseException) {
            listOf()
        }
    }
}

Here we create a JSON string from a list of DemoObjects.

We might use such a 'heresy' when we do not want to create a separate table for our DemoObject and link it to our object with a one-to-many relationship.

Those of you who have been paying attention may have noticed that we have added a Task class to the list of entities in the AppDatabase class.

Creating a relationship between two objects

This example demonstrates how to create a relationship between two objects.

The Task class looks like this:

@Entity(
    foreignKeys = [ForeignKey(
        entity = User::class,
        parentColumns = ["userId"],
        childColumns = ["taskId"],
        onDelete = ForeignKey.CASCADE,
    )]
)
data class Task(
    @PrimaryKey
    val taskId: String = "",
    val description: String,
    val userId: String, // Foreign key
)

The essence is quite clear. Even those with minimal knowledge of relational databases will recognise the concept of a foreign key. In the example, multiple tasks can belong to a single user, so we need to establish a one-to-many relationship between the two objects.

This can be achieved in our Task class by adding an ID that references the User table. When creating a Task object, the corresponding userId must also be provided.

It is also clear how to specify the foreign key in the @Entity annotation. You need to specify which class it points to - in this case, the User class - along with the ID of the User class and the ID of the Task class.

The final onDelete parameter specifies what should happen to the task records in the database when the associated user is deleted.

In the example, ForeignKey.CASCADE tells the database that when a user record is deleted, all tasks associated with that user should also be deleted. Of course, other configurations can be specified.

To make querying easier, we can create a class that contains both the user and the list of tasks.

data class UserWithTasks(
    @Embedded val user: User,
    @Relation(
        parentColumn = "userId",
        entityColumn = "taskId"
    )
    val tasks: List<Task>,
)

The code snippet above is fairly self-explanatory and requires little additional explanation.

What does the UserDao look like?

@Dao
interface UserDao {

    @Query("select * from User")
    suspend fun getUsers(): List<User>

    @Query("select * from User where userId = :id")
    suspend fun getUser(id: String): User?

    @Query("SELECT * FROM User where userId = :id")
    suspend fun getUserWithTasks(id: String): UserWithTasks


    @Query("select * from User")
    fun getUsersFlow(): Flow<List<User>>

    @Query("select * from User where userId = :id")
    fun getUserFlow(id: String): Flow<User?>

    @Query("SELECT * FROM User where userId = :id")
    fun getUserWithTasksFlow(id: String): Flow<UserWithTasks>


    @Insert(onConflict = OnConflictStrategy.REPLACE)
    suspend fun insertUser(user: User)

    @Update(onConflict = OnConflictStrategy.REPLACE)
    suspend fun updateUser(user: User)

    @Delete
    suspend fun deleteUser(user: User)

    @Query("delete from User")
    suspend fun deleteAll()

}

The getUserWithTasks() method allows you to easily query the UserWithTasks object, which contains the user and the list of tasks.

In summary, we have managed to link the two tables with relatively little code.

However, it is important to pay close attention to the correct specification of the IDs used for the relationship: both the type and the name are critical. A common mistake is to use id instead of userId. If this happens, Room will immediately inform you that it cannot perform the relationship mapping.

Some additional methods have been added to the DAO that return a flow type and are not suspended functions. Their role will be discussed later in the section on synchronisation.

For completeness, let's look at the TaskDao, which is very similar to the UserDao:

@Dao
interface TaskDao {

    @Query("select * from Task where taskId = :id")
    suspend fun getTask(id: String): Task?

    @Query("select * from Task")
    suspend fun getTasks(): List<Task>


    @Query("select * from Task where taskId = :id")
    fun getTaskFlow(id: String): Flow<Task?>

    @Query("select * from Task")
    fun getTasksFlow(): Flow<List<Task>>
    

    @Insert(onConflict = OnConflictStrategy.REPLACE)
    suspend fun insertTask(task: Task)

    @Update(onConflict = OnConflictStrategy.REPLACE)
    suspend fun updateTask(task: Task)

    @Delete
    suspend fun deleteTask(task: Task)

    @Query("delete from Task")
    suspend fun deleteAll()

}

Earlier we mentioned that we would provide an example of how to create the database and integrate it with a dependency injection framework. There are many excellent dependency injection frameworks, and among them, we will introduce Hilt, which is recommended by Google and probably the most widely used.

In a nutshell, Hilt works with annotations and is optimised for Android.

If we want to use a dependency in an Activity or Fragment, we just need to annotate it with @AndroidEntryPoint. For ViewModels the @HiltViewModel annotation is used.

The heart of the framework is the modules, where we define how to instantiate the objects we want to inject later.

Database creation module

@Module
@InstallIn(SingletonComponent::class)
class DatabaseModule {

    @Singleton
    @Provides
    fun provideDataBase(
        @ApplicationContext context: Context,
        configuration: Configuration,
    ): AppDatabase {
        return Room.databaseBuilder(context, AppDatabase::class.java, configuration.appDatabaseName)
            .build()
    }

    @Singleton
    @Provides
    fun provideUserDao(database: AppDatabase): UserDao {
        return database.userDao()
    }

    @Singleton
    @Provides
    fun provideTaskDao(database: AppDatabase): TaskDao {
        return database.taskDao()
    }

}

Here we tell the dependency injection framework how to instantiate the AppDatabase object, as well as how to provide the UserDao and TaskDao.

In short, an object becomes accessible in the dependency graph if there is a method in one of the modules annotated with @Provides or @Binds whose return type matches the given object.

Note: The UserDao and TaskDao can already be used in our ViewModels or UseCases. However, we prefer to add another layer in front of the DAOs. For example:

<

interface UserStorage {

    suspend fun getUser(): User?

    suspend fun saveUser(user: User)

}

@Singleton
class UserStorageImpl @Inject constructor(
    private val userDao: UserDao,
): UserStorage {

    override suspend fun getUser() = userDao.getUser()

    override suspend fun saveUser(user: User) {
        userDao.insertUser(user)
    }
}

This way, we do not call the database methods directly on the DAOs, and we can include additional logic.

Dependency injection for storage:

@Module
@InstallIn(SingletonComponent::class)
abstract class StorageModule {

    @Binds
    abstract fun provideUserStorage(userStorage: UserStorageImpl): UserStorage

    @Binds
    abstract fun provideTaskStorage(userStorage: TaskStorageImpl): TaskStorage

}

Data synchronisation method

We already know how to store data in the database. However, there may be a requirement to upload the data stored in the database offline to the server regularly - when an internet connection is available, of course.

Several questions immediately arise: when, how and what should we synchronise? Should it happen automatically when the Internet is available? Or should there be an indicator that informs the user about the Internet connection and whether there is data to synchronise? How do we decide what data to upload?

In the next section, we will present a case where we automatically detect if there is an internet connection and if there is data to synchronise. The result could be a visible indicator within the application showing the state of the synchronisation process. If the conditions are met, this could include displaying a synchronisation screen with the data to be uploaded and performing the synchronisation itself.

Is there an Internet connection?

class InternetConnectivityListener(private val context: Context) {

    val networkState = MutableStateFlow(NetworkState.UNKNOWN)

    private val connectivityManager: ConnectivityManager =
        context.getSystemService(Context.CONNECTIVITY_SERVICE) as ConnectivityManager

    private val networkCallback = object : ConnectivityManager.NetworkCallback() {
        override fun onAvailable(network: Network) {
            super.onAvailable(network)
            networkState.value = NetworkState.CONNECTED
        }

        override fun onLost(network: Network) {
            super.onLost(network)
            networkState.value = NetworkState.NOT_CONNECTED
        }
    }

    init {
        registerNetworkCallback()
    }

    private fun registerNetworkCallback() {
        val networkRequest = NetworkRequest.Builder()
            .addCapability(NetworkCapabilities.NET_CAPABILITY_INTERNET)
            .build()
        connectivityManager.registerNetworkCallback(networkRequest, networkCallback)
    }

    fun unregisterNetworkCallback() {
        connectivityManager.unregisterNetworkCallback(networkCallback)
    }
}

interface GetNetworkStateUseCase {

    fun invoke(): Flow<NetworkState>
}

class GetNetworkStateUseCaseImpl @Inject constructor(
    @ApplicationContext
    private val applicationContext: Context,
) : GetNetworkStateUseCase {

    private val internetConnectivityListener by lazy {
        InternetConnectivityListener(applicationContext)
    }

    override fun invoke(): Flow<NetworkState> {
        return internetConnectivityListener.networkState
    }
}

In the example, we register an internet connectivity listener, introduce an enum that reflects the current state of the internet, and make it accessible via a use case.

An important point to note is that this returns a Flow type. Android Flow is part of the Kotlin Coroutines and is a library for handling asynchronous data streams. Using Flow, we can create data streams and respond to data changes in real-time.

In Android, Flow is typically used in scenarios where continuously updated data needs to be retrieved or monitored, such as database changes, the results of network calls, or other events. In this case, it is used to monitor changes in internet connectivity.

What do we synchronise?

interface HasUnSyncedDataUseCase {

    fun invoke(): Flow<Boolean>
}

class HasUnSyncedDataUseCaseImpl @Inject constructor(
    private val userStorage: UserStorage,
    private val taskStorage: TaskStorage,
) : HasUnSyncedDataUseCase {

    override fun invoke(): Flow<Boolean> {
        val flows = listOf(
            userStorage.getUsersFlow(DataSyncState.IN_LOCAL_DATABASE),
            taskStorage.getTasksFlow(DataSyncState.IN_LOCAL_DATABASE),
        )

        return combine(flows) {
            val hasUnSyncedUsers = it[0].isNotEmpty()
            val hasUnSyncedTasks = it[1].isNotEmpty()

            hasUnSyncedVendors || hasUnSyncedModifiedVendors
        }
    }
}

This is where the database methods written in DAOs that return a flow come into play. Using the use case above, we can easily be notified when a new record is added to the database.

Let's look at the DataSyncState enum:

enum class DataSyncState {
    SYNCED_TO_CLOUD,
    IN_LOCAL_DATABASE,
    UNDEFINED,
}

This enum allows us to keep track of whether a particular database record has been synchronised or not. When a record is created, this enum is set to IN_LOCAL_DATABASE. When synchronisation is completed, the enum value is set to SYNCED_TO_CLOUD, indicating that the record has been uploaded.

Where the two meet:

interface GetCurrentSyncStateUseCase {

    fun invoke(): Flow<SyncStateModel>
}

class GetCurrentSyncStateUseCaseImpl @Inject constructor(
    private val hasUnSyncedDataUseCase: HasUnSyncedDataUseCase,
    private val getNetworkStateUseCase: GetNetworkStateUseCase,
) : GetCurrentSyncStateUseCase {

    override fun invoke(): Flow<SyncStateModel> {
        return combine(getNetworkStateUseCase.invoke(), hasUnSyncedDataUseCase.invoke()) { networkState, hasUnSyncedData ->
            when (networkState) {
                NetworkState.UNKNOWN, NetworkState.NOT_CONNECTED -> {
                    SyncStateModel(
                        state = SyncState.NO_INTERNET,
                    )
                }
                NetworkState.CONNECTED -> {
                    if (hasUnSyncedData) {
                        SyncStateModel(
                            state = SyncState.LAST_UPDATE,
                            lastUpdateTime = "Last update",
                        )
                    } else {
                        SyncStateModel(
                            state = SyncState.UP_TO_DATE,
                        )
                    }
                }
            }
        }
    }
}

enum class SyncState {
    UP_TO_DATE,
    NO_INTERNET,
    LAST_UPDATE;
}

data class SyncStateModel(
    val state: SyncState,
    val lastUpdateTime: String? = null
)

As shown, we can simply combine the two dynamically changing data points. The GetCurrentSyncStateUseCase can easily be linked to a UI indicator that informs the user if there is something to synchronise and if it is currently possible.

What data needs to be synchronised?

We can list on a screen for the user what data needs to be synchronised. For example, we can add the following method to the UserDao, which will only query the user records that have not yet been synchronised:

@Query("select * from Batch where dataSyncState = :syncState")
suspend fun getUsers(syncState: DataSyncState): List<User>

If we want to upload the data, we can do it as follows:

interface UploadUnSyncedDataUseCase {

    suspend fun invoke(): Job
}

class UploadUnSyncedDataUseCaseImpl @Inject constructor(
    private val userStorage: UserStorage,
    private val taskStorage: TaskStorage,
    private val uploadUserUseCase: UploadUserUseCase,
    private val uploadTaskUseCase: UploadTaskUseCase,
    private val updateLastSyncDateUseCase: UpdateLastSyncDateUseCase,
) : UploadUnSyncedDataUseCase {

    override suspend fun invoke() =
        CoroutineScope(Dispatchers.IO + SupervisorJob()).launch {
            // Sync users
            // Find all un synced user
            val unSyncedUsers = userStorage.getAllUser(DataSyncState.IN_LOCAL_DATABASE)

            // Upload all un synced user
            val userSyncJobs = unSyncedUsers.map { user ->
                async {
                    uploadUserUseCase.invoke(user)
                }
            }
            val userSyncResults = mutableListOf<Result<UserDTO>>()
            userSyncResults.addAll(userSyncJobs.awaitAll())

            // Sync tasks
            // Find all un synced task
            val unSyncedTasks = taskStorage.getAllTask(DataSyncState.IN_LOCAL_DATABASE)

            // Upload all un synced task
            val taskSyncJobs = unSyncedTasks.map { task ->
                async {
                    uploadTaskUseCase.invoke(task)
                }
            }
            val taskSyncResults = mutableListOf<Result<TaskDTO>>()
            taskSyncResults.addAll(taskSyncJobs.awaitAll())

            updateLastSyncDateUseCase.invoke()
        }
}

Here we fetch all the unsynchronised data and upload it asynchronously. The UploadUserUseCase and UploadTaskUseCase contain, as their names suggest, the API calls responsible for the upload. In addition, if the upload is successful, these cases can update our database to reflect the new synchronisation state.

With the code snippets above, we were able to tell the user what data needs to be uploaded and the current internet status, list the unsynchronised data, and finally, successfully upload the data.

Offline login

Here we introduce another interesting requirement arising from the offline app mode. How can we ensure that the user is authenticated to the server when logging in, and at the same time automatically log them out every day, but still allow them to log in offline? Clearly, these two requirements are contradictory. If there is no internet, we cannot make a call to the server, but the login must still work without an internet connection.

The solution is as follows. During the login, we check if there is an internet connection. If there is, the normal login process takes place. If the login is successful, we store the username/password pair associated with the user in EncryptedSharedPreferences using a hash-based approach. The hashing and use of EncryptedSharedPreferences is necessary due to the sensitive nature of the password. This ensures that the most recently logged-in user can log in offline.

To achieve a daily logout, we start a worker after a successful login that logs the user out after 24 hours.

Learn how Zebra devices simplify Android app development for specific hardware, including label printing, QR code scanning, Kiosk Mode and PDF creator.

At LogiNet, we assist you with both native and cross-platform mobile application development. Our professional experts are proficient in various technologies: creating native iOS and Android applications using Kotlin and Swift, and cross-platform solutions using Flutter. Our mobile development team also participates in resource outsourcing projects. Learn more about our mobile solutions!

Discover how we lead our partners to success

We partner with product owners and founders by developing products from scratch or growing the existing product. These real-world examples highlight how we help companies innovate and succeed.