Developing with ImmuDB: practical examples in a Python and Django environment

Ensuring data integrity in databases and traceability is becoming increasingly crucial in today’s digital systems, especially in contexts where immutability is a core requirement. ImmuDB offers an open-source solution precisely for this challenge. In the first part of this article, we introduced what ImmuDB is, when it is worth using, and what alternative solutions exist. In the second part of this article, we dive deeper into the technical aspects: using a Python-based Django project as an example, we examine the practical applicability of ImmuDB, the challenges encountered, and insights gained during development.

When ordering a product online, it’s common for the package to pass through multiple hands - couriers, subcontractors, warehouses - all involved in the delivery process. In such scenarios, audit trails in databases become essential: every transfer and shift in responsibility to be recorded in a verifiable and tamper-proof manner. An immutable database like ImmuDB enables exactly this. It ensures that each handover is reliably and permanently traceable, clearly mapping the chain of events and supporting the clarification of potential liabilities. The task of the database, in this scenario, is to log every transition: from whom, to whom, and in what condition the product was passed along.

Python is well-suited for this use case, especially if you're already comfortable not just with the language but also with a framework that allows relatively easy creation of a custom database connector. Within Python, Django is one of the most mature frameworks for this purpose.

Versions:

• Python >= 3.13

• Django >= 5.1

• ImmuDB = 1.9.6

• ImmuDB-py = Latest

Challenges of integrating ImmuDB

ImmuDB Python integration: opportunities and limitations in practice

Since ImmuDB is built in Go, it would be a natural choice to also use the Go language for development. However, this is precisely why exploring an alternative, widely adopted language such as Python may provide more insightful results. For using ImmuDB with Python, the recommended library is immudb-py, which communicates with the database via the gRPC protocol - a setup often explored in Python gRPC tutorial scenarios.

This, however, introduces challenges from the very first step: at the time of writing this article, the gRPC protocol does not support several commonly used data types such as UUID—despite the fact that ImmuDB itself does. This means that by choosing Python, we may lose access to certain functionality that the database otherwise supports.

Beyond the gRPC layer, all further capabilities depend on the maturity and maintenance of the immudb-py library. If Codenotary releases a new version of ImmuDB with additional features, we may only gain access to them later - or not at all - if the necessary updates are not implemented in the Python library. While this is not a guaranteed limitation, it remains a risk worth considering.

SQL in ImmuDB – capabilities and limitations

Putting aside the concerns mentioned above, it is encouraging to see that our key–value pair-based database can also be used - at least in part - as a traditional relational database. That said, developers with prior experience in relational systems may encounter a number of unexpected behaviours if they dive into ImmuDB’s SQL layer without proper due diligence. Below, we outline a few key considerations that - while not exhaustive - can help avoid common pitfalls.

Data types

The current documentation for ImmuDB SQL supports only a limited set of fundamental data types:

• Boolean

• 64-bit integer

• UTF-8 string

• BLOB (binary large object)

• Timestamp (microsecond precision, timezone-agnostic, UTC-based)

• IEEE-754 64-bit floating-point number

• UUID (128-bit)

• JSON (RFC 8259 compliant)

This may appear modest in contrast to both the wide variety of predefined types in Postgres and the option for developers to define their own types, such as enums. However, it is important to keep in mind that ImmuDB is relatively new, and many existing alternatives, particularly Blockchain-based solutions, offer even fewer options.

Foreign keys and table relationships

In SQL, relationships between tables via foreign keys - such as one-to-one or one-to-many - are typically expressed using some variation of the JOIN keyword, which represents different set operations. ImmuDB supports only one of these operations: the intersection, the INNER JOIN, which is, admittedly, the most commonly used form. This limitation is particularly important to note, as ImmuDB currently does not allow for the explicit definition of foreign keys at the table schema level. In other words, what would typically be a foreign key is merely a plain value in ImmuDB, with no database-level enforcement or referential integrity checks.

Queries

When constructing complex queries, the main query often needs to be broken down into multiple logical subqueries.

In ImmuDB, the following rules apply:

• All subqueries must contain the same number and type of fields.

• Duplicate results are automatically discarded.

• Subqueries can only be used in the FROM clause, not directly after SELECT.

Supported aggregate functions:

• COUNT

• MIN/MAX

• AVG

• SUM

Supported SQL clauses:

• ORDER BY

• HAVING

• GROUP BY

• LIMIT

When using these, specifying a column number is not accepted, a concrete column name must always be provided. An exception is the COUNT function, where the opposite is true: a specific column name cannot be given.

Table modification

Due to the need for tracking data changes, modifying the table structure is subject to certain technical limitations.

• A new column can only be added if it is nullable, since there would be no value to assign to existing records - setting a default value is not possible at the ImmuDB level.

• To delete a column, its associated indexes must first be removed.

• Changing a column’s data type or constraints is not supported, which can pose a significant limitation during development.

The next section outlines approaches that can partially address these constraints.

Using ImmuDB with Django

Django’s ORM (Object-Relational Mapping) infrastructure is mature and provides extensive support for rapid and efficient development. However, this feature-rich ecosystem can present certain challenges when used with ImmuDB’s current SQL implementation, which is not particularly feature-rich. These Django ORM limitations become especially evident when attempting to integrate with systems that do not fully support the same SQL operations or data structures.

Here are a few typical discrepancies that may cause issues:

• At the migration level, Django allows unrestricted modification of columns.

• Django does not limit itself to generating only INNER JOIN SQL statements.

• Django may support more data types than ImmuDB.

These differences do not mean that the two technologies cannot be used together. Rather, the developer must adapt - either by simplifying Django models or by finding workarounds for certain implementations.

Solutions

Integrating ImmuDB into a Django framework requires subclassing certain Django base classes. Although no comprehensive documentation is available on this integration, the error messages encountered during the development of the ImmuDB Django connector are generally informative. Additionally, Django’s open-source code makes it easy to inspect the methods of the relevant base classes.

It is advisable to place the database implementation in a separate Django app within the project. Let’s assume the name of the project is tracking, and the ImmuDB integration resides in the immudb_backend folder. In the main application’s settings.py file, we can specify which Django app communicates with the database. To do this, the DATABASES variable should be configured with the following dictionary value:

Technical foundation of the basic setup

DATABASES = {
   "default": {
       "ENGINE": "immudb_backend",
       "USER": environ.get("DATABASE_USER", "immudb"),
       "PASSWORD": environ.get("DATABASE_PASSWORD", "immudb"),
       "NAME": environ.get("DATABASE_NAME", "defaultdb"),
       "HOST": environ.get("DATABASE_HOST", "immudb"),
       "PORT": "3322",
   }
}

DATABASES = {
   "default": {
       "ENGINE": "immudb_backend",
       "USER": environ.get("DATABASE_USER", "immudb"),
       "PASSWORD": environ.get("DATABASE_PASSWORD", "immudb"),
       "NAME": environ.get("DATABASE_NAME", "defaultdb"),
       "HOST": environ.get("DATABASE_HOST", "immudb"),
       "PORT": "3322",
   }
}

DATABASES = {
   "default": {
       "ENGINE": "immudb_backend",
       "USER": environ.get("DATABASE_USER", "immudb"),
       "PASSWORD": environ.get("DATABASE_PASSWORD", "immudb"),
       "NAME": environ.get("DATABASE_NAME", "defaultdb"),
       "HOST": environ.get("DATABASE_HOST", "immudb"),
       "PORT": "3322",
   }
}

It is worth noting that the default values specified here match the default values of the ImmuDB database itself. Therefore, strictly speaking, even environment variables are not required for development purposes.

Next, to implement ImmuDB, we need to create the following files and subclass the corresponding Django classes:

• base.py: BaseDatabaseWrapper (+ custom cursor)

• client.py: BaseDatabaseClient

• features.py: BaseDatabaseFeatures

• introspection.py: BaseDatabaseIntrospection

• operations.py: BaseDatabaseOperations

• schema.py: BaseDatabaseSchemaEditor

The full implementation includes a number of technical nuances that we won’t go into here. However, in the following sections, we will present the most important and instructive experiences gathered during the development process.

Customizing the BaseDatabaseWrapper

This class defines the highest-level functions required for database communication. It is where we override methods such as execute, commit, and get_new_connection. We also specify our own subclassed components here, such as client_class and features_class.

Additionally, this is where we map ImmuDB-specific types and operations.

For instance, since ImmuDB interprets LIKE expressions as regular expressions by default, an example entry in the operators map could be 'regex': 'LIKE %s'. Similarly, the data_types map can be overridden to define how Django ORM field types are translated at the database level, for example, mapping 'DateTimeField': 'TIMESTAMP' to treat Django DateTimeFields as SQL TIMESTAMPTs. However, this mapping does not occur automatically, we must define how to convert a Django-provided date value into a TIMESTAMP format that ImmuDB can understand.

All ImmuDB-specific logic should be encapsulated in a custom database cursor class.

The cursor’s key method is execute, which also needs to be invoked in the DatabaseWrapper's own execute method. This function handles, among other things, date conversions, specific queries such as SELECT 1 or GROUP BY 1, converting UPDATE statements into UPSERTs in ImmuDB’s expected format, and rewriting SET expressions.

An additional challenge may arise from reserved keywords in ImmuDB’s user management system, which requires defining special mappings to avoid conflicts.

field_name_mapping = {
      'password': 'user_pass',  # PASSWORD is a reserved keyword in ImmuDB
}

field_name_mapping = {
      'password': 'user_pass',  # PASSWORD is a reserved keyword in ImmuDB
}

field_name_mapping = {
      'password': 'user_pass',  # PASSWORD is a reserved keyword in ImmuDB
}

We then apply this custom cursor in the DatabaseWrapper class as follows:

def create_cursor(self, name=None):
      return CursorWrapper(self.connection)

def create_cursor(self, name=None):
      return CursorWrapper(self.connection)

def create_cursor(self, name=None):
      return CursorWrapper(self.connection)

DatabaseSchemaEditor: custom logic

This class is responsible for generating the SQL statements that execute migrations during Django ORM development, based on the various migration operations.

Here, we can define the SQL statements used to create, modify, and drop tables, such as:

sql_create_table = (
      "CREATE TABLE IF NOT EXISTS %(table)s"
      "(%(definition)s,"
"PRIMARY KEY (%(primary_key)s))"
)

sql_create_table = (
      "CREATE TABLE IF NOT EXISTS %(table)s"
      "(%(definition)s,"
"PRIMARY KEY (%(primary_key)s))"
)

sql_create_table = (
      "CREATE TABLE IF NOT EXISTS %(table)s"
      "(%(definition)s,"
"PRIMARY KEY (%(primary_key)s))"
)

One of the most important pieces of logic to override is table modification. During development - especially local development - databases typically contain little data, or that data is stored in an importable format. This approach allows for quick database resets without data loss. Following this philosophy, we can implement a table-replacement logic that is not natively supported by ImmuDB.

Let’s say that during development a field was set as nullable, but later we decide it would be better not to allow NULL values in that column. ImmuDB does not support such modifications directly, so we first need to create a duplicate of the table with the updated field set to non-nullable, then copy over all values to the new table, and finally delete the original one.

If we want to change the column’s type, copying may not always be feasible and data loss may occur.

Another challenge is how to handle previously NULL values when copying data to the new table. Django offers a built-in solution: during migration, it prompts the developer to provide a default fallback value for such cases. If you don’t want to specify a default either, the safest approach may be to manually delete all records from the table before applying the changes.

Features and other supporting files

Several other files require smaller-scale adjustments. Among these, one of the more important is features.py, where you can explicitly define ORM- and SQL-specific flags, such as:

supports_left_outer_join = False
supports_right_outer_join = False
supports_full_outer_join = False
supports_native_bulk_update = False

supports_left_outer_join = False
supports_right_outer_join = False
supports_full_outer_join = False
supports_native_bulk_update = False

supports_left_outer_join = False
supports_right_outer_join = False
supports_full_outer_join = False
supports_native_bulk_update = False

In operations.py, among other things, you can override the previously mentioned bulk update feature and implement it manually using a loop, since ImmuDB’s SQL implementation does not support this operation natively.
For introspection, you only need to define the following:

class DatabaseIntrospection(BaseDatabaseIntrospection):
   """Database introspection for ImmuDB."""

   def get_table_list(self, cursor):
       """Return a list of table and view names in the current database."""
       cursor.execute("SELECT * FROM TABLES()")
       return [TableInfo(row[0], 't') for row in cursor.fetchall()]

class DatabaseIntrospection(BaseDatabaseIntrospection):
   """Database introspection for ImmuDB."""

   def get_table_list(self, cursor):
       """Return a list of table and view names in the current database."""
       cursor.execute("SELECT * FROM TABLES()")
       return [TableInfo(row[0], 't') for row in cursor.fetchall()]

class DatabaseIntrospection(BaseDatabaseIntrospection):
   """Database introspection for ImmuDB."""

   def get_table_list(self, cursor):
       """Return a list of table and view names in the current database."""
       cursor.execute("SELECT * FROM TABLES()")
       return [TableInfo(row[0], 't') for row in cursor.fetchall()]

Only client.py surpasses it in terms of simplicity:

class DatabaseClient(BaseDatabaseClient):
   """Database client for ImmuDB."""
   executable_name = 'immudb'

class DatabaseClient(BaseDatabaseClient):
   """Database client for ImmuDB."""
   executable_name = 'immudb'

class DatabaseClient(BaseDatabaseClient):
   """Database client for ImmuDB."""
   executable_name = 'immudb'

Summary: ImmuDB’s position among databases

ImmuDB addresses a critical and timely need, positioned somewhere between traditional databases and Blockchain technologies. While its maturity still falls short of today’s standard relational databases and its language integrations remain limited, its overall usability is not in question. Integrating ImmuDB into a Django-based Python environment requires several days of development effort and may not work seamlessly with every implementation. Still, in contexts where immutability is essential, it can confidently be placed in the “usable” category.