Back in June, we wrote about running Django on Dolt. We described our journey from Dolt as "Git for Data" to what we are today: a MySQL compatible relational database that is 99% SQL compliant. To fulfill our vision as a drop-in replacement for MySQL, we have to not only speak the dialect, but be able to support software like Django that integrates and builds on top of MySQL. For Dolt, that means doing more than mimicking MySQL features, we must expose Dolt's branch-and-merge versioning model. In the first part of this blog series, we talked about modeling Commits and Branches as Django Models to expose Dolt-specific functionality to the application layer. In this blog, we'll zoom out and talk about how to use Branches and Commits to version your application.

Django + Dolt

Django is an open source framework for building dynamic web apps. Django's database-centric design makes it an ideal integration for Dolt. Running with on a single branch, Dolt behaves exactly as Django would expect a MySQL database to behave. However, Dolt provides a fully-featured version control system that operates orthogonally to the SQL data model. From Django's perspective, each branch behaves like a separate database, but in reality they share a common storage layer capable of efficient diffs and merges. Django is probably best known for its Object-Relational Mapping (ORM), but it has many more features for building flexible, composable architectures. We'll see how we can use some of Django's unique features to build a version-controlled application, or to add version control to an existing Django project.

Using Middleware for Branches

In the previous post we made this model for Dolt branches:

class Branch(models.Model):
    """ Expose the `dolt_branches` system table """
    name = models.TextField(primary_key=True)
    hash = models.TextField()
    latest_committer = models.TextField()
    latest_committer_email = models.TextField()
    latest_commit_date = models.DateTimeField()
    latest_commit_message = models.TextField()

    class Meta:
        managed = False
        db_table = "dolt_branches"
        verbose_name_plural = "branches"
    ...

We also wrote a few utility methods for switching branches and merging branches together. This is great if we only wanted to use the ORM, but what we really want is to expose branches at the UI layer and version each request and response from the server. Django's Middleware framework is going to give us just what we need. A bit of background from the docs: "Middleware is a framework of hooks into Django’s request/response processing. It’s a light, low-level “plugin” system for globally altering Django’s input or output." As requests are processed, request handlers use the ORM to connect to the database and retrieve the data they need to provide responses. We can use middleware to alter the state of that connection and choose which branch in the database we want to connect to. The following class does exactly that:

class DoltBranchMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        return self.get_response(request)

    def process_view(self, request, view_func, view_args, view_kwargs):
        branch = self.get_branch(request)
        branch.checkout()
        return view_func(request, *view_args, **view_kwargs)

    def get_branch(self, request, *view_args, **view_kwargs):
        if "dolt-branch" in request.session:
            return request.session.get("dolt-branch")
        return "master"

This simple class globally alters our view of the database and in effect versions the entire application. The details of the get_branch() method are an important detail to the implementation. In order for requests to the application to choose a branch they must encode that branch choice somewhere. Here we use a cookie-based session to store the users current branch. Switching branches in the application is as simple as updated this state in the cookie.

Auto Dolt Committing with Signals

Commits are the next Dolt feature we want to integrate into request/response logic. Making a Commit saves the state of the application and allows us return to it later, or to undo its changes. In order to make this useful to application users, we want to make a new commit every time we create or update an object. We'll use another Django feature to accomplish this. Signals are Django's way of broadcasting data across the application. From the Django docs: "In a nutshell, signals allow certain senders to notify a set of receivers that some action has taken place." Django allows us to register for signals and execute custom logic whenever the specified event occurs. For this example we've structured our signal handlers within a context manager:

from django.db.models.signals import m2m_changed, post_save, pre_delete
from dolt.models import Commit

class AutoDoltCommit:
    """
    Context Manager for automatically creating Dolt Commits
    when objects are written to the database
    """

    def __init__(self, request, branch):
        self.request = request
        self.commit = False

    def __enter__(self):
        # Connect our receivers to the post_save and post_delete signals.
        post_save.connect(self._handle_update, dispatch_uid="dolt_commit_update")
        m2m_changed.connect(self._handle_update, dispatch_uid="dolt_commit_update")
        pre_delete.connect(self._handle_delete, dispatch_uid="dolt_commit_delete")

    def __exit__(self, type, value, traceback):
        if self.commit:
            self._commit()

        # Disconnect change logging signals. This is necessary to avoid recording any errant
        # changes during test cleanup.
        post_save.disconnect(self._handle_update, dispatch_uid="dolt_commit_update")
        m2m_changed.disconnect(self._handle_update, dispatch_uid="dolt_commit_update")
        pre_delete.disconnect(self._handle_delete, dispatch_uid="dolt_commit_delete")

    def _handle_update(self, sender, instance, **kwargs):
        """
        Fires when an object is created or updated.
        """
        self.commit = True

    def _handle_delete(self, sender, instance, **kwargs):
        """
        Fires when an object is deleted.
        """
        self.commit = True

    def _commit(self):
        msg = self._get_commit_message()
        Commit(message=msg).save()

Context managers are used in conjunction with Python's with statement to execute a block of code within a certain context. For our purposes, this means we'll handle update and delete signals that occur within our context manager. This gives us a tidy way to package up a group of changes into a Dolt Commit. We'll use this context manager in another piece of middleware:

class DoltAutoCommitMiddleware(object):

    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        # Process the request with auto-dolt-commit enabled
        with AutoDoltCommit(request):
            return self.get_response(request)

When routed through this middleware, every request that alters the database will be encapsulated in its own Commit. As users alter the database, a granular log of their changes will track the updates. Any individual commit can be inspected, or reverted if necessary.

Conclusion

Dolt is unlike any other database. It's the only SQL database you can branch, merge, diff, push and pull. We're still early in our journey, but we believe this is the future. Integrating Dolt and Django is a great use case to show the power of version control for databases. Nautobot, a network source-of-truth application, is a perfect example of this. A Dolt plugin for Nautobot allows users to stage changes to application state on branches, and merge them into production once peer-review is completed. Providing these features in the database solves an entire class of problems outside of application code. We're very excited about this integration and other possibilities that Dolt creates. If you have an idea for integrating Dolt into your application, or if you want to learn more, get in touch with us on Discord!