Dolt is the world’s first version-controlled relational database. Dolt lets you branch, fork, clone, merge, and diff your relational data, in all the same ways Git lets you work with your files.

We’ve been testing out generative AI app builders lately to see how well they can build web apps using a Dolt database as the backend datastore. Check out our previous posts for details on how Replit, Lovable, and Vercel v0 fared. Otherwise, read on to find out how Bolt performed.

TL;DR#

Let’s get right to it… how does Bolt compare to Lovable, Replit, and Vercel? Overall, I found Bolt easy to use, and unlike Lovable and Vercel, it didn’t come across as clumsy and unreliable. Bolt generally produced correct code and didn’t trip up much. There was just one time in my testing that Bolt broke my app and it was able to fix it on the next iteration. That’s dramatically better than Lovable and Vercel, which routinely broke my app and often took four or five prompts before they could fix it. However, it’s still not as good as Replit. Replit’s App Testing feature is a game changer and gives Replit a huge advantage over Bolt, Lovable, and Vercel. The ability to do automatic UI acceptance testing to catch obvious problems before a human has to get involved is really the gold standard. We’ve actually been beating this drum for a while here at DoltHub… agents need tests! Without some sort of testing, agents just don’t have a way to validate the work they do and prevent them from pushing out garbage.

The only frustrating point I hit with Bolt was getting my external database hooked up to my app. In my very first prompt, I told Bolt I had an external database that I needed to use for my app, but even in Bolt’s very first response, it tried to talk me out of using my own external database and instead using the default BoltDB that’s built into the platform. I get that Bolt has crafted their experience around their own database and has made that database available by default, but it was frustrating when Bolt didn’t listen to me after I explicitly told it using my own external database was a nonnegotiable requirement. I also hit problems getting Bolt to connect to my external database. This was very similar to the problems I hit with Lovable, although Replit and Vercel didn’t have these same issues.

Building an Inventory Tracking Web App#

Just like in the previous blogs with Lovable, Replit, and Vercel, I’m building the same simple inventory tracking website to test out the Bolt experience. The web app allows you to do basic CRUD operations on inventory items, and uses Dolt’s version control features to let you see the history of all inventory changes, revert changes, or drill in and see the history of all changes for an individual inventory item. Here’s a screenshot of the final UI Bolt created for me:

Launch a Dolt SQL Server#

The first thing we need to do is get a Dolt SQL server up and running for our web app to use. I’m using Hosted Dolt to run my Dolt SQL server. Hosted Dolt gives you a fully managed experience for running a Dolt server, including automatic backups, support for horizontal scaling through read replicas, automatic version upgrades, and more.

After logging into the Hosted Dolt website, I clicked on the Deployments link, and then on the Create Deployment button. A “deployment” is simply an instance of Dolt running on Hosted Dolt.

I named my deployment for this app “bolt-demo” and accepted the defaults on the other pages of the Create Deployment UI. After a few minutes, Hosted Dolt had spun up a Dolt server, ready to be used in my Bolt experiment.

If you’re following along, keep the deployment page open so that you can grab the database information later and tell Lovable how to connect.

If you scroll down on the deployment page, you’ll see instructions for connecting to the Hosted Dolt instance through the mysql tool. Use that command to open a SQL shell to our new Dolt server and run the following command to create the database, demo_db, on the server that we’ll be using for our application:

CREATE DATABASE demo_db;

Getting Started in Bolt#

Now that we’ve got a Dolt database server running, let’s jump over to Bolt and get started building our app!

Here’s how I got started with Bolt, using the same prompt I used with Lovable, Replit, and Vercel.

A few moments later, Bolt comes back with a response confirming the details of the app I’m building, but it surprised me that despite my clear instruction to use my existing Dolt database, Bolt was questioning my instruction and trying to get me to use the included BoltDB instead.

In response, I let Bolt know that I was in fact serious about using my existing Dolt database and gave it all the connection information. After that Bolt showed me a plan for how it would build the app and one by one ticked off the boxes, then gave me a summary of the initial app it had created.

Connecting a Database#

Now that we’ve got the app started, let’s get it hooked up to our Dolt database. I already supplied the database connection information to Bolt, so the first thing I need to do is instruct it to create our database schema and insert some sample data. I also asked it to use the dolt_commit() stored procedure to create Dolt commits after schema initialization and after inserting sample data.

Bolt claimed victory and told me it had added a new “Setup DB” button to my web app that would create the initial schema and populate some test data. I had asked for this to be done once, not to be added as a feature to the app, but that was a minor detail, since I could easily have it remove the button once I used it.

However, when I went to test the new “Setup DB” button, it didn’t work. I told Bolt about the “invalid peer certificate” error message I received, and it came back with this response:

Okay, that all sounds reasonable and makes sense, so I tried again to initialize the schema, but no dice. Bolt still wasn’t able to connect to my external database. I let Bolt know that it was still failing and here’s what it responded with:

I was surprised by Bolt trying to turn off SSL. In my very first prompt, I told it that my external database required SSL/TLS, but Bolt decided to ignore that requirement and disable SSL, and as expected, Bolt was unable to connect after it disabled a secure connection. At this point, Bolt gave up and started moving the app over to BoltDB without asking for me to approve that change. This was the most frustrating part of working with Bolt. I had given it very clear requirements on what I needed to do, but when it hit problems, it gave up and started dropping requirements. I expect an app builder to prioritize the explicit requirements I give it and work with me to get around problems, and at minimum to confirm any changes in approach that go against the requirements I gave it.

I canceled Bolt’s work there when it started going off the rails, and explained that it was a nonnegotiable requirement to use my external Dolt database. I also gave it the CA cert file from Hosted Dolt, since I figured that was why it couldn’t validate the server’s certificate. Other app builders had been able to connect to an external Dolt database without that, but Lovable had also needed it. Somewhat oddly, I couldn’t just drop the CA cert PEM file into the chat – Bolt kept telling me that was an invalid file type, so I ended up having to open up the file, copy the contents and then paste it in. I was pretty disappointed with this experience for a number of reasons: 1) Bolt decided to drop a requirement I had given it without first confirming that change with me, 2) Bolt should allow you to upload a PEM file directly instead of having to copy/paste the contents, 3) Bolt should have understood the “peer certificate error” enough to know to ask me if I had a CA cert file it could use to validate the server’s certificate, and 4) if Bolt had done any rudimentary acceptance testing, it would have easily caught these connection issues and could have saved me quite a bit of time and frustration. In the end, Bolt was able to connect to my Dolt database once I gave it the CA cert information, but it was definitely a rough start to my Bolt experience.

Publishing#

Publishing with Bolt was fast and easy, just like with the other app builders I tested. None of the app builders I tried out hit any issues with publishing my simple web app and the experience in each was pretty much the same – a simple one-click publishing experience that took just a minute or two to roll the application out.

Adding Version Control Features#

Now that the basic app is built and deployed, let’s take a look at adding some new features that use Dolt’s version control APIs. These APIs allow us to look back in history to see exactly how our data has changed, quickly compute diffs of data and schema between any two commits, and audit who made changes.

Commit History#

The first feature I’m adding is a log of all the changes in our inventory data. This is the same type of information you’d get from running git log in a Git project. In a Dolt database, we just need to run SELECT * FROM dolt_log; to read the commit log from the dolt_log system table.

Bolt handled this smoothly and gave a nice outline of its plan as well as a summary of what it built and how it works.

Here’s the UI it created:

The UI works correctly, looks nice, and when I tested the “reset to commit” action, it gave me a confirmation prompt and then correctly reset the commit history. The confirmation dialog is a bit basic and lacking style, but it does the job. For the record, Replit and Lovable both created a much nicer, more custom pop up confirmation dialog.

Now that support for resetting to an older commit is working correctly, let’s add support for reverting a specific commit. Just like in Git, resetting to a commit moves the database to be exactly as it was at that commit, so any changes from more recent commits are gone. In contrast, with revert, a new commit is created that has the opposite changes from a specific commit, in order to cancel out that commit – the commit still stays in the commit history, but the changes it made have been undone. Here’s the prompt I fed Bolt:

Bolt broke the work down into a structured plan again, and gave a summary of the changes it made.

Technically I asked for a context menu, but Bolt gave me buttons. No biggie, the buttons look good and are probably a little easier to use than having to open a context menu anyway. Here’s the new UI:

Bolt again created a super basic pop up confirmation dialog to confirm the revert action. It did a good job coming up with accurate text to describe the operation, but the styling was a bit too basic. I’m sure I could have asked Bolt to improve those pop-ups if I really wanted to make this app a little more stylish, but the default, simple pop-ups were okay for this experiment.

The revert functionality also worked correctly on the first try. Adding the commit history page turned out to be a pretty easy task for Bolt and went very smoothly.

Item History#

The commit history UI gives us a nice view of all changes across our database, but I’d also like to see all the changes for a specific inventory item. There are a couple ways we can do this with Dolt, but I asked Bolt specifically to use the dolt_history_<tablename> system table to see all rows in the table at each commit in the database. Here’s the prompt I gave Bolt.

Bolt handled this in stride and again laid out a structure plan, executed each step, then gave me a high-level summary of what changed and how the new page works.

Unfortunately, when I tested out the item history, the UI just went blank. It started to load for a split second, and then when totally blank. Unlike some of the other app builders, Bolt had so far been more successful at creating working code. This was the first major bug I hit using Bolt. It still made me miss Replit’s awesome App Testing feature that did basic acceptance testing and would have caught issues like this.

To Bolt’s credit, it was able to quickly fix the issue after just a single prompt. The item history UI worked correctly after that fix. Here’s the final UI.

Displaying Diffs#

Let’s add one last version-control feature to our app. The commit history view gives us a nice log of all the changes to the database, but it doesn’t show us exactly what row data changed. One of Dolt’s superpowers is computing fast diffs between any two points in the database’s commit graph. There are several APIs in Dolt for working with diffs. Here we’ve asked Bolt to use the dolt_diff() table function to compute the diff from before and after each commit.

Here’s the exact prompt I gave Bolt:

Bolt handled this one smoothly, too. Here’s the summary it gave me describing all the changes it made:

The new feature worked perfectly the first time I tested it. The UI does a nice job highlighting the data change to make it easy to read the diff. Here’s an example of the UI:

Final Thoughts#

Bolt did a good job building my simple web app. Ignoring the initial issues I had getting it to connect to an external database, Bolt came across as more trustworthy and reliable than Lovable and Vercel. However, I still strongly prefer Replit, primarily due to its awesome App Testing feature that does preliminary UI acceptance testing on your behalf to help find and fix any issues. Agents that can produce code on your behalf are a seriously powerful tool, but if they can’t do anything to validate their work before they kick it over the wall to you, then you’re just going to waste your time debugging their broken code.

We think the same rule applies for agent making changes to data stores. Without tests, those agents are just going to produce junk that you have to debug. That’s why we created a built-in test engine for Dolt databases that allows you to codify tests against your data.

If you want to talk more about generative AI tooling, or if you’re just curious about using a version-controlled relational database, come by our Discord and say hello!