PRODUCTS

KEYWORDS

DoltLite 100

As many of you know, I’m building DoltLite as my passion project. We recently announced DoltLite works on DoltHub. Now you can share DoltLite databases easily. I wanted some example DoltLite databases, so I put an agent on it, and before I knew it, I had 100 sample DoltLite databases pushed to DoltHub. Enjoy!

DoltLite Discover

DoltHub Is a DoltLite Killer Feature#

One of the main problems in the SQLite ecosystem is the sync problem. Since SQLite is just a file on your disk, how do you sync versions of the data between machines? Several sync engines have been built around SQLite, including Turso, PowerSync, ElectricSQL, and cr-sqlite.

But they all have one problem: what do you do if there are conflicts? The answer was always conflict-free replicated data types (CRDTs). CRDTs define how to merge concurrent updates so replicas converge. What if your application isn’t amenable to CRDTs? Optimistic locking. Last write wins. And you lose.

DoltLite has a different answer: Git-style conflict detection and resolution powered by fast diff at database scale. Your sync engine can be DoltHub with a pull request workflow to share changes. In the event of a conflict, resolve it using your own rules or knowledge, just as you would when merging code in Git. Data is never lost during sync. You get to keep whatever data model you want. Version-controlled data is syncable data.

DoltHub a bit heavy for you? DoltLite ships with a headless remote server for you to build the same sync process on top of.

Data Sharing Was Tedious#

We founded DoltHub on the idea that data was too hard to share and version control would fix that. In 2019, we launched Dolt and DoltHub as Git and GitHub for data.

We immediately proceeded to publish open databases on DoltHub: WordNet, ImageNet, IRS, and NOAA. This was tedious work, often taking a week or two to parse the data into a usable form and import it into a SQL database.

After a year or so, we realized version-controlled imports of existing data were not that exciting to people. We launched data bounties to create novel databases using Dolt and DoltHub. We eventually killed this idea too and focused our energy on being a database company.

But DoltHub still exists, and we still like the idea of sharing version-controlled data in the Git/GitHub model. Before agents, data sharing was just expensive and tedious.

Now Data Sharing Is a Prompt#

As with so much else in software, AI has made data collection and sharing trivial. Below, in three acts, are some of the prompts I used to build this set of 100 DoltLite databases. I used Grok, so this is not even a state-of-the-art model.

See If It Works#

The first act is “See if it works.” I have the agent find some open DoltLite databases and give me a list. Then I ask the agent to do the import and inspect the results. At first, it made a Dolt database, not a DoltLite database, but eventually I got a DoltLite database.

See if it works

Refine the Process#

Once I’m happy with the results, I go a bit wider by asking for ten at a time. With this many, I realize I need READMEs. The agent messes that up, so I course-correct a bit more.

Refine the process

Batch#

Now, the process is dialed in and I can just let the agent cook. It worked for a few hours getting to 100 DoltLite databases.

Batch

Conclusion#

Data sharing used to be hard. It’s now a few prompts away. Enjoy the future with all these sample DoltLite databases. The next step a few years ago was data bounties. What would agentic data bounties look like? Stay tuned.

Have any data you want on DoltHub? Let us know on our Discord.