Tim Sehn

March 30, 2020

Dolt is Git for data. Instead of versioning files, Dolt versions tables. DoltHub is a place on the internet to share Dolt repositories. As far as we can tell, Dolt is the only database with branches. How would you use such a thing? One of the hard t...

Zach Musgrave

March 27, 2020

Overview In this blog post, we discuss an approach for simulating an outbreak of COVID-19 in the US Congress. This is a long technical article about data sets, epidemiology, and simulation. Feel free to jump straight to the results of the simulatio...

Matt Jesuele

March 25, 2020

In the previous article in this series, we took a deep look at the overall system architecture of DoltHub, the online data community powered by the Dolt version-controlled database. In this article, we'll zoom in on the front end and see how the code...

Tim Sehn

March 23, 2020

We adopted Bash Automated Testing System (Bats) to test the Dolt command-line. As of March 10, 2020 we are up to 473 tests, though 55 are skipped because they currently fail. The tests define desired behavior so we're constantly working to get skippe...

Zach Musgrave

March 20, 2020

Dolt is Git for data. It's a SQL database that lets you branch, merge, and fork your data just like you would a Git repository. In previous blog posts we announced how you can use special system tables to query the history of your database. Today, we...

Tim Sehn

March 18, 2020

Here at DoltHub, we've been working on COVID-19 data since February 5, 2020. First, we started importing John Hopkins data and then we worked on assembling the largest open, regularly-updated set of case details from Singapore, Hong Kong and South Ko...

Tim Sehn

March 16, 2020

As part of our effort to track data related to the Novel Coronavirus (COVID-19), we wanted to scrape a JavaScript-enabled website on Coronavirus from Hong Kong. Moreover, you'll notice that the website from Hong Kong uses lazy loading based on scroll...

Tim Sehn

March 12, 2020

On Saturday, February 29, this transpired in our company chat room: Tim/Brian Google Chat Snippet A project was born. We had time series data for confirmed cases, deaths, and recoveries segmented by location sourced from John Hopkins but we did no...

Matt Jesuele

March 11, 2020

In our introductory article for this series, we took a high-level look at the technology stack and architecture behind DoltHub, the online home for Dolt data repositories. In this article, we'll delve a little deeper and discuss how the pieces of the...

Brian Hendriks

March 7, 2020

In this blog post I want to give an introduction to some core concepts used to implement fast querying of databases. These techniques were implemented in Dolt and produced significant performance improvements. Database internals The B-Tree is a cor...

Tim Sehn

March 6, 2020

Dolt is Git for Data. Learn about the options for versioning data catalogs, data pipeline version tools, and version controlled databases. The Dolt database versions data and schema with full audit history, diffs, and rollbacks.

Brian Hendriks

March 5, 2020

In the first part of this two part blog I covered NOAA's "Global Hourly Surface Data" dataset and how it is modeled in Dolt. Dolt is git for data, and for this dataset we model a day of observations as a single commit in the commit graph. In this se...

Brian Hendriks

March 2, 2020

The National Oceanic and Atmospheric Administration, NOAA, publishes weather measurements taken from stations around the world. It started in 1901 with a handful of stations, and there are more than 35,000 stations today. Most of these stations provi...

Zach Musgrave

February 28, 2020

Dolt is Git for data. We built Dolt to help teams collaborate on data sets using the forking, branching, and merging workflows that Git popularized. These workflows are what enable software engineers to collaborate on source code, and they're what en...

Oscar Batori

February 26, 2020

In our previous blog post we examined some freely available licensing tools for open data from Creative Commons. To briefly recap a license specifies the terms under which copyrightable material is made available for public access, sharply distinct f...

Oscar Batori

February 24, 2020

Introduction Dolt is a data format. DoltHub is a collaboration platform for data stored in the Dolt format. When sharing copyrighted content the terms of that sharing are governed by a license. In this post we highlight some common licenses attached...

Tim Sehn

February 23, 2020

John Hopkins University Center for Systems Science and Engineering began collecting, tabulating, and publishing Novel Coronavirus (COVID-19) data on January 31, 2020. We started importing this dataset into Dolt on February 5, 2020. This blog will exp...

Matt Jesuele

February 20, 2020

Towards the end of last month, we launched a totally reworked and redesigned version of DoltHub, our web application for hosting and collaborating on Dolt repositories. Now that we've had a little while to iron the kinks out, it seems like a good tim...

Oscar Batori

February 17, 2020

Background We are excited to announce the launch of our documentation site. The goal of Dolt and DoltHub is to enable developers and the data community with radically better data infrastructure. High quality documentation should empower users by all...

Zach Musgrave

February 14, 2020

Happy Valentines Day from all of us at DoltHub! You are the reason we do what we do! It you. In honor of the holiday, we want to talk about how much we love making queries faster. We're going to examine how our SQL engine makes a query plan and exp...