Something has shifted on the web.
When I started writing this article, it was originally going to be an exploration of a recently launched version-controlled database that boasted a similar feature set to our version controlled database Dolt.
We do this sort of comparison a lot. I’m naturally curious about anyone with a similar data model to us building cool new things, and there’s this spirit of colaboration when it comes to exploring this design space together. I was excited to do a frank comparison between the two.
But then as I worked my way through their website, my excitement cratered. It became obvious that the website had been AI-generated with minimal human review. Worse, at the time of my investigation, I was unable to prove that the closed-source, managed-hosting, invite-only platform they were advertising even existed.
It’s a bizarre feeling, to be looking at a polished website for a software project and questioning whether or not there’s even any software. There was no code I could actually see or run, and no reasonable metric by which I could actually evaluate them.
Ultimately, I decided that I couldn’t in good conscience promote this project.
When I told my team this, they suggested a second project I could write about instead. This one was at least provably real: it was open-source spread across multiple GitHub repos. But as I explored it, I realized it wasn’t much better: their documentation was at best vague and opaque and at worst outright missing, and after combing over the entire thing I still struggled to understand how the project actually worked, or why I would use it over any of its alternatives. The demos technically worked, but that was the best thing I could say about it. I decided this other project wasn’t worth writing about either.
I didn’t want it to turn out this way. I want to be excited by exciting new tech. Even when software projects are competing, there’s still an underlying spirit of collaboration, the desire to learn from each other, and the culture of game recognizing game. But I strongly suspected that there wasn’t actually anything to learn here, and I’d mostly just wasted my time.
Dolt By Comparison#
When we started DoltHub, our founder Tim recognized that our success relied on organic interest in our projects. We needed a corpus of work that prospective users could find and engage with and share. But an important part of that strategy was that the writing had to be genuine and high-quality. Readers can recognize authenticity and they hate having their time wasted.
That’s why we at DoltHub have been publishing 3-5 blogs every week since 2020, and each one is something that we think is worth talking about. The fact that we’ve maintained this cadence without devolving into blog spam is a testament to how much we have to say.
We feel like it’s been a pretty successful strategy. People respond to our blogs, even if it’s just to skewer me for my bad Golang opinions.
It’s one of the things that I outlined in DoltHub’s AI Policy earlier this year. We’re open about the ways that we use AI and the ways that we don’t, and part of that is identifying when AI is helpful and when it’s not.
Even for our repos that are predominantly AI-assisted, our communication around those repos is still human-crafted because we want to ensure that our communication is useful. We want users reading our documentation and working through our demos to be able to understand what Dolt is, how it works, and how it can be used. We have a deep understanding of the technology and use that understanding to make the onboarding experience as good as possible.
But reading through these other projects, I don’t feel that same level of care.
What Changed#
It didn’t used to be like this. Once upon a time, you could get a pretty good impression of a software project’s level of activity and polish just by looking at their online footprint. This was because documentation and code artifacts used to scale with the amount of thought and effort that went into creating them. If a project has demos and documentation and a fleshed-out website, you can be reasonably confident that there’s some amount of real development happening. And if nothing else, you would be able to safely assume that the project, you know… actually exists.
This was because write velocity used to be the bottleneck. Designing a website isn’t free. Making a tech demo isn’t free. But over the past year, generative AI has advanced to the point where you don’t need web design skills to make a website. And you don’t need to have a real code design to make software. AI will fill in the gaps for you as best it can.
This has hampered the ability of prospective users to assess code projects. They might now be required to wade through a theoretically limitless amount of generated documents only to realize that in doing so, they’ve put infinitely more thought into the project than its creator.
This is obviously not sustainable. So there’s value in learning to recognize such projects early, so users can avoid wasting their time.
I’m outlining the recurring patterns that I’ve noticed in such “AI slop” projects, drawing specific examples from these projects so that others can learn to recognize them sooner.
I won’t be naming these projects or sharing other details that could uniquely identify them. This is because if I’m right, I don’t want to give these projects free publicity. And if I’m wrong, I don’t want to besmirch a legitimate project. But it also doesn’t matter which exact projects I’m referring to, because these projects by their very nature aren’t unique. The Internet is full of them now.
If you’re a developer, you can also interpret this article as a guide on “what not to do.” I briefly wondered whether this advice would make it easy for “AI slop” projects to obscure their sloppiness, but after thinking about it, I’m not convinced these issues can be papered over without avoiding the design shortcuts that produced them in the first place. Most of these issues stem from fundamental gaps in the underlying design. AI has gotten very good at implementing the prompt its been given, but if that prompt reflects a bad design or has incomplete information, then there’s only so much it can do. It’s no better than a human developer with a client who wants something vague or impossible. Even the best tooling won’t help you build a good house on bad foundation.
And that’s why simply knowing these red flags isn’t enough to hide them. Ultimately the only fix for these symptoms is to address those design gaps. So if someone wants to “hide” these symptoms, and the way they do is by improving their design… then mission accomplished.
(Source: https://xkcd.com/810/, available under CC BY-NC 2.5)
Symptom 1: Meaningless Tables#
AI generated copy loves tables. I have to imagine this is exacerbated when the prompt specifically asks for the project to be compared with its competition. But if the AI doesn’t have enough information to make a meaningful comparison, it will settle for a meaningless comparison instead.
This typically means:
- The qualities being compared are vague and qualitative, as opposed to objective and measurable.
- The values in the table are not meaningfully comparable with each other, and often restate basic information about the projects being compared.
It’s easy to imagine how this comes about: the AI knows a good comparison has tables, so it attempts to extrapolate one based on the information it has available. But this is an impossible task because the data it needs to make a proper comparison doesn’t exist.
In this specific example, the site was comparing its own project to multiple other platforms, including Dolt. But because every data point wasn’t actually comparable, it ultimately failed to make a meaningful argument for any of the projects in the comparison.
Symptom 2: Demos that don’t Demonstrate Anything#
Their website had a demo that ran in the browser and claimed to connect to a running demo server and access its API. Inspecting the page source revealed that no such connection was actually taking place, and no API calls were actually made. Instead, it loaded a single static resource, and the rest of output was generated entirely locally.
If a demo was designed this way on purpose, I would call it deceitful. But I don’t think the demo was designed this way on purpose. I don’t think it was designed at all. The AI had only a vague outline of what the project was meant to do. A real demo would require answering questions about how to expose a demo instance to unauthenticated users. Without answers to these questions, a fake demo was all that it could generate.
Demos are a fantastic way to onboard prospective users. They allow users to familiarize themselves with your feature set, your APIs, and what kinds of tasks your project excels at. But to do this well, the demo has to be good. It has to be explainable and relatable. Making a good demo often requires context about not only what problems your project is trying to solve, but how it solves them.
Symptom 3: Extremely Opinionated APIs to do Simple Tasks#
Both projects were heavily marketed as tools for specific machine-learning or AI workflows, despite the fact that none of their features were specific to machine-learning or AI. These assumed use-cases were baked into command names and API endpoints, and documentation for these features describe them in that context, often obscuring how these features are actually implemented or their relationship with their underlying data model. When I took the time to piece together how these features worked, I often realized they were doing something simple or generic to accomplish their goal.
I suspect this happens when the developer doesn’t have a good mental link between their objective and the code concepts that will achieve it. This is similar to a pattern I’ve noticed in AI-generated code comments, where comments reference the use case specified in the original prompt even when the module being commented is more general-purpose. Since the context provided to the LLM only mentions the use case and not the underlying design, everything gets produced in the context of that use case. This also often leaves the developer unable to critically evaluate the final design created by the AI.
Symptom 4: It Reinvents Every Wheel#
This is similar to the previous one: if the prompter doesn’t have a clear vision of the underlying code components, they’re also less able to identify whether these components would be meaningfully different from already existing tools. So they don’t just reinvent the wheel, they reinvent multiple wheels and bundle them all together. Their code ends up incorporating many unrelated designs into a single project.
In this case, one of the projects I was looking at had entire pages boasting various add-on services that were bundled into the main binary. But these pages were themselves sparse on actual detail, mostly reiterating standard features of these services. Nothing on the site explained the benefit of reimplementing these services and bundling them together. There very well could be benefits to this approach, but the site didn’t even attempt to explain it.
This leads nicely into the next symptom.
Symptom 5: Documentation that Says Nothing#
Both of these projects were overflowing with splashy marketing pages, elaborate demos, and high-level documentation. But none of these provided any enlightenment into how the code was actually designed. I could follow along with a demo, running commands and observing the output, but little to help me understand what was actually happening or how I could leverage the projects to do something novel.
Symptom 6: Optimistic Features and Pricing#
The first project I looked at was particularly egregious here. Their pricing pages lists three different paid tiers, none of which are available yet. They also offered a free tier, but it was invite only, with a prompt to email support for an account.
The second project was worse: multiple pages on the website proudly boasted a laundry list of features, but many of these features were listed as “enterprise only”, despite the enterprise tier not existing yet.
In both cases, these paid tiers had no expected pricing. Of course they didn’t, because the paid tier was entirely wishful thinking.
How to Fix This#
Project developers and users have a common goal in making sure that the project can be understood and evaluated. Code that can’t be understood is useless. This is true regardless of whether or not the code is created with AI.
As AI models become more advanced, we’re learning more about what kinds of things they can and can’t do. It’s possible that even more advanced models will diminish some of these symptoms. But I’m not convinced they’ll disappear entirely, because these are symptoms aren’t caused by flaws in the model, but by the fact that the model is being asked to do work that it simply doesn’t have the context or framing to do.
Fixing each of these issues requires answering questions about your project, questions like:
- How does my project actually work?
- What common building blocks does my project use?
- What actually makes my project different from projects that have come before?
- What features am I bringing, and what are the tradeoffs?
These are easy questions if you have a robust design and actually understand what they’re trying to accomplish, and hard questions otherwise. Being able to answer these questions will shape your documentation and make it easier to bring users on board.
In Conclusion#
I don’t know what the future of coding is going to look like. But I know that we’ll continue to communicate directly about our work, to make useful resources that we think are worth reading, and to leverage our deep understanding of databases and distributed version control in order to make something real and useful.
That’s all for now. If you have any questions or thoughts or feelings, you can always join our Discord and drop us a line; we want to hear what you have to say.