The fate of “small” open source

The most popular npm package I’ve written so far is blob-utilWhich is ~10 years old and still gets 5+ million weekly downloads.

This is a small collection of utilities to do the job BlobIs in JavaScript. I wrote this because I found that PouchDB users were extremely confused about how to work with it Bloband how to convert them into strings, ArrayBuffers, etc.

Given that almost 80% of developers are now using AI in their regular work, blob-util This is almost certainly something that most developers would happily have an LLM prepared for them. sure, you can Use blob-utilBut then you would be taking on an additional dependency with unknown performance, maintenance, and supply-chain risks.

And of course, Claude will happily spit out whatever Blob Utilities you’ll need when prompted:

> Write a utility function for me in TypeScript to convert a Blob to an ArrayBuffer. It should return a promise.

function blobToArrayBuffer(blob: Blob): Promise {
  return new Promise((resolve, reject) => {
    const reader = new FileReader();

    reader.onload = () => {
      if (reader.result instanceof ArrayBuffer) {
        resolve(reader.result);
      } else {
        reject(new Error('Failed to read Blob as ArrayBuffer'));
      }
    };

    reader.onerror = () => {
      reject(reader.error);
    };

    reader.readAsArrayBuffer(blob);
  });
}

Cloud’s version is close enough blob-util version (not surprising, since that’s probably what it was trained on!). Although it is too verbose, it is being unnecessarily scrutinized readAsArrayBuffer really gives you a ArrayBuffer (Although it makes TypeScript happy). To be fair, it also directly improves my implementation. rejecting with an error rather than being more awkward onerror events.

I think some people will see this as progress: fewer dependencies, more robust code (even if it’s a bit more verbose), faster turnaround time compared to the old “search npm, find a package, read the docs, install it” approach.

I’m not overly proud of this library, and I don’t particularly care whether the download numbers go up or down. But I think something is lost with the AI ​​approach. when i wrote blob-utilI adopted a teacher’s mindset: The README contains a charming and whimsical tutorial that shows Kirby in all his glory. (At the time I had a thing for putting Nintendo characters in all my stuff.)

The goal wasn’t just to provide a utility to solve your problem (although it does that) – the goal was also teach People learn how to use JavaScript effectively, so you have an understanding to solve other problems in the future.

I don’t know where we’re headed with AI (well, about 80% of us; for the remaining holdouts, I salute you and wish you speedy progress!), but I think it’s a future where we value immediate answers rather than teaching and understanding. There’s less reason to use something like this blob-utilWhich means there’s less reason to write it in the first place, and therefore less reason to educate people about the problem space.

There is still a movement towards putting documentation into it llms.txt file, so you can simply point an agent at it and save your brain cells the effort of understanding English prose. (Is this even documentation now? What Is Documentation?)

conclusion

I still believe in open source, and I’m still doing it (in fits and starts). But one thing has become clear to me: the era of small, low-cost libraries. blob-util Is over. They had already been phased out due to Node.js and the adoption of more and more of their functionalities by browsers (see). node:glob, structuredCloneetc), but LLMs are the last nail in the coffin.

This means there’s less opportunity to use these libraries as springboards for user education (Underscore.js had this same philosophy), but maybe that’s okay. If there is no need to find a library for grouping objects in an array, it is probably unnecessary to learn about the mechanics of such a library. Many software developers would argue that asking a candidate to invert a binary tree is pointless, as it never comes up in daily work, so perhaps the same could be said for utility libraries.

I’m still trying to figure out what Type In this new age open source is worth writing about (hint: the ones an LLM can’t just spit out on command), and that’s where education is most lacking. My current thinking is that the most value is in larger projects, more inventive projects, or more specific topics that are not covered in the LLM’s training data. For example, I look back at my work fuite and various memory-leak-hunting blog posts, and I’m quite satisfied that LLM couldn’t reproduce it, because it requires novel research and creative techniques. (Though who knows: maybe someday an agent will be able to poke his head around a Chrome heap snapshot until he discovers a leak. I’ll believe it when I see it.)

There’s been a lot of discussion lately about where open source fits in the world of LLM, but I still see people pushing the boundaries. For example, many naysayers think there’s no point in writing a new JavaScript framework because LLMs are trained too much on React, but then the tireless Dominic Gannaway writes Ripple.js, which is another JavaScript framework (and with some new ideas, to boot!). This is something I love to see: humans laughing at the machine, carrying on with their human thing.

So if there’s any conclusion to this winding blog post (forgive my squid human brain; I didn’t use LLM to write this), it’s just this: yes, LLM has made some types of open source obsolete, but there’s still plenty of open source left to write about. I’m excited to see what kind of innovative and unexpected things you all come up with.