infra

Infra was the company I joined when I left Datadog back in 2021. Jeff Morgan and Michael Chiang were the two founders and they had completed YC W21 on the idea of making RBAC in kubernetes easier. It was a compelling story for me as I was building the first round of training on Kubernetes at Datadog.

We spent about a year working on Infra, but finding Product Market Fit was a challenge. They had a great idea, a great team, and great advisors. But it was a small market.

As we were building, we saw a related idea that was more general that could leverage what we worked on. Making SSH key distribution easier in companies. We started focusing on that, calling it keypair. But more and more we were finding that companies were rarely allowing their employees to SSH directly into their servers, instead relying on CI/CD pipelines. For instance only a tiny handful of folks at Datadog were allowed to SSH directly into production servers. If there was a problem, then the machine was destroyed and another VM was spun up with the right code in its place.

We were getting ready to attend a security event in Seattle and then KubeCon in Amsterdam and just a few weeks before, Patrick started working on an AI tool. Jeff had given the directive before that that we should all be using AI tools to help us all be more productive in what we do. Local AI was tempting but the existing tools, like LM Studio and Oobabooga, were terrible. We knew we could do better, leveraging our experience with Infra and Keypair as well as Docker.

With both Infra and Keypair, we struggled to get even 1000 stars on the GitHub repo, but within weeks we had over 100,000 stars on Ollama. It was, at the time, one of the fastest growing repos on all of GitHub.

The Infra github repo is still there if you want to check it out, but it hasn’t been updated in 3+ years. You’ll find a little overlap in some of the functionality and concepts between Infra and Ollama, especially around layers in a Docker like hub.