Own the parts of a data platform that hold what your business knows: the raw data, the definitions of your core records, the history of how they changed, and the checks that say a number is fit to use. Rent the machinery underneath, because databases, storage and compute are commodities that many vendors sell well. Build the whole thing yourself only when the data is the advantage and you can pay to keep it running for years. For most growing businesses the answer is a small owned core on bought parts.
The question usually arrives as all or nothing: a vendor’s platform, or a team building one from scratch. In my experience it is really a question about layers, and each layer has a different answer.
What should you always own?
- The raw copy. Data as it arrived from each source, in a format you can read without the vendor. If the only copy is the cleaned one inside someone else’s product, you can’t rebuild it when the cleaning turns out to be wrong.
- The model. What a customer, an order or an account is in your business, how each is identified, and how they relate. This is your knowledge written down, and it should outlive any tool.
- The history. What you knew and when, and which version of your rules produced each figure.
- The checks. The rules that decide whether a number can be used, written somewhere you can read and run them.
None of these needs exotic software. On my current market-data platform they live in plain files and an ordinary Postgres database. Files hold the raw and cleaned data; Postgres holds the configuration and the record of every run. The code I wrote joins them. The database and the file formats are things anyone can download for nothing, which is exactly why I chose them.
What should you rent?
Almost everything with a commodity price: databases, storage, compute, scheduling, monitoring, sign-in. Vendors run these better than a small team can, and you can move between them as long as your data sits in open formats and your model is yours. Keep a clear line between your knowledge and their machinery, and the machinery becomes replaceable.
Do the vendor’s terms matter as much as the price?
More, often. When I surveyed news sources for that platform, the licence ruled out nearly every one before price came into it. Some forbid building a database from their data at all. Others require you to delete everything when your subscription ends. Data held on those terms is rented, however long you keep it. If your business would be in trouble the day a contract lapses, you don’t own that data, and your platform shouldn’t pretend you do.
I now record each source’s retention rule as a field on the source itself, with a link to the terms and the date I read them. “May we keep this?” is then a query, rather than a paragraph in a contract nobody rereads.
When does building the whole platform make sense?
When the data is the product, or close to it. If what you sell depends on combining sources in a way nobody else does, keeping a history nobody else keeps, or answering questions a vendor’s tools can’t, then the platform is the business, and renting it means renting your advantage. It also makes sense when the rules you work under say exactly where data may live and who may touch it, and no vendor fits them.
It makes less sense than people think when the motive is control for its own sake. My first attempt at that platform grew a web app of sixty thousand lines and an API of sixteen thousand. I left nearly all of it behind, because the foundation underneath was wrong, and the size of what sat on top had made that harder to see.
Every part you build is a part you maintain. Own what you know, and rent what anyone could sell you.
What are the signs you have bought too much?
- You can’t export your full history in a format another tool can read.
- The definition of a core record lives in a vendor’s settings screen, and one person knows where.
- Changing a calculation means raising a ticket with someone else.
- Your AI tools can see only what the vendor chooses to expose.
And the signs you have built too much?
- A large share of engineering time goes on keeping your own plumbing alive rather than on what it carries.
- You have written your own scheduler, dashboard tool or login system, and a vendor would have sold you a better one.
- The platform needs the person who built it in the room, because nothing about how it runs is written down.
How should you decide?
Draw your platform as layers and mark each one. The raw copy, the model, the history and the checks are yours: open formats, in a place you control. Storage, compute and the tools around them are rented, on terms you have actually read. Then ask of every remaining piece whether a vendor could sell it to your competitor as easily as to you. If they could, buy it. If they couldn’t, that piece is your advantage, and it is the part worth building well.
Whichever way each layer falls, write the decision down with its reason. The next person to ask “why did we build this?” deserves a better answer than a shrug, and so does the next vendor who offers to replace it.