We are processing petabytes of data every day; only a few kilobytes are relevant. Storage is cheap, but processing is still expensive, especially when the users need to wait for it.
Back in the monolithic/RDBMS platform era, we cared about not to store too much data and optimizing the queries as much as we could. Now with our de-normalized, microservices-based data pipelines, we are storing everything we can just in case.
As we design our next-gen data platforms, we need to think deeply about the consumption layer and have well-defined maps (aka APIs) where users can quickly discover the data treasures. Data and product thinking need to converge.
Founder & CEO at InTheValley
I help founders like you fix engineering teams that should be moving faster. If you're scaling a startup, you've probably felt the pain: great people on paper, but execution feels slow. I've been building remote teams for Bay Area startups since 2005 — engineers you can trust who actually deliver and know how to use AI to ship faster, not slower.
Latest posts by Leo Celis (see all)
- The Wrong AI Tool Is As Bad As the Wrong Hire - 04/15/25
- COVID, Remote Work & AI - 04/08/25
- The Tech Downturn is Here – It is Not About Budgets - 04/01/25