We became really good at Python, Django and MySQL.
The back-end couldn’t handle the data volume coming from the Facebook Ads API. We turned to Celery distributed queue manager to solve this scaling issue.
And so we became really good at Celery.
Slowly and steadily things started to fell apart. We did what any other engineering team would have done: we turned to a more stable technology, Java.
We became really good at Java.
Then we hit an evolution point: users wanted to manage campaigns across channels (Google, Twitter, Pinterest.)
Given the monolithic architecture, we couldn’t just plug another Ads API. We will have to clone the entire platform and adapt it to the new channel (which never happened in the Ad Tech industry.)
What was the decision? Yes, another technology. We threw the platform away and designed a microservices-based architecture running on AWS.
We became really good at Elasticsearch, Cassandra, ElastiCache, RDS, SQS, SNS, Docker, Consul, Terraform, Kinesis, etc., etc.
As an engineer, you need to be aware that you are becoming temporarily good at something. You are not practicing karate for a decade to get to 10th dan. You are practicing different martial arts until they don’t work anymore. If you are a ninja now, tomorrow you need to be a samurai.
As an entrepreneur, CTO or engineering manager, the challenge is not choosing the right technology. The challenge is to identify the components in your platform that won’t change, use a trusted technology to build them, and invest properly to get it right, once.