TLDR; Sometimes stars do align! For me these stars are open source, opportunity and people.

I joined Datastax to head the engineering effort to build data streaming offerings based on open source project Apache Pulsar. A new career opportunity should have a good balance between problems that can leverage your existing strengths, and throw some new challenges. With your strength you can make an impact immediately and help move the needle at your new job while new challenges are what helps you learn new skills, get you excited and help you grow into your career. Let’s talk about the stars and how they map to strengths and challenges .

Open Source: DataStax is one of the known brands in the world when it comes to open source and big data. DataStax engineers are the leading contributors to Apache Cassandra, a free and open source, distributed NoSQL database. Many world-renowned brands such as Netflix, EBay and Apple rely on Cassandra to serve their gigantic user base. And through its cloud native Astra DB offering, DataStax makes it dead easy for developers to use Cassandra-as-a-Service (CaaS).

More Open Source: When DataStax looked at its customer base they found that the majority of these customers use some sort of messaging system to pipe data to and from Cassandra. So they looked for an open source messaging system that shared the most important traits of Cassandra: geo replication, hybrid and multi cloud deployability, high scalability. Apache Pulsar checks all of these boxes, so DataStax decided to build product offerings based on this very popular messaging system. DataStax has already hired several committers to Apache Pulsar and related projects Bookkeeper and Zookeeper.

My initiation into open source development was at JBoss and Red Hat, where I worked on enterprise offerings of several Java based middleware open source projects such as application server, portal platform and in-memory distributed data cache. I understood the value of working with the OSS community–for OSS is a positive sum game when all players in the community participate in a true OSS spirit.

Opportunity: It’s a known fact that more data is generated each year but the majority of these data remain siloed. It’s operationally challenging to bring different sources into one place in near real time. Enterprises which are good at this have significant advantage over their competitors but they do so at a heavy cost. It’s exciting to have a chance to work on a project that helps enterprises solve this problem efficiently. Imagine a product offering where developers and enterprises can use and operate streaming as easily as they currently use a database as a service.

This is a great opportunity full of challenges. It’s non-trivial work to provide technology as a service in a multi-tenant, multi-cloud deployment with a single digit latency, high uptime and enterprise grade security.

People: It’s a privilege to work with people with whom I have before and who have continued to mentor me. Chet Kapoor, Ed Anuff and Sam Ramji were there at Apigee when they trusted me enough to acquire InstaOps, a mobile APM startup that I had co-founded. I learned a lot about building enterprise software in the cloud for developers during my four years tenure at Apigee. On top of that I have followed Jonathan Ellis, co-founder of DataStax, very closely since the early days of DataStax. He is an amazing technologist and I am grateful to be working closely with him.

I believe this new role is a great opportunity to channel my experiences working with OSS, building developer centric products and building multiple startups to help launch a new product enterprise software as a service from the scratch. The challenge is to do so in a cost effective way in a space that is constantly evolving.

On a lighter note, I am still debating what to call this service. Is it a PaaS (Pulsar as a Service) or is it a SaaS (Streaming as a Service) ? I guess it’s both at the same time!