12 July 2019

Engineering Products vs. Engineering Primitives

Throughout the duration of my career, I’ve noticed that you can divide engineers into two groups, primitive engineers and product engineers. The primitive engineers enjoy building primitives that are used within various systems. A primitive1 is a building block2 of software that can be used by other pieces of software to build a larger part of a system. Examples of these primitives could be anything from an internal database3, a rate limiting service, or even something such as a job queuing service. The other group, product engineers, then use these primitives while building out systems and products for their end-users. These engineers are the ones who work more closely with product managers and designers in order to build a product for users.

In my career, I’ve always leaned towards product engineering, because I enjoy being close to the user. When I worked on the Networking Products team at DigitalOcean and helped build the Cloud Firewalls and Virtual Private Cloud products, people often assume that I helped build the firewall and VPC daemons that live on the hypervisors. In reality, other engineers created the primitives that talked to the hypervisors and handled the networking rules. Then my team used these primitives in order to create a product for our users. Our systems had the responsibility of storing and updating the state of things such as firewall rules and making sure they were applied correctly on the hypervisor, while also ensuring that they were correctly being shown back to the user. It’s a joint collaboration between the primitive engineers and the product engineers, everybody has a part.

There is an intersection of product and primitive engineering (often in the cloud computing space), in which a primitive is so valuable internally that it is open-sourced and/or developed into a product for customers. This is how various software and products were released to the public, such as Amazon DynamoDB, Google Cloud Spanner, and Kubernetes4. What makes this intersection interesting is when a primitive is first created but then transforms into a full-fledged product offering, it indirectly causes the primitive engineer to assume a hybrid role as both a primitive and product engineer.

Expanding more on this thought of primitives as a product, a fair amount of SaaS5 that is used within developer tooling can be thought of as a “primitive as a service”. When developing a product, you have the classic “build” vs. “buy” situation for a lot of functionality. Do you have the time to build your own transactional email service? Proably not. Mailgun, SendGrid, and Postmark are primitives for sending transactional emails. Similarly, Stripe and Braintree are primitives for online payments and billing for products. Lastly, Amazon S3, Google Cloud Storage, and Azure Storage are primitives for object storage. Additionally, a lot of these services have been battle-tested and are known to scale6, so it allows engineers to spend more time focusing on building out the actual system and product. Knowing these primitives exist and how to use them is invaluable to all engineers.

While a lot of the focus of this article has been back-end systems focused, this distinction also applies to the front-end as well. From a front-end engineering perspective, this distinction can also be seen, especially when thinking about design systems. In various engineering organizations, there normally are entire teams dedicated to maintaining a design system for the user interfaces7. These front-end engineers responsible for building and maintaining the various components of a design system can be seen as primitive engineers. The front-end engineers who then use these components to build out the user interfaces for their products can be seen as product engineers. In some situations, an engineer may both build and maintain these design system components and build out products8, thus becoming a hybrid, similar to what we discussed earlier.

As with a lot of things, product and primitive engineering are not mutually exclusive. A product engineer may build a new primitive for their system that does not yet exist, which can then be extracted and reused across an engineering organization. Or, like we discussed earlier, a primitive engineer may create something that evolves into a product. An engineer may be working on both products and primitives at the same time. In conclusion, while there are differences in what product and primitive engineers create, one is not smarter or better than the other. A healthy and successful engineering organization requires a mixture of both product and primitive engineers, who have a symbiotic relationship. What kind of engineer do you think you are: primitive or product?

Special thanks to Kelly Sutton and Jacky Alcine for reading early drafts of this post.

  1. At first thought, “primitive” seems like a misnomer within software given that they usually are not so simple under the hood. 

  2. I believe I first read of this definition in The Amazon Tax on Stratechery. 

  3. Dynamo by Amazon is a good example of this. 

  4. Kubernetes has seen a rise in adoption rates within various engineering organizations, so Google created Google Kubernetes Engine, a hosted Kubernetes option. 

  5. Software as a Service. 

  6. Of course, this adds risk to a system by adding an external piece of software to the infrastructure, but this still sounds less risky than creating the primitive yourself and ensuring that it scales and stays up.  

  7. My current employer, GitHub, is an example of this. 

  8. I haven’t met anybody yet who has done this extensively, but I am sure they exist. 

Did you enjoy reading this?

Feel free to let me know on Twitter. You can also subscribe to my newsletter for more of my writing.