Platform thinking: tips when designing a cloud-native platform
A lot of engineers have heard this following (often untrue) tale sound. As a software engineer you get the opportunity to design an innovative platform that has the potential to reach millions of users. The platform plays perfectly into CSR (corporate social responsibility). You have no legacy software to work with and you have complete control over how you want to design the platform. The only requirements to the platform are that it is built in a future-proof way and that security and privacy are built into the platform from the start. How does that sound to you?
Well, if someone would say this to me I’d say there’s no such thing. Being a software engineer, 95% of the time you have to build on whatever has been built already. Nonetheless, this is exactly the opportunity we had and I’m going to tell you all about it in this blog. Here we go!
Creating a payroll giving platform called Loongift in collaboration with The Social Handshake
Working at an agency that helps its customers with their digital transformations, we rarely have startups as customers. Of course, hiring external consultants costs money, which is something startups rarely have access to. Once in a blue moon, there’s a startup with a really innovative idea that has managed to get some funding but has no IT knowledge. The name of that startup is The Social Handshake, and that is where we came in.
We designed and built a platform called LoonGift, in which doing good is made easy by enabling employees to directly donate part of their salary to a charity of their choice. The amount you want to donate is deducted directly from your pay stub and your employer has no access or insights into which charities you donate to. At the end of a fiscal year, you get tax benefits over the amount that you donated to charity. It’s easy for participants to sign up and change their donations. Another added benefit is that employers pay for the use of the platform, so the entire donation from the employee goes to the charity. Also, the charity cannot ask you for more money or send you any other information if you haven’t signed up for it and the charities don’t pay retention costs or acquisition costs.
The concept behind LoonGift is called payroll giving. Payroll giving has already been popular in several countries throughout Europe, for example in Germany and England. Nothing like this existed in the Netherlands, however.
Google Cloud as the basis for the platform
So how did we go about doing this? Given the fact that nothing like this exists in the Netherlands yet and given the fact that the concept has proved itself in other European countries, we had a unique opportunity to design a platform that can potentially be huge from scratch! Truly cloud-native. Since we are a Google Cloud partner and the opportunities are endless on the Google Cloud platform
Unfortunately, we did not have unlimited time and resources to build this amazing platform. Like in all software projects, we had to make choices about where to focus our time and energy on and which features could be delayed. Building such a platform from scratch is tricky because it’s a constant tradeoff between ‘good-enough-for-now’ and ‘future-proof’. Luckily for you, I’ll share our tips, tricks and struggles here, so you don’t have to find out the hard way!
Steps you need to take in platform thinking
So, you want to design a platform that is future proof. How do you go about this?
1. Think ahead
It’s impossible to know everything you want in the future. There are multiple known unknowns. You inherently build in some technical debt from the start. This does not mean that you can just start building and see where you end up! You have to have an idea of what you’re designing for. Together with our UX and UI designers, we made a prototype of ‘the dream’ first. ‘The dream’ was a prototype of the complete platform, validated by potential end-users. With ‘the dream’ in mind, we had an idea of what we should work towards and how functionalities should work in the future. After we prototyped ‘the dream’, we could decide on what our MLP (Minimum Lovable Product) was going to look like. With ‘the dream’ done, we could also make architectural and technical decisions that were perhaps not strictly necessary for the MLP but are crucial to long-term success for LoonGift.
2. Make the inevitable technical debt replaceable
Even with designing with the future in mind, it’s impossible to know everything beforehand. So, with these known unknowns, make sure that the technical debt you’re creating is as replaceable as possible. This means that you might want to replace some functionality, even in another programming language. So, obviously, like most modern applications, adopt a microservice design. Make sure that every microservice is responsible for its data and its own data only. Modifying another services’ data is never allowed. This way, you make sure that you can relatively easily replace a service if needed.
3. Security by design
Sometimes, security, user-friendliness, time, and budget might be a little bit of a trade-off, but security should never suffer. Especially if you work with sensitive data, think about the unhappy flows upfront. What realistically can go wrong, will go wrong. There are still enough edge cases you have not thought about yet, I promise. Make sure that the edge cases you can think of are covered. In these cases, with a limited budget and high ambitions, I like to use managed services as much as possible. Managed services on cloud platforms have built-in security.
4. Prepare for ‘smart data things’
Machine learning is hot right now (obviously). And even if you don’t have a specific use case in your mind right now, prepare for the future! Make data flow freely throughout your microservice architecture. Have every service produce data, and have another service pick the data up. Every service that has subscribed to the data from the producing service (and, of course, is allowed to see the data) can do whatever they want with it. This is a perfect architecture to combine data from different sources to potentially perform machine learning or large scale data analysis on!
We got the takeaways on platform thinking: contact us so we can help you with it!
Most of the things I told you in this blog we found out the hard way. We were so excited to design a new platform from scratch that we wanted to use the newest technologies. With the newest, I mean really the newest — several products and managed services that we used were still in beta. I shouldn’t need to tell you that using stuff that’s in beta can cause some troubles in production environments, especially if resources are limited. We pulled it off, but for the sake of your sanity, I would advise against it.
Also, try to challenge the stakeholders. They obviously have some sort of idea of what should be built and how it should be built, but it’s good to bring in some perspective. As you (the programmer) are the expert on the technical side, you should advise the customer when you see a better way to build a feature.
Pages turned and lessons learned, the result is really something to be proud of. A scalable, innovative, cloud-native, future-proof platform with no infrastructural maintenance (thank you, managed services!). Go check it out on Loongift and if you’re in the Netherlands, check out if your employer can join too!