Over the last few years as a founder of a small startup, I have implemented a ton of internal processes to work more efficiently and save costs. What’s interesting about this process is I often didn’t know the formal term or didn’t know what the industry called it. One such example is that I had adopted the microservices architecture without consciously knowing that it’s formally called that. In this post, I will tell my story on
- My goal to minimise costs and save development time
- The actions I took to do so
- The result and some of the drawbacks of it
- How does it relate to micro-services
Part 1 of the story is; I have lost count of the no of processes, I have implemented at My Day To-Do to improve work efficiency and reduce development time. My Day To-Do is a very small, mostly one developer based startup where time is very critical. Hence, I am always looking for ways to improve efficiency by minimising development by not having to write the same code twice.
Part 2 of this story; I wanted the My Day To-Do family of apps to have two sets of functionality i.e. 1) that makes it unique and 2) common features shared by all apps in the family. Please keep in mind that functionality 1 and 2 are not mutually exclusive. Of the two, getting the daily weather was one of the features that could be shared across multiple apps. Therefore, I built it as a standalone REST web service that serve resources via endpoints. At the time of building this, my first motivation was…
Apple introduced App Transport Security from iOS 9 onwards which meant that iOS apps cannot call a non-secure url i.e. you can consume REST resourced via HTTPS only. This meant, if My Day To-Do tries to get the weather info from the OpenWeather API, it must do so via a secure connection ( HTTPS call). Now, I realised that you can only consume the OpenWeather API via a secure URL if you are on one of paid accounts. Due to the early stages of the startup, I thought that it wouldn’t be a good idea to spend money on a paid account. I was in a bit of a bind! I wanted both the app to have this feature without me spending too much money on it! So what do I do?
II came across something called CloudFlare and the solution seemed pretty obvious to me. This was my thought process,
- “What do we want? information about the weather!
- How do we get it? typically JSON response
- Does the iOS app care about where the json is coming from? Not at all, it just needs JSON, that’s it
- so if we had a custom standalone REST API that gave weather data, would it matter? No!
- Can we secure a web url for our purpose with CloudFlare? Yes”
My solution was to build a Python/Django based REST API, deploy it to the shared hosting we already pay for and secure it with CloudFlare. By CloudFlare, I mean use it’s free tier. So the operating costs still remain low. This would not only save costs but also improve data access speeds for getting certain information. I will be honest, at the time, I didn’t consciously think that I was solving a distributed systems problem, rather, just as something to keep costs low. At least that was my original rationale behind it.
My plan as of late 2016/early 2017 was for My Day To-Do to give a host of information to it’s users as soon as they woke up in the day. Remember, My Day To-Do has the alarms feature whereby it can wake up the user, greet, announce the day todo and the local weather? Therefore local news, sports news, current events were all part of the information that I wanted My Day To-Do to relay to users. Therefore I decided, why not use the same approach? So instead of just having one proxy, why not have N proxies, all secured by CloudFlare. This way each proxy can serve a certain piece of info that maybe used by one or more apps. Great, however when it came to the implementing, I didn’t actually build multiple Python/Django REST web services. I chose to add the logic to serve the news, current trends etc via multiple endpoints from the same REST web service.
…and we had it
This was the setup at My Day To-Do,
- user data was stored on the Cloud platform i.e. Firebase
- data was being fetched from stand lone web services secured with CloudFlare
- the native iOS app UI could be done by a HTML5 front-end programmer
and all this architecture and setup was done by one programmer who only did what seemed logical to him, without consciously thinking about distributed systems! So did I just unknowingly adopt the Microservices architecture?
During my undergraduate degree, I fell in love with the whole “Separation of concerns” concept. So much so that every time I see a problem, technical or otherwise, I try to decompose it into small sub-problems. Also, growing up, I heard “united we stand divided we fall”, entirely too often. When united, the sub-problems stand, but if we divide them, they fall i.e. we can solve them. Of course this approach may not always work, however it’s a mindset. It’s often my mindset into finding solutions to a problem.
It was either all that and/or the distributed systems concepts got into my mind a bit too well during my undergraduate degree. I did score a high distinction in that subject, but never realised the impact it had on my mind.
The advantages of such a solution are really those that come with a micro-services architecture. However first, let me outline what I thought the advantages were before I realised it was a distributed systems problem,
- All the code to handle external API calls is in our custom REST web app proxy. Therefore, the app doesn’t care what external API we are getting the data from, it just calls the proxy
- I can hire people on a project basis i.e. short one-off contracts just to work on the proxy web service without affecting the iOS app
- Compared to an iOS UI dev, I could probably hire a slightly (cheaper) front-end developer to improve the iOS app UI
- This was one brilliantly admirable decentralised architecture. I am blushing right now, as I think about this and type ☺️
Disadvantages of this approach
In addition to me increasing my workload N times, there were other disadvantages to this, relative to the company size of i.e.
- Being the only developer working with many technologies, I will loose time during the programming language context switch. I mean, every time, I move from coding iOS to coding Python/Django, I will loose some time
- I simply wouldn’t have enough time to keep track of the different moving parts of the application
- Testing for graceful failure! How do these systems fail? Do they fail in a way that it would just annoy the user? I did not test all that
- It will cost more to hire developers for so many different technologies. Also, to hire good developers, we need experts to interview them, have we got that in place?
Understand the Pros (formally)
In this section, I will look at the pros and cons of my solution from the formal MicroService architecture standpoint. The pros of a MicroService architecture are,
- Highly maintainable
- Independently deployable
- Loosely coupled
- Organised around business capabilities
- Owned by a small team
In the My Day To-Do architecture, we can keep adding functionality to the REST web service without touching the iOS app. So that covers the highly maintainable and independently deployable part. Loosely coupled, well….that’s self explanatory. Organised around business capabilities, that too is obvious given how I described the feature early on. Owned by a small team, I built these solutions in hopes the interns working with me will look after them. However, I haven’t been very lucky with the development interns that have worked with me. Therefore point 5 is actually a disadvantage in our solution
The cons (formally)
- Dealing with the complexity of creating a distributed system
- Inter-service communication and deal with partial failure
- Requests that span multiple service is hard
- Ensure/test services communicate with each other
- Cross team communication must be good for inter-service interaction
- IDE’s are fixated on building monolithic apps instead of distributed apps
- Deployment complexity
- Increased memory consumption
Of the disadvantages, the deployment complexity is the only thing that I have noticed the most. This is especially worse, when I have to do an aforementioned context switch.
It’s just incredible how some concepts that you learn in university stay in your mind long after you finish university. In my case, I just realised that I was doing something without even realising it originated from what I learned years before.
If you would like to know more about microservices architecture and distributed systems have a read of the links below
As usual, if you find any of my posts useful, support us by buying or even trying one of our products and leave us a review on the app store.