Tired of Deployments, built my own Heroku

This article is brought to you by the first deployment service, dedicated specifically to Python & Django Appliku.com.
Never manage a server again.
Deploy your Django app in 5 minutes.

TLDR: I got tired of deploying my Django projects, went to Heroku, and got burnt by their invoices, built my own SaaS to deploy apps to existing clouds.

Story begins with custom software development agency

6 Years ago I started yet another agency where we were building custom solutions for businesses that needed automate time-consuming tasks:

• Semi-automated task distribution for a large service company,
• Big e-commerce businesses,
• Generation of construction documentation
• Many smaller projects.

It was an interesting journey.

I loved what we’ve built. The money we saved. Manual labor we got rid of(well, not everyone was happy about the last part of course).

It lasted for 5 years and I loved most of what we had to do.

But one thing was annoying every single time.

Every now and then I needed to spin up a new project and set up a test environment and then production one.

Deployment was a pain every time

During that time I have experimented with different deployment approaches and tools:

• Used Fabfile,
• Added CI/CD – first Atlassian Bamboo, then GitLab
• Used Supervisord, then Docker,
• Bare-metal servers, Digital Ocean, Linode.

At some point, I came up with a number of scripts that used docker-compose and GitLab CI to start deployment on new commits pushed to the repository.

But for each new project, a lot of manual work had to be done, it was human-error prone and took a full day to set everything up:

• .gitlab-ci.yml
• docker-compose.yml
• set up Nginx and let’s encrypt
• databases
• list of small things goes on

I loved it when a new client project came up, but I feared going over that process again and again.

I thought there has to be a better way.

Let’s give Heroku a try

At that time we came up with a couple of our own side projects, where we use Heroku.

It was such a pleasant experience.

I was “WOW”-ing on every step.

I literally spent zero time on anything DevOps.

I moved a couple more projects there, that was easy. None of those projects were generating any revenue since it was hobby sites.

Then the time came and I received SMS from the bank about Heroku charging my card. At that moment all excitement went away.

Here comes Heroku Invoice

In 2 months Heroku spendings climbed from $15/mo up to $80/mo. For hobby projects.

I have immediately disabled those sites. One I moved off, as the only one that we were interested in.

This coincided with a drastic change in my personal financial situation. I was completely broke for several months.

Getting back on track, seeking for the idea

When I solved my financial issues, I had time to think about what to do next.

We have finished with custom software development.

What was left after doing that business clearly was the pain from deployments and pain from paying for Heroku.

I wanted to get back to some of our hobby projects.

I had time to make it the right way and enjoy doing it.

I wanted something as pleasant as Heroku, but I didn’t want to pay that much for that.

Light bulb

It became clear to me: I need to build my own Heroku.

At that time on IndieHackers, I stumbled upon Hatchbox.io for Ruby and heard about Laravel Forge for PHP.

I treated that as validation for a similar tool but for Python/Django.

I came up with a name for it a month later: Appliku.

I started from bash scripts for building Docker, spun up a Django up to manage them, and iterated on that for 17 months to this moment (Sun, 16th Aug 2020).

What do I have now?

• Automated provisioning of servers on both AWS and DigitalOcean
• Server setup (Docker, Nginx, Let’s Encrypt)
• Building apps from the source code on GitHub Repo
• An almost as good interface as Heroku
• Heroku Config Vars Sync for those who are trying to move off Heroku step by step.

What is under the hood?

Django application is the core of the product.

The frontend is built with Angular based on mdbootstrap.com Angular Package.

Tons of background tasks run by Celery.

When a user requests creation of a new server – Appliku talks via API to Digital Ocean or AWS to provision it.

When Server is provisioned, with help of the Paramiko module Appliku connects to the server over SSH and runs the setup script.

When the setup is complete – you can deploy your app.

App creation consists of naming your app, picking repository and branch, and the server you want to deploy to.

The only requirement for the app is having a Procfile.

I figured that since I was moving off of Heroku, there must be others who do the same. And I liked this idea of having processes in the code itself rather than specifying them in UI.

Procfile is a text file where you specify what processes are needed to be running for your app to operate.

web: python gunicorn wsgi:app
release: python manage.py migrate
celery: celery worker -Q default
web: python gunicorn wsgi:app
release: python manage.py migrate
celery: celery worker -Q default
web: python gunicorn wsgi:app
release: python manage.py migrate
celery: celery worker -Q default

Enter fullscreen mode Exit fullscreen mode

“web” tells us that this process should accept HTTP connections.

“release” is what get’s executed on every release.

Other processes have no special meaning.

Appliku asks what programming language you use to run the app. Right now supported languages are Python, PYPY, PHP, and Node.

For complex builds, we allow writing your own Dockerfile.

Heroku Sync is a feature that pulls config vars of the app from Heroku API every minute and updates config vars for the related app in Appliku and rebuilds it.

This has proven to be useful for those who moved part of Heroku Dynos to Digital Ocean with the purpose of reducing expenses. Usually, this is done with memory-hungry background workers.

When you hit Deploy.

Appliku makes your server to:

• go to your GitHub repo
• clone the code
• create docker-compose file(based on Procfile and processes you enabled to run)
• build the image
• start it

Also, it requests Let’s Encrypt SSL certificate for appname.applikuapp.com domain or any other custom domain you have added.

By making your server do that we ensure that all your code is only stored on your server. The only piece of information from your code we store is the contents of the Procfile.

After the app is deployed it runs without much of the interaction with our app.

If you push new commits – we run the build/deploy process.

Every day we run certbot to check if any of the certificates are about to expire.

Couple words about Nginx configuration.

Nginx vhost is setup in a way that it respects cache headers from your app.

proxy_cache_path /home/app/_cache/sitename.com levels=1:2 keys_zone=sitename.com:10m max_size=1g inactive=60m use_temp_path=off;
location / {
        proxy_cache sitename.com;
        proxy_cache_revalidate on;
        proxy_cache_min_uses 1;
        proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
        proxy_cache_background_update on;
        proxy_cache_lock on;
        proxy_cache_bypass $cookie_nocache ;
        add_header X-Cache-Status $upstream_cache_status;
        proxy_pass http://127.0.0.1:8003;
        proxy_read_timeout 600;
        proxy_send_timeout 600;
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
proxy_cache_path /home/app/_cache/sitename.com levels=1:2 keys_zone=sitename.com:10m max_size=1g inactive=60m use_temp_path=off;
location / {
        proxy_cache sitename.com;
        proxy_cache_revalidate on;
        proxy_cache_min_uses 1;
        proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
        proxy_cache_background_update on;
        proxy_cache_lock on;
        proxy_cache_bypass $cookie_nocache ;
        add_header X-Cache-Status $upstream_cache_status;

        proxy_pass http://127.0.0.1:8003;
        proxy_read_timeout 600;
        proxy_send_timeout 600;
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
proxy_cache_path /home/app/_cache/sitename.com levels=1:2 keys_zone=sitename.com:10m max_size=1g inactive=60m use_temp_path=off;
location / {
        proxy_cache sitename.com;
        proxy_cache_revalidate on;
        proxy_cache_min_uses 1;
        proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
        proxy_cache_background_update on;
        proxy_cache_lock on;
        proxy_cache_bypass $cookie_nocache ;
        add_header X-Cache-Status $upstream_cache_status;

        proxy_pass http://127.0.0.1:8003;
        proxy_read_timeout 600;
        proxy_send_timeout 600;
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }

Enter fullscreen mode Exit fullscreen mode

This way you can use dummy caching on the application side (like Dummy Cache Backend in Django) in order to generate proper cache headers and the actual caching will happen before the request reaches the backend.

Closing thoughts.

I have built this whole thing alone with the help of my wife that took ownership of most non-engineering tasks.

It is a fun thing to work on. I have learned a lot about Docker, Nginx, Paramiko, Django itself.

I think that this app can help many people besides me to:

• ship their idea faster
• avoid doing DevOps
• solve the frustration for beginners “I built the app, how do I publish it?”

Business-model is simple – you pay Appliku fixed fee for service and pay directly to cloud providers for their services.

There is a free plan on Appliku and AWS gives 1 year for new users.

If you are learning to code or seeking for a (mostly) free and enjoyable way to run your app – I really hope in these 17 months I built a solution that can give you that.

P.S.

I really hope you that hearing about my journey and what I built was interesting and doesn’t look like a promotional thing.

It was a really interesting journey, full of excitement, frustration (Hello, AWS Docs!), and the idea of making someone’s work easier.

I shared all this here to tell that even big and complex things can be done if you believe in it and if you show up every day (even for 30 minutes) and persistently work on it.

If the app helps someone or the article itself motivates somebody to get up and create something – I will be extremely happy.

Thanks for taking time to read this.

P.P.S.

Nearly forgot about the link to the app: https://appliku.com/

原文链接：Tired of Deployments, built my own Heroku