Why I Stick With Django - A Story About My Failed Project
So today I was scratching my head for good three hours over Django REST Framework.
While I furiously googled possible solutions, I kept asking why do I torture myself with this complicated framework?
Don't get me wrong.
Django is a great web development framework.
But is very opinionated and has a very steep learning curve.
I started learning Django three years ago in 2016 after I finished I now failed a large side project.
Website, which provides risk analytic coverage of European and US stocks.
The project was hacked together with WordPress, PHP, Python, and cron jobs.
The core analytics were done in Python, which fed an FTP server with daily stock data, scheduled with windows task scheduler.
On the server-side, I had a Cron job, that executed a PHP script, that loaded batch files into the WordPress database.
Sounds hacky right?
Yeah. I didn't know how to code back then.
Now, that I know a little better, I admire how I came up with such overcomplicated architecture.
After my project went life, I realized, that there is no way I can scale and improve that architecture.
So, I looked over the internet for web backend frameworks, that allow me to execute python scripts natively.
I found out about Flask and Django.
My first impression on Flask was positive.
A simple framework for quick prototypes, but difficult for going full-stack, as it didn't provide any support for functionalities, that WordPress provided out of the box, such as user author basic templating system.
Django offered much more.
Integrated user management, authentication, REST framework, MVC approach, basic templating, and routing system.
I was excited.
I can finally develop apps, that can incorporate machine learning and super complex data analytic tasks natively.
But then I read Django's official documentation.
I read it again, and again...and many times over again.
My mind was overwhelmed.
The difficulty was 1000x higher than learning how to deploy a WordPress blog and hack the WordPress template with custom PHP snippets.
The best way to learn is to try it myself and fail.
So, I tried to follow official Django documentation and create a basic todo app according to their step by step guide.
I immediately identified many new concepts, that require a significant change in the mindset.
Looking from the perspective of a non-programmer person, who grew up on WAMP stack (Apache, MySQL, PHP).
Deployment
There is no one-click installation program to set up the whole stack on your hosting.
You have to figure out by yourself how to set up and configure Nginx and Gunicorn.
Database management (Model)
Because Django uses ORM, the whole database model is defined in the model's file.
You just define table structure, columns, their data types, and relationships.
All changes are deployed by calling two commands - make migration and migrate.
No more fiddling with MyPHPAdmin. Heck yeah!
View
A new concept for me.
Views are files where you define your app logic.
That includes data extraction from the database with ORM or raw SQL queries.
Data manipulation like conditional filtering, sorting, parsing.
Finally, you return the processed data to the Django template for rendering.
Templates
A refreshing way of creating HTML output.
I don't have to write tons of code to do simple loops and variable transformations like in PHP.
I still don't know how to efficiently develop dynamic pages using the Django templating system, but I believe it will be less painful than with PHP.
Url Routing
Also a new concept.
You tie predefined URL paths to views and templates.
Each URL redirects you to the Django template and each template is filled with data from the corresponding view.
Everything is tightly coupled together.
I guess it has to be. Otherwise, it wouldn't be server-side rendering.
REST Framework
The only thing, that works as expected.
There is no need to learn additional concepts.
Pull data from the model to view.
Tie view to URL and define the JSON object structure in the Django serializer file.
Conclusion
The architecture works out of the box if you use it as Django documentation tells you to use it - as a simple CRUD application.
But simplicity ends when you start to have more complex models, that require heavy data transformation.
Then you have to figure out how to do complex query sets with multiple conditional joins, subqueries, with constructions and aggregation.
Despite all the quirks and hurdles.
There is no other option if you want to use analytical python libraries in your app.
That's why I didn't give up on Django and started to learn it again 3 months ago.
It's a painful experience, but I have to sacrifice prototyping speed for the long term scalability of the project.
Furthermore, once I finish my first Django web application, then I will be able to create anything I want in the future.
That's the basic value proposition of the Django framework - promise and hope, that once I overcome the steep barrier of entry, it will give me new powers and open new possibilities.
To end this article, I should say something more about my failed side project.
It wasn't technological debt, that killed it.
It is supposed to solve my problem with tracking hundreds of Stocks daily, calculate their risk measures, compare them side by side, and watch for any news articles, that appear.
I thought other people have the same problem, but I was wrong.
My risk measures were too advanced to comprehend.
Ordinary inexperienced stock traders just need actual price, moving average, and nothing else.
Simply if you don't understand something, you won't believe it.
So, I failed to get initial traction from early adopters.
Eventually, faith sealed my project forever as my only source of Stock data - Yahoo finance API - closed their shop so my website stopped updating itself with new stock data.
But I learned a lot in the process - hacking WordPress templates, working with HighCharts JS, scripting in python, scripting in PHP.
Mainly I didn't give up and finished my first large side project and its pain points led me to Django.
