Twenty Questions

This post covers scaffolding an MVP for the golinx project - a simple link redirector. For more info, see the project intro.

The Python world has quite a few different web frameworks, and I don’t want to enumerate them here. I chose Flask because I knew I’d be working with it in a job and had some prior experience. It’s pretty light-weight, but I’ve gained a lot of development experience since last using it and question the typical usage of the framework.

The application versus the project

If golinx were a movie, the web app would be the lead role, and the other posts in this series about tooling, testing, deployment, etc would be the critical but underappreciated roles that scroll by unnoticed in the credits. The reason why I’m writing this series is to shine light and understand these supporting roles that are so crucial to building a blockbuster web app, just like how the writing, directing, editing and filming can make or break a movie.

For me, the payoff in this series is the experience of standing something reliable up from scratch, knowing how to interchange each role, and distilling patterns for future use. With that, let’s talk about the starring role: the web app itself.

Web app

In my mind, the web app is the starring role because it’s the essence of what the project says it does. Golinx is a URL redirector, and the logic that defines what it means to be a URL redirector lives within the lines of the web app. To touch on the original introduction, I’ll need something that can:

• Create, read, update and destroy link-redirects (say, a LinkController).
• Store the link-directs (say, a LinkModel).
• Provide users a way to interact with link-redirect records (say, jinja templates).
• An actual redirect function that matches a path to its destination.
• Finally and eventually, some kind of user management.

Flask got the part because I have some prior experience and knew I’d be using it for my current job. With some more experience under my belt, I feel equipped to look at it up close with a different lens. I still like some of what I see, but find other choices in the framework odd (more below).

One other note: I’m avoiding using additional packages that probably already accomplish a lot of what I’m trying to do. I understand I’m reinventing the wheel, but that’s kind of the point.

Layout

This is going to evolve, but the app will follow a basic MVC setup with REST-like controllers, and a directory layout along the lines of:

├── auth.py
├── BUILD
├── controllers
│   ├── BUILD
│   ├── __init__.py
│   ├── link_controller.py
│   └── resource_controller.py
├── __init__.py
├── main.py
├── models
│   ├── base_model.py
│   ├── BUILD
│   ├── data
│   ├── db.py
│   ├── __init__.py
│   ├── link_model.py
│   ├── schema.sql
│   └── user_model.py
├── server.py
├── static
│   └── style.css
├── templates
│   ├── auth
│   ├── base.html
│   └── link
└── utils.py

It’d be fun to wire up some kind of SPA that interacts with the server through an API, but that’ll come later. For now, the UI layer is going to be basic web pages. I’m starting with the Flask tutorial, and tweaking for my needs.

Controllers

Riffing on the idea of RESTful resources, I’ve got a fairly simple approach for vanilla CRUD-operations on a top-level ResourceController using Flask’s MethodView class, which maps HTTP verbs to their corresponding actions. One minor drawback to this approach is that it works well for APIs, but not so great for pure HTML applications, because HTML doesn’t support PUT and DELETE requests. Because of this, I have to wire up POST routes that proxy to these actions.

One thing that naturally arises are nested resources and other associations between distinct entities (e.g. mapping from a user to their links). I’m toying around with how to represent this in a way that doesn’t introduce circular dependencies or leaky abstractions (i.e. where the information needed to make an association flows in one direction).

Models

The guiding light for me with models is to keep everything interchangeable. In the Python world, that means utilizing DBAPI2 compliant libraries (available for virtually any database you’d want to use). Starting with SQLite is a no-brainer, but I want to drop something more exotic in, like rqlite. As with controllers, there’s a simple BaseModel that handles CRUD operations on records. My primary challenge with the models is in efficiently managing access to a database connection. Right now the instance hangs onto a reference to a connection, but this has obvious drawbacks:

• a connection should really only be used in one place at a time to avoid messy transactions, or unexpected closures.
• if you utilize a pool, you can request a connection and give it back when done.
• models are lighter if they don’t have to worry about connection semantics.

Critique of Flask

Flask does a lot of things for you: provides routing, handles basic security (e.g. CSRF), has an extensive ecosystem and knowledge online, and offers many ways to get things done. Even so, after spending a short while getting to know it again, there are a few things that irk me about it. These are all my opinion, I would love to understand counterarguments, and find out better ways to accomplish what I’ve portrayed below.

Use of globals

Flask offers a few globals in the AppContext and RequestContext, which are ways to pass data between your functions and methods (e.g. g, request). This just seems like such an opportunity to inadvertently screw things up, especially in a larger code base (imagine name collisions for a variable set by two unrelated modules).

Import-time instantiation

It seems very common in Flask to instantiate your app and other utilities when they’re imported. Consider this example from the tutorial:

from flask import (
    Blueprint, flash, g, redirect, render_template, request, url_for
)
from werkzeug.exceptions import abort

from flaskr.auth import login_required
from flaskr.db import get_db

bp = Blueprint('blog', __name__)

I understand bp is created to provide pretty route decorators later on and may be justifiable in isolation, but it sets a pattern that is likely often repeated throughout. In aggregate, importing modules in your app will be slow and even risky (imagine eagerly generating some connection to a third party service at import).

Doing things should be reserved for functions and methods, not as a side effect of importing.

Harder to test

Because Flask hangs so much information off of globals provided by the framework at runtime, it makes it unintuitive to test (this is my own opinion at least). Consider how heavy this setup step from the docs is in order to run a simple unit test:

@pytest.fixture
def client():
    db_fd, flaskr.app.config['DATABASE'] = tempfile.mkstemp()
    flaskr.app.config['TESTING'] = True

    with flaskr.app.test_client() as client:
        with flaskr.app.app_context():
            flaskr.init_db()
        yield client

    os.close(db_fd)
    os.unlink(flaskr.app.config['DATABASE'])

Before you can even test a handler, you have set up a test context, and in this particular example intitialize a database. This seems way too heavy for a fast write / build / test cycle. It would be fine for end-to-end or regression tests, but is an absolute crutch to merely confirm a single handler’s behavior. This is necessary because Flask doesn’t call your handler with the request. Instead, makes it available via globals, such as in this example from the tutorial:

from flask import request

@bp.route('/register', methods=('GET', 'POST'))
def register():
    # Let's check out some stuff from the request global...
    if request.method == 'POST':
        username = request.form['username']
        password = request.form['password']
        ...

IMHO, it would be so much more ergonomic if the handlers looked like this:

@bp.route('/register')
class RegistrationResource(MethodView)
    def post(req: RegistrationRequest) -> RegistrationResponse:
        do_something_with(req.username, req.password)
        ...

Then you wouldn’t need to do any fancy fixtures to get a simple test going, you just craft a RegistrationRequest and pass it into the function, and would have type-information about its properties rather than some error-prone dict. My simple brain can hold that info all at once, but trying to think about wiring up a ton of stubs and other ancillary modules just to test one function blows my mind.

Further, if you need to mock the RegistrationRequest, then you can actually autospec it to constrain what methods or properties were available. I’m not sure how you would achieve that with a dict.

Overly reliant on environment variables

Flask provides its own opinionated way to do configuration, and the default methods rely heavily on assumptions about what environment variables have been set. Just give me a well-documented configuration class that I can use to instantiate my app, and I’ll inject everything it needs from my own set up (ok, Flask does have that). I understand a lot of this is up to how a project implements configuration, and can be solved with the factory pattern; however, because most projects will follow defaults it leads to head scratching moments where you’re asking yourself what magical env var you need to set to get some desired behavior.

Closing

Nothing’s perfect, and Flask is great example of this. In my opinion, it is great for smaller, single-person projects but does not scale well without very careful attention to detail, which itself is prone to human error. In Golinx I’ll be working very hard to code around the above issues as much as possible.