Improving_Flask_Performance.MD

2 critical problems in Flask app performance

1. CPU usage
2. Database Optimization queries (overloaded database)

SOLUTIONS

Logging Database Performance and Optimizing queries

Before we can optimize queries , we need to know which is time-consuming

#config.py

import os
basedir = os.path.abspath(os.path.dirname(__file__))

class Config:
    SECRET_KEY = os.environ.get('SECRET_KEY')
    SQLALCHEMY_TRACK_MODIFICATIONS = False
    SQLALCHEMY_RECORD_QUERIES = True
    FLASKY_SLOW_DB_QUERY_TIME = 360

#views.py

#Write a view function that returns and log database queries that takes up to 2 mins


from flask import Blueprint, current_app
from flask_sqlalchemy import get_debug_queries

main = Blueprint('main', __name__)

@main.after_app_request
def after_request(response):
    for query in get_debug_queries():
        if query.duration >= current_app.config['FLASKY_SLOW_DB_QUERY_TIME']:
            current_app.logger.warning(
                'Slow query: %s\nParameters: %s\nDuration: %fs\nContext: %s\n'
                % (query.statement, query.parameters, query.duration,
                   query.context))
    return response

Top solution

• Adding more indexes to your SQLAlchemy models
• Use Flask-Cache extension to cache long queries.
• Database replication : Setup a master database that generally only supports write operations. A slave database gets copies of the data from the master database and only supports read operations.

Check out the following articles that have been helpful to me on how to implement DB replication.

1. There is no official support yet with Flask-SQLalchemy but you can customize it, check out this guide here on stackoverflow
2. Check out this medium article Ishan Joshi
3. Check out this Github's gist solution
4. Flask extension Flask Replicated by Peter Denim
5. Improving flask performance by Alexey Smirnov

CPU Optimization

First identify poor-performing functions and consider setting up the asynchronous background or periodic tasksusing Celery

How to indentify time consuming tasks

1. Werkzeug can optionally enable Python Profiler for each request
2. Install Flask-profiler which measures endpoints defined in your flask application and provides you fine-grained report through a web interface. I will always recommend the second option

Option 1 : Werkzeug can optionally enable Python Profiler for each request

Set up a profiler that watches your running application, records the functions that are called, how long each takes to execute, and gives a detailed report showing the slowest task. However, Profiling is typically done only in a development environment.

WHY?

A source code profiler makes the application run much slower than normal because it has to observe and take notes on all that is happening in real-time.

#app/__init__.py

from config import config
from flask import Flask
def create_app(config_name):
    app = Flask(__name__)
    app.config.from_object(config[config_name])
    config[config_name].init_app(app)
    #---existing code
    return app

#main.py 
import os
from app import create_app
app = create_app(os.getenv('FLASK_ENV'))


@app.cli.command()
@click.option('--length', default=35,
              help='Number of functions to include in the profiler report.')
@click.option('--profile-dir', default=None,
              help='Directory where profiler data files are saved.')
def profiler(length, profile_dir):
    """Start the application under the code profiler."""
    from werkzeug.contrib.profiler import ProfilerMiddleware
    app.wsgi_app = ProfilerMiddleware(app.wsgi_app, restrictions=[length],
                                      profile_dir='./profile_folder')
    app.run()

When you start your application, the console will show the profiler statistics for each request and it will include the slowest 35 functions specified by the --length option. The profile data for each request is saved to a file in the profile_folder directory

Flask Profiler

I recommend this because it will give you an interactive web UI that answer questions such as :

• Where are the bottlenecks in my application?
• Which endpoints are the slowest in my application?
• Which are the most frequently called endpoints?
• What causes my slow endpoints? In which context, with what args and kwargs are they slow?
• How much time did a specific request take?

#app.py
import os
from flask import Flask
app = Flask(__name__)

from flask_profiler import Profiler


app.config["flask_profiler"] = {
    "enabled": app.config["DEBUG"],
    "storage": {
        "engine": "sqlalchemy",
        "db_url": "postgresql://user:pass@localhost:5432/flask_profiler"
    },
    "basicAuth":{
        "enabled": True,
        "username": os.getenv('profiler_username'),
        "password": os.getenv('profiler_password')
    },
    "ignore": [
        "^/static/.*"
    ]
}

Profiler(app)

The dashboard is also using SQLAlchemy as its available background. The dashboard is available under localhost:5000/flask-profile using your default credentials( profiler_username and profiler_password)