TIL: Python Performance, REST API Design, Rich Terminal Trees, and Python Launcher

12 Requests Per Second in Python

12 requests per second in Python

Detailed analysis of Python web application performance optimization:

The Challenge:

• Performance Bottlenecks: Identifying what limits Python web app throughput
• Realistic Benchmarking: Testing with real-world scenarios, not just hello world
• Optimization Strategies: Practical techniques for improving performance
• Infrastructure Considerations: Hardware and deployment factors

Performance Analysis:

Baseline Measurements:

• Flask Application: Basic REST API with database operations
• Load Testing: Using realistic request patterns and data
• Bottleneck Identification: CPU, I/O, memory, and network constraints
• Profiling Tools: cProfile, line_profiler, and memory profilers

Optimization Techniques:

Database Optimization:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

# Connection pooling
from sqlalchemy.pool import QueuePool
engine = create_engine(
    'postgresql://...', 
    poolclass=QueuePool,
    pool_size=20,
    max_overflow=30
)

# Query optimization
# Before: N+1 queries
users = User.query.all()
for user in users:
    print(user.profile.name)  # Additional query per user

# After: Eager loading
users = User.query.options(joinedload(User.profile)).all()
for user in users:
    print(user.profile.name)  # No additional queries

Caching Strategies:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

import redis
from functools import wraps

cache = redis.Redis()

def cached(expiration=3600):
    def decorator(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            key = f"{func.__name__}:{hash(str(args) + str(kwargs))}"
            result = cache.get(key)
            if result is None:
                result = func(*args, **kwargs)
                cache.setex(key, expiration, result)
            return result
        return wrapper
    return decorator

Key Insights:

• Database is Usually the Bottleneck: Optimize queries first
• Connection Pooling: Essential for concurrent request handling
• Caching: Dramatic improvements for repeated operations
• Async Processing: Use Celery for heavy background tasks

Designing Beautiful REST + JSON APIs

Oktane17: Designing Beautiful REST + JSON APIs - YouTube

Comprehensive guide to REST API design principles:

Core Design Principles:

Resource-Oriented Design:

Consistent Naming:

• Plural Nouns: Use /users not /user
• Hierarchical: /users/123/posts for nested resources
• Lowercase: Consistent casing throughout API
• Hyphen Separation: Use /user-profiles not /userProfiles

REST Must Be Hypertext-Driven

REST APIs must be hypertext-driven » Untangled

Roy Fielding’s clarification on true REST principles:

HATEOAS (Hypermedia as the Engine of Application State):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

{
  "id": 123,
  "name": "John Doe",
  "email": "[email protected]",
  "_links": {
    "self": {"href": "/users/123"},
    "posts": {"href": "/users/123/posts"},
    "edit": {"href": "/users/123", "method": "PUT"},
    "delete": {"href": "/users/123", "method": "DELETE"}
  }
}

API Evolution:

• Discoverability: Clients discover available actions through links
• Loose Coupling: Clients don’t hardcode URLs
• Version Independence: Links adapt to API changes
• Self-Documenting: API structure evident from responses

HTTP Status Codes:

Rich Terminal Tree Views

Rendering a tree view in the terminal with Python and Rich

Creating beautiful terminal interfaces with the Rich library:

Rich Library Features:

• Styled Output: Colors, bold, italic, underline in terminal
• Complex Layouts: Tables, panels, columns, and trees
• Progress Bars: Beautiful progress indicators
• Syntax Highlighting: Code syntax highlighting in terminal

Tree Rendering:

Basic Tree Structure:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

from rich.console import Console
from rich.tree import Tree

console = Console()

# Create root node
tree = Tree("📁 Project Root")

# Add branches
src_branch = tree.add("📁 src")
src_branch.add("🐍 main.py")
src_branch.add("🐍 utils.py")

tests_branch = tree.add("📁 tests")
tests_branch.add("🧪 test_main.py")
tests_branch.add("🧪 test_utils.py")

tree.add("📄 README.md")
tree.add("📄 requirements.txt")

console.print(tree)

File System Tree:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

import os
from pathlib import Path

def create_file_tree(directory: Path) -> Tree:
    tree = Tree(f"📁 {directory.name}")
    
    for item in sorted(directory.iterdir()):
        if item.is_dir():
            # Recursively add subdirectories
            subtree = create_file_tree(item)
            tree.add(subtree)
        else:
            # Add files with appropriate icons
            icon = "🐍" if item.suffix == ".py" else "📄"
            tree.add(f"{icon} {item.name}")
    
    return tree

Advanced Features:

• Styling: Custom colors and styles for different node types
• Interactive: Combine with click events for navigation
• Live Updates: Dynamic tree updates for monitoring
• Large Datasets: Efficient rendering of large hierarchies

Python Launcher (Rust)

python-launcher - crates.io

Rust-based Python version launcher and manager:

What It Does:

• Version Detection: Automatically detects appropriate Python version
• PEP 514 Compliance: Follows Python launcher specification
• Fast Performance: Rust implementation for speed
• Cross-Platform: Works on Windows, macOS, and Linux

Key Features:

Automatic Version Selection:

1
2
3
4
5
6

# Checks pyproject.toml, .python-version, or uses default
python3 script.py

# Explicit version specification
python3.9 script.py
python3.10 -m pip install package

Configuration:

1
2
3
4

# pyproject.toml
[tool.python-launcher]
default = "3.10"
prefer-active-venv = true

Advantages:

• Performance: Faster startup than shell-based launchers
• Reliability: Robust version detection and selection
• Standards Compliance: Follows Python packaging standards
• Modern Implementation: Written in modern Rust with good error handling

Each resource demonstrates different aspects of Python ecosystem optimization - from web application performance tuning to API design best practices, terminal UI enhancement, and development tool innovation.