Practical Logging in Python: From print() to Structured Logs

Something crashed at 3am. I checked the logs. This is what I found:

Error connecting
Retrying...
Done
10
User not found

No timestamps. No context. "User not found"—which user? When? What endpoint? I spent 2 hours just figuring out which part of the code produced these messages.

That was the day I stopped using print() for anything beyond quick debugging.

What's Wrong with print()

For quick scripts, print() is fine. But in production:

• No timestamps—when did this happen?
• No severity levels—is this info or an emergency?
• No filtering—can't show only errors
• No file output—everything vanishes when the terminal closes

Good logs answer: when, where, what, and how bad.

Logging Basics

Python's logging module is built-in. Here's the minimum you need:

import logging

logging.basicConfig(level=logging.INFO)

logging.debug("Detailed stuff, usually hidden")
logging.info("Normal operation, good to know")
logging.warning("Something unexpected, but not broken")
logging.error("Something broke")
logging.critical("Everything is on fire")

With level=logging.INFO, debug messages are hidden. Bump it to DEBUG when troubleshooting, set to WARNING in production to reduce noise.

The killer feature: you can leave debug logs in your code forever. They're silent until you need them.

Adding Timestamps and Context

basicConfig works for scripts. For real apps, set up proper formatting:

import logging

logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)

logging.info("Server started")

Output:

2025-03-22 10:30:00,123 - root - INFO - Server started

Now you know exactly when it happened.

Logging to Files

Console output disappears when the process dies. Write to files too:

import logging

logger = logging.getLogger("myapp")
logger.setLevel(logging.DEBUG)

# Console handler - only warnings and above
console = logging.StreamHandler()
console.setLevel(logging.WARNING)

# File handler - everything
file_handler = logging.FileHandler('app.log')
file_handler.setLevel(logging.DEBUG)

# Format
formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
console.setFormatter(formatter)
file_handler.setFormatter(formatter)

logger.addHandler(console)
logger.addHandler(file_handler)

Now your console stays clean (just warnings and errors) while the file captures everything for later analysis.

Rotating Logs

Log files grow forever. I've seen servers crash because a log file consumed all disk space.

from logging.handlers import RotatingFileHandler

handler = RotatingFileHandler(
    'app.log',
    maxBytes=5*1024*1024,  # 5MB
    backupCount=3           # Keep 3 old files
)
logger.addHandler(handler)

When app.log hits 5MB, it becomes app.log.1, and a fresh app.log starts. Old files get rotated out. Total disk usage stays bounded.

Structured Logging (JSON)

Text logs are great for humans reading a terminal. They're terrible for searching across thousands of servers.

Modern log aggregation tools (ELK, Datadog, CloudWatch) work better with JSON:

import json
import logging

class JsonFormatter(logging.Formatter):
    def format(self, record):
        log_data = {
            "timestamp": self.formatTime(record),
            "level": record.levelname,
            "message": record.getMessage(),
            "module": record.module,
        }
        if record.exc_info:
            log_data["exception"] = self.formatException(record.exc_info)
        return json.dumps(log_data)

handler = logging.StreamHandler()
handler.setFormatter(JsonFormatter())
logger.addHandler(handler)

logger.info("User logged in", extra={"user_id": 123})

Output:

{"timestamp": "2025-03-22 10:30:00", "level": "INFO", "message": "User logged in", "module": "auth"}

Now you can query: "show me all ERROR logs from the auth module in the last hour."

Logging Exceptions

When catching exceptions, use logger.exception()—it automatically includes the stack trace:

try:
    risky_operation()
except Exception:
    # ❌ Loses the stack trace
    logger.error("Operation failed")
    
    # ✅ Includes full traceback
    logger.exception("Operation failed")

Quick Tips

• Use __name__: logger = logging.getLogger(__name__) automatically tags logs with the module name
• Don't log secrets: Never log passwords, API keys, or personal data
• Be specific: "Failed to connect to database" is better than "Error"
• Include IDs: "User 12345 failed to login" helps trace issues

Switching from print() to logging feels like extra work. Until the first time you're debugging a production issue at 2am and your logs actually tell you what happened. Then it feels like a superpower.