DevOps
Smarter Python Configs: an OOP Framework
Wisaroot Lertthaweedech Dev.to (EN Zone)
1 views
Originally published on wisl.dev.
Update: this framework grew into bakefile, a task runner: the same base, service, and instance inheritance, rearchitected and published as a library. The story behind it: Developer Workflows Need an Abstraction.
Managing configurations across multiple environments and services is one of those things that should be simple, but rarely is.
If you've worked with .env files, you know the drill: every environment (e.g., sit, uat, canary, prod) has its own set of config files, and every service (be it Cloud Run, Dataflow, or others) duplicates a large portion of the same values. Add CI/CD pipelines into the mix, and the friction becomes obvious. Slight changes in config require regenerating .env files, updating deployment scripts, and often maintaining separate CI/CD logic per service and per environment.
This setup creates redundancy and slows down delivery.
So I built an OOP-based configuration framework in Python, built on top of Pydantic, to simplify and centralize configuration management. Here's how it works.
Why do configs get so redundant?
In most systems, we deal with multiple environments, such as sit (system integration test), uat (user acceptance test), canary, and prod (production).
Some teams might use different naming conventions like dev, qa, stage, or preprod, but the core challenge remains the same: Each environment requires its own config setup, often with small differences.
Additionally:
Each service (e.g., Dataflow job, Cloud Run service) needs its own config.
A large portion of the config values are shared (Docker registry URLs, GCP credentials, etc).
CI/CD pipelines often need to be duplicated to reflect these separate configurations.
That's a lot of repetitive work for something that's logically hierarchical and composable.
The tell-tale smell: a config change that touches three environments means three .env edits, three deployment-script checks, and three chances to get it wrong. Hierarchy collapses all of that into one edit at the right layer.
What does a layered config look like?
Layered composition: the base layer holds shared values, service and instance mixins add specifics, and the environment mixin flips the switches
Instead of treating configs as raw key-value pairs in .env, I defined them as Python classes, layered to reflect how configs are structured in reality.
The framework is built on Pydantic's BaseModel for type-safe config validation, and Object-Oriented Programming (OOP) to encourage composition and reuse.
Let's walk through the layers.
Base Layer: BaseConfigs
Contains shared, global settings like:
env: environment identifier (e.g., sit, prod)
is_localhost: true/false for local development
logging_level_per_module
Local GCP credentials path
Also provides template methods such as:
setup_environment()
deploy()
assert_deploy()
Each environment-specific mixin (e.g., ProdConfigsMixin) will override env = "prod" and anything else environment-specific.
Service Layer: e.g. DataflowConfigsMixin, CloudRunConfigsMixin
Contains service-specific variables:
For example, in DataflowConfigsMixin:
dataflow_job_name
dataflow_gcp_project
dataflow_subnetwork
Also contains service-specific behavior like:
deploy() method for triggering Dataflow job
assert_deploy() logic tailored to Dataflow logs or job status
For a Cloud Run-based service, you'd swap in CloudRunConfigsMixin.
Instance Layer: e.g. DataflowJobAConfigsMixin
Overrides any instance-specific config:
dataflow_job_name = "dataflow_job_a"
assert_deploy() might validate output in BigQuery or check an API result
Each job or service instance gets its own mixin, where you specialize the behavior.
How do the pieces come together?
Here's how all the pieces assemble in configs.py:
sit_configs = SitCommonConfigs()
uat_configs = UatCommonConfigs()
canary_configs = CanaryCommonConfigs()
prod_configs = ProdCommonConfigs()
configs = get_configs(
prod_configs=prod_configs,
canary_configs=canary_configs,
uat_configs=uat_configs,
sit_configs=sit_configs,
)
get_configs() returns the config object for the current environment, determined by a single ENV variable in .env.
Example of an environment-specific class:
class ProdCommonConfigs(
DataflowJobAConfigsMixin,
ProdDataflowConfigsMixin,
DataflowConfigsMixin,
ProdConfigsMixin,
BaseConfigs,
):
pass
What does the CLI give us?
We provide a small CLI tool (ac, short for Abacus Configs) with a few essential commands:
ac echo_configs
Outputs all resolved configs for the current environment:
> ac echo_configs
export ENV=sit
export IS_LOCALHOST=True
export DATAFLOW_GCP_PROJECT=production-project
...
You can use this in shell sessions or pipelines with:
> eval $(ac echo_configs)
This means you can dynamically generate environment variables from your Python config at runtime: no more handcrafting .env files for CI/CD.
eval $(ac echo_configs) inside a pipeline step means the pipeline itself carries zero environment-specific values. The same YAML deploys sit, uat, and prod: only the ENV variable changes.
ac deploy
Triggers the appropriate deploy() method for the current config. Could be deploying a Dataflow job or pushing to Cloud Run, depending on the config object's class structure.
ac assert_deploy
Runs validations defined in the config class:
Check for logs
Confirm data landed in a target system
Make an API call to confirm behavior
This fine-grained control is often needed to test service correctness post-deployment.
Why does this work?
Because all config objects implement the same interface (deploy, assert_deploy, etc.), CI/CD pipelines no longer need to be duplicated per service or environment.
Here's what changes:
Environment-specific behavior is encoded in the config object
CI/CD pipelines can simply run ac deploy and ac assert_deploy in the right environment
Adding a new service or environment requires only a new config class, not a whole new pipeline
On top of that:
Zero duplication: shared logic and values are defined once, and reused through inheritance
Type-safe: configs are validated early using Pydantic
Flexible overrides: environment, service, and instance layers allow for precise customizations
Testable & extensible: logic like deploy() can easily be unit tested or extended
Unified CI/CD: one pipeline logic to rule them all
Where to take it next
This configuration system has significantly reduced overhead for our dev teams. It's made configs declarative, validated, and reusable, and has unified deployment across environments and services.
If you're finding yourself copying .env files, duplicating CI pipelines, or fighting config drift, this might be worth exploring in your Python projects.
Read original: https://dev.to/wisl/smarter-python-configs-an-oop-framework-2ggj
← Previous
Apple M6: Why There Will Never Be an M6 Pro
Next →
The Model Is Not the Product. Here's What Actually Is.
Related
🚀 Kubernetes: From YAML to Running Pods
DevOps
1
DEV Community
Web Traffic Inspector Dash
DevOps
1
Dev.to (EN Zone)
Three production bugs in one night, and all three were hidden by empty catch blocks I wrote myself
DevOps
0
Reddit r/webdev
DaemonCore Academy: Into the Microsoft Store we go!
DevOps
7
Dev.to (EN Zone)
Comments0
No comments yet — be the first