Frontend
Stop Building SEO Reports by Hand: Automating Client Reporting With APIs, SQL, and Scheduled Jobs
Pallab Mondal Dev.to (EN Zone)
3 views
If you manage SEO for more than a couple of websites, you eventually hit the same wall.
The SEO work itself isn't always the slow part.
The reporting is.
Every month, you open Google Search Console. Then Google Analytics. Maybe PageSpeed Insights. You export CSV files, adjust date ranges, copy numbers into a spreadsheet, update charts, write a summary, check that nothing is obviously wrong, and finally turn everything into a PDF.
Then you do it again for the next client.
And the next one.
At first, this feels manageable. After all, you only need an hour or two per report.
But multiply that by 10, 20, or 50 clients and the problem becomes obvious.
You're not really doing SEO reporting anymore.
You're maintaining a manual data pipeline.
The good news is that most of this process can be automated with fairly ordinary tools: APIs, Python, SQL, scheduled jobs, and a reporting template.
You don't need a massive data platform.
You need a reliable pipeline.
This article walks through how I'd approach building one.
The Real Problem Isn't the Report
A typical SEO report contains several different types of information:
Organic clicks
Impressions
Click-through rate
Average search position
Top queries
Top landing pages
Organic sessions
Conversions
Technical issues
Period-over-period changes
Recommendations
The difficult part isn't displaying those numbers.
The difficult part is getting the right data, consistently, for the right date range, then turning it into something a client can understand.
A manual process might look like this:
Google Search Console
↓
CSV export
↓
Spreadsheet
↓
Manual calculations
↓
Charts
↓
Written summary
↓
PDF
↓
Client
An automated process can look more like this:
┌──────────────────┐
│ Google Search │
│ Console API │
└────────┬─────────┘
│
▼
┌──────────────┐ ┌───────────────┐
│ Google │──▶│ Data │
│ Analytics API│ │ Collection │
└──────────────┘ └───────┬───────┘
│
▼
┌─────────────┐
│ PostgreSQL │
└──────┬──────┘
│
▼
┌─────────────┐
│ Calculations│
└──────┬──────┘
│
▼
┌─────────────┐
│ Report │
│ Generator │
└──────┬──────┘
│
▼
PDF / HTML
│
▼
Client
That distinction is important.
You're not automating a PDF.
You're automating the system that produces the PDF.
Decide What the Report Should Answer
Before touching an API, decide what questions your report should answer.
Clients rarely care about a number just because it exists.
They care about questions such as:
Did organic traffic grow?
Did search visibility improve?
Which pages generated the most traffic?
Which pages lost traffic?
Are rankings moving in the right direction?
Did organic visitors generate leads or sales?
What changed compared with last month?
What should we work on next?
A practical overview of what belongs in an SEO report is also worth keeping nearby when designing the template.
The answers determine what data you actually need.
For example, if your goal is to report on organic growth, Search Console data might be enough for search performance.
If you want to connect SEO to business results, you'll probably also need Google Analytics data.
If you're reporting on technical SEO, you'll need another data source entirely.
The first lesson is simple:
Don't collect data just because an API makes it available.
Collect data because it answers a useful question.
Start With Google Search Console
Google Search Console is usually the best place to start.
The Search Console API gives you access to performance information such as:
Clicks
Impressions
CTR
Average position
Queries
Pages
Countries
Devices
Search appearance
You can request a date range and optionally group the results by dimensions such as query or page.
Conceptually, a response might look like this:
{
"rows": [
{
"keys": ["seo automation"],
"clicks": 124,
"impressions": 2400,
"ctr": 0.0516,
"position": 8.4
},
{
"keys": ["automated seo reporting"],
"clicks": 97,
"impressions": 1800,
"ctr": 0.0538,
"position": 10.2
}
]
}
You can then transform that response into your own internal format.
For example:
keyword = {
"query": row["keys"][0],
"clicks": row["clicks"],
"impressions": row["impressions"],
"ctr": row["ctr"],
"position": row["position"]
}
Once the data is inside your application, you shouldn't care too much about where it originally came from.
That's an important architectural decision.
Your report generator shouldn't need to know whether a metric came from Search Console, Analytics, a CSV file, or a database.
Compare Periods Instead of Showing Snapshots
A report saying:
Organic clicks: 14,320
isn't particularly useful.
Is that good?
Was it 12,000 last month?
Was it 20,000?
Without context, the number means very little.
Instead, calculate changes.
For example:
def percent_change(current, previous):
if previous == 0:
return None
return ((current - previous) / previous) * 100
Then:
current_clicks = 14320
previous_clicks = 12000
change = percent_change(current_clicks, previous_clicks)
print(f"{change:.1f}%")
The result is:
19.3%
Now your report can say:
Organic clicks increased 19.3% compared with the previous period.
That's much more useful than displaying two disconnected numbers.
You can use the same approach for:
Impressions
Sessions
Conversions
Revenue
Landing-page traffic
Keyword groups
Technical issues
The comparison itself often contains more information than the raw metric.
Then Bring in GA4
Search Console tells you what happened in Google Search.
Google Analytics can help answer what happened after someone arrived.
Depending on the reporting goal, you might collect:
Organic sessions
Engaged sessions
Users
Conversions
Revenue
Landing pages
Engagement metrics
A simple landing-page dataset might look like this:
[
{
"page": "/seo-guide",
"sessions": 1820,
"conversions": 42
},
{
"page": "/technical-seo",
"sessions": 1240,
"conversions": 31
},
{
"page": "/keyword-research",
"sessions": 980,
"conversions": 18
}
]
You can combine this with Search Console information.
For example:
Search Console
↓
Which pages receive search visibility?
GA4
↓
Which pages receive organic visitors?
Conversions
↓
Which pages contribute to business outcomes?
That gives you a much stronger report.
Instead of:
This page ranks well.
You can potentially say:
This page is receiving significant organic visibility and is also contributing to conversions.
That's the difference between an SEO report and a collection of SEO metrics.
Authentication Is Where the Fun Begins
APIs are convenient once authentication works.
Getting authentication configured is often the annoying part.
For Google APIs, you'll typically need to deal with:
A Google Cloud project
API access
Credentials
OAuth or service-account authentication
Appropriate permissions
Property/account access
For a single internal project, you can keep this fairly simple.
For an agency platform, things become more interesting.
You may have:
Client A
├── Search Console property
├── GA4 property
└── Credentials
Client B
├── Search Console property
├── GA4 property
└── Credentials
Client C
├── Search Console property
├── GA4 property
└── Credentials
Your database might contain something like:
CREATE TABLE clients (
id SERIAL PRIMARY KEY,
name TEXT NOT NULL,
search_console_property TEXT,
ga4_property_id TEXT
);
Credentials should not simply be stored as plaintext database fields.
Use environment variables, a secret manager, or encrypted credential storage appropriate for your deployment.
For example:
GOOGLE_APPLICATION_CREDENTIALS=/secrets/google-service-account.json
The exact authentication architecture will depend on whether you're building a personal tool, an agency platform, or a SaaS product.
Don't Generate the PDF Directly From the API
This is one of the architectural mistakes I'd avoid.
You don't want code that effectively does this:
API request
↓
PDF generation
It works initially.
Then the report becomes difficult to maintain.
Instead, create layers.
API
↓
Raw data
↓
Normalized data
↓
Metrics
↓
Report model
↓
Template
↓
PDF
For example, your normalized object could look like:
report_data = {
"period": {
"current": "2026-08-01/2026-08-31",
"previous": "2026-07-01/2026-07-31"
},
"organic_search": {
"clicks": 14320,
"impressions": 248000,
"ctr": 0.0577,
"position": 9.8
},
"analytics": {
"organic_sessions": 17640,
"conversions": 412
}
}
Now your PDF generator doesn't need to understand Google APIs.
It just receives structured data.
That makes testing much easier.
You can even generate a report from a fake dataset without making a single external API call.
Store Historical Data
If you're serious about automation, don't rely entirely on live API calls.
Store historical data.
A simple PostgreSQL table might look like:
CREATE TABLE seo_daily (
id BIGSERIAL PRIMARY KEY,
client_id INTEGER NOT NULL,
date DATE NOT NULL,
clicks INTEGER DEFAULT 0,
impressions INTEGER DEFAULT 0,
ctr NUMERIC,
position NUMERIC,
organic_sessions INTEGER DEFAULT 0,
conversions INTEGER DEFAULT 0
);
Then your reporting system can query historical data:
SELECT
date,
SUM(clicks) AS clicks,
SUM(impressions) AS impressions,
SUM(organic_sessions) AS sessions,
SUM(conversions) AS conversions
FROM seo_daily
WHERE client_id = 42
GROUP BY date
ORDER BY date;
This gives you several advantages.
Historical comparisons
You can compare:
Month over month
Quarter over quarter
Year over year
Custom periods
Faster reports
Your report generator doesn't have to request everything from external APIs every time.
Debugging
If an API returns strange data today, you can inspect exactly what was stored previously.
Reproducibility
You can regenerate an old report using the data that existed when the report was originally produced.
That's valuable when clients ask:
"Can you resend the report from six months ago?"
Keep Raw and Processed Data Separate
A useful pattern is to distinguish between raw data and processed data.
For example:
Raw API response
↓
Normalization
↓
Business calculations
↓
Report data
Raw data might look messy.
That's okay.
The normalized layer should be predictable.
For example:
{
"date": "2026-08-31",
"source": "search_console",
"metric": "clicks",
"value": 14320
}
Then your calculation layer can work with consistent structures.
This also means you can change your report design without changing your API integration.
That's exactly what you want.
Build the Report Around Decisions
A common reporting mistake is starting with charts.
Instead, start with decisions.
Imagine these two sections.
Version A
Organic clicks: 14,320
Impressions: 248,000
CTR: 5.77%
Average position: 9.8
Version B
Organic clicks increased 19% compared with the previous period.
The strongest growth came from informational pages,
while two commercial landing pages lost visibility.
The next priority is improving internal linking to the
declining commercial pages and refreshing their content.
Version B is much more useful.
The numbers can still be included.
But they support the explanation.
A good report should answer:
So what?
That's the question that turns analytics into strategy.
Add Simple Anomaly Detection
You don't need machine learning to make automated reporting smarter.
Start with rules.
For example:
def detect_change(current, previous, threshold=20):
if previous == 0:
return None
change = ((current - previous) / previous) * 100
if abs(change) >= threshold:
return {
"change": change,
"significant": True
}
return {
"change": change,
"significant": False
}
Then you can flag situations like:
Clicks ↓ 28%
Conversions ↓ 34%
or:
Impressions ↑ 41%
Clicks ↑ 3%
The second example is interesting.
Visibility increased substantially, but clicks barely moved.
That could justify investigating:
Search intent
SERP features
Titles
Meta descriptions
Ranking distribution
Query mix
The system doesn't need to diagnose everything automatically.
It just needs to surface things worth investigating.
Schedule the Whole Pipeline
Once your pipeline works manually, schedule it.
A simple architecture might be:
Every Monday at 06:00
↓
Fetch API data
↓
Validate response
↓
Store raw data
↓
Normalize data
↓
Calculate metrics
↓
Detect anomalies
↓
Generate report
↓
Upload PDF
↓
Send email
For a small deployment, cron may be enough.
Example:
0 6 * * 1 /usr/bin/python3 /app/run_reports.py
For larger systems, you might eventually use:
Celery
Cloud Scheduler
GitHub Actions
AWS EventBridge
Google Cloud Scheduler
Kubernetes CronJobs
A managed workflow system
Don't start with the most complicated scheduler you can find.
Start with the simplest reliable option.
Make Failures Boring
Automated systems fail.
APIs timeout.
Credentials expire.
Rate limits happen.
A property gets removed.
A database connection fails.
The goal isn't to eliminate every failure.
The goal is to make failures easy to understand.
For example:
import logging
logger = logging.getLogger(__name__)
try:
data = fetch_search_console_data()
except Exception:
logger.exception("Search Console request failed")
You should ideally track:
START
↓
FETCH
↓
VALIDATE
↓
STORE
↓
CALCULATE
↓
GENERATE
↓
SEND
↓
SUCCESS
If something breaks:
START
↓
FETCH
↓
ERROR
You immediately know where to look.
Good logging is one of those boring engineering practices that becomes incredibly useful once you have dozens of automated jobs running.
Add Retries Carefully
Temporary API failures shouldn't necessarily kill the entire report.
A simple retry mechanism can help:
import time
def retry(func, attempts=3, delay=2):
for attempt in range(attempts):
try:
return func()
except Exception:
if attempt == attempts - 1:
raise
time.sleep(delay * (attempt + 1))
But don't blindly retry every error.
A retry makes sense for things like:
Temporary network errors
Timeouts
Some server-side failures
It doesn't fix:
Invalid credentials
Invalid property IDs
Permission problems
Malformed requests
Good retry logic should distinguish temporary problems from permanent ones.
Generate Different Report Formats From the Same Data
Once you have a clean report model, you can generate multiple outputs.
For example:
Normalized report data
│
├── PDF
├── HTML
├── Email
├── Dashboard
├── JSON
└── Spreadsheet
This is another reason not to couple your calculations directly to the PDF.
Your report model might be:
report = {
"summary": {},
"organic_search": {},
"landing_pages": [],
"conversions": {},
"technical": {},
"recommendations": []
}
The PDF template consumes it.
An email template consumes it.
A dashboard consumes it.
A future API endpoint consumes it.
One data model.
Multiple outputs.
A Practical Report Structure
If I were building an automated client report today, I'd probably start with something like this.
1. Executive Summary
Three to five important observations.
Not 20 metrics.
For example:
Organic traffic increased 18% this month.
Search visibility also improved, with impressions increasing 31%.
However, conversions grew only 4%, suggesting that traffic quality
should be investigated before scaling content production.
2. Organic Search Performance
Include:
Clicks
Impressions
CTR
Average position
Period comparison
A line chart can show the trend.
3. Top Landing Pages
Show:
Page
Clicks
Sessions
Conversions
/seo-guide
3,420
4,120
86
/technical-seo
2,870
3,180
72
/keyword-research
2,140
2,630
51
But don't stop there.
Highlight unusual changes.
For example:
/technical-seo
Traffic: +38%
Conversions: +52%
That's much more useful than simply ranking pages from highest to lowest.
Don't Automate the Human Explanation Too Aggressively
Automation can easily produce terrible writing.
For example:
Organic clicks increased by 19.3%, indicating a positive performance trend and suggesting that SEO visibility has improved during the reporting period.
Technically correct.
Painfully robotic.
A better approach is to use structured templates and leave room for human judgment.
For example:
if clicks_change > 15 and conversions_change > 10:
summary = (
"Organic search improved strongly this period, "
"with both traffic and conversions moving higher."
)
If traffic rises but conversions fall:
if clicks_change > 15 and conversions_change < -10:
summary = (
"Search traffic increased, but conversions declined. "
"The next step is to investigate whether the additional "
"traffic is reaching the right landing pages."
)
These aren't meant to replace an SEO professional.
They're meant to handle the repetitive 80%.
The human should still be able to edit the final interpretation.
Where AI Actually Helps
AI can be useful here, but I'd give it a narrow job.
Don't send raw API responses to an AI model and ask:
"Write my SEO report."
That's an easy way to get invented conclusions.
Instead, calculate the facts yourself.
Then provide structured information.
For example:
{
"organic_clicks_change": 19.3,
"organic_sessions_change": 17.8,
"conversions_change": 4.1,
"top_growth_page": "/seo-guide",
"declining_page": "/technical-seo",
"technical_issues": 3
}
Then ask the model to interpret those facts.
You can constrain it further:
Use only the supplied metrics.
Do not invent causes.
Do not invent rankings.
Do not claim that a change was caused by an algorithm update
unless that information is explicitly provided.
Return:
1. Three observations
2. Two risks
3. Three recommended next actions
That's a much safer use of AI.
The application owns the facts.
The model helps turn structured facts into readable language.
The Architecture I'd Build
For a small SEO agency, I'd keep the first version relatively boring.
Something like:
┌──────────────────┐
│ Search Console │
└────────┬─────────┘
│
▼
┌──────────────────┐
│ Data Collector │
└────────┬─────────┘
│
┌────────┴─────────┐
│ │
▼ ▼
┌──────────────┐ ┌──────────────┐
│ Raw Storage │ │ GA4 Collector│
└──────┬───────┘ └──────┬───────┘
│ │
└────────┬─────────┘
▼
┌──────────────┐
│ PostgreSQL │
└──────┬───────┘
│
▼
┌──────────────┐
│ Calculations │
└──────┬───────┘
│
▼
┌──────────────┐
│ Report Model │
└──────┬───────┘
│
┌────────┴─────────┐
▼ ▼
┌───────────┐ ┌───────────┐
│ PDF │ │ HTML │
└───────────┘ └───────────┘
The stack could be surprisingly simple:
Python
PostgreSQL
Google APIs
Jinja2
HTML/CSS
PDF renderer
Cron
Object storage
You don't need Kubernetes on day one.
You don't need a distributed event system.
You don't need 17 microservices.
You need a reliable pipeline.
What I Would Automate First
If you're starting from zero, I'd build it in this order.
Step 1: Automate data collection
Get Search Console data without manual exports.
Step 2: Store the data
Put it somewhere persistent.
Step 3: Add GA4
Connect search performance with website behavior.
Step 4: Calculate comparisons
Month over month is enough initially.
Step 5: Add anomaly detection
Flag large changes automatically.
Step 6: Create a report model
Separate data from presentation.
Step 7: Build one good template
Don't create five formats immediately.
Step 8: Schedule it
Make the pipeline run without manual intervention.
Step 9: Add AI carefully
Use it for interpretation, not fact generation.
This order matters.
If the underlying data isn't reliable, a beautifully designed AI-generated report doesn't solve anything.
It just produces unreliable information faster.
The Biggest Reporting Mistake
The easiest metrics to automate aren't necessarily the most valuable ones.
It's very easy to create a report containing:
Keywords: 4,281
Backlinks: 1,204
Domain Rating: 62
Pages Crawled: 8,291
Impressions: 248,000
It looks impressive.
But what did the business actually gain?
A better report might contain fewer numbers:
Organic traffic: +18%
Organic conversions: +14%
Revenue from organic traffic: +21%
Three commercial pages lost visibility.
Next priority:
refresh those pages and strengthen internal links.
That's a business conversation.
The purpose of SEO reporting isn't to prove that you've collected a lot of data.
It's to explain what happened and what should happen next.
A Useful Rule for Automated SEO Reporting
Here's the rule I'd keep on the wall:
Automate collection. Automate calculation. Automate repetition. Keep judgment human.
APIs are excellent at collecting information.
SQL is excellent at storing and aggregating it.
Python is excellent at transforming it.
Schedulers are excellent at repeating it.
Templates are excellent at formatting it.
But deciding what matters?
That's where human expertise still earns its keep.
From Spreadsheet Chore to Reporting System
The first time you automate an SEO report, the result might not look revolutionary.
Maybe you save 30 minutes.
That's fine.
Then you automate another client.
Then another.
Soon the same system is collecting data, calculating changes, generating charts, producing reports, and preparing summaries for dozens of properties.
That's when the economics change.
Instead of spending Monday morning copying metrics into spreadsheets, you can spend that time looking at the exceptions.
Which pages are growing?
Which pages are declining?
Where are conversions moving?
What changed?
What should we test next?
That's where SEO work becomes more interesting.
The goal isn't to eliminate reporting.
The goal is to eliminate the repetitive parts that don't require your attention.
Build the pipeline once.
Let the machine collect the numbers.
Then spend your time explaining what they mean.
Read original: https://dev.to/raisereturn/stop-building-seo-reports-by-hand-automating-client-reporting-with-apis-sql-and-scheduled-jobs-7g7
← Previous
StyleSmuggler: Unauthenticated RCE via Adobe Commerce Failed Payment Email Rendering
Next →
A ticker is not an identity
Related
Beyond APIs: Building a Privacy-First Drug Interaction Tool with WebGPU and WebLLM
Frontend
0
DEV Community
Why Amazon Deprecated MOBI for Kindle and How In-Browser EPUB Conversion Works
Frontend
0
DEV Community
How to Deploy a Web Project with Tencent EdgeOne Makers Using GitHub
Frontend
0
DEV Community
Why Enterprise SEO Advice Can Backfire on Small Sites and What to Do Instead
Frontend
1
Dev.to (EN Zone)
Comments0
No comments yet — be the first