Your bid limit controls payload, not priced project rows

Your daily research job is about to fetch proposals, and the biggest-looking limit isn't the one that prices project rows.

maxItems caps emitted projects. maxBidsPerProject caps the bids nested inside them, while start-event quantity depends on memory.

That distinction matters before you enable enrichment: a thousand nested bids aren't a thousand separately priced project records.

Example configuration: validate the project ceiling and proposal limit before either reaches the scheduler.

import assert from 'node:assert/strict'; import { ApifyClient } from 'apify-client'; import { z } from 'zod'; const ResearchInput = z.object({ mode: z.literal('search'), query: z.string().min(1), includeDetails: z.boolean(), includeBids: z.boolean(), maxItems: z.number().int().min(1).max(10000), maxBidsPerProject: z.number().int().min(1).max(1000), }).strict(); const run_input = ResearchInput.parse({ mode: 'search', query: 'python', includeDetails: false, includeBids: true, maxItems: 100, maxBidsPerProject: 20, });

This validator supports the search workflow you're scheduling, with explicit enrichment switches so a later default change can't silently alter your configuration.

Unexpected keys fail validation. A misspelled spending control deserves an error before breakfast.

Priced events follow the dataset boundary

The published result event counts a record emitted into the default dataset. Individual bids live inside that project record, so their presence doesn't create additional result events.

Published event	Quantity follows	Budget consequence
`apify-default-dataset-item`	Emitted project records	Use `maxItems` for the result-event ceiling
`apify-actor-start`	Starts and allocated memory	Account for memory on every attempt

There isn't a separate published bid-request event.

Enabling includeBids adds one request per project. Enabling includeDetails independently adds one project-page request per project, even though both enrich the same emitted record.

Those requests still matter. They add work that can affect platform usage, while larger nested payloads also increase storage and processing work in your own pipeline.

Keep the branches visible when you design the budget:

maxItems ------------> project rows ------> result events memory + run starts ---------------------> start events includeBids ---------> extra requests ----> platform usage maxBidsPerProject ---> nested payload ----> downstream work

For proposal research, a useful payload limit depends on what you intend to read or analyze afterward. Returning more text has little value if your next stage discards it.

Compute the ceiling before paying to start

Use integer microdollars for event arithmetic, then convert to dollars at the API boundary. This keeps the budget comparison in integer units.

The start rule is one event per GB, with a minimum of one event. Fractional-memory rounding isn't specified, so the estimator below accepts only whole-GB allocations and checks an explicit start-event quantity against that restricted model.

Guessing deserves a rejection here.

This function combines the published FREE and GOLD prices with the validated project limit.

const EVENT_PRICES = { FREE: { result: 990, start: 100_000 }, GOLD: { result: 490, start: 30_000 }, } as const; type Tier = keyof typeof EVENT_PRICES; type MemoryAccounting = { memoryGb: number; startEventQuantity: number; }; function eventCeiling( rawInput: unknown, tier: Tier, memory: MemoryAccounting, ) { const input = ResearchInput.parse(rawInput); if (!Number.isSafeInteger(memory.memoryGb) || memory.memoryGb < 1) { throw new Error('Only whole-GB memory accounting is supported'); } if (memory.startEventQuantity !== Math.max(1, memory.memoryGb)) { throw new Error('Start-event quantity does not match memory'); } const price = EVENT_PRICES[tier]; const eventMicroUsd = input.maxItems * price.result + memory.startEventQuantity * price.start; if (!Number.isSafeInteger(eventMicroUsd)) { throw new Error('Event ceiling exceeds safe integer arithmetic'); } return { input, eventMicroUsd, platformUsage: 'unpriced' as const }; }

The memory argument describes the actual run allocation; it doesn't configure that allocation. Keep both settings together operationally, and require a budget review whenever the deployed memory changes.

maxBidsPerProject belongs in validation and payload planning. Multiplying it by the result price would manufacture a charge event that the Store doesn't publish.

Platform usage remains unpriced here. You pay it at your plan's rates in addition to event fees, so this function can't provide a whole-run quote.

Reserve attempts across the schedule

Your recurring budget needs a defined accounting period and an allowance for recovery attempts, because retries purchase another start even when the research query hasn't changed.

A failed attempt can also have emitted chargeable rows before it stopped. Deduplicating those rows later doesn't undo their collection cost.

These executable checks verify the supplied price examples, then reserve event fees for a chosen scheduling period.

const memoryAccounting = { memoryGb: 1, startEventQuantity: 1, }; for (const [tier, maxItems, expectedUsd] of [ ['FREE', 100, 0.199], ['GOLD', 100, 0.079], ['FREE', 1000, 1.09], ['GOLD', 1000, 0.52], ] as const) { const quote = eventCeiling( { ...run_input, maxItems }, tier, memoryAccounting, ); assert.equal(quote.eventMicroUsd / 1_000_000, expectedUsd); } const schedule = { scheduledStarts: 30, recoveryStarts: 5 }; const eventAllowanceMicroUsd = 10_000_000; const perAttempt = eventCeiling(run_input, 'FREE', memoryAccounting); const reservedEventMicroUsd = (schedule.scheduledStarts + schedule.recoveryStarts) * perAttempt.eventMicroUsd; if (reservedEventMicroUsd > eventAllowanceMicroUsd) { throw new Error('Schedule exceeds its event-fee allowance'); }

Preflight can reject an oversized schedule before any start charge occurs. A platform-side limit only becomes relevant once you've requested a run.

For a live scheduler, persist the reservation and consume an attempt before launch. Make that operation atomic when workers can overlap, or each worker can approve itself against the same remaining balance.

Fewer, larger runs can spread start fees across more project records. You also accept less frequent observations and a larger recovery task, which may be a poor trade for research focused on changing bid activity.

Freshness has a budget too.

Cap the attempt at launch

Preflight catches an oversized configuration; a platform-side spending limit constrains what the attempt may charge while it runs. Use both, with a separate allowance for platform usage.

The REST start endpoint accepts maxTotalChargeUsd, so this helper places the event ceiling directly on the launch request.

function cappedStartRequest( input: z.infer<typeof ResearchInput>, eventMicroUsd: number, ) { const token = process.env.APIFY_TOKEN; if (!token) throw new Error('APIFY_TOKEN is required'); const url = new URL( 'https://api.apify.com/v2/acts/' + 'nimbryxo-studio~freelancer-com-project-budget-and-bids-scraper/runs', ); url.searchParams.set( 'maxTotalChargeUsd', String(eventMicroUsd / 1_000_000), ); return new Request(url, { method: 'POST', headers: { Authorization: `Bearer ${token}`, 'Content-Type': 'application/json', }, body: JSON.stringify(ResearchInput.parse(input)), }); }

This boundary doesn't guarantee that every requested record will arrive, and the event calculation still can't price unknown platform usage.

Keep the estimate, charge limit, and platform-usage allowance as separate configuration values. Their purposes overlap, but treating them as interchangeable makes the budget look more complete than it's.

Repeatedly restarting a capped attempt until it “works” creates an unbudgeted schedule with unusually poor manners.

Preserve partial rows and their unfinished state

An interrupted run can leave useful dataset rows behind. Their existence says nothing about whether ingestion finished, especially when a downstream report happily accepts whatever rows it receives.

The collector below checks terminal status before accepting completion. It retains rows with run provenance and deduplicates a current-state view by projectId, choosing the freshest available observation.

Nested bid data stays opaque because budgeting and recovery don't require reading individual bid properties.

const ProjectRow = z.object({ projectId: z.string().min(1), scrapedAt: z.string().datetime({ offset: true }).optional(), bids: z.array(z.unknown()).optional(), }).passthrough(); type Observation = z.infer<typeof ProjectRow> & { runId: string; runSucceeded: boolean; fetchedAt: string; }; function observationTime(observation: Observation) { return Date.parse(observation.scrapedAt ?? observation.fetchedAt); } async function collectResearch(previous: Observation[]) { const quote = eventCeiling(run_input, 'FREE', memoryAccounting); const request = cappedStartRequest(quote.input, quote.eventMicroUsd); const response = await fetch(request); if (!response.ok) throw new Error(`Start failed: ${response.status}`); const started = z.object({ data: z.object({ id: z.string().min(1) }), }).parse(await response.json()); const client = new ApifyClient({ token: process.env.APIFY_TOKEN }); const finished = await client.run(started.data.id).waitForFinish(); if (!finished || ![ 'SUCCEEDED', 'FAILED', 'TIMED-OUT', 'ABORTED', ].includes(finished.status)) { throw new Error(`Completion unresolved for run ${started.data.id}`); } const datasetResponse = await fetch( `https://api.apify.com/v2/datasets/${finished.defaultDatasetId}` + '/items?format=json&clean=true', { headers: { Authorization: request.headers.get('Authorization')! } }, ); if (!datasetResponse.ok) throw new Error('Dataset retrieval failed'); const projects = z.array(ProjectRow).parse(await datasetResponse.json()); const complete = finished.status === 'SUCCEEDED'; const fetchedAt = new Date().toISOString(); const incoming: Observation[] = projects.map(row => ({ ...row, runId: finished.id, runSucceeded: complete, fetchedAt, })); const recovered = new Map<string, Observation>(); for (const observation of [...previous, ...incoming]) { const existing = recovered.get(observation.projectId); if ( !existing || observationTime(observation) >= observationTime(existing) ) { recovered.set(observation.projectId, observation); } } return { runRecord: { runId: finished.id, status: finished.status, complete }, observations: [...recovered.values()], }; }

Your scheduler supplies previously stored observations and persists both returned values. Write the run record even when complete is false, so usable partial rows retain their incomplete-ingestion marker.

Keep that ledger independent of the current project table. Replacing an observation mustn't erase evidence that an earlier ingestion remained unfinished.

Production workers also need a durable checkpoint containing the started run ID before waiting. If retrieval or validation throws, leave that checkpoint unresolved and recover the existing run before purchasing another attempt.

The map serves a current-state view. A historical bid study needs separate observation history, because deduplicating every collection by project identity would erase the changes you're trying to measure.

Successful collection doesn't establish research coverage

Apply these boundaries when scheduling Freelancer.com Project Budget & Bids Scraper, particularly before proposal enrichment becomes a daily habit.

Run completion, ingestion completion, and research coverage need separate states. Combining them into a single green indicator makes an inexpensive collection look like a complete answer.

A restrictive filter can exhaust matches within maxPages, while reaching maxItems establishes only that the emission ceiling was reached. Neither row-count outcome proves that your research covered the market.

Proposal availability adds another boundary. Some projects hide their bid lists, some bidders leave proposal text blank, and sealed bids use a zero amount as a hidden-value placeholder.

Missing proposals therefore can't establish missing competition. Likewise, that placeholder can't support a claim about unusually cheap bids.

For your report, distinguish successful execution from successful retrieval, then assess population coverage and public proposal availability separately. A terminal status answers the execution question; your ingestion checkpoint and research method answer the rest.