AI News Automation — Ingestion-to-Email Pipeline
A daily source-to-email pipeline turns AI articles and video transcripts into source-linked digests. Per-stage results distinguish collection, processing, generation, and delivery failures across a repeatable run.
Problem & context
Collect articles and transcripts, persist source identities, generate digest entries, and compose a source-linked email with stage-level run results.
My contribution
source collection and repository methods, digest orchestration, and plaintext/HTML email composition and delivery integration.
Generated locally without a model request or SMTP delivery. Source URLs identify the synthetic sample’s references.
Inspect the workflow.
The product output is a multipart email digest. Inspect the formatter and run instructions in the repository.
Failure path: No recent digests are available, article ranking returns no result, or email configuration is missing.
Generation/configuration raises ValueError and send_digest_email returns a success:false error result. Other exceptions reach the daily runner's outer error handler; a successful formatter sample does not prove email delivery.
An engineering decision.
stage results preserve where processing stopped and distinguish digest failures from delivery failures. They aid inspection but do not provide durable retries or exactly-once execution.
Implemented approach
Separate collection, content processing, digest generation, and email delivery, with a results dictionary and per-stage counts.
Alternative & tradeoff
Treat the entire daily job as one opaque success/failure operation.
Follow the system.
Verified behavior & limitations.
What the evidence establishes
- Executed locally: the actual digest_to_html formatter and EmailDigestResponse.to_markdown method rendered two synthetic entries and retained their source links.
- Source review: repository methods check stored GUIDs or video IDs before creating source records.
- Source review: the daily runner records scraping, processing, digest, email, duration, and overall success fields.
- Source review: digest processing counts failed or absent model results; email setup validates recipient and sender configuration before SMTP use.
Boundaries & remaining work
- The sample text is explicitly synthetic. No scraper, model, ranking, database, scheduler, or SMTP send was executed to produce it.
- Identity lookup before insert is a source-level duplicate check, not proof of exactly-once ingestion under concurrency.
- The source snapshot's model integration and the deterministic email formatter are separate verification boundaries; this sample proves formatting only.
Inspect the source.
References are pinned to the reviewed revision. Public sample screens demonstrate the interface with fixtures; source review and executed checks are labelled separately.