Messy text in, tidy data out | Master AI Automation in 4 hours Master AI Automation in 4 hours Course About Ayush Modules Sample chapter Toolbox The Microcap Minute Classroom / Module 08: Documents, Data & Research / Chapter 3 Messy text in, tidy data out Watch first, then read. Same lesson, your pace. What you will learn – The extraction prompt pattern (schema + rules + examples) – Handling missing fields honestly – Scaling from one document to a hundred The pattern Half of real-world data work is un-gluing information from prose: prices inside listings, dates inside emails, marks inside report cards. AI does this superbly, if you demand structure. The reusable recipe: Extract data from the text below into JSON with EXACTLY this schema: { “item”: string, “price_inr”: number | null, “date”: “YYYY-MM-DD” | null, “seller”: string | null } Rules: 1. Use null when a field is absent, never guess. 2. Normalise all dates to YYYY-MM-DD. 3. Output ONLY the JSON array, no commentary. Example input/output: <<>> TEXT TO PROCESS: <<>> Three design choices make it robust: an explicit schema (the contract), null-for-missing (Module 1’s anti-hallucination permission again), and output-only (no chatty wrapper breaking automation). One worked example (few-shot, Module 2) resolves most ambiguity about edge cases. Honest holes The null rule deserves its own paragraph. Real documents lack fields; weak prompts invite the model to invent plausible ones (“price not stated… let’s say ₹999”). Nulls also make downstream filtering trivial, every spreadsheet and script understands empty. Missing-and-marked beats present-but-false, in data as in life. Scaling up One document: any chat works. A hundred: Batch via API (Modules 4-6): loop over files, same extraction prompt each, off-peak hours, free models first Chunk long files , extract per page/section if outputs start truncating Validate mechanically , load results; flag rows where price is null or date failed parsing; re-run only failures Sample-check , manually verify 5 random rows against sources before trusting the pile That last step is Module 7’s auditor persona applied to data. Try it yourself Paste three different real classifieds/emails (or fabricate messy samples) into one extraction run using the template above. Load the output at jsonlint.com. Then scale-test: ask for CSV instead (“same data, comma-separated”) and import into Sheets. Finally break it on purpose: remove the null rule, rerun on your sparsest sample, and watch invented values appear. Log before/after in learn/extraction-notes.md . Key takeaways – Extraction = explicit schema + null-permitted + output-only + one example. – Marking missing as null prevents invention and simplifies downstream use. – Scale via API loops, chunking, validation flags and random sampling. Download the exercise sheet (PDF) Module workbook (PDF) ← Prev: Spreadsheets on easy mode Next: Research like a pro → Classroom / Module 08: Documents, Data & Research / Chapter 3 Messy text in, tidy data out What you will learn – The extraction prompt pattern (schema + rules + examples) – Handling missing fields honestly – Scaling from one document to a hundred The pattern Half of real-world data work is un-gluing information from prose: prices inside listings, dates inside emails, marks inside report cards. AI does this superbly, if you demand structure. The reusable recipe: Extract data from the text below into JSON with EXACTLY this schema: { “item”: string, “price_inr”: number | null, “date”: “YYYY-MM-DD” | null, “seller”: string | null } Rules: 1. Use null when a field is absent, never guess. 2. Normalise all dates to YYYY-MM-DD. 3. Output ONLY the JSON array, no commentary. Example input/output: <<>> TEXT TO PROCESS: <<>> Three design choices make it robust: an explicit schema (the contract), null-for-missing (Module 1’s anti-hallucination permission again), and output-only (no chatty wrapp
Messy text in, tidy data out
Written by
in