How to Prepare Your Data for a Custom ML Model
Better data beats a better model almost every time. You can do most of this preparation before a single ML engineer is involved, and it will cut the project time and cost.
- Published
Define the target first
Write down exactly what the model should predict and how you would check it: "units of each SKU sold per store per week" or "whether this ticket is a billing issue". An ambiguous target produces an ambiguous model.
Collect and consolidate
- Pull history from every system into one place with consistent IDs and dates.
- Keep raw exports untouched; clean in copies.
- Note known gaps: outages, system migrations, periods with unusual events.
Clean and label
- Remove duplicates and test records.
- Standardise units, currencies and categories.
- Label examples consistently — write a labelling guide and have two people label a sample to check agreement.
- Keep a hold-out set aside that nobody tunes on.
Avoid leakage
Leakage is when training data includes information that would not be available at prediction time — like a "refund issued" flag when predicting refunds. It produces models that look brilliant in testing and fail in production. For every input, ask: would we know this at the moment we need the prediction?
Frequently asked questions
How much data is enough?
It depends on the task. We run a feasibility check on your data before quoting the full build.
Can we use synthetic data?
Sometimes to supplement, rarely to replace real data. Evaluation must use real data.
Is personal data a problem?
Minimise it, pseudonymise where possible, and make sure you have a lawful basis for use.
