Data Engineering · Case Study
An 8-phase Python pipeline that transformed 13k+ raw USDA food records into 1,900+ production-ready ingredients — organized into a 3-tier category hierarchy and tagged for dietary restrictions at 95%+ accuracy.
13k+
Raw USDA records in
1,900+
Production ingredients out
95%+
Diet-tag accuracy
17
Top-level categories
29
Mid-level groups
10+
Diet patterns tagged
USDA's food datasets — FoodData Central and the legacy FNDDS — contain over 13,000 records. They include raw whole foods, processed variants, industrial compounds, laboratory standards, and dozens of near-duplicate entries at varying levels of specificity.
The startup needed a clean, categorized ingredient database for recipe matching, nutrition display, and dietary restriction filtering. No off-the-shelf dataset matched the shape we needed, and manual curation at this scale would have taken weeks and introduced inconsistencies.
The pipeline needed to be reproducible, auditable, and fast enough to re-run as the USDA data or category schema evolved.
e.g. Proteins · Grains · Produce · Dairy · Fats & Oils
e.g. Leafy Greens · Root Vegetables · Citrus · Tree Fruits
e.g. Spinach · Carrot · Lemon · Granny Smith Apple
The hierarchy was designed for the product's filtering UI: Level 1 powers broad dietary filters ("Show me Proteins"), Level 2 drives subcategory browsing, and Level 3 is what appears in recipe ingredient lists and search.
Before
After
Demo video coming soon
A walkthrough of the pipeline and the product it powers is being prepared.