Nutrition

How we built the photo macro scanner (and what it still gets wrong)

Computer vision, 12,000 dishes, and a humble lesson.

Our photo scanner reads a plate of food and estimates calories, protein, carbs, and fat. It works well for most home-cooked meals. It's also wrong in interesting ways we want to be honest about.

Under the hood: a single Gemini vision call, prompted to identify visible ingredients with rough portion estimates, then a deterministic macro lookup against a 12,000-row nutrition table.

What works: clearly plated single dishes — a bowl, a plate, a sandwich. Familiar staples like rice, chicken, salmon, broccoli, eggs. Good lighting.

What doesn't: stews and soups (we can't see what's underneath), sauces with hidden fat (caesar dressing reads as romaine), and rare ingredients the model has never seen.

The lesson we keep relearning: the model is a very good guesser. It's not a measurement tool. We surface the uncertainty in the UI now — you'll see a 'low confidence' chip when the model isn't sure.

Keep reading