A Smartphone Photo Can Check Japanese Food Labels for Allergens

From one stained note to a life-safety label

Before the morning batch, a small confectioner's ingredient note lies on the worktable, marked by oil and hurried corrections: flour, butter, egg, almonds. A new supplier's chocolate contains an emulsifier. A contracted cookie base may carry soy. A seasonal product borrows yesterday's label, and the person responsible for checking it is also producing, shipping and keeping the books. Shirushi-san promises a shorter path: point a phone at the note, turn it into structured ingredients, flag allergen candidates, and draft the Japanese label.

The phrase “check allergens with a photo” merges two different jobs. One feature reads handwritten or printed recipe notes and turns them into product data. Another photographs an actual finished label and compares what is printed with the registered composition to look for omissions. This is not primarily a consumer app that points at a supermarket package and chemically discovers what is inside. It is a manufacturer's system for preparing and checking declarations.

IT Farbridge was established in January 2026 and announced the formal launch at 9:10 a.m. on July 21. The browser service works on phones, tablets and computers. A free tier covers up to ten labels and ten specification sheets. The all-in-one plan is ¥14,800 a month or ¥148,000 a year before tax; two narrower plans are ¥9,800 a month. Low setup cost and no installation are aimed squarely at makers that cannot maintain a full-time regulatory department.

“AI reads; rules decide”

The published workflow has four layers. A multimodal model first extracts ingredient names from one or several overlapping photographs and removes duplicates. The user can correct the transcription. A fixed dictionary and rule engine then compares the confirmed text with allergen and additive masters. The system assembles Japan's standardized label block, previews it at physical size and exports PDF, PNG, CSV or printer-language files. Product specifications, supplier requests, version history, legal changes and recall records can then be tied to the same item.

Restricting generative AI to input assistance and proposals is a meaningful safety choice. Image recognition is probabilistic. Even a seemingly high per-item success rate compounds across many critical strings, and a beautifully formatted output can conceal a one-character error. A deterministic engine returns the same result for the same confirmed input and master. Its warning can be traced to a named rule rather than to the opaque phrasing of a language model.

Yet deterministic does not mean correct. If the AI reads komugiko, wheat flour, as komeko, rice flour, a perfect rule engine will process the wrong ingredient perfectly. If pistachio is absent from a master, it will be missed consistently. If a supplier changed a margarine formula while the customer retained an old specification, repeatability simply produces a repeatable error. The architecture moves the hardest question from “Can AI reason about law?” to “Are the input, dictionary, version and approval current?” That is progress, but not proof.

The hardest ingredients are the ones absent from the note

Food labeling is not keyword circling. The Consumer Affairs Agency's April 2026 handbook demonstrates the work with an ordinary croquette. Breadcrumbs, mirin, soy sauce, textured vegetable protein and margarine must be opened into first- and second-level components using supplier specification sheets. Wheat gluten used to form textured protein may not be visible from the ingredient's retail name. An emulsifier can be soy-derived. Processing aids and carry-over additives may still require an allergen declaration even when the additive itself would otherwise be omitted.

Japanese rules also distinguish prescribed alternative and extended names. “Peanut” can stand for groundnut, and “wheat flour” identifies wheat; the nyu character in “emulsifier,” “lactic acid” or “lactic-acid bacteria” does not mean milk. Edamame and black soybeans fall within soy, while azuki does not. A fish called “trout” is not automatically within the defined salmon scope. Egg white and yolk contain the character for egg, but they still require an explicit egg declaration. A literal search is not enough.

The durable value of software therefore lies less in photogenic OCR than in provenance. Which supplier document and version supported the conclusion? How far was a compound ingredient decomposed? Which products inherit a changed raw material? Who reviewed the change, and when? Which regulation version governed release? Many labeling failures do not begin with bad arithmetic. They begin where an old relationship between documents was never updated.

Japan started with five mandatory items in 2001

Japan's allergen-labeling system predates this AI wave by a quarter-century. The former Health Ministry conducted national studies from fiscal 1996 through 1999 to identify foods associated with immediate and severe reactions. In April 2001, rules under the Food Sanitation Act made five allergens mandatory—egg, milk, wheat, buckwheat and peanut—and recommended 19 more. The original scope was 24 items, built from Japanese case frequency and severity rather than copied wholesale from another jurisdiction.

The list moved with the diet. Banana entered the recommended category in fiscal 2004. Shrimp and crab became mandatory in fiscal 2008. Cashew and sesame were added to the recommended group in fiscal 2013, and almond followed in 2019. Walnut moved to the mandatory category in 2023, with its transition ending in March 2025. In 2024, macadamia was added to the recommended group and matsutake removed.

The legal frame changed as well. The Food Labeling Act, enacted in 2013 and effective in April 2015, unified labeling provisions previously split across the Food Sanitation Act, JAS Act and Health Promotion Act. Since June 2021, businesses conducting a voluntary recall for a safety-related labeling violation—such as a missing allergen or bad use-by date—must notify authorities, and the national government publishes the recall. A label is no longer merely the last piece of package copy. It is a regulated safety record.

The April 2026 change to 29

Only 111 days before Shirushi-san's launch, the system changed again. On April 1, the Consumer Affairs Agency added cashew to the statutory category, bringing mandatory allergens to nine. There is a two-year transition, but the agency asks companies to relabel as soon as practicable. The same notification added pistachio to the recommended group because national surveillance continued to record a substantial number of cases. It also urged prompt disclosure in light of cross-reactivity between pistachio and cashew.

Counts across countries are not directly interchangeable. Japan has nine statutory items and 20 recommended items, revisited through national case surveillance roughly every three years. The United States named eight major food groups through the 2004 FALCPA and made sesame the ninth effective in 2023. The European Union requires clear indication of 14 groups under Regulation 1169/2011, including typographic emphasis in ingredient lists. An exporting SME cannot maintain one universal allergen field; the covered groups, terminology and presentation must be versioned by destination.

The launch-page conflict: 28 versus 29

The most consequential public audit finding is simple. Shirushi-san's product page repeatedly describes an “allergen 28-item” master paired with 12 additive uses. The Consumer Affairs Agency's current handbook explicitly says 29 and even recommends package wording such as “allergens: 29 items covered.” The launch release recalls walnut's transition in 2025 but omits the cashew and pistachio changes that took effect three months before launch.

Public copy cannot tell us whether pistachio is absent from the live engine or whether the marketing page is stale. We therefore do not claim that the software fails to detect pistachio. But a mismatch in the advertised count is material for a product whose central value proposition is automatic legal updating. Before contracting, a buyer should require the current master list, effective date, change log and a live test containing pistachio, cashew and walnut.

The release also left an editorial placeholder—literally a note requesting a Small and Medium Enterprise Agency or Economic Census source—after asserting that food manufacturing is dominated by smaller firms. The proposition may be directionally sound; the unfinished citation is still revealing. A company selling evidence and version control should apply the same review discipline to its own public material. Safety maturity is visible not only in architecture but in copy, master data and change records.

Why an omitted line is an accident, not a clerical error

Food allergy can cause hives or swelling, but it can also progress to respiratory distress, falling blood pressure, loss of consciousness and anaphylactic shock. Japan does not have a precise national patient count; the agency's handbook cites estimates of 5–10% among infants and 1–3% among school-age children. The trigger and threshold differ by person, and even one person's reaction can vary with physical condition. The package line is a decision tool for whether to eat.

Japan's fiscal 2024 national surveillance collected 6,033 physician-reported immediate-reaction cases. Of those, 2,052 involved accidental ingestion; labeling errors were linked to harm in 6.4% of that accidental-ingestion subset. The denominator matters, but so does the trend: the report found no large reduction in accidental ingestion caused by labeling mistakes compared with earlier surveys.

Recall data show the operational side. From the June 2021 start of mandatory notification through March 2024, authorities published 4,841 Food Labeling Act recalls. Among 4,181 completed cases tabulated by reason, allergens accounted for 2,429, or 58.1%, the largest category. Causes included data-entry omissions, printing failures, wrong-label application, missing labels and use of the wrong ingredient. Correctly analyzing a formula cannot prevent a worker from applying another product's artwork. A strong digital system must close the loop at packaging, lot and release—not stop at the recipe.

What a photograph cannot see: cross-contact

A recipe photograph records ingredients added intentionally. Allergens also enter unintentionally. Plain chocolate can follow peanut chocolate on a shared line. Wheat dust can travel across a weighing room. Shrimp and crab may share fishing, transport or processing equipment. Cleaning reduces residue but may not eliminate it. This is cross-contact, described in Japanese guidance as unintended mixing.

The official handbook prioritizes prevention: clean thoroughly, schedule allergen-free products before products containing allergens, use dedicated tools where possible, and control zones and air pressure for powders. When diligent controls cannot exclude mixing, a specific warning such as “Products containing peanut are also manufactured on this line” may be appropriate. A vague “may contain” or “may be present” statement is not permitted merely as a catch-all. Excessive warnings unnecessarily narrow food choices for allergic people.

This is the boundary of photo OCR. A note cannot reveal the product that ran immediately before, the validated cleaning result, airborne dust or a supplier's shared transport. The company says its service distinguishes contamination notices, but the software can do so only if people continuously maintain facility, equipment, sequence, cleaning and logistics data. An empty field is not evidence of an allergen-free line.

A camera is not an ELISA test

A smartphone sees characters on paper or packaging. It does not measure allergenic protein in the food. Regulatory investigation can use two screening methods with different characteristics and confirmatory testing to evaluate specified allergens. Immunoassays such as ELISA—and, for suitable targets, DNA-based methods—produce a different kind of evidence from OCR.

Laboratory testing is not a complete replacement for documentation either. Processing can alter proteins and extraction efficiency, so a measured result may not perfectly equal the true quantity. A test examines a sample at a point in time. A compliant label requires the full recipe, supplier specifications, production records, order by weight, additive functions and legally prescribed names. Documentary control and physical testing cover different blind spots.

The responsible claim is therefore not “one photo proves safety.” It is that the service may accelerate input, increase the number of discrepancies surfaced and consolidate an audit trail. A low-complexity product on a dedicated line may need primarily documentary review. A shared line, imported compound ingredient, previous incident or unusual process may justify a site audit, expert review or analytical test.

Putting a human last is not enough

IT Farbridge says an AI-read ingredient remains marked “unconfirmed,” and any unconfirmed item blocks mass production and release until a person approves it. That is a valuable control. Simply displaying AI output invites automation bias: a hurried employee accepts what looks polished. Persistent status, named approval, timestamps, before-and-after changes and a hard gate make review visible.

Approval can still become ceremonial. If one person writes the note, takes the photo, corrects the transcript and approves the label, there is no independent second set of eyes. Higher-risk products should separate preparer and approver and compare supplier specifications, production records and the physical pack as distinct evidence. Low-confidence OCR, manually corrected fields, compound materials and recently changed regulations deserve escalated review.

The contract draws a clear boundary. The service does not guarantee legality, accuracy or completeness; the customer makes the final decision. When liability applies, the general cap is the fees paid over the previous 12 months. The standard 99.9% uptime target describes service availability, not 99.9% decision accuracy. Those terms are not unusual for low-cost software, but they mean a maker is buying assistance—not transferring recall or health-harm responsibility to the vendor.

A recipe is a trade secret

For a small maker, proportions, suppliers, costs and process are intellectual property. Before sending photographs to an external AI service, a buyer should know which model provider processes them, in which countries, for how long, whether they are used for training and whether provider staff can access them. The public privacy policy says foreign cloud, AI and analytics providers may handle data abroad, but it does not name the AI processor, state a retention period for recipe images or clearly resolve training use.

Published controls include two-factor authentication, passkeys, tenant separation, encryption in transit and encrypted daily backups. SSO, detailed audit logs, dedicated environments, on-premises operation and data-location requirements are quoted separately for enterprise users. These are useful starting points, not substitutes for independent certification, penetration-test evidence, key management, deletion confirmation, backup-retention limits and a current subprocessor list.

Accuracy and confidentiality belong in the same purchase review. Sending every high-resolution supplier sheet may improve extraction while enlarging the trade-secret footprint. Redacting too much may hide the compound ingredients that matter. Data minimization, role-based access, retention, export and deletion should be designed around the product lifecycle rather than left to a generic cloud default.

A launch is the beginning of proof

Supervisor Makoto Tajima brings substantial domain history. He worked at the Agriculture Ministry's Food Research Institute, became a professor and president at Jissen Women's University, led food-science societies and Consumer Commission food-labeling panels, and headed JAXA's space-food certification team. That experience can strengthen dictionaries, explanatory content and regulatory review.

Expert supervision is not a product benchmark. As of July 21, public material provided no handwriting character-error rate, ingredient-level recall, allergen-warning sensitivity or specificity, blind comparison, independent audit, named deployed customer or measured reduction in mistakes. The company is recruiting three “first partners,” offering six months free and a permanent 50% discount afterward in exchange for permission to publish the customer name and outcome data. The proposition is still building its evidence base.

The advertised surface is also exceptionally broad: labels, specifications, nutrition estimation, halal and vegan self-checks, medical-diet support, food-poisoning prevention, legal monitoring, supplier portals and management insight. Breadth can be attractive to a small company that wants one tool. It can also diffuse the validation resources of a six-month-old vendor. The buyer's question should be narrower: is the exact workflow we will use tested, maintained, auditable and designed to fail closed?

Fifteen tests before adoption

The best pilot is not a clean note selected by the vendor. Mix legacy forms, bad handwriting, supplier PDFs, correct labels and labels with one deliberate omission; do not disclose the answer key. Average speed matters less than the dangerous false negative—the system saying “none” when an allergen is present. A false alarm costs review time. A miss can reach a patient.

Small makers need a stop line, not magic

There is real room to improve a labeling process still divided among paper, spreadsheets and memory. Photo input can be a practical bridge for companies that cannot begin with a pristine database. Dictionary checks can serve as a second set of eyes. Linking specifications to labels can shorten the search for products affected by a legal or supplier change. Blocking release until a person confirms the transcription is closer to safety engineering than ordinary text generation.

The public evidence does not support a victory declaration. The item-count conflict, absence of performance metrics, lack of mature customer results, very broad product scope and external handling of recipe data all require buyer scrutiny. Since the company's own terms say legality, accuracy and completeness are not guaranteed, the launch headline's “without mistakes” language should be read as an aspiration, not a warranty.

A good labeling system does not remove responsibility from a maker. It makes responsibility executable: gather the current specification, expose a dangerous change, stop one person's assumption with independent evidence, record approval, and refuse shipment when the facts are incomplete. A smartphone photograph does not create safety. The control system connecting that photograph to supplier data, the production line, testing, current law and a second human decision does.