Beyond
the Answer

Mathematics presents an unusually unforgiving test for a conversational machine. A fluent paragraph can conceal a lost minus sign. A cited formula can be genuine while the substitution beneath it is wrong. A perfectly correct solution can still be poor teaching if it removes every decision the learner was supposed to make.

That tension sits at the center of a new public experiment by Kanazawa Institute of Technology, commonly known as KIT. Its Research Center for Basic Education in Science and Engineering has released Mathko and Quanta, generative-AI chatbots designed to support the study of mathematics and physics. Users can type ordinary questions or construct fractions, integrals, matrices and roots with a TeX-based equation editor. The systems search KIT’s own learning pages, use retrieved material to produce an answer and display reference numbers that lead back to the source pages.

The service is available not only to current KIT students but also to prospective students, high-school pupils, adult learners and students elsewhere. That openness turns a campus tool into public educational infrastructure. It also makes questions about accuracy, instructional design and data handling more consequential than they would be in a small, supervised trial.

The source library came before the chatbot

The most important part of the project may be the part that predates modern generative AI. KIT Mathematics Navigation began in 2004 as a web resource spanning high-school mathematics and the material commonly encountered in the first two university years. Physics Navigation followed from 2015. The pages are meant to be consulted like a reference work: formulas, concepts and worked approaches are broken into discrete units, with links leading learners backward to prerequisites and onward to related ideas.

KIT describes the design as “one page, one item of knowledge.” Connections among those items form a knowledge graph. If a student reaches vector analysis but discovers that the underlying differentiation is uncertain, the links provide a route back and then forward again. This “link-back” pattern was built for human navigation long before it became a retrieval layer for a language model.

The university also had conversational ambitions before ChatGPT transformed the market. A 2021 engineering-education paper reported a rules-based chatbot that advised students on differentiation calculations and connected them to web materials. The current project therefore belongs to a longer institutional history: first organize the knowledge, then link it, then add a dialogue through which learners can find it.

What Mathko and Quanta actually do

Mathko is reached from the mathematics site and is intended to cover topics including calculus, linear algebra, differential equations, sequences and series. KIT’s examples include the chain rule, substitution integration, matrices, Taylor expansion and vector analysis. Quanta sits on the physics side, where the stated range includes mechanics, electromagnetism, waves and thermodynamics; the university names Newton’s laws, free fall, force vectors and the Biot–Savart law among possible subjects.

A “thinking mode” can be switched on for more detailed responses to difficult questions, at the cost of speed, or off for shorter, faster replies. This is the product label and the behavior KIT describes. It should not be interpreted as disclosure of the model’s private reasoning or as evidence that a particular reasoning-model architecture is deployed.

The usage instructions contain a more important qualification. When the system finds a relevant page, it uses KIT content. When it does not, it may answer from the AI’s general knowledge. The instructions warn that responses are reference information, may be inaccurate and require the user’s own final judgment. A source-linked answer is therefore a designed outcome, not an unconditional property of every response.

Retrieval narrows the problem; it does not solve it

The architecture described by KIT is retrieval-augmented generation, or RAG. A user’s query is used to retrieve relevant material; the question and that material are then supplied to a large language model, which composes the response. Compared with asking a general-purpose model to rely on whatever it absorbed during training, retrieval can constrain the subject matter, bring in institution-selected explanations and make checking easier. KIT’s knowledge graph can help reveal conceptual relationships beyond simple keyword matching.

But RAG creates a chain of dependencies. Retrieval can miss the necessary page or select an adjacent but inapplicable one. A model can overlook a condition in correct source material, merge two incompatible explanations, mishandle notation or introduce an algebraic mistake while writing the final answer. If no relevant page is found and general model knowledge takes over, the institutional grounding becomes thinner still.

The prototype’s five-model test

A 2025 paper by Professor Keita Nishioka and colleagues offers a view of the development path. Titled “Creating a Chatbot that Collaborates with Math and Physics Websites—Validation of Open Source LLMs and Building RAG in a Local Environment,” it described a prototype assembled with Ollama and Open WebUI on a computer carrying an NVIDIA GeForce RTX 3080 Ti with 12GB of graphics memory. That constrained the researchers to models of roughly eight billion parameters or fewer.

They compared Llama 3.2, Phi-4-mini, Gemma 3, DeepSeek-R1 and Qwen 3 on problems pitched around the level of KIT mathematics and physics examinations. In that limited exercise, Gemma 3 and Qwen 3 were the strongest of the five; Gemma 3 was notable for doing well at a smaller size. The paper also recorded instances in which DeepSeek-R1 answered a Japanese question in Chinese. The authors treated quantitative correctness evaluation as work still to be done.

Those findings are useful history, not a benchmark for the service released in 2026. The test set was not presented as a comprehensive standardized evaluation, and KIT’s launch announcement does not identify the production foundation model, embedding model or inference server. Nor does it confirm that the public deployment retains the prototype’s entirely local arrangement. Local open-source inference mattered to the earlier work because it offered avenues for cost control, privacy and customization; the released system’s exact stack remains undisclosed in the official material reviewed for this article.

Correctness is necessary; tutoring is more

An accurate final answer is only the floor for an educational system. A tutor must decide whether to reveal a solution, offer one hint, ask the learner to state an assumption or diagnose the misconception behind a wrong step. The best intervention depends on what the student already understands and what the assignment is supposed to assess.

Consider an integration exercise. Producing a flawless derivation may help a student check completed work, yet deprive another student of the central learning task: recognizing which substitution to choose and why. A more productive exchange might direct the learner to a prerequisite page, ask for the next transformation and withhold the completed expression until after an attempt. The public descriptions do not establish whether Mathko or Quanta systematically diagnoses misconceptions, stages hints or adjusts feedback to a demonstrated level of mastery.

KIT identifies learning-history- and learning-status-based support as a future goal. That wording matters. Personalization is not a verified current feature. If introduced, it would require more than remembering prior questions: the institution would need to explain what it records, how it estimates competence, how an incorrect estimate can be challenged and whether teachers can see or correct the resulting learner profile.

The distinction can shape evaluation. A system may score well on final-answer accuracy while encouraging dependency. Conversely, a chatbot that sometimes refuses to provide a full solution and instead elicits reasoning may feel less efficient while producing better learning. Educational value must therefore be tested with transfer problems, unaided later performance and the quality of student reasoning—not merely satisfaction or the number of chats completed.

A public tool creates a data duty

KIT’s usage page says that questions and answers are logged. It also says those logs are not used as training data for the AI model, and instructs users not to enter personal information or anything they would not want a third party to know. These are meaningful disclosures. “Not used for model training,” however, answers only one question about a recorded conversation.

The public instructions reviewed by Japan.co.jp do not specify a retention period, who is authorized to inspect the logs, a deletion process, the provider or location of the deployed service, or whether aggregated or de-identified records might be used in the planned educational-effect research. These are unresolved disclosure questions, not evidence of misuse. They become especially important when a service explicitly welcomes high-school students and other members of the public.

Data practice can itself become part of the curriculum. Before students begin, an instructor can explain why names, student numbers, private assessment material and unpublished research should stay out of prompts; what a conversation log is; and why model training is only one possible use of data. Japan’s Ministry of Education, Culture, Sports, Science and Technology published Version 2.0 of its generative-AI guidance for primary and secondary education in December 2024. It does not directly govern a university service, but its emphasis on age- and context-sensitive use, current knowledge and information handling is pertinent to a tool open to high-school learners.

Classroom rules cannot be delegated to the bot

Once the system is public, use will not remain inside a supervised classroom. A workable policy needs more precision than either a blanket ban or unrestricted permission. Can students use it to revisit definitions, request another example or audit a completed derivation? Can they use it to generate the final work submitted for credit? Must they preserve the prompt, answer and source pages? The boundary depends on the learning objective, and the instructor—not the chatbot—must draw it.

One constructive approach is to turn the output into an object of verification. Students can check each transformation, track dimensions and units, test a result against limiting cases, and compare the stated assumptions with those in the linked KIT page. If the chatbot fails, they can identify the first invalid step and explain why the answer still sounded plausible. Such work uses the system to strengthen judgment instead of outsourcing it.

Accessibility also needs practical testing. An equation editor may make sophisticated notation possible, but the service should be assessed with keyboards, screen readers and mobile devices, and with learners who are less fluent in formal syntax. A public URL is not by itself proof of equal access.

The decisive evidence has not yet been produced

The development is part of a Japan Society for the Promotion of Science Grants-in-Aid for Scientific Research (KAKENHI) Basic Research (C) project, number 25K06650. Its Japanese title describes the development of a mathematics and science education-support chatbot using generative AI and its introduction into an e-learning system. Nishioka, a professor in KIT’s Mathematical Sciences, Data Science and AI curriculum, is the principal investigator.

KIT says the next work includes tighter integration with KIT STEM Navigation, personalization based on learning history and status, higher answer accuracy, additional functions and verification of educational effects. That agenda appropriately treats the release as the start of research rather than its conclusion. No published learning-gain result accompanied the September 7 announcement.

The necessary evidence is multidimensional. Accuracy should be reported by subject, level and problem type, with separate measures for retrieval relevance and mathematical correctness. Educational studies should ask whether learners can later solve related problems without the bot, whether misconceptions persist, whether time on task changes and whether effects differ between novices and stronger students. Reliability needs observation over time as models and content change. Qualitative research should examine whether students understand the reference links or treat them as decorative proof.

Japan.co.jp’s assessment is that KIT’s durable advantage is not a fashionable model name. It is the collection of concise, linked educational pages built since 2004 and the institutional knowledge represented by their graph. The system’s credibility will depend on resisting the temptation to treat grounding as a guarantee, making data practices more explicit, and judging success through student reasoning rather than fluent output.

Mathko and Quanta can help a learner reach an answer and its source. The more ambitious question is whether they can help that learner leave with a method, an informed doubt and the ability to proceed alone. The public release provides the instrument. Careful measurement must now determine whether it provides the education.