Nilgün Şengöz

As AI systems become more prevalent in medical diagnosis, the need for transparency and interpretability grows exponentially. In my research, I focus on Explainable AI (XAI) methods that help clinicians understand how algorithms reach their decisions.

Gradient-weighted Class Activation Mapping (Grad-CAM) is one of the key techniques I use to visualize which parts of a histopathological image the model focuses on when making a diagnosis. This visual feedback is crucial for building trust between AI systems and healthcare professionals.

The challenge is not just building accurate models. Doctors must understand them, trust, and ultimately use to improve patient outcomes. This is where XAI bridges the gap between algorithmic power and clinical practice.

In our recent study on paratuberculosis diagnosis, we demonstrated that Grad-CAM heatmaps closely aligned with the regions pathologists identified as diagnostically relevant, validating the model's reasoning process.

Research context: This work was conducted at the University of Nottingham under a TÜBİTAK 2219 international postdoctoral fellowship, in collaboration with the Computational Optimisation and Learning (COL) Lab, under the supervision of Prof. Dr. Ender Özcan (University of Nottingham), with contributions from co-researcher Dr. Jeremie Clos. Presented at PATAT 2026, the 15th Conference on the Practice and Theory of Automated Timetabling, University of Nottingham.

The Question

Large language models are routinely described as text tools: prose generators, code writers, conversational agents. That framing undersells what these systems are at a computational level. During my postdoctoral research, I have been exploring a different question: what happens when you treat an LLM not as an output device, but as a reasoning engine embedded inside a classical optimisation algorithm?

This question led to LLM-STAR, a framework developed in the context of examination timetabling, a classic NP-hard combinatorial scheduling problem. The work was presented at PATAT 2026 under the title "Large Language Model-Based Explainable Hyper-heuristics for Examination Timetabling".

Why Timetabling?

University timetabling requires assigning courses to rooms and time slots such that no student or lecturer has two commitments simultaneously, room capacities are respected, and dozens of soft preferences are satisfied as fully as possible. At scale, meaning a university with hundreds of courses and thousands of students, this becomes computationally intractable for exact methods. Classical approaches search a vast solution space using mathematically defined moves. They evaluate, but they do not reason.

The Architecture

The LLM-STAR loop has five steps: State Encoding → LLM Reasoning → Candidate Repairs → Evaluator Scoring → State Update. The critical step is "Candidate Repairs". Where a classical metaheuristic generates neighbour solutions by random perturbation, the LLM proposes semantically motivated repairs: reasoning about patterns such as: two courses sharing the majority of enrolled students create a structural conflict when scheduled in parallel, beyond what the hard constraint captures.

This kind of soft-constraint reasoning is difficult to formalise explicitly but straightforward to elicit from a language model. Steps 1 and 4 are classical (deterministic). Steps 2 and 3 are where the LLM contributes semantic reasoning.

How the Framework Is Structured

The framework is built as a hyper-heuristic: rather than solving the problem directly, the language model selects and adapts among low-level heuristics during the search. Detailed experimental results, benchmark comparisons and ablation analyses are reported in the PATAT 2026 paper. The full slide deck is available for download in the Talks section.

The explainability dimension is central to the design rather than incidental. Because the language model's heuristic selection at each step can be logged and inspected, the search produces a natural audit trail. In regulated environments, this auditability carries operational value alongside solution quality.

Broader Applicability

The finding is that LLMs can serve as first-class algorithmic components in combinatorial search. The class of problems to which this applies is large: airline crew assignment, hospital operating room scheduling, military mission planning, port berth allocation, urban traffic signal optimisation. Each is a variant of the same structure: resources assigned to time slots under hard and soft constraints, at a scale that defeats exact methods.

Note for researchers: Full methodological details will accompany the formal publication. Academic correspondence on LLM-guided search, hybrid optimisation, or combinatorial scheduling is welcome.

Research context: This work has been published in El-Cezeri Journal of Science and Engineering (ECJSE), Vol. 13, No. 2, pp. 146–160, 2026. Co-authors: G. Karaman, M. S. Çeliker, N. Y. Çan. DOI: 10.31202/ecjse.1747013

Why Standard Detectors Fail

Camouflage is one of the oldest and most persistent challenges in visual perception. Its effectiveness relies on a straightforward principle: match target appearance to background statistics so thoroughly that the human visual system finds no foothold. Standard object detection architectures are optimised for their dominant training data: objects that are visually distinct from backgrounds, with clear edges, consistent texture, and reasonable contrast. Military camouflage specifically engineers against all three of these properties.

The result: naive application of standard models produces detection rates near chance level on well-camouflaged targets. The problem is not model capacity. It is misaligned inductive bias.

The Attention-Based Approach

Our published work integrates attention mechanisms into established segmentation and classification backbones, specifically Attention U-Net and ResNet-50, to re-frame the detection task. Rather than asking whether there is an object matching a given class, the attention-augmented model is guided toward regions whose properties are inconsistent with the surrounding background.

This re-framing is the core insight. A camouflaged target cannot match the background perfectly; there will always be statistical residuals: micro-scale shadow inconsistencies, subtle texture regularity, edge artefacts at boundary regions.

Attention gates allow the network to suppress irrelevant background activation and concentrate representational capacity on regions where camouflage artefacts are strongest. Full architectural details, dataset description and quantitative results are reported in the published paper.

Explainability

Given my broader research programme in Explainable AI, I paid particular attention to whether the model's attention maps were interpretable by domain experts. Grad-CAM visualisations showed consistent focus on boundary regions and texture discontinuities that human experts also identified as informative. This alignment between model attention and expert judgement is a necessary, though not sufficient, condition for operational trust.

Transferability

The attention-based approach is not domain-specific. The model learns to find statistical inconsistencies in visual data. The same problem appears in medical imaging (subtle lesions in complex tissue), industrial inspection (surface defects that blend with material texture), remote sensing (concealed installations), and infrastructure security (foreign object detection).

The same attention-based approach applies to each of these problems with adaptation. The domain changes what "anomaly" means. The algorithm for finding anomalies remains the same.

Looking back across my published work, I notice a pattern I did not plan but can now articulate clearly. My publications span histopathological image classification, military camouflaged object detection, LLM-based combinatorial optimisation, hybrid neural architectures, and agricultural image analysis. On the surface, these seem unrelated. On closer inspection, they share a common structure: each involved designing an algorithm to find structure in data that is not obvious, then explaining why that structure exists.

The Distinction That Matters

Consider two researchers. The first is a "medical imaging AI specialist." Invaluable in that context, but when the clinical application changes, their algorithmic toolkit must adapt to new domain knowledge. The second researcher asks first: what is the structure of this problem at the level of the algorithm? They recognise that detecting a tumour in histopathological tissue and detecting a concealed object in a complex visual background are, algorithmically, the same problem, and apply the same attention-based architecture to both.

The domain changes what "anomaly" means. The algorithm for finding anomalies transfers.

Evidence from My Own Research

CLAHE contrast enhancement, developed for veterinary histopathology, transfers directly to military vision, because both involve low-contrast targets in complex backgrounds. XGBoost hybrid ensembles, developed for rotten fruit detection, apply to medical image classification, because both combine structural and appearance features under class imbalance. Grad-CAM attention visualisation, developed for clinical XAI, transfers to defense AI, because both require operator-interpretable evidence maps for high-stakes decisions.

Each transfer emerged organically, not from planning, but from recognising that two problems with different application contexts were structurally identical at the algorithmic level.

What Algorithm Architecture Means in Practice

Problem abstraction before domain engagement. Before reading the domain literature, ask: what is the formal structure of this problem? Classification with class imbalance? Detection with low signal-to-noise? Combinatorial problem with hard and soft constraints? The formal structure determines which algorithmic family is relevant. The domain determines the data characteristics and validation criteria.

Method transfer as a first hypothesis. When facing a new problem, the first hypothesis is always: has this algorithmic structure been solved elsewhere? If so, does the solution transfer, and if not, where does it break?

Generalisability as a research contribution. A paper reporting "97% accuracy on dataset X in disease Y" has made a domain contribution. A paper showing "architecture Z outperforms baseline B on low-salience detection across three domains" has made an algorithmic contribution. The second type has a longer citation half-life.

A clarification: this is not an argument against domain expertise. It is an argument that the most productive AI researchers tend to be those whose primary fluency is in the underlying algorithms, engaging domain knowledge as a constraint and validation mechanism rather than a primary lens.

My research in Explainable AI originated in a clinical context: making deep learning diagnostics trustworthy to pathologists. That context remains important; the CRC Press volume I co-edited in 2024 addresses it directly. Over the course of this research, I have become increasingly convinced that the XAI problem is not primarily a healthcare problem. It is a consequence of deploying powerful AI systems in any context where human accountability is required, and as regulatory frameworks mature, that context is becoming essentially everywhere.

The Regulatory Shift

The EU AI Act, which entered into force in 2024, establishes a risk-based framework imposing transparency requirements on AI systems in proportion to potential harm. High-risk applications, including medical diagnosis, credit scoring, employment screening and critical infrastructure, are subject to requirements including human oversight, model behaviour documentation, and the ability to explain individual decisions.

Globally, similar frameworks are emerging. The UAE National AI Strategy 2031 explicitly prioritises trustworthy and ethical AI. NATO AI principles require meaningful human control over autonomous systems. The FDA has issued guidance on AI medical devices. The direction of travel is consistent: AI systems making consequential decisions must be explainable.

Where XAI Requirements Are Most Acute

Healthcare & Medical Devices: EU MDR and FDA guidance require AI transparency in clinical decision support. Diagnosis, triage, and treatment recommendation systems must provide auditable reasoning to clinicians and regulators.

Defence & Autonomous Systems: NATO AI principles and international humanitarian law require meaningful human control over autonomous weapons. Explainable decision trails are a legal prerequisite.

Financial Services: GDPR Article 22 and regional governance frameworks require right-to-explanation for automated credit, insurance, and fraud decisions.

Smart City & Infrastructure: AI systems managing urban resources such as traffic, utilities and emergency response require auditable reasoning for public accountability and operator override capability.

The Methodological Toolkit

Grad-CAM produces spatial heatmaps for image classification tasks, answering where the model was looking. SHAP provides feature attribution for tabular data, grounded in game-theoretic foundations. LIME offers instance-level explanation of any black-box model when global interpretability is impossible. Attention visualisation reveals what transformer-based architectures attended to at each step.

Choosing the right method depends on three factors: the model architecture, the nature of the explanation required (spatial vs. feature vs. instance), and the audience (clinician vs. regulator vs. operator). These methods are not interchangeable.

An Honest Assessment

The most widely used post-hoc explanation methods are approximations. They reveal something true about model behaviour, but they do not provide a complete or guaranteed-faithful account. There is active and legitimate debate in the research community about whether these approximations are reliable enough for high-stakes operational deployment.

This debate is pushing the field toward intrinsically interpretable architectures: models designed for transparency from the outset of the design process. Within five years, XAI will likely be a prerequisite for deployment approval in regulated environments rather than an optional add-on.

I served as a co-editor of Explainable Artificial Intelligence (XAI) in Healthcare (CRC Press, Taylor & Francis, 2024). The volume brings together twelve contributed chapters on deploying XAI methods in clinical and other high-stakes settings, covering disease diagnosis, medical imaging, drug discovery and precision medicine. Academic correspondence on XAI methodology, cross-sector applications, or the regulation-explainability interface is welcome.