India's AI Safety Architecture Has a Blind Spot

India is building its AI Safety Institute in a hub-and-spoke model but risks developing a blind spot. An AI system can pass a safety test in Bengaluru and still fail in a village in Telangana. The people and communities who are least represented in datasets, least attractive as commercial markets and hardest to reach institutionally may also be the people least likely to be included in AI safety assessments.

A company building a language model for Indian-language speakers may have little commercial incentive to evaluate performance among Telugu-speaking tribal communities in Adilabad or Mahabubnagar if those communities are not part of its target market. Assessment samples therefore tend toward urban, literate, and digitally connected populations not because of deliberate exclusion, but because that is where the market is and it is what is represented digitally on the internet.

When AISI anchors to such assessments, it becomes a governance problem, especially when the same technology moves beyond the market for which it was developed and into public services.

An AI system used for welfare delivery, agricultural advice or public legal services can encounter communities that were absent from its training data, validation samples or pilot programmes. A system can therefore satisfy conventional benchmarks while still performing poorly in the environments where the state ultimately deploys it.

India’s emerging AI safety architecture needs to account for this gap. It has to differentiate system-level safety assessment from community-level assessment. System-level safety assessment asks whether a model or application meets technical standards: whether it is accurate, robust, explainable, secure or biased in measurable ways.

Community-level impact assessment asks a different set of questions. Does the system work in the language actually spoken by users? Do people understand its recommendations? Do local officials use it as intended? Does unreliable connectivity alter outcomes? Does the introduction of the system change how citizens access a service? Are there harms that appear only after deployment? These questions require access to communities over time, not simply access to datasets.

India’s current AI Safety Institute (AISI) is housed within India AI mission’s Safe & Trusted AI pillar. As of September 2026, 13 projects are ongoing under AISI. 8 projects were selected in 2024 and 5 were selected in 2025 covering areas including bias mitigation, algorithmic auditing, explainable AI and privacy-preserving AI. But the architecture is still taking shape. The government has been building the institute through projects and partnerships, rather than through a fully mature national testing infrastructure. That makes the next stage of institutional design particularly important. The question is not whether India should have a national AI safety capability. It is how that capability can acquire evidence from communities that a central institution is unlikely to reach consistently on its own.

Solution - A Statutory Body With Inclusive Safety as Mandate

As a solution, AISI should be established as a statutory body with a mandate for inclusive risk and impact assessment. It requires systematic documentation of how AI systems actually perform when deployed among populations that were not part of the training, validation, or pilot sample. These require different methodologies, different institutional positions relative to the communities being assessed, and different relationships with local civil society. Telangana provides a tractable entry point for designing this mechanism. The state has announced an AI Research and Collaboration Network and holds existing agreements with universities across its districts. This infrastructure is sufficient to support a community-level impact assessment mechanism that operates as a state-level spoke within the national AISI architecture, covering assessment territory the national design does not, while feeding findings back into it.

Lessons From Connecticut’s QuantumCT, a Direction for India’s AISI

Community-level impact assessment requires trust relationships that can only be built locally and over time; a state mechanism is better placed than a national one to build them. On institutional design, Connecticut’s QuantumCT initiative offers a relevant template, though it was built for a different purpose. Funded at USD 15 million under the National Science Foundation’s Regional Innovation Engines programme, it connects Yale University and the University of Connecticut as research anchors with Southern Connecticut State University, which leads on workforce and community engagement. The structure is tiered and the funding staged: pilot projects receive one year of seed funding and in-kind support before competing for multi-year external funding. What makes this relevant is not the domain but the architecture — a research anchor holding the methodology, a network extending it across a geography, and a third tier connecting the system to communities.

A Three-Tier Model for Telangana

The national institution would not need to know every community. The state network would not need to reinvent national safety standards. And civil-society organisations would not need to develop their own technical evaluation frameworks.

Applied to Telangana, the three tiers would function as follows. The first tier would be a research anchor. An institution such as IIIT Hyderabad or the University of Hyderabad could develop and maintain the technical methodology, establish assessment protocols and coordinate with the national AISI. The second tier would be a statewide research network. Universities and research institutions across Telangana could adapt the methodology to local conditions and identify communities and public services requiring assessment. The third tier would be community organisations. Civil-society groups with established field relationships could conduct structured assessments among target populations and report findings to the research anchor.

For this mechanism to function within India’s emerging AI governance architecture, two things need to happen.

1. The IndiaAI Safety Institute should explicitly distinguish system-level safety assessment from community-level impact assessment in its future partner and evaluation frameworks. Community assessment should have its own methodology, evaluation criteria and reporting requirements.
2. Telangana should position its existing AI research and collaboration infrastructure as a potential state-level spoke within the national architecture.

The spoke should not become another regulator. Nor should it duplicate the national institute’s technical functions. Its job would be narrower: identify populations and deployment contexts that are underrepresented in conventional testing; assess AI systems in those settings; document harms and unexpected effects; and feed the evidence back into the national system.

The reporting cycle should be public and regular. Findings could identify the system tested, the population assessed, the deployment context, observed failures and recommended mitigations. Over time, this would create something India currently lacks: an evidence base on how AI performs across the country’s social and geographic diversity after it leaves the laboratory.

The Larger Opportunity

India’s AI governance architecture is still being built. That makes this the right moment to decide what “safe” should mean. An AI system should not be considered adequately assessed simply because it performs well on the populations for which data is abundant. Nor should inclusion mean merely adding more demographic categories to a benchmark.

For public-facing AI, safety should also mean understanding what happens when a system encounters people who were never part of its development process. That is why community-level assessment should not be treated as an optional extension of AI safety. It should become part of the country’s safety infrastructure. The objective is not to decentralise AI regulation. It is to decentralise the evidence on which AI governance depends.

India’s national AI safety institution can set the standards. But if it wants to know whether AI is safe for the people who will actually use it, it will need institutions close enough to those people to ask.