Skip to main content

LLD: AI Audit Run API

1. Overview​

Runs the AI-powered audit analysis for a single Transformer Centre (TC) for a given month. This is API 1 of the AI Audit feature — it performs GPS-based installation classification, remark-based consumption analysis, computes projected audit values, and stores all results in intermediate (staging) collections without modifying real data.

The flow:

  1. Validates eligibility — TC must exist, not be AUDIT_FAILED, and not already frozen by a prior AI audit
  2. Cleans up any existing unfrozen AI audit result for this TC + month
  3. Fetches all installations in the section using $geoNear with the TC's GPS as the reference point
  4. Classifies installations into four sets based on tagging status and GPS proximity (Sets A, B, C, Remaining)
  5. Derives remark fields for each installation (remarkType, aiSuggestsConsumption)
  6. Computes original and projected audit values (before vs. after AI suggestions)
  7. Persists results to ai-audit-result and ai-audit-installation collections
  8. Returns the AI audit result header with original/computed comparison and summary counts

Important: This API does NOT modify the real transformer or installation documents. Changes are committed only when the user freezes the result (covered in a separate LLD).

LLDRelationship
LLD-ai-audit-schemas.mdData model — defines ai-audit-result and ai-audit-installation schemas
LLD-audit-tc.mdStandard reaudit — reuses computeAuditFields.js formulas

2. Endpoint​

RouteMethod
POST /audit/transformers/:id/ai-auditPOST

Single route — user type (Admin vs AE) resolved from request.user.userType. AE users are validated against their assigned section.

3. Request​

3.1 Headers​

HeaderRequiredDescription
AuthorizationYesBearer <sessionToken>

3.2 Path Parameters​

ParamTypeRequiredDescription
idstringYesThe id field of the transformer document (e.g., TC-xxx)

3.3 Query Parameters​

ParamTypeRequiredDefaultDescription
monthstringNoCurrent month (YYYY-MM)Month to run AI audit for

4. Response​

4.1 Success (200)​

{
  "success": true,
  "message": "AI audit completed",
  "data": {
    "aiAuditResultId": "6651a3f2b1c2d3e4f5a6b7c8",
    "tcId": "TC-xxx",
    "tcNumber": "11F9020",
    "tcName": "TC 11F9020",
    "month": "2025-10",
    "meterConstant": 0.1,
    "meterConstantChanged": false,
    "tcConsumption": 1256,
    "previousMonthLossPercentage": 200,
    "previousToPreviousMonthLossPercentage": 25,
    "original": {
      "installationConsumption": 1563,
      "installationCount": 20,
      "lossPercentage": 200,
      "auditStatus": "AUDITED",
      "billingCount": {
        "unbilled": 10,
        "mnr": 5,
        "vacant": 3,
        "zeroConsumption": 8,
        "doorlock": 2,
        "abnormal": 1,
        "subnormal": 4,
        "billCancellation": 0
      },
      "revenueParams": {
        "arr": null,
        "atc": null,
        "demand": 480000,
        "collection": 460000,
        "billingEfficiency": 96,
        "collectionEfficiency": 92
      },
      "overloaded": false,
      "totalInstallationSanctionedLoadInKVA": 85
    },
    "computed": {
      "installationConsumption": 1100,
      "installationCount": 18,
      "lossPercentage": 15,
      "auditStatus": "AUDITED",
      "billingCount": { "unbilled": 8, "mnr": 3, "vacant": 2, "zeroConsumption": 6, "doorlock": 1, "abnormal": 1, "subnormal": 3, "billCancellation": 0 },
      "revenueParams": { "arr": null, "atc": null, "demand": 420000, "collection": 400000, "billingEfficiency": 94, "collectionEfficiency": 90 },
      "overloaded": false,
      "totalInstallationSanctionedLoadInKVA": 72
    },
    "computedTagging": {
      "installationConsumption": 1150,
      "installationCount": 18,
      "lossPercentage": 8.3,
      "auditStatus": "AUDITED",
      "billingCount": { "unbilled": 8, "mnr": 3, "vacant": 2, "zeroConsumption": 6, "doorlock": 1, "abnormal": 1, "subnormal": 3, "billCancellation": 0 },
      "revenueParams": { "arr": null, "atc": null, "demand": 420000, "collection": 400000, "billingEfficiency": 94, "collectionEfficiency": 90 },
      "overloaded": false,
      "totalInstallationSanctionedLoadInKVA": 72
    },
    "computedRemarks": {
      "installationConsumption": 1120,
      "installationCount": 20,
      "lossPercentage": 10.8,
      "auditStatus": "AUDITED",
      "billingCount": { "unbilled": 10, "mnr": 5, "vacant": 3, "zeroConsumption": 8, "doorlock": 2, "abnormal": 1, "subnormal": 4, "billCancellation": 0 },
      "revenueParams": { "arr": null, "atc": null, "demand": 480000, "collection": 460000, "billingEfficiency": 96, "collectionEfficiency": 92 },
      "overloaded": false,
      "totalInstallationSanctionedLoadInKVA": 85
    },
    "installationTaggingSummary": {
      "total": 25,
      "changesSuggestedByAI": 8,
      "noChangesRequired": 17
    },
    "installationRemarksSummary": {
      "total": 5,
      "consumptionSuggested": 4
    }
  }
}

4.2 Not Found (404)​

{
  "success": false,
  "message": "Transformer not found"
}

4.3 Forbidden (403)​

When AE requests AI audit on a TC outside their section:

{
  "success": false,
  "message": "TC does not belong to your assigned section"
}

4.4 Bad Request (400) — TC Not Eligible​

{
  "success": false,
  "message": "TC is not eligible for AI audit"
}

4.5 Bad Request (400) — Already Frozen​

{
  "success": false,
  "message": "AI audit already completed and frozen for this TC"
}

5. Preconditions​

Before running the AI audit, all of the following must pass. If any fail, return the corresponding error immediately.

#CheckError
1TC must exist for the given id + month404 — "Transformer not found"
2If userType is AE, TC must belong to AE's section403 — "TC does not belong to your assigned section"
3TC must NOT be AUDIT_FAILED — must have meterConstant (non-null, non-zero), tcConsumption (non-null), lossPercentage (non-null), and no remark400 — "TC is not eligible for AI audit"
4lossPercentage must be within -100% to 100% — values outside this range indicate a TC meter issue, not a tagging issue400 — "TC is not eligible for AI audit as the loss percentage is abnormal..."
5lossPercentage must NOT be within -5% to 10% — TC loss is within acceptable limits, AI audit not necessary400 — "TC loss percentage is within acceptable limits..."
6TC must NOT already have auditedByAI: true (set during freeze — permanently prevents re-audit)400 — "AI audit already completed and frozen for this TC"
7If an unfrozen ai-audit-result exists for this TC + month, DELETE it AND all linked ai-audit-installation docs, then proceed with fresh analysis. If frozen result exists (safety net), return error.400 or cleanup

6. Business Logic​

6.1 Default Month​

IF month not provided:
  month = current date formatted as "YYYY-MM"

6.2 Fetch Transformer (Step 1)​

const transformer = await transformerModel
  .findOne({ id, month })
  .lean();

if (!transformer) {
  return 404 — "Transformer not found";
}

6.3 AE Section Validation (Precondition 2)​

Same pattern as auditTC.js:

IF userType === "AE":
  1. Fetch user doc → extract user.location.sectionCode
  2. Verify transformer.location.sectionCode === user.location.sectionCode
  3. If mismatch → 403 Forbidden

6.4 AUDIT_FAILED Check (Precondition 3)​

if (transformer.remark || !transformer.meterConstant || transformer.tcConsumption == null || transformer.lossPercentage == null) {
  return 400 — "TC is not eligible for AI audit";
}

6.4b Abnormal Loss Percentage Check (Precondition 4)​

if (transformer.lossPercentage > 100 || transformer.lossPercentage < -100) {
  return 400 — "TC is not eligible for AI audit as the loss percentage is abnormal (outside -100% to 100%)";
}

Loss % outside -100% to 100% indicates a data anomaly (e.g., meter constant mismatch, bad readings) that GPS-based retagging cannot fix. AI audit is blocked to avoid producing misleading suggestions.

6.4c Already AI-Audited Check (Precondition 5)​

if (transformer.auditedByAI === true) {
  return 400 — "AI audit already completed and frozen for this TC";
}

This check is on the transformer document itself — the auditedByAI flag is set to true during freeze and permanently prevents re-running AI audit on this TC for this month.

6.5 Existing Result Cleanup (Precondition 6) (Step 2)​

const existingResult = await aiAuditResultModel.findOne({ tcId: id, month });

if (existingResult) {
  if (existingResult.frozen) {
    // Safety net — should not reach here if 6.4b passed, but guard anyway
    return 400 — "AI audit already completed and frozen for this TC";
  }
  // Unfrozen — delete and re-run fresh
  await aiAuditInstallationModel.deleteMany({
    aiAuditResultId: existingResult._id.toString(),
  });
  await aiAuditResultModel.deleteOne({ _id: existingResult._id });
}

6.6 Fetch Installations Using Two Targeted Queries (Step 3)​

Instead of fetching all ~17K installations in the section, run two small parallel queries that target only relevant documents:

QueryFilterIndex usedExpected size
taggedInstallations{ tcId: installationTcId, month }tcId_1_month_1~200 docs
candidateInstallations{ location.sectionCode, month, tcId: {$ne}, outsideTCRadius: true }location.sectionCode_1_month_1_tcId_1small subset

candidateInstallations are untagged installations already confirmed (via outsideTCRadius: true) to be outside their current TC's range — these are the only valid Set C candidates.

const installationProjection = {
  id: 1, tcId: 1, tcNumber: 1, gpsCoordinates: 1, outsideTCRadius: 1,
  accountId: 1, rrNumber: 1, consumerName: 1, mrCode: 1, readingDay: 1,
  tariff: 1, sanctionedLoad: 1, consumption: 1, demand: 1, collection: 1,
  unbilled: 1, mnr: 1, vacant: 1, zeroConsumption: 1, doorLock: 1,
  emin: 1, emax: 1, avgConsumption6Months: 1,
};

const [taggedInstallations, candidateInstallations] = await Promise.all([
  installationModel.find({ tcId: installationTcId, month }, installationProjection).lean(),
  installationModel.find({
    "location.sectionCode": transformer.location.sectionCode,
    month,
    tcId: { $ne: installationTcId },
    outsideTCRadius: true,
  }, installationProjection).lean(),
]);

Why not one big query? Real sections contain ~17K installations per month. The old single query fetched all of them; the vast majority were Remaining (untagged + out of range) which are irrelevant and never shown in the UI. The two-query approach fetches only what can possibly end up in setA/B/C.

Why not $geoNear? The $geoNear stage scans the entire 2dsphere index before applying the query filter, causing >60s timeouts on 8M+ docs.

computeDistanceMeters helper (gpsConstants.js): Standard Haversine formula, takes two [longitude, latitude] pairs, returns distance in meters rounded to 2 decimal places.

6.7 Classify Installations into Sets A, B, C, Remaining (Step 4)​

The transformer id has a "TC-" prefix but installation.tcId does not. Strip the prefix for comparison (same pattern as auditTC.js):

const installationTcId = id.replace(/^TC-/, "");

Classification rules:

SetConditioncurrentTaggingStatusaiSuggestionMeaning
Ainstallation.tcId === installationTcId AND distanceFromTC <= thresholdTAGGEDNO_CHANGECurrently tagged, within range — keep as-is
Binstallation.tcId === installationTcId AND distanceFromTC > thresholdTAGGEDUNTAGCurrently tagged, out of range — AI suggests untagging
Cinstallation.tcId !== installationTcId AND distanceFromTC <= threshold AND outsideTCRadius !== falseUNTAGGEDTAGWithin range of this TC AND not confirmed within range of its current TC — AI suggests tagging
Remainingeverything elseUNTAGGEDNO_CHANGEOut of range, or already correctly placed at its current TC

outsideTCRadius on Set C: This field on the installation indicates whether the installation is outside the radius of its currently tagged TC.

  • true → outside its current TC's radius — valid TAG candidate
  • false → within its current TC's radius — correctly placed, do NOT suggest TAG (falls to Remaining)
  • null → unknown (no GPS or field not set) — fall back to TAG (same as pre-fix behaviour)

For Set C installations, capture the source TC info:

// Set C only
sourceTcId: installation.tcId || null,    // the other TC's id (or null if truly untagged)
sourceTcNumber: installation.tcNumber || null, // the other TC's number

Since the two queries already pre-filter by tagging status, classification simplifies to a distance check:

const setA = []; // TAGGED + within range → NO_CHANGE
const setB = []; // TAGGED + out of range → UNTAG

for (const inst of taggedInstallations) {
  addDistance(inst);
  if (inst.distanceFromTC <= GPS_RANGE_THRESHOLD_METERS) setA.push(inst);
  else setB.push(inst);
}

const setC = []; // UNTAGGED + within range → TAG
// (candidateInstallations already have outsideTCRadius:true from the query filter)

for (const inst of candidateInstallations) {
  addDistance(inst);
  if (inst.distanceFromTC <= GPS_RANGE_THRESHOLD_METERS) setC.push(inst);
}

// Remaining (untagged + out of range) is never fetched — excluded at query time.

6.8 Derive Remark Fields for Each Installation (Step 5)​

For every installation (all four sets), derive remark-related fields:

function deriveRemarkType(inst) {
  if (inst.unbilled)                          return "UNBILLED";
  if (inst.mnr)                               return "MNR";
  if (inst.vacant)                            return "VACANT";
  if (inst.zeroConsumption)                   return "ZERO_CONSUMPTION";
  if (inst.doorLock)                          return "DOORLOCK";
  if (inst.emin != null && inst.emin > 0)     return "ABNORMAL";
  if (inst.emax != null && inst.emax > 0)     return "SUBNORMAL";
  return null;
}

For each installation:

const remarkType = deriveRemarkType(inst);
const hasRemark = remarkType !== null;
const avgConsumption6Months = inst.avgConsumption6Months || null;
const aiSuggestsConsumption = hasRemark && avgConsumption6Months != null && avgConsumption6Months > 0;

6.9 Compute "Original" Audit Values (Step 6)​

These are the current values on the transformer document — a snapshot before AI audit changes anything:

const original = {
  installationConsumption: transformer.installationConsumption,
  installationCount: transformer.installationCount,
  lossPercentage: transformer.lossPercentage,
  auditStatus: transformer.auditStatus,
  billingCount: transformer.billingCount,
  revenueParams: transformer.revenueParams,
  overloaded: transformer.overloaded,
  totalInstallationSanctionedLoadInKVA: transformer.totalInstallationSanctionedLoadInKVA,
};

6.10 Compute Projected Audit Values for All Toggle Combinations (Step 7)​

The frontend can independently toggle the two AI suggestion tables (Installation Tagging and Remarks). The API precomputes all four combinations so the frontend can update the displayed loss% live as the user toggles.

computed fieldTagging tableRemarks tableTagged setConsumption
originalOFFOFFA + B (current)actual
computedTaggingONOFFA + C (retagged)actual
computedRemarksOFFONA + B (current)avg for remarked installs
computedONONA + C (retagged)avg for remarked installs

Helper function — computes all audit fields for a given set:

// useAvg=true  → substitute avgConsumption6Months for installations with remarks
// useAvg=false → use actual consumption for all installations
function computeAuditValues(taggedSet, useAvg) {
  const installationConsumption = taggedSet.reduce((sum, inst) => {
    if (useAvg) {
      const remarkType = deriveRemarkType(inst);
      const avg = inst.avgConsumption6Months;
      if (remarkType !== null && avg != null && avg > 0) return sum + avg;
    }
    return sum + (inst.consumption || 0);
  }, 0);

  const installationCount = taggedSet.length;

  const effectiveTcConsumption = (transformer.tcConsumption || 0) * (transformer.meterConstant || 1);
  let lossPercentage = null;
  if (effectiveTcConsumption !== 0) {
    lossPercentage = ((effectiveTcConsumption - installationConsumption) / effectiveTcConsumption) * 100;
    lossPercentage = Math.round(lossPercentage * 100) / 100;
  }

  const billingCount = {
    unbilled: 0, mnr: 0, vacant: 0, zeroConsumption: 0,
    doorlock: 0, abnormal: 0, subnormal: 0, billCancellation: 0,
  };
  for (const inst of taggedSet) {
    if (inst.unbilled) billingCount.unbilled++;
    if (inst.mnr) billingCount.mnr++;
    if (inst.vacant) billingCount.vacant++;
    if (inst.zeroConsumption) billingCount.zeroConsumption++;
    if (inst.doorLock) billingCount.doorlock++;
    if (inst.emin != null && inst.emin > 0) billingCount.abnormal++;
    if (inst.emax != null && inst.emax > 0) billingCount.subnormal++;
  }

  const totalInstallationSanctionedLoadInKVA = taggedSet.reduce(
    (sum, inst) => sum + (inst.sanctionedLoad?.kw || 0), 0
  );
  const overloaded = transformer.tcCapacity
    ? totalInstallationSanctionedLoadInKVA > transformer.tcCapacity
    : false;

  const demand = taggedSet.reduce((sum, inst) => sum + (inst.demand || 0), 0);
  const collection = taggedSet.reduce((sum, inst) => sum + (inst.collection || 0), 0);
  const revenueParams = { demand, collection, arr: null, atc: null, billingEfficiency: null, collectionEfficiency: null };
  if (demand > 0) {
    revenueParams.billingEfficiency = Math.round((installationConsumption / demand) * 100 * 100) / 100;
  }
  if (installationConsumption > 0) {
    revenueParams.collectionEfficiency = Math.round((collection / installationConsumption) * 100 * 100) / 100;
  }

  return {
    installationConsumption, installationCount, lossPercentage,
    auditStatus: "AUDITED", billingCount, revenueParams, overloaded,
    totalInstallationSanctionedLoadInKVA,
  };
}

const newTaggedSet = [...setA, ...setC];
const originalTaggedSet = [...setA, ...setB];

const computed = computeAuditValues(newTaggedSet, true);       // tagging + remarks
const computedTagging = computeAuditValues(newTaggedSet, false); // tagging only
const computedRemarks = computeAuditValues(originalTaggedSet, true); // remarks only

6.11 Create ai-audit-result Document (Step 8)​

Create one document in the ai-audit-result collection:

const aiAuditResult = await aiAuditResultModel.create({
  // Identity
  tcId: id,
  tcNumber: transformer.number,
  tcName: transformer.name,
  month,
  location: transformer.location,

  // TC Properties (snapshot)
  tcCapacity: transformer.tcCapacity,
  meterConstant: transformer.meterConstant,
  meterConstantChanged: transformer.meterConstantChanged || false,
  readingDay: transformer.readingDay,
  readingMR: transformer.readingMR,
  gpsCoordinates: transformer.gpsCoordinates,
  reading: transformer.reading,

  // TC Detail Properties
  tcMake: transformer.tcMake,
  serialNumber: transformer.serialNumber,
  timsCode: transformer.timsCode,
  dtlms: transformer.dtlms,
  dtr: transformer.dtr,
  feeder: transformer.feeder,
  executionType: transformer.executionType,

  // Meter Detail Properties
  meterMake: transformer.meterMake,
  meterSerial: transformer.meterSerial,
  ctRatio: transformer.ctRatio,

  // Original audit values (before AI)
  original,

  // Computed AI audit values (after GPS + remarks analysis)
  computed: {
    installationConsumption: newInstallationConsumption,
    installationCount: newInstallationCount,
    lossPercentage: newLossPercentage,
    auditStatus: newAuditStatus,
    billingCount: newBillingCount,
    revenueParams: newRevenueParams,
    overloaded: newOverloaded,
    totalInstallationSanctionedLoadInKVA: newTotalSanctionedLoad,
  },

  // TC consumption (read-only, from transformer)
  tcConsumption: transformer.tcConsumption,
  previousMonthLossPercentage: transformer.previousMonthLossPercentage,
  previousToPreviousMonthLossPercentage: transformer.previousToPreviousMonthLossPercentage,

  // Freeze meta
  frozen: false,
});

6.12 Create ai-audit-installation Documents (Step 9)​

For EVERY installation from Step 4 (all four sets), create one document in ai-audit-installation. Use insertMany for bulk performance.

const aiAuditResultId = aiAuditResult._id.toString();

const installationDocs = [];

function buildInstallationDoc(inst, currentTaggingStatus, aiSuggestion, sourceTcId, sourceTcNumber) {
  const remarkType = deriveRemarkType(inst);
  const hasRemark = remarkType !== null;
  const avg = inst.avgConsumption6Months || null;
  const aiSuggestsConsumption = hasRemark && avg != null && avg > 0;

  return {
    // Link fields
    aiAuditResultId,
    tcId: id,
    month,

    // Installation identity
    accountId: inst.accountId,
    installationId: inst.id,
    rrNumber: inst.rrNumber,
    consumerName: inst.consumerName,
    mrCode: inst.mrCode,
    readingDay: inst.readingDay,
    tariff: inst.tariff,
    sanctionedLoad: inst.sanctionedLoad,
    gpsCoordinates: inst.gpsCoordinates,

    // Installation data
    consumption: inst.consumption,
    demand: inst.demand,
    collection: inst.collection,

    // Table 1: Tagging fields
    distanceFromTC: inst.distanceFromTC,
    withinRange: inst.distanceFromTC <= GPS_RANGE_THRESHOLD_METERS,
    currentTaggingStatus,
    aiSuggestion,
    sourceTcId: sourceTcId || null,
    sourceTcNumber: sourceTcNumber || null,

    // Table 2: Remarks fields
    hasRemark,
    remarkType,
    avgConsumption6Months: avg,
    aiSuggestsConsumption,

    // Billing flags
    unbilled: inst.unbilled,
    mnr: inst.mnr,
    vacant: inst.vacant,
    doorLock: inst.doorLock,
    zeroConsumption: inst.zeroConsumption,
    emin: inst.emin,
    emax: inst.emax,
  };
}

// Set A — TAGGED + within range → NO_CHANGE
for (const inst of setA) {
  installationDocs.push(buildInstallationDoc(inst, "TAGGED", "NO_CHANGE", null, null));
}

// Set B — TAGGED + out of range → UNTAG
for (const inst of setB) {
  installationDocs.push(buildInstallationDoc(inst, "TAGGED", "UNTAG", null, null));
}

// Set C — UNTAGGED + within range → TAG (capture source TC)
for (const inst of setC) {
  installationDocs.push(buildInstallationDoc(inst, "UNTAGGED", "TAG", inst.tcId, inst.tcNumber));
}

// Remaining (UNTAGGED + out of range) is NOT persisted — it is never shown in
// the UI and does not affect any computed value. Excluding it reduces insertMany
// from ~17K docs to ~200-400, which is the primary performance fix.

await aiAuditInstallationModel.insertMany(installationDocs);

6.13 Build and Return Response (Step 10)​

// Installation tagging summary
const changesSuggestedByAI = setB.length + setC.length; // UNTAG + TAG suggestions
const noChangesRequired = setA.length; // TAGGED + within range — no action needed

// Installation remarks summary
const allRemarkInstallations = installationDocs.filter(d => d.hasRemark);
const consumptionSuggested = installationDocs.filter(d => d.aiSuggestsConsumption);

return {
  statusCode: 200,
  success: true,
  message: "AI audit completed",
  data: {
    aiAuditResultId,
    tcId: id,
    tcNumber: transformer.number,
    tcName: transformer.name,
    month,
    meterConstant: transformer.meterConstant,
    meterConstantChanged: transformer.meterConstantChanged || false,
    tcConsumption: transformer.tcConsumption,
    previousMonthLossPercentage: transformer.previousMonthLossPercentage,
    previousToPreviousMonthLossPercentage: transformer.previousToPreviousMonthLossPercentage,
    original,
    computed: {
      installationConsumption: newInstallationConsumption,
      installationCount: newInstallationCount,
      lossPercentage: newLossPercentage,
      auditStatus: newAuditStatus,
      billingCount: newBillingCount,
      revenueParams: newRevenueParams,
      overloaded: newOverloaded,
      totalInstallationSanctionedLoadInKVA: newTotalSanctionedLoad,
    },
    installationTaggingSummary: {
      total: setA.length + setB.length + setC.length,
      changesSuggestedByAI,
      noChangesRequired,
    },
    installationRemarksSummary: {
      total: allRemarkInstallations.length,
      consumptionSuggested: consumptionSuggested.length,
    },
  },
};

7. Data Flow Diagram​

┌─────────────────────────────────────────────────────────────────────────┐
│                POST /audit/transformers/:id/ai-audit                    │
└──────────────────────────────┬──────────────────────────────────────────┘
                               │
                               ▼
                ┌──────────────────────────┐
                │  Preconditions:           │
                │  1. TC exists?            │
                │  2. AE section match?     │
                │  3. Not AUDIT_FAILED?     │
                │  4. Not frozen?           │
                │  5. Delete unfrozen?      │
                └────────────┬─────────────┘
                             │
                             ▼
                ┌──────────────────────────┐
                │  $match: fetch ALL        │
                │  installations in section │
                │  + Haversine distance     │
                └────────────┬─────────────┘
                             │
              ┌──────────────┼──────────────────────┐
              ▼              ▼                      ▼
     ┌──────────────┐ ┌──────────────┐    ┌──────────────────┐
     │ Classify into │ │ Derive remark│    │ Snapshot original│
     │ Sets A/B/C/   │ │ fields for   │    │ audit values     │
     │ Remaining     │ │ each install │    │ from transformer │
     └──────┬───────┘ └──────┬───────┘    └────────┬─────────┘
            │                │                      │
            └────────────────┼──────────────────────┘
                             │
                             ▼
              ┌──────────────────────────────┐
              │ Compute projected values:     │
              │  new tagged set = A + C       │
              │  effective consumption (with  │
              │  avg for remarked installs)   │
              │  → loss%, billingCount, etc.  │
              └──────────────┬───────────────┘
                             │
              ┌──────────────┼──────────────────────┐
              ▼              ▼                      ▼
     ┌──────────────┐ ┌──────────────────┐ ┌──────────────────┐
     │ Create        │ │ Create           │ │ Build response   │
     │ ai-audit-     │ │ ai-audit-        │ │ with original vs │
     │ result doc    │ │ installation     │ │ computed + summ. │
     │ (1 insert)    │ │ docs (bulk)      │ │                  │
     └──────────────┘ └──────────────────┘ └──────────────────┘

8. Validation Schema (AJV)​

8.1 Path Parameters Schema​

{
  type: "object",
  properties: {
    id: {
      type: "string",
      transform: ["trim"],
    }
  },
  required: ["id"]
}

8.2 Query String Schema​

{
  type: "object",
  properties: {
    month: {
      type: "string",
      pattern: "^\\d{4}-(0[1-9]|1[0-2])$",
      errorMessage: { pattern: "month must be in YYYY-MM format" }
    }
  },
  additionalProperties: false,
  errorMessage: {
    additionalProperties: "Unsupported properties in query parameters"
  }
}

8.3 Response Serialization​

const responseSerialization = {
  "2xx": responseSerializer("2xx", {
    type: "object",
    properties: {
      aiAuditResultId: { type: "string" },
      tcId: { type: "string" },
      tcNumber: { type: "string" },
      tcName: { type: "string" },
      month: { type: "string" },
      meterConstant: { type: ["number", "null"] },
      meterConstantChanged: { type: "boolean" },
      tcConsumption: { type: ["number", "null"] },
      previousMonthLossPercentage: { type: ["number", "null"] },
      previousToPreviousMonthLossPercentage: { type: ["number", "null"] },
      original: {
        type: "object",
        properties: {
          installationConsumption: { type: ["number", "null"] },
          installationCount: { type: ["number", "null"] },
          lossPercentage: { type: ["number", "null"] },
          auditStatus: { type: ["string", "null"] },
          billingCount: {
            type: "object",
            properties: {
              unbilled: { type: "number" },
              mnr: { type: "number" },
              vacant: { type: "number" },
              zeroConsumption: { type: "number" },
              doorlock: { type: "number" },
              abnormal: { type: "number" },
              subnormal: { type: "number" },
              billCancellation: { type: "number" },
            },
          },
          revenueParams: {
            type: "object",
            properties: {
              arr: { type: ["number", "null"] },
              atc: { type: ["number", "null"] },
              demand: { type: ["number", "null"] },
              collection: { type: ["number", "null"] },
              billingEfficiency: { type: ["number", "null"] },
              collectionEfficiency: { type: ["number", "null"] },
            },
          },
          overloaded: { type: "boolean" },
          totalInstallationSanctionedLoadInKVA: { type: ["number", "null"] },
        },
      },
      computed: {
        type: "object",
        properties: {
          installationConsumption: { type: ["number", "null"] },
          installationCount: { type: ["number", "null"] },
          lossPercentage: { type: ["number", "null"] },
          auditStatus: { type: ["string", "null"] },
          billingCount: {
            type: "object",
            properties: {
              unbilled: { type: "number" },
              mnr: { type: "number" },
              vacant: { type: "number" },
              zeroConsumption: { type: "number" },
              doorlock: { type: "number" },
              abnormal: { type: "number" },
              subnormal: { type: "number" },
              billCancellation: { type: "number" },
            },
          },
          revenueParams: {
            type: "object",
            properties: {
              arr: { type: ["number", "null"] },
              atc: { type: ["number", "null"] },
              demand: { type: ["number", "null"] },
              collection: { type: ["number", "null"] },
              billingEfficiency: { type: ["number", "null"] },
              collectionEfficiency: { type: ["number", "null"] },
            },
          },
          overloaded: { type: "boolean" },
          totalInstallationSanctionedLoadInKVA: { type: ["number", "null"] },
        },
      },
      installationTaggingSummary: {
        type: "object",
        properties: {
          total: { type: "number" },
          changesSuggestedByAI: { type: "number" },
          noChangesRequired: { type: "number" },
        },
      },
      installationRemarksSummary: {
        type: "object",
        properties: {
          total: { type: "number" },
          consumptionSuggested: { type: "number" },
        },
      },
    },
  }),

  "4xx": responseSerializer("4xx"),
  "5xx": responseSerializer("5xx"),
};

9. Route Config​

{
  routeId: "aiAuditTC",
  authentication: {
    // No excludeFrom — requires full auth (validation + verification)
  },
  authorization: {
    action: "read",
    resource: "audit"
  }
}

10. File Structure​

api/src/entities/audit/
  audit.route.v1.js                    — Add new route entry
  controller/
    fetchAuditSummary.js               — Existing
    fetchTransformers.js               — Existing
    fetchTransformerDetail.js          — Existing
    fetchInstallations.js              — Existing
    auditTC.js                         — Existing
    fetchConsumptionTrend.js           — Existing
    fetchRevenueTrend.js               — Existing
    aiAuditTC.js                       — New: handler, schema, config exports
  helpers/
    computeAuditFields.js              — Existing (reuse formulas for reference)
    recomputeAuditSummaries.js         — Existing
    resolveLocationFilter.js           — Existing
    lossBucketRanges.js                — Existing
    gpsConstants.js                    — New: GPS_RANGE_THRESHOLD_METERS + computeDistanceMeters (Haversine)
    deriveRemarkType.js                — New: derive remarkType from billing flags

common/src/schemas/
  transformer.model.js                 — Existing (10 new fields + 2dsphere index per schemas LLD)
  installation.model.js                — Existing (1 new field + 2dsphere index per schemas LLD)
  aiAuditResult.model.js               — New (defined in schemas LLD)
  aiAuditInstallation.model.js         — New (defined in schemas LLD)

11. Error Handling​

ScenarioStatusError Type
Invalid month format400ValidationError (AJV)
TC is AUDIT_FAILED (no meterConstant, no tcConsumption, or has remark)400BadRequestError
AI audit already frozen for this TC + month400BadRequestError
Invalid/expired session401UnauthorizedError
AE accessing TC outside their section403ForbiddenError
TC not found for given id + month404ResourceUnavailableError
AE user doc not found500DatabaseError
DB connection failure500DatabaseError
Installation find fails500DatabaseError
insertMany fails (duplicate key on re-run race)500DatabaseError

12. Performance Notes​

12.1 DB Operations​

OperationCollectionTypeCount
Find TCtransformerRead1
Find user (AE only)userRead0 or 1
Find existing resultai-audit-resultRead1
Delete old installationsai-audit-installationWrite0 or 1 (deleteMany)
Delete old resultai-audit-resultWrite0 or 1
Find installations (section+month)installationRead1
Create resultai-audit-resultWrite1
Bulk insert installationsai-audit-installationWrite1 (insertMany)

Total: 3-4 reads + 2-4 writes.

12.2 Memory​

  • find with section + month filter returns ~200-500 installations per section (average ~22 per TC, ~20 TCs per section). Loading all into memory is acceptable for demo scale.
  • Haversine distance computation for ~500 docs is negligible (~1ms).
  • insertMany with ~200-500 documents is well within MongoDB's bulk write limits.

12.3 Index Usage​

QueryIndex Hit
transformerModel.findOne({ id, month })_id or needs { id: 1, month: 1 } index
aiAuditResultModel.findOne({ tcId, month }){ tcId: 1, month: 1 } unique index — exact match
installationModel.find({ sectionCode, month }){ location.sectionCode: 1, month: 1, tcId: 1 } — compound index, exact match
aiAuditInstallationModel.deleteMany({ aiAuditResultId }){ aiAuditResultId: 1, accountId: 1 } — prefix match
aiAuditInstallationModel.insertMany(...)N/A (write)

12.4 Latency Estimate​

For a section with ~135 installations (tested on Atlas from local machine):

  • Find installations (section+month): ~500-1000ms (network to Atlas)
  • Haversine distance computation: ~1ms (in-memory)
  • Classification + remark derivation: ~5ms (in-memory)
  • Audit value computation: ~5ms (in-memory)
  • insertMany for ~135 docs: ~500-1000ms (network to Atlas)
  • Other DB operations (find TC, find user, find/delete existing result, create result): ~500-1000ms
  • Total: ~2-4s (dominated by Atlas network round-trips)
  • Observed: ~3.3s for section S01 (135 installations, 7 months seed data)

13. Key Design Decisions​

13.1 Why $match + Haversine Instead of $geoNear?​

$geoNear was the original design — it computes spherical distance server-side using the 2dsphere index. However, $geoNear must be the first aggregation stage and scans the entire 2dsphere index before applying the query filter. With 9.3M installation documents, this caused >60s timeouts.

The fix: use a standard find query filtered by { location.sectionCode, month } (hits the existing compound index, returns ~200-500 docs), then compute Haversine distance in application code. For ~500 docs, the in-app computation is ~1ms — negligible compared to network latency.

The Haversine formula uses Earth's mean radius (6,371 km) and produces results accurate to within ~0.5% of $geoNear's WGS84 spherical calculation — more than sufficient for a 5km threshold.

13.3 Why Staging Collections?​

The AI audit produces a "what-if" analysis. Results are stored in intermediate collections (ai-audit-result, ai-audit-installation) so the user can review and optionally adjust before committing. Real data is only modified on freeze.

13.4 Why Delete-and-Recreate on Re-run?​

When an unfrozen result already exists, we delete it entirely and re-run from scratch. This is simpler than diffing/patching individual installation documents and ensures the result always reflects the latest state of the transformer and installation data.

13.5 tcId Prefix Stripping​

The transformer id field uses a "TC-" prefix (e.g., "TC-abc123"), but the installation tcId field stores only the raw ID (e.g., "abc123"). The id.replace(/^TC-/, "") pattern is already established in auditTC.js and is reused here for consistency.

14. Testing Guide​

14.1 Test Data​

The seed API (POST /seed/ai-audit-test-data) created test data in section S01 with 15 TCs and ~135 installations per month across 7 months (2025-09 to 2026-03). All installations have GPS coordinates and 133 have avgConsumption6Months pre-computed.

Eligible TCs for AI audit (month 2025-10): TC-01 through TC-07, TC-10, TC-11, TC-12, TC-14 (10 total — AUDITED, no remark, has meterConstant + tcConsumption, auditedByAI=false).

Ineligible TCs (AUDIT_FAILED): 4 TCs with remarks — use these to test error cases.

14.2 How to Test​

Prerequisites​

  1. Start the API server: yarn api-dev
  2. Get a session token by logging in:
curl -s -X POST http://localhost:3000/dtcea-mumbai-demo/v1/auth/login \
  -H 'Content-Type: application/json' \
  -d '{"loginUserName": "<username>", "loginPassword": "<password>"}' | jq '.data.sessionToken'
  1. Set the token for convenience:
TOKEN="<paste-session-token-here>"

Test 1: Happy Path — Run AI Audit on an Eligible TC​

curl -s -X POST \
  'http://localhost:3000/dtcea-mumbai-demo/v1/audit/transformers/TC-01/ai-audit?month=2025-10' \
  -H "Authorization: Bearer $TOKEN" | jq .

Expected response:

  • success: true, message: "AI audit completed"
  • data.aiAuditResultId — non-null ObjectId string
  • data.tcId: "TC-01", data.month: "2025-10"
  • data.original — snapshot of current TC audit values (installationConsumption, installationCount, lossPercentage, billingCount, etc.)
  • data.computed — projected values after AI suggestions (may differ from original if installations were retagged or remark consumption was substituted)
  • data.installationTaggingSummary.total — count of ALL installations in the section (not just tagged to this TC)
  • data.installationTaggingSummary.changesSuggestedByAI — count of TAG + UNTAG suggestions
  • data.installationRemarksSummary.total — count of installations with remarks
  • data.installationRemarksSummary.consumptionSuggested — count of remarked installations where avgConsumption6Months > 0

Test 2: Re-trigger (Delete-and-Recreate)​

Run the same request again:

curl -s -X POST \
  'http://localhost:3000/dtcea-mumbai-demo/v1/audit/transformers/TC-01/ai-audit?month=2025-10' \
  -H "Authorization: Bearer $TOKEN" | jq .

Expected: Same success response. The old staging docs are deleted and recreated fresh.

Test 3: TC Not Found (404)​

curl -s -X POST \
  'http://localhost:3000/dtcea-mumbai-demo/v1/audit/transformers/TC-NONEXISTENT/ai-audit?month=2025-10' \
  -H "Authorization: Bearer $TOKEN" | jq .

Expected: { "success": false, "message": "Transformer not found" } with status 404.

Test 4: TC Not Eligible — AUDIT_FAILED (400)​

Find a TC with a remark in section S01:

# First find an AUDIT_FAILED TC
mongosh "mongodb+srv://demo:GhKYzU37veEYEmrc@energyaudit.0l6duna.mongodb.net/dtcea-mumbai-demo" --quiet --eval '
  db.transformer.findOne({"location.sectionCode": "S01", month: "2025-10", remark: {$ne: null}}, {id:1, remark:1, _id:0})
'

Then hit the API with that TC's id:

curl -s -X POST \
  'http://localhost:3000/dtcea-mumbai-demo/v1/audit/transformers/<FAILED_TC_ID>/ai-audit?month=2025-10' \
  -H "Authorization: Bearer $TOKEN" | jq .

Expected: { "success": false, "message": "TC is not eligible for AI audit" } with status 400.

Test 5: Invalid Month Format (400)​

curl -s -X POST \
  'http://localhost:3000/dtcea-mumbai-demo/v1/audit/transformers/TC-01/ai-audit?month=2025-13' \
  -H "Authorization: Bearer $TOKEN" | jq .

Expected: 400 validation error — "month must be in YYYY-MM format".

14.3 DB Verification After a Successful Run​

After running Test 1 successfully, verify the DB state:

Check ai-audit-result collection (should have 1 doc)​

db.getCollection("ai-audit-result").find({ tcId: "TC-01", month: "2025-10" }).pretty()

Verify:

  • frozen: false, frozenOn: null
  • original block has the same values as the transformer doc's current audit fields
  • computed block has the projected values (may differ if installations retagged)
  • location.sectionCode: "S01"
  • TC properties snapshot present (tcCapacity, meterConstant, gpsCoordinates, reading, etc.)

Check ai-audit-installation collection (should have N docs = all installations in section S01 for that month)​

// Count total
db.getCollection("ai-audit-installation").countDocuments({
  aiAuditResultId: "<aiAuditResultId-from-response>"
})

// Check classification breakdown
db.getCollection("ai-audit-installation").aggregate([
  { $match: { aiAuditResultId: "<aiAuditResultId-from-response>" } },
  { $group: { _id: { status: "$currentTaggingStatus", suggestion: "$aiSuggestion" }, count: { $sum: 1 } } }
])

Expected groups:

  • { TAGGED, NO_CHANGE } — Set A (tagged to this TC, within 5km)
  • { TAGGED, UNTAG } — Set B (tagged to this TC, outside 5km)
  • { UNTAGGED, TAG } — Set C (not tagged, within 5km)
  • { UNTAGGED, NO_CHANGE } — Remaining (not tagged, outside 5km)

Check remarks analysis​

// Installations with remarks
db.getCollection("ai-audit-installation").countDocuments({
  aiAuditResultId: "<aiAuditResultId-from-response>",
  hasRemark: true
})

// Installations where AI suggests using avg consumption
db.getCollection("ai-audit-installation").countDocuments({
  aiAuditResultId: "<aiAuditResultId-from-response>",
  aiSuggestsConsumption: true
})

Check GPS distances​

// Verify distance values are populated and reasonable
db.getCollection("ai-audit-installation").find(
  { aiAuditResultId: "<aiAuditResultId-from-response>" },
  { accountId: 1, distanceFromTC: 1, withinRange: 1, currentTaggingStatus: 1, aiSuggestion: 1, _id: 0 }
).sort({ distanceFromTC: 1 }).limit(10)

Verify: distanceFromTC is in meters, withinRange is true when distanceFromTC <= 5000.

Verify re-trigger cleanup​

After running Test 2 (re-trigger), verify:

// Should still be exactly 1 result doc (old one deleted, new one created)
db.getCollection("ai-audit-result").countDocuments({ tcId: "TC-01", month: "2025-10" })
// Expected: 1

// The aiAuditResultId should be DIFFERENT from Test 1 (new ObjectId)

14.4 Cleanup After Testing​

To reset the staging data and allow re-testing (including re-testing auditedByAI flag behavior):

// Delete all AI audit staging data for TC-01
db.getCollection("ai-audit-result").deleteMany({ tcId: "TC-01" })
db.getCollection("ai-audit-installation").deleteMany({ tcId: "TC-01" })

// Or nuke all staging data
db.getCollection("ai-audit-result").deleteMany({})
db.getCollection("ai-audit-installation").deleteMany({})