Skip to main content

LLD: Backfill Average Consumption for Remarked Installations

1. Objective​

Pre-compute the 6-month rolling average consumption for every installation that has a billing remark (unbilled, MNR, vacant, door lock, zero consumption, abnormal, subnormal). The computed value is stored in the avgConsumption6Months field on the installation document and is consumed by the AI Audit feature to suggest consumption overrides.

2. Background​

The AI Audit flow uses avgConsumption6Months in two places:

  1. AI Audit Run — determines aiSuggestsConsumption flag: 
    aiSuggestsConsumption = hasRemark && avgConsumption6Months != null && avgConsumption6Months > 0
  2. AI Audit Freeze — when a user accepts the AI suggestion, the freeze endpoint substitutes the installation's reported consumption with avgConsumption6Months for the reaudit computation.

Without this backfill, all remarked installations would have avgConsumption6Months: null and the AI could never suggest consumption overrides.

3. Script​

Config name:  installation_backfillAvgConsumption
File:         script/src/entities/installation/backfillAvgConsumption.js

In config.json:

{
  "scriptRunArray": ["installation_backfillAvgConsumption"],
  "input": {
    "installation": {
      "backfillAvgConsumption": {
        "months": ["2025-09", "2025-10", "2025-11", "2025-12", "2026-01", "2026-02", "2026-03"]
      }
    }
  }
}

If months is omitted or null, the script auto-discovers all distinct months from the installation collection.

Exports​

export const tags = [];

export async function mainFunction(input) { ... }

4. Remark Detection​

An installation "has a remark" if any of the following billing flags are set:

FlagCondition
unbilled=== true
mnr=== true
vacant=== true
doorLock=== true
zeroConsumption=== true
emin (abnormal)!= null && > 0
emax (subnormal)!= null && > 0

MongoDB filter for remarked installations:

{
  month: M,
  $or: [
    { unbilled: true },
    { mnr: true },
    { vacant: true },
    { doorLock: true },
    { zeroConsumption: true },
    { emin: { $gt: 0 } },
    { emax: { $gt: 0 } },
  ],
}

Helper function:

function hasRemark(inst) {
  return (
    inst.unbilled === true ||
    inst.mnr === true ||
    inst.vacant === true ||
    inst.doorLock === true ||
    inst.zeroConsumption === true ||
    (inst.emin != null && inst.emin > 0) ||
    (inst.emax != null && inst.emax > 0)
  );
}

5. Average Consumption Formula​

For a given installation (accountId) in month M:

previousMonths = [M-1, M-2, M-3, M-4, M-5, M-6]

validConsumptions = installation docs for this accountId in previousMonths
                    WHERE consumption != null AND consumption > 0

avgConsumption6Months = sum(validConsumptions) / count(validConsumptions)
                        rounded to 2 decimal places

If count(validConsumptions) === 0 → avgConsumption6Months = 0

Key rules:

  • Only months with non-null, non-zero consumption contribute to the average
  • The denominator is the number of months with data, not always 6
  • If an installation exists for only 3 of the 6 previous months, average over those 3
  • If the data starts at 2025-09 and we're processing 2025-09, there are 0 prior months -- result is 0

Month Utility​

function getPrevious6Months(month) {
  const [year, mon] = month.split("-").map(Number);
  const months = [];
  for (let i = 1; i <= 6; i++) {
    let m = mon - i;
    let y = year;
    if (m <= 0) { m += 12; y -= 1; }
    months.push(`${y}-${String(m).padStart(2, "0")}`);
  }
  return months;
}

Example: getPrevious6Months("2026-01") returns ["2025-12", "2025-11", "2025-10", "2025-09", "2025-08", "2025-07"].

6. Algorithm​

Phase 1: Resolve target months​

Input: config.months (optional string array, e.g., ["2025-09", "2025-10"])
If not provided → query installationModel.distinct("month") to get all available months
Sort ascending so earlier months are processed first (order doesn't affect correctness)

Phase 2: For each target month, find remarked installations​

const remarked = await installationModel.find(
  {
    month,
    $or: [
      { unbilled: true },
      { mnr: true },
      { vacant: true },
      { doorLock: true },
      { zeroConsumption: true },
      { emin: { $gt: 0 } },
      { emax: { $gt: 0 } },
    ],
  },
  { accountId: 1, _id: 0 },
).lean();

const remarkedAccountIds = remarked.map((d) => d.accountId);

Log: [2025-10] Phase 2: Found 85,432 remarked installations

Phase 3: Fetch historical consumption data in bulk​

const previousMonths = getPrevious6Months(month);

const historicalDocs = await installationModel.find(
  {
    accountId: { $in: remarkedAccountIds },
    month: { $in: previousMonths },
    consumption: { $gt: 0 },
  },
  { accountId: 1, consumption: 1, _id: 0 },
).lean();

This is one query per month that fetches all historical data at once instead of per-installation queries.

Log: [2025-10] Phase 3: Fetched 287,104 historical docs across 6 previous months

Phase 4: Compute averages in memory​

// Group historical docs by accountId
const historyByAccount = new Map();
for (const doc of historicalDocs) {
  if (!historyByAccount.has(doc.accountId)) {
    historyByAccount.set(doc.accountId, []);
  }
  historyByAccount.get(doc.accountId).push(doc.consumption);
}

// Build bulk update operations
const bulkOps = [];
let nonZeroAvgCount = 0;
let zeroAvgCount = 0;

for (const accountId of remarkedAccountIds) {
  const consumptions = historyByAccount.get(accountId) || [];

  let avg = 0;
  if (consumptions.length > 0) {
    const sum = consumptions.reduce((a, b) => a + b, 0);
    avg = Math.round((sum / consumptions.length) * 100) / 100;
  }

  if (avg > 0) nonZeroAvgCount++;
  else zeroAvgCount++;

  bulkOps.push({
    updateOne: {
      filter: { accountId, month },
      update: { $set: { avgConsumption6Months: avg } },
    },
  });
}

Log: [2025-10] Phase 4: Computed averages — 72,891 with data, 12,541 with zero avg

Phase 5: Bulk write updates​

if (bulkOps.length > 0) {
  const result = await installationModel.bulkWrite(bulkOps, { ordered: false });
  log.info(`[${month}] Phase 5: bulkWrite — matched: ${result.matchedCount}, modified: ${result.modifiedCount}`);
}

7. Internal Structure​

mainFunction(input)
├── resolveMonths(input.months)                   // Phase 1
├── for each month:
│   ├── findRemarkedInstallations(month)          // Phase 2
│   ├── fetchHistoricalConsumption(accountIds)    // Phase 3
│   ├── computeAverages(accountIds, historicalDocs) // Phase 4
│   └── bulkWriteUpdates(bulkOps)                 // Phase 5
└── log final summary

Key functions​

getPrevious6Months(month) --> string[]​

Returns the 6 month strings preceding the given month. Handles year boundaries.

hasRemark(inst) --> boolean​

Returns true if the installation has any billing remark flag set.

findRemarkedInstallations(month) --> string[]​

Queries the installation collection for all accountIds in the given month that have at least one remark flag. Returns an array of accountId strings.

fetchHistoricalConsumption(accountIds, previousMonths) --> Map<string, number[]>​

Queries the installation collection for historical docs and groups them by accountId. Returns a Map where each key is an accountId and the value is an array of non-zero consumption values.

computeAverage(consumptions) --> number​

Computes the average of an array of consumption values, rounded to 2 decimal places. Returns 0 if the array is empty.

8. Fields Touched​

CollectionFieldActionReason
installationavgConsumption6MonthsSet to computed average (or 0)Pre-computed value for AI Audit

Fields NOT touched​

  • All other installation fields (consumption, unbilled, mnr, vacant, doorLock, zeroConsumption, emin, emax, tcNumber, location, etc.) are read-only inputs
  • updatedAt / createdAt — will be updated by Mongoose timestamps (acceptable since this is a legitimate data backfill)

9. Edge Cases​

ScenarioHandling
Month "2025-09" (first month in dataset)0 prior months have data, avgConsumption6Months = 0
Installation exists for only 3 of 6 prior monthsAverage over those 3 (denominator = 3, not 6)
All prior months have 0 or null consumptionavgConsumption6Months = 0
Installation has no docs in any prior monthavgConsumption6Months = 0
No remarked installations for a monthLog warning, skip to next month
Script re-runIdempotent — overwrites avgConsumption6Months with freshly computed values

10. Performance Estimate​

OperationData sizeExpected time
Find remarked installations (per month)~330K–660K installations, 10–20% remarked~2–5 seconds
Fetch historical docs (per month)~200K–400K docs across 6 months~5–15 seconds
In-memory grouping + average computation~33K–66K accountIds<1 second
bulkWrite (per month)~33K–66K updateOne ops~5–10 seconds
Total (7 months)~1.5–3.5 minutes

Notes:

  • The data covers 7 months (2025-09 to 2026-03) with ~3.3M total installations
  • Only remarked installations are processed (estimated 10–20% of total)
  • The { accountId: 1, month: 1 } unique index supports efficient lookups for both the historical query and the bulkWrite filter
  • ordered: false on bulkWrite allows parallel execution of update operations

11. Dependencies​

DependencyImport path
installationModel../../../../common/src/schemas/installation.model.js
log../../utils/logger.js
assignDefaults../../utils/functions.js

No new dependencies required.

12. Error Handling​

ScenarioHandling
No installation docs for a monthLog warning, skip that month
No remarked installations for a monthLog info, skip to next month (no bulkWrite)
Historical query returns 0 docsAll averages will be 0, which is correct
bulkWrite partial failureordered: false ensures other updates still apply; log failures
Script re-runIdempotent — $set overwrites previous values

13. Logging​

Each month should produce log output like:

[2025-09] Phase 2: Found 54,321 remarked installations
[2025-09] Phase 3: Fetched 0 historical docs across 6 previous months (months: 2025-03..2025-08)
[2025-09] Phase 4: Computed averages — 0 with data, 54,321 with zero avg
[2025-09] Phase 5: bulkWrite — matched: 54,321, modified: 54,321

[2025-10] Phase 2: Found 58,102 remarked installations
[2025-10] Phase 3: Fetched 142,587 historical docs across 6 previous months (months: 2025-04..2025-09)
[2025-10] Phase 4: Computed averages — 41,203 with data, 16,899 with zero avg
[2025-10] Phase 5: bulkWrite — matched: 58,102, modified: 58,102
...

Summary: Processed 7 months, updated 412,847 installations total

14. Testing Strategy​

After running the script, execute the following tests. All tests must pass.

Connection:

mongosh "<MONGO_CONNECTION_STRING>" --db dtcea-mumbai-demo --quiet

Test 1: All remarked installations have avgConsumption6Months set​

Verify that no remarked installation has a null avgConsumption6Months after the backfill.

var remarkedWithoutAvg = db.installation.countDocuments({
  $or: [
    { unbilled: true },
    { mnr: true },
    { vacant: true },
    { doorLock: true },
    { zeroConsumption: true },
    { emin: { $gt: 0 } },
    { emax: { $gt: 0 } },
  ],
  avgConsumption6Months: null,
});

print("Remarked installations missing avgConsumption6Months:", remarkedWithoutAvg);
print(remarkedWithoutAvg === 0 ? "PASS" : "FAIL");

Pass criteria: remarkedWithoutAvg === 0. Every remarked installation must have a value (0 or positive).

Test 2: Non-remarked installations are untouched​

Verify that installations without any remark flag still have avgConsumption6Months: null (or the field does not exist).

var nonRemarkedWithAvg = db.installation.countDocuments({
  unbilled: { $ne: true },
  mnr: { $ne: true },
  vacant: { $ne: true },
  doorLock: { $ne: true },
  zeroConsumption: { $ne: true },
  $and: [
    { $or: [{ emin: null }, { emin: { $exists: false } }, { emin: { $lte: 0 } }] },
    { $or: [{ emax: null }, { emax: { $exists: false } }, { emax: { $lte: 0 } }] },
  ],
  avgConsumption6Months: { $ne: null, $exists: true },
});

print("Non-remarked installations with avgConsumption6Months set:", nonRemarkedWithAvg);
print(nonRemarkedWithAvg === 0 ? "PASS" : "FAIL");

Pass criteria: nonRemarkedWithAvg === 0. The script should not touch non-remarked installations.

Test 3: First month (2025-09) should have all zeros​

Since 2025-09 is the first month in the dataset, there is no prior data. All remarked installations in this month should have avgConsumption6Months === 0.

var nonZeroInFirstMonth = db.installation.countDocuments({
  month: "2025-09",
  $or: [
    { unbilled: true },
    { mnr: true },
    { vacant: true },
    { doorLock: true },
    { zeroConsumption: true },
    { emin: { $gt: 0 } },
    { emax: { $gt: 0 } },
  ],
  avgConsumption6Months: { $gt: 0 },
});

print("Remarked installations in 2025-09 with non-zero avg:", nonZeroInFirstMonth);
print(nonZeroInFirstMonth === 0 ? "PASS" : "FAIL");

Pass criteria: nonZeroInFirstMonth === 0.

Test 4: Spot-check a single installation's average​

Pick one remarked installation from a later month and manually verify the average computation.

// Find a remarked installation in 2026-01 with a non-zero average
var sample = db.installation.findOne({
  month: "2026-01",
  avgConsumption6Months: { $gt: 0 },
  mnr: true,
});

if (!sample) {
  print("No sample found — try a different month or remark type");
} else {
  var prevMonths = ["2025-12", "2025-11", "2025-10", "2025-09", "2025-08", "2025-07"];

  var historicalDocs = db.installation.find({
    accountId: sample.accountId,
    month: { $in: prevMonths },
    consumption: { $gt: 0 },
  }).toArray();

  var consumptions = historicalDocs.map(d => d.consumption);
  var expectedAvg = consumptions.length > 0
    ? Math.round((consumptions.reduce((a, b) => a + b, 0) / consumptions.length) * 100) / 100
    : 0;

  print("Account:", sample.accountId, "Month:", sample.month);
  print("Historical months with data:", consumptions.length);
  print("Consumptions:", JSON.stringify(consumptions));
  print("Expected avg:", expectedAvg, "Stored avg:", sample.avgConsumption6Months);

  var pass = sample.avgConsumption6Months === expectedAvg;
  print(pass ? "PASS" : "FAIL");
}

Pass criteria: Stored avgConsumption6Months matches the manually computed average.

Test 5: avgConsumption6Months is never negative​

Consumption values should always be positive, so the average should never be negative.

var negativeAvg = db.installation.countDocuments({
  avgConsumption6Months: { $lt: 0 },
});

print("Installations with negative avgConsumption6Months:", negativeAvg);
print(negativeAvg === 0 ? "PASS" : "FAIL");

Pass criteria: negativeAvg === 0.

Test 6: Later months have more non-zero averages than earlier months​

As more historical data becomes available, more installations should have non-zero averages. Verify the trend is non-decreasing.

var months = db.installation.distinct("month").sort();
var prevCount = 0;
var pass = true;

months.forEach(m => {
  var count = db.installation.countDocuments({
    month: m,
    avgConsumption6Months: { $gt: 0 },
  });
  if (count < prevCount) {
    print("FAIL:", m, "has", count, "non-zero avgs, less than previous month's", prevCount);
    pass = false;
  }
  print(m, ":", count, "installations with non-zero avg");
  prevCount = count;
});

print(pass ? "PASS" : "FAIL");

Pass criteria: Non-zero average count is non-decreasing across months. Prints PASS.

Test 7: Cross-validate total remarked count per month​

Verify the script processed the expected number of installations per month by comparing remarked counts.

var months = db.installation.distinct("month").sort();

months.forEach(m => {
  var remarkedCount = db.installation.countDocuments({
    month: m,
    $or: [
      { unbilled: true },
      { mnr: true },
      { vacant: true },
      { doorLock: true },
      { zeroConsumption: true },
      { emin: { $gt: 0 } },
      { emax: { $gt: 0 } },
    ],
  });

  var withAvg = db.installation.countDocuments({
    month: m,
    avgConsumption6Months: { $exists: true, $ne: null },
  });

  var match = remarkedCount === withAvg;
  print(m, "- remarked:", remarkedCount, "with avg:", withAvg, match ? "OK" : "FAIL");
});

Pass criteria: For every month, the remarked count equals the count of docs with avgConsumption6Months set.

Test Summary​

#TestWhat it validates
1Remarked installations have avg setNo remarked installation left with null
2Non-remarked installations untouchedScript only affects remarked installations
3First month has all zerosCorrect behavior when no prior data exists
4Spot-check single installationManual verification of the average formula
5No negative averagesSanity check on computed values
6Non-zero avgs increase over timeMore historical data produces more non-zero averages
7Remarked count matches avg countEvery remarked installation was processed

All 7 tests must print PASS for the script to be considered successful.

15. Pre-Run Sample IDs (100 Remarked Installations)​

Captured before the script was run. Use these to verify avgConsumption6Months was set correctly after execution.

Note: Only months 2025-09, 2025-10, 2025-11 have remarked installations in the current DB. Month 2025-12 has data but no remark flags set yet.

2025-09 (30 samples — expect avgConsumption6Months = 0, no prior data)​

_idaccountId
69c3b092c68efe0446c1ea680001000012
69c3b092c68efe0446c1ea6b0001000003
69c3b092c68efe0446c1ea880001000052
69c3b092c68efe0446c1ea8d0001000112
69c3b092c68efe0446c1ea9a0001000123
69c3b092c68efe0446c1eaa40001000090
69c3b092c68efe0446c1eaaf0001000148
69c3b092c68efe0446c1eab70001000147
69c3b092c68efe0446c1eab90001000152
69c3b092c68efe0446c1eaba0001000162
69c3b092c68efe0446c1eac10001000138
69c3b092c68efe0446c1eacf0001000169
69c3b092c68efe0446c1ead10001000182
69c3b092c68efe0446c1eae10001000253
69c3b092c68efe0446c1eaf30001000201
69c3b092c68efe0446c1eb000001000245
69c3b092c68efe0446c1eb0f0001000244
69c3b092c68efe0446c1eb160001000265
69c3b092c68efe0446c1eb1b0001000273
69c3b092c68efe0446c1eb1e0001000279
69c3b092c68efe0446c1eb240001000301
69c3b092c68efe0446c1eb2a0001000292
69c3b092c68efe0446c1eb2b0001000300
69c3b092c68efe0446c1eb2f0001000258
69c3b092c68efe0446c1eb300001000259
69c3b092c68efe0446c1eb330001000272
69c3b092c68efe0446c1eb490001000296
69c3b092c68efe0446c1eb960001000396
69c3b092c68efe0446c1eba80001000447
69c3b092c68efe0446c1ebb00001000416

2025-10 (35 samples — expect avgConsumption6Months >= 0, up to 1 prior month)​

_idaccountId
69c3b23cb87afb735d12bfd10001000003
69c3b23cb87afb735d12bfee0001000052
69c3b23cb87afb735d12bff30001000112
69c3b23cb87afb735d12bffc0001000101
69c3b23cb87afb735d12bffe0001000109
69c3b23cb87afb735d12c0000001000123
69c3b23cb87afb735d12c0150001000148
69c3b23cb87afb735d12c01d0001000147
69c3b23cb87afb735d12c01f0001000152
69c3b23cb87afb735d12c0200001000162
69c3b23cb87afb735d12c0270001000138
69c3b23cb87afb735d12c02c0001000130
69c3b23cb87afb735d12c0310001000157
69c3b23cb87afb735d12c0350001000169
69c3b23cb87afb735d12c0370001000182
69c3b23cb87afb735d12c0440001000239
69c3b23cb87afb735d12c0460001000250
69c3b23cb87afb735d12c0470001000253
69c3b23cb87afb735d12c0590001000201
69c3b23cb87afb735d12c0630001000238
69c3b23cb87afb735d12c0660001000245
69c3b23cb87afb735d12c0750001000244
69c3b23cb87afb735d12c0810001000273
69c3b23cb87afb735d12c0840001000279
69c3b23cb87afb735d12c08a0001000301
69c3b23cb87afb735d12c0900001000292
69c3b23cb87afb735d12c0910001000300
69c3b23cb87afb735d12c0950001000258
69c3b23cb87afb735d12c0960001000259
69c3b23cb87afb735d12c0990001000272
69c3b23cb87afb735d12c09d0001000283
69c3b23cb87afb735d12c0af0001000296
69c3b23cb87afb735d12c0fc0001000396
69c3b23cb87afb735d12c10e0001000447
69c3b23cb87afb735d12c1160001000416

2025-11 (35 samples — expect avgConsumption6Months >= 0, up to 2 prior months)​

_idaccountId
69c3b4a29c05c630e7dbd7940001000012
69c3b4a29c05c630e7dbd7950001000001
69c3b4a29c05c630e7dbd7960001000002
69c3b4a29c05c630e7dbd7970001000003
69c3b4a29c05c630e7dbd7980001000004
69c3b4a29c05c630e7dbd7990001000008
69c3b4a29c05c630e7dbd79a0001000009
69c3b4a29c05c630e7dbd79b0001000011
69c3b4a29c05c630e7dbd79c0001000039
69c3b4a29c05c630e7dbd79d0001000050
69c3b4a29c05c630e7dbd79e0001000053
69c3b4a29c05c630e7dbd79f0001000058
69c3b4a29c05c630e7dbd7a00001000005
69c3b4a29c05c630e7dbd7a20001000014
69c3b4a29c05c630e7dbd7a30001000015
69c3b4a29c05c630e7dbd7a50001000051
69c3b4a29c05c630e7dbd7a60001000054
69c3b4a29c05c630e7dbd7a70001000056
69c3b4a29c05c630e7dbd7a80001000060
69c3b4a29c05c630e7dbd7aa0001000007
69c3b4a29c05c630e7dbd7ab0001000010
69c3b4a29c05c630e7dbd7ad0001000047
69c3b4a29c05c630e7dbd7ae0001000057
69c3b4a29c05c630e7dbd7af0001000059
69c3b4a29c05c630e7dbd7b00001000062
69c3b4a29c05c630e7dbd7b10001000013
69c3b4a29c05c630e7dbd7b20001000041
69c3b4a29c05c630e7dbd7b30001000042
69c3b4a29c05c630e7dbd7b40001000052
69c3b4a29c05c630e7dbd7b50001000067
69c3b4a29c05c630e7dbd7b60001000068
69c3b4a29c05c630e7dbd7b70001000094
69c3b4a29c05c630e7dbd7b80001000106
69c3b4a29c05c630e7dbd7b90001000112
69c3b4a29c05c630e7dbd7ba0001000116

Post-Run Verification Query​

After running the script, verify all 100 samples have avgConsumption6Months set:

var sampleIds = [
  ObjectId("69c3b092c68efe0446c1ea68"), ObjectId("69c3b092c68efe0446c1ea6b"),
  ObjectId("69c3b092c68efe0446c1ea88"), ObjectId("69c3b092c68efe0446c1ea8d"),
  ObjectId("69c3b092c68efe0446c1ea9a"), ObjectId("69c3b092c68efe0446c1eaa4"),
  ObjectId("69c3b092c68efe0446c1eaaf"), ObjectId("69c3b092c68efe0446c1eab7"),
  ObjectId("69c3b092c68efe0446c1eab9"), ObjectId("69c3b092c68efe0446c1eaba"),
  ObjectId("69c3b092c68efe0446c1eac1"), ObjectId("69c3b092c68efe0446c1eacf"),
  ObjectId("69c3b092c68efe0446c1ead1"), ObjectId("69c3b092c68efe0446c1eae1"),
  ObjectId("69c3b092c68efe0446c1eaf3"), ObjectId("69c3b092c68efe0446c1eb00"),
  ObjectId("69c3b092c68efe0446c1eb0f"), ObjectId("69c3b092c68efe0446c1eb16"),
  ObjectId("69c3b092c68efe0446c1eb1b"), ObjectId("69c3b092c68efe0446c1eb1e"),
  ObjectId("69c3b092c68efe0446c1eb24"), ObjectId("69c3b092c68efe0446c1eb2a"),
  ObjectId("69c3b092c68efe0446c1eb2b"), ObjectId("69c3b092c68efe0446c1eb2f"),
  ObjectId("69c3b092c68efe0446c1eb30"), ObjectId("69c3b092c68efe0446c1eb33"),
  ObjectId("69c3b092c68efe0446c1eb49"), ObjectId("69c3b092c68efe0446c1eb96"),
  ObjectId("69c3b092c68efe0446c1eba8"), ObjectId("69c3b092c68efe0446c1ebb0"),
  ObjectId("69c3b23cb87afb735d12bfd1"), ObjectId("69c3b23cb87afb735d12bfee"),
  ObjectId("69c3b23cb87afb735d12bff3"), ObjectId("69c3b23cb87afb735d12bffc"),
  ObjectId("69c3b23cb87afb735d12bffe"), ObjectId("69c3b23cb87afb735d12c000"),
  ObjectId("69c3b23cb87afb735d12c015"), ObjectId("69c3b23cb87afb735d12c01d"),
  ObjectId("69c3b23cb87afb735d12c01f"), ObjectId("69c3b23cb87afb735d12c020"),
  ObjectId("69c3b23cb87afb735d12c027"), ObjectId("69c3b23cb87afb735d12c02c"),
  ObjectId("69c3b23cb87afb735d12c031"), ObjectId("69c3b23cb87afb735d12c035"),
  ObjectId("69c3b23cb87afb735d12c037"), ObjectId("69c3b23cb87afb735d12c044"),
  ObjectId("69c3b23cb87afb735d12c046"), ObjectId("69c3b23cb87afb735d12c047"),
  ObjectId("69c3b23cb87afb735d12c059"), ObjectId("69c3b23cb87afb735d12c063"),
  ObjectId("69c3b23cb87afb735d12c066"), ObjectId("69c3b23cb87afb735d12c075"),
  ObjectId("69c3b23cb87afb735d12c081"), ObjectId("69c3b23cb87afb735d12c084"),
  ObjectId("69c3b23cb87afb735d12c08a"), ObjectId("69c3b23cb87afb735d12c090"),
  ObjectId("69c3b23cb87afb735d12c091"), ObjectId("69c3b23cb87afb735d12c095"),
  ObjectId("69c3b23cb87afb735d12c096"), ObjectId("69c3b23cb87afb735d12c099"),
  ObjectId("69c3b23cb87afb735d12c09d"), ObjectId("69c3b23cb87afb735d12c0af"),
  ObjectId("69c3b23cb87afb735d12c0fc"), ObjectId("69c3b23cb87afb735d12c10e"),
  ObjectId("69c3b23cb87afb735d12c116"), ObjectId("69c3b4a29c05c630e7dbd794"),
  ObjectId("69c3b4a29c05c630e7dbd795"), ObjectId("69c3b4a29c05c630e7dbd796"),
  ObjectId("69c3b4a29c05c630e7dbd797"), ObjectId("69c3b4a29c05c630e7dbd798"),
  ObjectId("69c3b4a29c05c630e7dbd799"), ObjectId("69c3b4a29c05c630e7dbd79a"),
  ObjectId("69c3b4a29c05c630e7dbd79b"), ObjectId("69c3b4a29c05c630e7dbd79c"),
  ObjectId("69c3b4a29c05c630e7dbd79d"), ObjectId("69c3b4a29c05c630e7dbd79e"),
  ObjectId("69c3b4a29c05c630e7dbd79f"), ObjectId("69c3b4a29c05c630e7dbd7a0"),
  ObjectId("69c3b4a29c05c630e7dbd7a2"), ObjectId("69c3b4a29c05c630e7dbd7a3"),
  ObjectId("69c3b4a29c05c630e7dbd7a5"), ObjectId("69c3b4a29c05c630e7dbd7a6"),
  ObjectId("69c3b4a29c05c630e7dbd7a7"), ObjectId("69c3b4a29c05c630e7dbd7a8"),
  ObjectId("69c3b4a29c05c630e7dbd7aa"), ObjectId("69c3b4a29c05c630e7dbd7ab"),
  ObjectId("69c3b4a29c05c630e7dbd7ad"), ObjectId("69c3b4a29c05c630e7dbd7ae"),
  ObjectId("69c3b4a29c05c630e7dbd7af"), ObjectId("69c3b4a29c05c630e7dbd7b0"),
  ObjectId("69c3b4a29c05c630e7dbd7b1"), ObjectId("69c3b4a29c05c630e7dbd7b2"),
  ObjectId("69c3b4a29c05c630e7dbd7b3"), ObjectId("69c3b4a29c05c630e7dbd7b4"),
  ObjectId("69c3b4a29c05c630e7dbd7b5"), ObjectId("69c3b4a29c05c630e7dbd7b6"),
  ObjectId("69c3b4a29c05c630e7dbd7b7"), ObjectId("69c3b4a29c05c630e7dbd7b8"),
  ObjectId("69c3b4a29c05c630e7dbd7b9"), ObjectId("69c3b4a29c05c630e7dbd7ba"),
];

// All 100 should have avgConsumption6Months set (not null)
var missing = db.installation.countDocuments({
  _id: { $in: sampleIds },
  $or: [
    { avgConsumption6Months: null },
    { avgConsumption6Months: { $exists: false } },
  ],
});
print("Sample IDs missing avgConsumption6Months:", missing, "/ 100");
print(missing === 0 ? "PASS" : "FAIL");

// 2025-09 samples should all be 0
var sept = sampleIds.slice(0, 30);
var septNonZero = db.installation.countDocuments({
  _id: { $in: sept },
  avgConsumption6Months: { $gt: 0 },
});
print("2025-09 samples with non-zero avg:", septNonZero, "/ 30");
print(septNonZero === 0 ? "PASS" : "FAIL");