IT audit evidence: measuring controls from live data, not screenshots

IT audit evidence is the proof that a control actually operated during the audit period, not a statement that it exists. The weakest form of that proof is a screenshot taken the week before the auditor arrives. The strongest form is a measurement pulled from the systems themselves, for every machine and every account, with a timestamp, a method and a gap list. This article explains why screenshot evidence keeps failing, what "measured from live data" means in practice, how to handle controls you cannot measure yet, and how to hand an auditor a pack they can open and verify on their own.

Why screenshots fail as IT audit evidence

A screenshot answers one question: what did one screen show at one moment. An auditor needs to answer a different question: did the control work across the whole estate for the whole period.

The gap between those two questions is where most audit findings come from.

• Sample of one. A screenshot of BitLocker turned on for a laptop says nothing about the other four hundred laptops.
• Point in time. It cannot show whether the setting drifted in March and was fixed in April.
• No method. A picture of a console has no record of which query produced it, which filter was applied, or what was excluded.
• Easy to produce, easy to doubt. Screenshots can be cropped, edited or taken from a test environment.
• Expensive to repeat. Every audit cycle restarts the same scramble, run by the people who are busiest keeping the systems up.

The usual response is a shared folder of PNGs and spreadsheets named by control number. It satisfies the request and leaves everyone unsure what it proves.

What "measured from live data" means

Live-data evidence has four properties. If any one is missing, the result is a screenshot in a different format.

1. Population, not sample. The measurement covers every object the control applies to: every endpoint, every account, every share. Where coverage is incomplete, the uncovered part is listed, not hidden.
2. Repeatable method. The same query, run again, gives the same kind of answer. The method is recorded with the result.
3. Timestamped and retained. Each measurement has a time, and past measurements are kept, so the control can be shown operating across the period.
4. Gap list included. The result is not just "pass" or "fail". It is "412 of 420 endpoints compliant; these 8 are not, and here is why".

A simple test: could the auditor re-run this measurement, or watch you re-run it, and expect the same shape of result? If yes, it is evidence. If it depends on someone remembering how they did it last time, it is not.

Examples of measurable controls

The right-hand column is harder to build once and much easier to repeat forever.

"Not measured" is a finding, not a pass

The most common quiet failure in audit programs is treating silence as compliance. A control with no data behind it gets a green mark because nobody found anything wrong, and nobody found anything wrong because nobody looked.

An evidence system needs three states:

• Pass: measured, and the result meets the control.
• Fail: measured, and the result does not meet the control.
• Not measured: no data, or data too old or too partial to support a conclusion.

"Not measured" must surface as prominently as a fail. It should never collapse into pass by default, and never hide behind a filter that only shows controls with results. The honest estate map is the one that shows where you are blind, and auditors trust that map far more than a wall of green.

A related rule: coverage counts. If the collector that produces the measurement is missing from part of the estate, those endpoints are "not measured" even if every other one passes. Report both numbers.

Mapping controls to data sources

Most IT general controls map to data you already generate. A practical mapping for a Windows-centric estate:

• Access control: directory group membership, local admin membership per endpoint, privileged access requests and their expiry, session logs.
• Change management: the console's own admin trail, policy version history, deployment records with install verification.
• Vulnerability and patch management: per-endpoint missing updates, rescan results, time from release to install, exceptions with owners. See patch management rings for why "installed" should only count after a rescan.
• Endpoint protection: encryption status and key escrow, application control policy state, hardening baseline drift. BitLocker inventory with key escrow is a good first control because the data is unambiguous.
• Data protection: classification coverage, DLP policy decisions per channel, incidents with outcomes.
• Backup and recovery: not just that backups ran, but that a restore was tested and when. An untested backup is "not measured" for recovery purposes.
• Logging and monitoring: logs exist, are retained for the required period, and someone reviewed them.

For each control, record the source, the query, the refresh interval and the owner. That table is itself evidence of a working program.

Building the evidence pack

An evidence pack is the deliverable the auditor takes away. Unlike a folder, it can be verified without you in the room.

A good pack has:

• Scope statement: which controls, which period, which population, and what was excluded and why.
• Per-control results: the measurement, the time it was taken, the method, and the pass/fail/not-measured verdict.
• Gap lists: every object that failed or was not measured, with an identifier the auditor can look up.
• Trend across the period: not only the final state but the measurements taken along the way.
• Integrity: the pack is sealed once produced. A hash or signature lets the auditor confirm nothing changed between the time it was generated and the time they opened it.
• Provenance: which system produced it, which version, which account triggered it, and that action recorded in the admin trail.

Keep the pack small enough to open and specific enough to check. An auditor who can spot-check five entries from a gap list against the live system will accept the rest.

Scheduled, not scrambled

Produce the pack on a schedule, not on request. Monthly or quarterly runs cover the period by design, and the delivery history itself proves the evidence process ran continuously.

Checklist: is your IT audit evidence defensible?

• Every control in scope has a named data source and a written query or method.
• The measurement covers the full population, and coverage gaps are reported as "not measured".
• No control is marked pass without a measurement behind it.
• Each result carries a timestamp and past results are retained for the audit period.
• Fail results come with a gap list of specific objects, not a count alone.
• Administrative changes to the systems producing evidence are logged with actor, object and time.
• The evidence pack is sealed with a hash or signature at generation time.
• Packs are produced on a schedule with a delivery history, not assembled by hand before the audit.
• Backup evidence includes a restore test, not only job completion.
• Role-based access limits who can generate, view and export packs, and those actions are themselves audited.
• The auditor can re-run or watch a measurement and get the same shape of result.

If several of these are missing, the program depends on people rather than data, and it will fail when those people are busy or gone.

How Tracemark does it

Tracemark's Governance & Audit product measures IT audit controls from the live data its agent and console already collect across the estate, and it treats "not measured" as its own state that is never a pass.

• Controls measured from live data. Each control is evaluated against data the console already holds from its other products, such as BitLocker state and key escrow, patch status that counts only after a rescan, temporary admin access requests and their expiry, Device Control decisions, application control rules, hardening drift, classification levels and DLP outcomes. A control without a measurement is reported as not measured, beside the pass and fail results. It is never a pass.
• Evidence sealed into a pack. Results are sealed into an evidence pack the auditor can open.
• Scheduled reports with a delivery history. Reports run on a schedule and every delivery is recorded, so the program shows it operated across the period rather than once before the audit.
• An admin trail of every change. Every change made in the console is recorded with who did it, what changed and when. This covers the systems producing the evidence as well as the controls themselves.
• Role-based access. Each role sees only what it can use, so audit evidence is generated and viewed by the people meant to do so, and that access is part of the trail.

Because one agent delivers all nine products, the evidence for patching, encryption, device control, admin rights and classification comes from one source, not from six exports stitched together.

Frequently asked questions

What counts as IT audit evidence?

Anything that shows a control operated during the audit period. Strong evidence is a measurement across the full population, with a timestamp and method, that the auditor can verify. Weak evidence is a single screenshot, a policy document, or a statement that the control exists.

Do auditors accept automated evidence?

Yes, and usually they prefer it, provided the method is documented and the output can be checked. Evidence that covers every endpoint is stronger than a manual sample of ten. The auditor may still ask to see the measurement run.

How should I handle a control I cannot measure yet?

Report it as not measured, say why, and give a date for when measurement will exist. Do not mark it pass. A documented blind spot with a plan is a far smaller finding than a false pass discovered during testing.

How long should evidence be retained?

At least for the full audit period plus the auditor's review time, and often longer under your own policy or regulation. Keep measurements as they were taken rather than overwriting them, so the control can be shown operating over time.

Try it

Try Tracemark free for three months. Claim it at tracemark.ca