Design SIEM log retention around investigation needs. Use an eight-field worksheet and six-step retrieval test to assess search windows, late logs, access and cost.
Choosing a SIEM log retention period involves more than deciding how many days to keep data. An investigation can stall even when the logs still exist: the analyst may lack retrieval permissions, required fields may be missing, or nobody may know how long retrieval takes.
This guide helps IT and SOC teams design retention and retrieval together. Our SIEM comparison and cost guide covers product costs, while the log ingestion acceptance checklist covers initial integration checks. Here, the focus is investigating historical events after operations begin.
Start with the investigation window and acceptable waiting time
Define investigation requirements before choosing product settings. For each source, such as identity, endpoint or cloud audit logs, answer three questions:
How far back must you investigate? Identify what you need to establish about the period before a suspicious event was discovered.
When are results needed? Decide whether responders need immediate results or whether a next-business-day investigation is acceptable.
What must the evidence explain? Identify the fields that establish who did what, when and to which resource.
The joint logging guidance published by ACSC and partner agencies recommends determining retention from risk and applicable requirements, including the possibility of logs being overwritten when storage fills. A product default is not automatically your requirement. Source: Best practices for event logging and threat detection
Keeping everything longer can increase costs and access-management work. Check business needs, internal policies, contracts and applicable requirements, and document the purpose of retaining each type of information. This guide does not prescribe one retention period for every organization.
Separate readily searchable data from longer-term storage
Record these two periods separately in your requirements:
Readily searchable period: The period for which analysts can obtain results through their normal investigation interface within the required time.
Total retention period: The period for which data remains available for investigation, including data requiring a separate search job or retrieval procedure.
Product terminology and behavior differ. Azure Monitor Logs distinguishes analytics retention from long-term retention according to the table plan. Long-term Analytics and Basic data is available through search jobs, while Auxiliary data remains queryable throughout its total retention period. Do not assume every archive requires the same restoration procedure. Source: Azure Monitor Logs retention documentation
Be able to explain which logs can be searched, by whom, through which procedure and after how much waiting. Agree on retrieval time separately from retention duration, then measure it.
Eight fields for a log retention requirements worksheet
Use the following fields to organize requirements. Mark unknown values for confirmation and assign an owner instead of inventing numbers.
Source and scope: Product, accounts and devices covered, and excluded environments.
Investigation scenario: The activity to explain, such as suspicious authentication, permission changes or bulk access.
Required fields: Event time, user ID, resource, action, result and original event identifier.
Lookback requirement: How far back the investigation must reach, its justification and its approver.
Retrieval method and target time: Normal search, search job or restoration, and the agreed time to obtain results.
Retention clock: Whether retention starts at event time or ingestion time, including treatment of late arrivals.
Retrieval and preservation owners: Operator, approver, backup operator, permissions and destination for retrieved data.
Verification record: Last successful retrieval, measured results, evidence, unresolved gaps and next review date.
For an authentication investigation, counts of successful and failed sign-ins may be insufficient. You may need identifiers that link users and devices across sources, including after a username changes. Include time zones and validate required fields against your own investigation scenario.
Check the retention clock and late-arriving events
A stated retention period of 180 days can mean different remaining lifetimes for a delayed event depending on whether the clock starts when the event occurred or when it was ingested.
Google Security Operations documents retention based on ingestion time for raw logs and event time for UDM events. Confirm which data you are searching before estimating deletion dates from ingestion alone. Source: Google Security Operations data retention
Record event and ingestion timestamps separately. Test whether events resent after a forwarding interruption, and events affected by clock skew, appear in the intended search window. Extending retention does not recover data that has already been deleted. Microsoft specifies that an increase applies to previously ingested data that has not yet been removed.
A six-step historical log retrieval test
The following is an example for operational acceptance and configuration changes. Agree on frequency and pass criteria for your environment. Use approved test environments and data for tests involving configuration changes or deletion.
Select identifiable events. Include events within the normal search window, in longer-term storage and near the retention boundary. Record expected timestamps, counts and IDs first.
Use the assigned operator's permissions. An administrator-only test does not validate the on-call process. Check that the designated operator can complete the documented request, approval and retrieval steps.
Record queries and elapsed time. Capture request receipt, approval, processing start, search availability and investigation completion separately. Distinguish permission delays, system processing and operator work.
Reconcile the contents. Verify IDs, event time, ingestion time, required fields and counts, not just the presence of some results. Zero results require checking scope, permissions, time filters and missing data before concluding that no event occurred.
Correlate across sources. Follow the same user or device through identity, endpoint and cloud logs. Retrieving one source is insufficient if the other sources needed for the investigation do not cover the same period.
Account for costs and cleanup. Record search, retrieval and transfer quantities, operator work, destination permissions and retention or deletion procedures for copies. Follow evidence-preservation procedures where required.
A pass means the required events and fields can be investigated by the designated operator within the agreed time. A successful operation alone is insufficient. If the test fails, record whether the cause is missing logs, retrieval method, permissions, capacity or staffing rather than silently reducing the requirement.
Separate volume estimates from billable quantities
Consider a hypothetical average of 20 GB per day, total retention of 180 days and a readily searchable window of 30 days. Ignoring compression, deletion, duplication and format changes, the simple totals are 20 × 180 = 3,600 GB overall, 600 GB for the latest 30 days and 3,000 GB for the remaining 150 days. These assumptions illustrate the calculation; they are neither recommended retention periods nor a product price quote.
Actual storage and billing depend on compression, indexes, replicas, allowances, contractual retention arrangements and search or retrieval frequency. Check whether data moves between storage tiers or is copied into multiple stores. Measure peak ingestion and query load, then estimate capacity and cost separately.
Checks before changing retention settings
Shortening retention: Check active investigations, preserved evidence and internal or contractual requirements.
Extending retention: Establish which existing data is affected and confirm that deleted data is excluded.
Moving storage: Review retrieval procedures, permissions, encryption keys, transfer costs and exit arrangements.
Growing log volume: Check whether capacity limits could be reached before the retention deadline and who detects collection failures or gaps.
Repeat the same tests before and after a change. Update the justification, review date and owners in the worksheet. Preserve actual search results and retrieval records alongside configuration screenshots so the next operator can understand the evidence.
What to prepare for an implementation discussion
Bring a one-page summary of products, daily volume by source, historical investigation scenarios, acceptable waiting time and current retention and search methods. List unknowns as measurements or checks to complete.
If you are evaluating Yagura AI SIEM or AI SOC, use this worksheet in an implementation discussion to confirm source coverage, retention and retrieval conditions, and human responsibilities. This article does not guarantee a particular retention duration or retrieval time for Yagura products.
Related reading: Compare SIEM costs on equivalent assumptions / Test log ingestion before monitoring begins
The Japanese source article was researched on October 13, 2026. Microsoft and Google retention documentation was rechecked on October 16, 2026 for this translation. Product plans and behavior can change; check current documentation and your environment before configuration or contract decisions.
