Microsoft Sentinel: Analytic vs Basic vs Archive (2024)

Below is an article about three different types of logs that can be ingested and retained in Microsoft Sentinel's Log Analytics Workspace.

Pre-requisites

• Basic Knowledge around Microsoft Sentinel and Log Analytics Workspace

Overview

Microsoft Sentinel is considered to be the so called "expensive" SIEM platform in the market at the moment (which isn't true in the entirety). Microsoft Sentinel isn't an old-school traditional SIEM where it is feasible to dump all logs in one place and tick the checkbox for compliance.

Sentinel (and few other modern cloud SIEM platforms) follows a different approach where - you ingest "only the logs that are needful", rather than treating it as a storage box.

Logs from Log Analytics Workspace are primarily used by 3 components of Microsoft Sentinel:

• Analytic Rules
• Workbooks
• Hunting Queries (or manual threat hunting in Logs)

Analytic Logs

Analytic logs is the primary log type in Microsoft Sentinel. Until a year (or more) ago, Analytic logs was the only type of log that was supported in Log Analytics Workspace. Analytic Logs can be treated as the "hot storage" and has no restriction on using/manipulating the logs in Microsoft Sentinel.

Ingestion Cost

Below shows the "Ingestion" cost of 10GB/day of Analytic Log for 30 days:

See Also

Query SQL Database with query editor in the Azure portal - Azure SQL DatabaseAzure Monitor data platform - Azure MonitorSelect a table plan based on data usage in a Log Analytics workspace - Azure MonitorSupported Resource log categories for Azure Monitor

Retention Cost

Below shows the "Retention" cost of 10 GB/day of Analytic Log for 24 months:

Pros

✅ Logs are stored as "Hot Storage" and can be accessed anytime

✅ Can be used in Analytic Rules, Workbooks, and Hunting Queries

✅ No limitations in using KQL operators over the Analytic Log

✅ Longer log retention (maximum: 2 years)

Cons

❌ Expensive! 😬

PS: Remember - premium cars don't come cheap 🤷♂️

Basic Logs

Key thing I love about the Microsoft Sentinel team is that - they listen 🙂

After several organizations started facing a cost issue, the Microsoft Sentinel team came up with a workaround solution. Even though it goes against the desired approach - some organizations would still want to have logs that have a high volume (Firewall logs, DNS logs, etc.). To cover the use case, Microsoft introduced a new type of log - "Basic logs". Basic logs are still considered to be "hot storage", since they are accessible anytime, although it has some limitations on the usability.

Ingestion Cost

Below shows the "Ingestion" cost of 10GB/day of Analytic Log for 30 days:

Retention Cost

There is no Retention cost, since the fixed log retention is 8 days.

Pros

✅ Logs are stored as "Hot Storage" and can be accessed anytime

✅ Cheaper price

Cons

❌ Maximum log retention - 8 days

❌ Only a list of KQL operators can be used over the Basic Log

❌ Basic logs cannot be used in Analytic Rules, (most) Workbooks or in (most) Hunting queries

Archive Logs

Archive Logs were released in conjunction with Basic Logs. They are NOT of the type "hot storage" - but I wouldn't classify them as "cold storage" either. I like to call it as "warm storage" due to its easy accessibility and the capability to migrate the logs into hot storage in a few clicks. Their sole purpose is for long term log retention, and is not built to be used by Analytic Rules, Hunting Queries or Workbooks. The maximum log retention goes up to 12 years.

There is no ingestion cost for Archive Logs, since you CANNOT ingest logs directly into Archive Log table.

Ingestion Cost

There is no Ingestion cost, since the logs cannot be directly ingested into Archive Log Table.

Retention Cost

Below shows the "Retention" cost of 10 GB/day of Analytic Log for 24 months:

It wouldn't be a fair comparison to have Pros and Cons, since the purpose of Archive Logs is entirely different from Analytic and Basic Logs. Archive Logs still have the capability to run (very limited) threat hunting as "Search Jobs" - but they do come at a very minimal cost.

Winner?

• Analytic Log can be used in analytic rules, workbook and hunting queries with no limitations - but its expensive
• Basic logs are cheap - but it cannot be used in an analytic rule, (most) workbooks, and (most) hunting queries. On top of it - the maximum log retention is just 8 days.
• Archive logs are the cheapest of all, and can store logs up to 12 years - but it is not built for using it as a hot storage. It has "Search Jobs" for threat hunting, but it comes at a cost.

The key is knowing when to use what type of log table.

• Analytic logs should be used for high value security data that requires scheduled monitoring and alerting.
• Basic logs should be used for low detection value logs, but are valuable for investigating an incident (threat hunting)
• Since Basic logs have a 8 days log retention, Archive logs should be used to store the basic logs for a longer duration - to increase the scope of threat hunting when it is required.

Conclusion

Each log type has its purpose in Microsoft Sentinel, and using it the right way can save cost without compromising the security posture of an organization.

PS: If you are a huge organization using Microsoft Sentinel, and haven't heard of "Commitment Tier" - you are losing money down the drain 🙂