Forensics: 7. Creating Anomaly Detection Baseline Alerts
Posted: Tue Jul 31, 2012 6:28 am
A Traffic analysis in Forensics can be saved as an Intelligent Baseline Network Behavior Anomaly Detection Alert. (Baseline Alert or IB-NBAD)
Creating a new Anomaly Detection Alert
Two methods are available to create an Anomaly Detection: Single Item - Single Click, or Saving a Baseline Alert from Custom Forensics or Forensics Analysis.
Single Item - Single Click
Clicking on an item in a flow-field configuration screen and pressing the Anomaly Detection button will bring you into the "Forensics Template / Report / Alert" screen from where you can either save-and-go or choose to extend by tuning the filter at anytime
Saving a Baseline Alert from Custom Forensics or Forensics Analysis
Click on the top toolbar "Save" button in a "Forensics" or a "Custom Forensics" screen provides a method of reaching the Baseline Alert screen thats suits more complex baselines made up of multiple criteria and multiple dimensions.
The Forensics Template/Report/Alert Screen
The Anomaly Detection Baseline Alerting Parameters: includes Detection Header Information, "Definition", "Schedule", "Alert Action", "Delivery" and "Baseline Alert Criteria"
Command Buttons
Name of the Anomaly Detection Baseline Alert.
Description
Additional information for the Baseline Alert.
Report Type
Choose "Baseline Alert" Report Type to configure the Baseline Anomaly Detection .
Other available options allow you to save this report as a Template, a Scheduled Report, a Threshold Alert or an Intelligent Baseline Anomaly Detection Alert.
Category
The category is fixed as "Baseline Alert".
Add Link
Enabling the "Add Link" option will add an icon to the generated Baseline Alert. The icon provides a click back link to the "Forensics" screen from the generated Baseline Alert.
In order for the click back to function correctly the server must be correctly configured in the "Site Configuration" screen under "Administrator" in the "Configuration" panel.
Data Period
Data Period is set to Schedule Frequency - Baseline Alert is checked each interval.
Definition
The following options in "Custom Forensics" and "Forensics" Filter tab can be overwritten here.
Defines the alerting schedule instance checking start time point.
Schedule To
Defines the alerting schedule instance checking end time point.
Available if "Run Indefinitely" is un-ticked.
Run Indefinitely
The alerting schedule instance will be active indefinitely if it is ticked on.
Wait for Delayed
When the Baseline Alert includes multiple devices/exporters (routers or switches), the Baseline Alert will be delayed to run until all active device data has reached the scheduled time points. This is to ensure that alerts maintain integrity when dependent on multiple inputs.
Priority
Defines the priority of the alerting schedule.
Alert Category
Defines a category for the alert.
The alert category can be added in "Alert Category" screen, which can be entered by clicking the left menu "Category" under the Alert Administration in the "My Analytics" panel.
on No Data
Send an alert when there is no data for the defined traffic by "Forensics" if it is ticked on.
Alerting Level
Controls the sending of alerts to Email, Report Repositories and SNMP Traps and only sends those alerts above and including the Alerting Level when Anomaly Detection thresholds are breached.
Controls the sending of alerts to Email, Report Repositories and SNMP Traps and only sends those alerts where the learned baseline threshold is breached N times within the defined M minutes period.
Delivery
The Anomaly Detection Baseline Alert can be sent to one or more email addresses, the Report Repository or an SNMP Trap Server.
The SMTP server and its service port, sender address and subject must be configured properly to allow send schedule report.
The Anomaly Detection Baseline Alert can be sent to an SNMP trap server.
The SNMP trap server and its relative information must be configured properly in "Site Configuration" screen.
The SNMP Trap codes are defined in the "SNMP Trap Code" screen.
The Anomaly Detection Baseline Alert can be specified to save to a directory in the Report Repository with each scheduled time stamp as the report name affix or copied to a specified report name in the report repository for viewing or to enable other applications to refer to the automatically refreshed file.
Delivery to "Directory" and "File" have 3 shared attribute options.
The criteria of an Anomaly Detection Baseline Alert can be based on any one or more measurements, Bytes, bps, Packets, pps, Flows, Packet-Size, Count and TcpFlags. The measurement value in the criteria is compared against the learned baseline data, instead of the absolute value.
When creating an Intelligent Baseline Network Behavior Anomaly Detection the Alert levels are pre-configured to Default Criteria to allow an easy One-Click Anomaly Detection Creation. The Default Alert criteria thresholds are the most appropriate baseline to compare against. Flexible options extend the Default Alert criteria:
There are 3 threshold levels available "Information" (Green), "Warning" (Yellow) and "Critical" (Red).
The Anomaly Detection Baseline Build Learning Parameters includes "Baseline Build"
Over N Minutes Per Hour
The baseline learning schedule will calculate and merge the statistics for a hour only when there are over N minutes netflow data collected in this hour. The default is 30 minutes.
Send Alert if Exceptional Hour
A report will be sent to the recipients showing the traffic defined in the build versus the baseline that has been built as a background.
Statistics Average, Min, Max and Standard Deviation of any measurement can be added to learning criteria.
Defines the learning schedule instance start time point.
Priority
Defines the priority of the learning schedule.
Where a traffic analysis already has measurement or time fields in the Criteria in the Dimensions the analysis cannot be saved as an alert.Two main processes occur for each Anomaly Detection Baseline Alert.
- Learning
Continually builds per minute baseline statistics of all measurements - Bytes, bps, Packets, pps, Flows, Packet-Size, Count and TcpFlags for each statistical computation ie Sum, Average, Standard Deviation, Minimum and Maximum
The learning processes calculate the latest statistics for each Anomaly Detection Baseline and merges them hourly with previously learned data for each corresponding hour per weekday period. - Alerting
The alerting processes compare the current traffic against the learned statistics for each Anomaly Detection Baseline.
The alerting schedule is checked by default in 5 minutes intervals for each of the last 5 minutes. The minimum interval is 1 minute.
Creating a new Anomaly Detection Alert
Two methods are available to create an Anomaly Detection: Single Item - Single Click, or Saving a Baseline Alert from Custom Forensics or Forensics Analysis.
Single Item - Single Click
Clicking on an item in a flow-field configuration screen and pressing the Anomaly Detection button will bring you into the "Forensics Template / Report / Alert" screen from where you can either save-and-go or choose to extend by tuning the filter at anytime
The Anomaly Detection button is available from any of the Configuration screens; Device Group, Device, Interface Group, Interface, MPLS, Business Groups, Account, CostCentre, IPv4 and IPv6, IP Range, IP Allocation, IP Domain, IP Country Blocks, ASN, Applications, Protocol, Known Service Port, All Ports, QoS, Tos, Tos Precedence, PHB, PHB Class
Saving a Baseline Alert from Custom Forensics or Forensics Analysis
Click on the top toolbar "Save" button in a "Forensics" or a "Custom Forensics" screen provides a method of reaching the Baseline Alert screen thats suits more complex baselines made up of multiple criteria and multiple dimensions.
The Forensics Template/Report/Alert Screen
The Anomaly Detection Baseline Alerting Parameters: includes Detection Header Information, "Definition", "Schedule", "Alert Action", "Delivery" and "Baseline Alert Criteria"
Command Buttons
- Save New - Save as a new Baseline Alert. Where the Baseline Alert is based on an existing Baseline Alert, Schedule Report or a Template the original one will not be changed.
- Save Back - Save back to the original baseline alert after modifying some options.
- Report - Go back to Forensics screen to check or adjust the baseline alert traffic options in "Forensics".
- Filter - Go back to "Custom Forensics" to adjust the baseline alert traffic options in "Forensics".
- Suspend - Suspend this baseline alert.
- Resume - Recover checking this baseline alert.
- Cancel - Go back to the previous page.
- Delete - Delete this baseline alert as well as the learned baseline data if it exists.
Saving back a baseline alert will clean all learned baseline data if changes have been made to the criteria or dimensions of the defined traffic.Title
Name of the Anomaly Detection Baseline Alert.
Description
Additional information for the Baseline Alert.
Report Type
Choose "Baseline Alert" Report Type to configure the Baseline Anomaly Detection .
Other available options allow you to save this report as a Template, a Scheduled Report, a Threshold Alert or an Intelligent Baseline Anomaly Detection Alert.
Category
The category is fixed as "Baseline Alert".
Add Link
Enabling the "Add Link" option will add an icon to the generated Baseline Alert. The icon provides a click back link to the "Forensics" screen from the generated Baseline Alert.
In order for the click back to function correctly the server must be correctly configured in the "Site Configuration" screen under "Administrator" in the "Configuration" panel.
Data Period
Data Period is set to Schedule Frequency - Baseline Alert is checked each interval.
Definition
The following options in "Custom Forensics" and "Forensics" Filter tab can be overwritten here.
- "Aggregated Data",
- "Report Layout",
- "Bidirectional Reverse Criteria" and
- "Duplication"
Defines the alerting schedule instance checking start time point.
Schedule To
Defines the alerting schedule instance checking end time point.
Available if "Run Indefinitely" is un-ticked.
Run Indefinitely
The alerting schedule instance will be active indefinitely if it is ticked on.
Wait for Delayed
When the Baseline Alert includes multiple devices/exporters (routers or switches), the Baseline Alert will be delayed to run until all active device data has reached the scheduled time points. This is to ensure that alerts maintain integrity when dependent on multiple inputs.
Priority
Defines the priority of the alerting schedule.
Alert Category
Defines a category for the alert.
The alert category can be added in "Alert Category" screen, which can be entered by clicking the left menu "Category" under the Alert Administration in the "My Analytics" panel.
on No Data
Send an alert when there is no data for the defined traffic by "Forensics" if it is ticked on.
Alerting Level
Controls the sending of alerts to Email, Report Repositories and SNMP Traps and only sends those alerts above and including the Alerting Level when Anomaly Detection thresholds are breached.
- Critical - Alert on critical events.
- Warning - Alert on warning and critical events.
- Information - Alert on information,warning and critical events.
Information and warning events will still show in the screen when Critical alerting level is selected and Information events will still show in the screen when Warning alerting level is selected.over N Defined Event(s) and With M minutes
Controls the sending of alerts to Email, Report Repositories and SNMP Traps and only sends those alerts where the learned baseline threshold is breached N times within the defined M minutes period.
All events will still be shown in the Alert screenBaseline Anomaly Detections produce few False Positives but this option can help to fine tune some alerting.
Delivery
The Anomaly Detection Baseline Alert can be sent to one or more email addresses, the Report Repository or an SNMP Trap Server.
The SMTP server and its service port, sender address and subject must be configured properly to allow send schedule report.
The Anomaly Detection Baseline Alert can be sent to an SNMP trap server.
The SNMP trap server and its relative information must be configured properly in "Site Configuration" screen.
The SNMP Trap codes are defined in the "SNMP Trap Code" screen.
The Anomaly Detection Baseline Alert can be specified to save to a directory in the Report Repository with each scheduled time stamp as the report name affix or copied to a specified report name in the report repository for viewing or to enable other applications to refer to the automatically refreshed file.
Delivery to "Directory" and "File" have 3 shared attribute options.
- Private - only allows the report Owner and Administrator to browse the alert in the Report Repository.
- Shared - allows any user who is logged into CySight to browse the alert.
- Public - allows anyone to browse the given alert using a specified URL without being logged into CySight.
The criteria of an Anomaly Detection Baseline Alert can be based on any one or more measurements, Bytes, bps, Packets, pps, Flows, Packet-Size, Count and TcpFlags. The measurement value in the criteria is compared against the learned baseline data, instead of the absolute value.
When creating an Intelligent Baseline Network Behavior Anomaly Detection the Alert levels are pre-configured to Default Criteria to allow an easy One-Click Anomaly Detection Creation. The Default Alert criteria thresholds are the most appropriate baseline to compare against. Flexible options extend the Default Alert criteria:
| Measurement | > or < | Threshold | Statistic | Alert |
|---|---|---|---|---|
| bps | > | +100% | bps Max | Critical |
| bps | > | +50% | bps Max | Warning |
| bps | > | +10% | bps Max | Information |
| pps | > | +100% | pps Max | Critical |
| pps | > | +50% | pps Max | Warning |
| pps | > | +10% | pps Max | Information |
| Flows | > | +100% | Flows Max | Critical |
| Flows | > | +50% | Flows Max | Warning |
| Flows | > | +10% | Flows Max | Information |
| Count | > | +100% | Count Max | Critical |
| Count | > | +50% | Count Max | Warning |
| Count | > | +10% | Count Max | Information |
| TcpFIN | > | +100% | TcpFIN Max | Critical |
| TcpFIN | > | +50% | TcpFIN Max | Warning |
| TcpFIN | > | +10% | TcpFIN Max | Information |
| TcpSYN | > | +100% | TcpSYN Max | Critical |
| TcpSYN | > | +50% | TcpSYN Max | Warning |
| TcpSYN | > | +10% | TcpSYN Max | Information |
| TcpRST | > | +100% | TcpRST Max | Critical |
| TcpRST | > | +50% | TcpRST Max | Warning |
| TcpRST | > | +10% | TcpRST Max | Information |
| TcpPSH | > | +100% | TcpPSH Max | Critical |
| TcpPSH | > | +50% | TcpPSH Max | Warning |
| TcpPSH | > | +10% | TcpPSH Max | Information |
| TcpACK | > | +100% | TcpACK Max | Critical |
| TcpACK | > | +50% | TcpACK Max | Warning |
| TcpACK | > | +10% | TcpACK Max | Information |
| TcpURG | > | +100% | TcpURG Max | Critical |
| TcpURG | > | +50% | TcpURG Max | Warning |
| TcpURG | > | +10% | TcpURG Max | Information |
| TosCE | > | +100% | TosCE Max | Critical |
| TosCE | > | +50% | TosCE Max | Warning |
| TosCE | > | +10% | TosCE Max | Information |
The Anomaly Detection Baseline Build Learning Parameters includes "Baseline Build"
Over N Minutes Per Hour
The baseline learning schedule will calculate and merge the statistics for a hour only when there are over N minutes netflow data collected in this hour. The default is 30 minutes.
Send Alert if Exceptional Hour
A report will be sent to the recipients showing the traffic defined in the build versus the baseline that has been built as a background.
An Exceptional Hour is treated as abnormal traffic and will not be merged into the existing baseline.Baseline learning criteria
Statistics Average, Min, Max and Standard Deviation of any measurement can be added to learning criteria.
The hour will treated as an Exceptional Hour if any one of the statistics of the current hour is ascertained to be incorrect.Build From
Defines the learning schedule instance start time point.
Priority
Defines the priority of the learning schedule.