-
Notifications
You must be signed in to change notification settings - Fork 132
[security][9.0] Creates Automatic Migrations guide #713
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 7 commits
Commits
Show all changes
22 commits
Select commit
Hold shift + click to select a range
fc1317b
[security][9.0] Creates SIEM migrations guide
benironside 9642d8e
Merge branch 'main' into 586-siem-migration-guide
benironside ad9ec40
fix links
benironside d990ad1
fixes broken links
benironside bc4dbb5
Merge branch 'main' into 586-siem-migration-guide
natasha-moore-elastic fce4352
Apply suggestions from code review
natasha-moore-elastic ae769b1
fix build warning
natasha-moore-elastic 64f887b
responds to Jatin's feedback, other minor fixes
benironside 90668a3
Merge branch 'main' into 586-siem-migration-guide
benironside 221c6fa
Merge branch 'main' into 586-siem-migration-guide
benironside 2218382
Updates SIEM migration guide
benironside 98e0c95
Merge branch 'main' into 586-siem-migration-guide
benironside fa70757
Creates FAQ
benironside 4a0cc5e
Merge branch '586-siem-migration-guide' of https://github.com/elastic…
benironside a13bdb2
add index info, updates file name
benironside e127b06
updates front matter
benironside b707416
Apply suggestions from code review
benironside c3ff66c
Update solutions/security/get-started/automatic-migration.md
benironside 9a0eaf2
responds to reviews
benironside 71ca182
removes reference to elastic LLM
benironside bb36397
Update solutions/security/get-started/automatic-migration.md
nastasha-solomon 49d9ddd
Merge branch 'main' into 586-siem-migration-guide
benironside File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,106 @@ | ||
# AI-powered SIEM migration | ||
|
||
::::{warning} | ||
This feature is in technical preview. It may change in the future, and you should exercise caution when using it in production environments. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of GA features. | ||
:::: | ||
|
||
Elastic's AI-powered SIEM migration helps you quickly convert SIEM rules from the Splunk Processing Language (SPL) to the Elasticsearch Query Language ({{esql}}). It simplifies onboarding by matching your rules to Elastic-authored rules, if comparable rules exist. Otherwise, it automatically translates rules on the fly so you can verify and edit them instead of rewriting them from scratch. | ||
|
||
You can ingest your data before migrating your rules, or migrate your rules first, in which case the tool will recommend which data sources you need to power your migrated rules. | ||
|
||
::::{admonition} Requirements | ||
* A working [LLM connector](/solutions/security/ai/set-up-connectors-for-large-language-models-llm.md). | ||
* {{stack}} users: an [Enterprise](https://www.elastic.co/pricing) subscription. | ||
* {{Stack users}}: {{ml}} must be enabled. | ||
* {{serverless-short}} users: a [Security Complete](../../../deploy-manage/deploy/elastic-cloud/project-settings.md) subscription. | ||
|
||
:::: | ||
|
||
## Get started with AI-powered SIEM migration | ||
|
||
1. Find **Get started** in the navigation menu or use the [global search bar](/explore-analyze/find-and-organize/find-apps-and-objects.md). | ||
2. Under **Configure AI provider** select a model, or [add a new one](/solutions/security/ai/set-up-connectors-for-large-language-models-llm.md). For information on how different models perform, refer to the [LLM performance matrix](../../../solutions/security/ai/large-language-model-performance-matrix.md). | ||
3. Next, under **Migrate rules & add data**, click **Translate your existing SIEM rules to Elastic**, then **Upload rules**. | ||
4. Follow the instructions on the **Upload Splunk SIEM rules** flyout to export your rules from Splunk as JSON. | ||
|
||
:::{image} ../../../images/security-siem-migration-1.png | ||
:alt: the Upload Splunk SIEM rules flyout | ||
:width: 700px | ||
:screenshot: | ||
::: | ||
|
||
|
||
::::{note} | ||
The provided query downloads Splunk correlation rules and saved searches. Alternatively, as long as you export your results in a JSON format, you can use a different query. For example: | ||
|
||
``` | ||
| rest /servicesNS/-/-/saved/searches | ||
| search is_scheduled=1 AND eai:acl.app=splunksysmonsecurity | ||
| where disabled=0 | ||
| table id, title, search, description, action.escu.eli5, | ||
``` | ||
Which would download rules related to just the `splunksysmonsecurity` app. | ||
|
||
We don't recommend downloading all searches (for example with`| rest /servicesNS/-/-/saved/searches`) since most of the data will be irrelevant to SIEM rule migration. | ||
:::: | ||
|
||
5. Select your JSON file and click **Upload**. | ||
::::{note} | ||
If the file is large, you may need to separate it into multiple parts and upload them individually to avoid exceeding your LLM's context window. | ||
:::: | ||
|
||
6. After you upload your Splunk rules, SIEM migration will detect whether they use any Splunk macros or lookups. If so, follow the instructions which appear to export and upload them. Alternatively, you can complete this step later — however, until you upload them, some of your migrated rules will have a `partially translated` status. | ||
|
||
7. Click **Translate** to start the rule translation process. You don't need to stay on this page. A notification will appear when the process is complete. | ||
|
||
8. When migration is complete, click the notification or return to the **Get started** page then click **View translated rules** to open the **Translated rules** page. | ||
|
||
|
||
## The Translated rules page | ||
|
||
This section describes the **Translated rules** page's interface and explains how the data that appears here is derived. | ||
|
||
When you upload a new batch of rules, they are assigned a name and number, for example `SIEM rule migration 1`, or `SIEM rule migration 2`. Use the **Migrations** dropdown menu in the upper right to select which batch appears. | ||
|
||
The table's fields are as follows: | ||
|
||
* **Name:** The names of Elastic authored rules cannot be edited until after rule installation. To edit the name of a custom translated rule, click the name and select **Edit**. | ||
* **Status:** The rule's translation status. Hover over the tooltip next to the column title for more information. | ||
* **Risk Score:** For Elastic authored rules, risk scores are predefined. For custom translated rules, risk scores are defined as follows: | ||
* If the source rule has a field comparable to Elastic's risk score, we use that value. | ||
* Otherwise, if the source rule has a field comparable to Elastic's rule severity field, we base the risk score on that value according to these [guidelines](/solutions/security/detect-and-alert/create-detection-rule.md#custom-highlighted-esql-fields). | ||
* Otherwise, a default value is assigned. | ||
* **Rule severity:** For Elastic authored rules, severity scores are predefined. For custom translated rules, risk scores are based on the source rule's severity field. Splunk severity scores are translated to Elastic rule severity scores as follows: | ||
|
||
| Splunk severity | Elastic rule severity | | ||
| ------- | ----------- | | ||
| (Info) | Low | | ||
| (Low) | Low | | ||
| (Medium) | Medium | | ||
| (High) | High | | ||
| (Critical) | Critical | | ||
logeekal marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
* **Author:** Shows one of two possible values: `Elastic`, or `Custom`. Elastic authored rules are created by Elastic and update automatically. Custom rules are translated by the SIEM migration tool or your team, and do not update automatically. | ||
* **Integrations:** Shows the number of Elastic integrations that must be installed to provide data for the rule to run successfully. | ||
* **Actions:** Allows you to click **Install** to add a rule to Elastic. Installed rules must also be enabled before they will run. To install rules in bulk, select the check box at the top of the table before clicking **Install**. | ||
|
||
## Finalize translated rules | ||
|
||
Once you're on the **Translated rules** page, to install any rules that were partially translated or not translated, you will need to edit them. Optionally, you can also edit custom rules that were successfully translated to finetune them. | ||
|
||
:::{note} | ||
You cannot edit Elastic authored rules using this interface, but after they are installed you can edit them from the [**Rules**](/solutions/security/detect-and-alert/about-detection-rules.md) page. | ||
::: | ||
|
||
### Edit a custom rule | ||
|
||
Click the rule's name to open the rule's details flyout to the **Translation** tab, which shows the source rule alongside the translated — or partially translated — Elastic version. You can update any part of the rule. When finished, click **Save**. | ||
|
||
::::{note} | ||
If you haven't yet ingested your data, you will likely encounter `Unknown index` or `Unknown column` errors while editing. You can ignore these and add your data later. | ||
:::: | ||
|
||
### View rule details | ||
|
||
The rule details flyout which appears when you click on a rule's name in the **Translate rules** table has two other tabs, **Overview** and **Summary**. The **Overview** tab displays information such as the rule's severity, risk score, rule type, and how frequently it runs. The **Summary** tab explains the logic behind how the rule was translated, such as why specific {{esql}} commands were used, or why a source rule was mapped to a particular Elastic authored rule. | ||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.