-
Notifications
You must be signed in to change notification settings - Fork 475
Cloudtrail add actor and target #11245
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
241ed25
to
b5e160f
Compare
packages/aws/data_stream/cloudtrail/elasticsearch/ingest_pipeline/default.yml
Outdated
Show resolved
Hide resolved
6cef63a
to
5d506d5
Compare
...ta_stream/cloudtrail/_dev/test/pipeline/test-delete-network-acl-entry-json.log-expected.json
Show resolved
Hide resolved
...m/cloudtrail/_dev/test/pipeline/test-authorize-security-group-ingress-json.log-expected.json
Show resolved
Hide resolved
883391d
to
5c2dce2
Compare
...data_stream/cloudtrail/_dev/test/pipeline/test-put-bucket-replication-json.log-expected.json
Show resolved
Hide resolved
packages/aws/data_stream/cloudtrail/_dev/test/pipeline/test-put-object-json.log-expected.json
Show resolved
Hide resolved
d445e1e
to
8f088ee
Compare
🚀 Benchmarks reportPackage
|
Data stream | Previous EPS | New EPS | Diff (%) | Result |
---|---|---|---|---|
vpcflow |
6622.52 | 5376.34 | -1246.18 (-18.82%) | 💔 |
waf |
7042.25 | 4310.34 | -2731.91 (-38.79%) | 💔 |
To see the full report comment with /test benchmark fullreport
.../data_stream/cloudtrail/_dev/test/pipeline/test-delete-global-cluster-json.log-expected.json
Fixed
Show fixed
Hide fixed
.../data_stream/cloudtrail/_dev/test/pipeline/test-delete-global-cluster-json.log-expected.json
Fixed
Show fixed
Hide fixed
.../data_stream/cloudtrail/_dev/test/pipeline/test-delete-global-cluster-json.log-expected.json
Fixed
Show fixed
Hide fixed
...ges/aws/data_stream/cloudtrail/_dev/test/pipeline/test-delete-web-acl-json.log-expected.json
Fixed
Show fixed
Hide fixed
...ges/aws/data_stream/cloudtrail/_dev/test/pipeline/test-delete-web-acl-json.log-expected.json
Fixed
Show fixed
Hide fixed
...ges/aws/data_stream/cloudtrail/_dev/test/pipeline/test-delete-web-acl-json.log-expected.json
Fixed
Show fixed
Hide fixed
8f088ee
to
02e21e3
Compare
@romulets - We will begin reviewing this PR today. Apologies for the latency. |
For Examples:
The importance here is the second example where we are getting the identity of the In the first example, this would indicate that the authenticated user |
@romulets - By cross-referencing our attack scenario doc (reference https://github.com/elastic/ia-trade-team/issues/456#issuecomment-2444285976), I noticed the following
|
@terrancedejesus do you have examples of events for those you shared? |
@romulets examples can be found in the table here. Next to each event.action is a link to a saved search in mostly Tin's cluster. For a few we had to use our TRaDE cluster, the serverless project we gave you access to. |
0aea983
to
3c39730
Compare
|
Adding @tommyers-elastic, for on any comments on observability use case. |
Is there an shared schema defined for these new root-level namespaces ( |
4bb4b5f
to
68ba02f
Compare
/test benchmark fullreport |
1 similar comment
/test benchmark fullreport |
💔 Build Failed
Failed CI StepsHistory
|
|
Pinging @elastic/security-service-integrations (Team:Security-Service Integrations) |
eda4138
to
f728ca7
Compare
Background
Elastic Cloud Security Team has been focusing, this past year, on Cloud Detection and Response (CDR). One of the first steps towards the CDR vision is to enhance investigation workflows for the Cloud Security use-case in SIEM.
As part of enhancing investigation workflows it's necessary to be able to correlate events and entities. Meaning, if an alert is triggered on the ec2 instance i-000000000, it is of great value to easily be able to search all the events related to that entity, across multiple indices, with one query. Therefore we are working on extracting entities and enabling them to be correlated.
What is an entity?
An "entity" in our context refers to any discrete component within an IT environment that can be uniquely identified and monitored. This broad term encompasses both managed and unmanaged elements.
The term "entity" is broader than the current set of available fields under related. Although ip, user and hosts can be identities, there is a lack of space to represent messaging queues, load balancers, storage systems, databases and others. Therefore the proposal to add a new field.
The proposed structure
There are two fields being added on this PR:
actor.entity.id
captures entities that started the event, the actorstarget.entity.id
captures entities that were affected by the event. Being that created, updated, listed. We try to do as much as possible with the data present in the event.Decisions made on the Painless Script
Structure
The painless script turned very large. There are essentially three parts to it:
related
,actor
andtarget
).requestParameters
andrepsonseElements
, there is, usually a somewhat coherent structure per AWS service. I believe such separation brings better reading, creates a better headspace once working in a specific service and also breaks down the hugeif else
chain present in the previous state of the codeWhy TreeSet as datastructure to hold
related
,actor
,target
.There are two properties that this script must have:
Previously I had ensured both properties on "post processing", at the end of the script. Now it's ensured by the data structure itself.
I have not performance tested myself, but the usage of TreeSet should improve the time complexity of the algorithm, since we sort data on
add
, and previously we had tosort
afterwards. I couldn't find a reliable source for time complexity ofTreeSet.add
vsCollections.sort
- and honestly, the size of the list is so small that might not even matter.Amount of tests
The testing was essential to me to validate what I was doing, to verify each output. And I would like to keep the tests for future reference and ensuring we are not changing anything by mistake. But the tests are starting to get slow. Specially if you compare with other integrations, such as
okta
.