Skip to content

Refactored Taint Analysis #216

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 117 commits into from
Feb 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
117 commits
Select commit Hold shift + click to select a range
6c679d1
Taint analysis
Lipen Feb 9, 2024
103d20c
Remove unnecessary accept override
Lipen Feb 12, 2024
b1a09c9
fix: access flags for unknown methods
sergeypospelov Feb 12, 2024
3ae0bcd
fix: revert 0
sergeypospelov Feb 12, 2024
bf97bc3
fix: revert 1
sergeypospelov Feb 12, 2024
c5fbc8f
fix: revert 2
sergeypospelov Feb 12, 2024
781e65a
fix: revert 3
sergeypospelov Feb 12, 2024
82f2101
fix: TODO
sergeypospelov Feb 12, 2024
479be4e
fix: revert 5
sergeypospelov Feb 12, 2024
5c9bcf4
Debug messages
Lipen Feb 14, 2024
d57d3c4
Make handleControlEvent non-suspend
Lipen Feb 14, 2024
9eab66e
Move getOverrides
Lipen Feb 14, 2024
f7f6fc8
Add toString for units
Lipen Feb 14, 2024
4a61f44
Fix summary flows
Lipen Feb 14, 2024
6a8a61a
Fix logger import
Lipen Feb 14, 2024
8bf35bf
Move entry point for taint analysis
Lipen Feb 14, 2024
028fcf6
Delegate to basicConditionEvaluator
Lipen Feb 14, 2024
f75a0c3
Rename Runner
Lipen Feb 14, 2024
744679b
Use `computeIfAbsert`, switch to concurrent set
Lipen Feb 14, 2024
79527b9
Make all runner fields private, restore erased types
Lipen Feb 14, 2024
013d0f3
Reuse event instances
Lipen Feb 14, 2024
0d0be06
Remove comments
Lipen Feb 14, 2024
749d4d8
Remove IDEA-specific suppresses
Lipen Feb 14, 2024
789a7b1
Move NPE analysis entry point
Lipen Feb 14, 2024
4e43674
Move timeout to argument
Lipen Feb 14, 2024
4691ab7
Fix debug
Lipen Feb 14, 2024
d305cfc
Replace with TODO
Lipen Feb 14, 2024
aae64c5
Remove `isSkipped`
Lipen Feb 14, 2024
f41c335
Remove old unused code
Lipen Feb 14, 2024
83f0118
Replace with logger
Lipen Feb 14, 2024
f0c7c83
Remove
Lipen Feb 14, 2024
eef22f1
Remove
Lipen Feb 14, 2024
d8f1d35
Remove
Lipen Feb 14, 2024
08c81d1
Cleanup
Lipen Feb 14, 2024
f3bad35
Remove
Lipen Feb 14, 2024
88d35e9
Rename UniRunner
Lipen Feb 14, 2024
6045beb
Make pathEdges internal for extension function
Lipen Feb 14, 2024
0fefbb9
Remove
Lipen Feb 14, 2024
ef8f0f5
Revert "Delegate to basicConditionEvaluator"
Lipen Feb 14, 2024
1d393a2
Remove
Lipen Feb 14, 2024
6cd9c63
Replace globals with manager constructor argument
Lipen Feb 14, 2024
159866b
Move queueIsNotEmpty
Lipen Feb 14, 2024
05e4f20
Format
Lipen Feb 14, 2024
1e7529b
Use logger
Lipen Feb 14, 2024
4bd389a
Remove types
Lipen Feb 14, 2024
b63adb8
Do not add deps
Lipen Feb 14, 2024
67cf2de
Ignore events for non-existing runners
Lipen Feb 14, 2024
5df901e
Make handleControlEvent suspend
Lipen Feb 14, 2024
e625b2f
Use computeIfAbsent, remove types
Lipen Feb 14, 2024
6687372
Fix
Lipen Feb 14, 2024
b1dd664
Cleanup
Lipen Feb 14, 2024
62ed74d
Timeout 30s in tests
Lipen Feb 14, 2024
020df51
Types
Lipen Feb 14, 2024
8771a1b
Cleanup
Lipen Feb 14, 2024
00b20c4
Make handleControlEvent non-suspend again
Lipen Feb 14, 2024
8570449
Log timeout
Lipen Feb 14, 2024
0af092a
Add explanations for unexpected IsType and AnnotationType conditions
Lipen Feb 14, 2024
8dbe55b
Use isOnHeap
Lipen Feb 15, 2024
7463761
Revert unnecessary changes
Lipen Feb 15, 2024
bc82b6e
Add mocking tests for flow functions
Lipen Feb 16, 2024
94ccf10
Remove unnecessary converters to old DomainFact
Lipen Feb 16, 2024
ce50333
Add missing config for mocking tests
Lipen Feb 16, 2024
927dbb1
Remove trace logging
Lipen Feb 16, 2024
327af42
Cleanup
Lipen Feb 16, 2024
090adac
Extract aggregates
Lipen Feb 16, 2024
c031650
Add test with bidi runner and class unit resolver
Lipen Feb 16, 2024
901e255
Fix visited
Lipen Feb 16, 2024
9611d66
Start analysis of deps
Lipen Feb 16, 2024
6f1a8d0
Review fixes
Lipen Feb 16, 2024
8d531e5
Move toDomainFact
Lipen Feb 16, 2024
a83cf68
Report vulnerability for each triggered rule
Lipen Feb 16, 2024
b5a0db3
Increase coverage
Lipen Feb 16, 2024
459c9a1
Revert "Increase coverage"
Lipen Feb 16, 2024
e1b00e2
Remove old IFDS
Lipen Feb 16, 2024
e4c5cc8
Cleanup logs
Lipen Feb 16, 2024
eef7118
Types
Lipen Feb 16, 2024
ad5a03a
Format
Lipen Feb 16, 2024
a350539
Cleanup
Lipen Feb 16, 2024
c13092e
Add tests for condition evaluator
Lipen Feb 17, 2024
473c9f7
Fix type
Lipen Feb 17, 2024
b7f0da5
Add toString for AnyArgument
Lipen Feb 17, 2024
a81add4
Test ContainsMark condition via FactAwareConditionEvaluator
Lipen Feb 17, 2024
da3fdd5
Fix BaseAnalysisTest
Lipen Feb 17, 2024
261bff3
Add unused variable analysis
Lipen Feb 17, 2024
2e30243
Cleanup logger config
Lipen Feb 17, 2024
fe685fd
Fix trace graph merging
Lipen Feb 19, 2024
f95366b
Fix tests
Lipen Feb 19, 2024
365b4cc
Fix unused variable analysis
Lipen Feb 19, 2024
9a69046
Fix SARIF, rename some files and classes
Lipen Feb 19, 2024
8f5a871
Restore CLI
Lipen Feb 19, 2024
af8161e
Fix internal function name
Lipen Feb 19, 2024
c2d7351
Fix
Lipen Feb 19, 2024
ec55566
Remove unnecessary stuff
Lipen Feb 19, 2024
f893970
Fix NPE tests
Lipen Feb 19, 2024
e0839cc
Fix running analysis from Java
Lipen Feb 19, 2024
b506f74
Remove ElementAccessor's dummy constructor
Lipen Feb 19, 2024
8a6a3a0
Use non-nullable T
Lipen Feb 19, 2024
8ea7525
Convert ConditionSimplifier to object
Lipen Feb 19, 2024
fcad9bd
Compact toString
Lipen Feb 19, 2024
0ed6f48
Cleanup
Lipen Feb 19, 2024
8cd0891
Use any
Lipen Feb 21, 2024
89c1226
Cleanup
Lipen Feb 21, 2024
05e992d
Remove unnecessary NewVulnerability event for UnusedVariable analysis
Lipen Feb 21, 2024
0cabd92
Remove unnecessary entry points
Lipen Feb 21, 2024
b7a2230
Add "entry point" position resolvers
Lipen Feb 21, 2024
3ec515c
Remove comment
Lipen Feb 21, 2024
facb51a
Replace comment with TODO
Lipen Feb 21, 2024
62c8770
Remove comments
Lipen Feb 21, 2024
13e9b53
Unify code
Lipen Feb 21, 2024
0f4464b
Add JvmName
Lipen Feb 21, 2024
ece975a
Trace
Lipen Feb 21, 2024
d70ed76
Add PassThrough rule for Map::put
Lipen Feb 21, 2024
4b720d5
Use info logger level for found sinks
Lipen Feb 21, 2024
41ce66a
Add PassThrough for decodeBase64
Lipen Feb 21, 2024
3ec7809
Add PassThrough rule for Map::replace
Lipen Feb 21, 2024
ae76dd6
Fix ConditionMark evaluation, add adhoc for arrays
Lipen Feb 21, 2024
b39519f
Do not override tainted arrays
Lipen Feb 23, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 20 additions & 7 deletions buildSrc/src/main/kotlin/Dependencies.kt
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,8 @@ object Versions {
const val kotlinx_metadata = "0.5.0"
const val kotlinx_serialization = "1.4.1"
const val licenser = "0.6.1"
const val mockito = "4.8.0"
const val mockk = "1.13.3"
const val sarif4k = "0.5.0"
const val shadow = "8.1.1"
const val slf4j = "1.7.36"
const val soot_utbot_fork = "4.4.0-FORK-2"
Expand Down Expand Up @@ -134,6 +135,11 @@ object Libs {
)

// https://github.com/Kotlin/kotlinx.serialization
val kotlinx_serialization_core = dep(
group = "org.jetbrains.kotlinx",
name = "kotlinx-serialization-core",
version = Versions.kotlinx_serialization
)
val kotlinx_serialization_json = dep(
group = "org.jetbrains.kotlinx",
name = "kotlinx-serialization-json",
Expand Down Expand Up @@ -199,11 +205,11 @@ object Libs {
version = Versions.jdot
)

// https://github.com/mockito/mockito
val mockito_core = dep(
group = "org.mockito",
name = "mockito-core",
version = Versions.mockito
// https://github.com/mockk/mockk
val mockk = dep(
group = "io.mockk",
name = "mockk",
version = Versions.mockk
)

// https://github.com/JetBrains/java-annotations
Expand Down Expand Up @@ -275,6 +281,13 @@ object Libs {
name = "cwe${cweNum}",
version = Versions.juliet
)

// https://github.com/detekt/sarif4k
val sarif4k = dep(
group = "io.github.detekt.sarif4k",
name = "sarif4k",
version = Versions.sarif4k
)
}

object Plugins {
Expand Down Expand Up @@ -320,4 +333,4 @@ object Plugins {

fun PluginDependenciesSpec.id(plugin: Plugins.ProjectPlugin) {
id(plugin.id).version(plugin.version)
}
}
6 changes: 6 additions & 0 deletions jacodb-analysis/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,20 @@ plugins {
dependencies {
api(project(":jacodb-core"))
api(project(":jacodb-api"))
api(project(":jacodb-taint-configuration"))

implementation(Libs.kotlin_logging)
implementation(Libs.slf4j_simple)
implementation(Libs.kotlinx_coroutines_core)
implementation(Libs.kotlinx_serialization_json)
api(Libs.sarif4k)

testImplementation(testFixtures(project(":jacodb-core")))
testImplementation(project(":jacodb-api"))
testImplementation(kotlin("test"))
testImplementation(Libs.mockk)

// Additional deps for analysis:
testImplementation(files("src/test/resources/pointerbench.jar"))
testImplementation(Libs.joda_time)
testImplementation(Libs.juliet_support)
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,185 @@
/*
* Copyright 2022 UnitTestBot contributors (utbot.org)
* <p>
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package org.jacodb.analysis.config

import org.jacodb.analysis.ifds.AccessPath
import org.jacodb.analysis.ifds.ElementAccessor
import org.jacodb.analysis.ifds.Maybe
import org.jacodb.analysis.ifds.onSome
import org.jacodb.analysis.ifds.toPath
import org.jacodb.analysis.taint.Tainted
import org.jacodb.api.cfg.JcBool
import org.jacodb.api.cfg.JcConstant
import org.jacodb.api.cfg.JcInt
import org.jacodb.api.cfg.JcStringConstant
import org.jacodb.api.cfg.JcValue
import org.jacodb.api.ext.isAssignable
import org.jacodb.taint.configuration.And
import org.jacodb.taint.configuration.AnnotationType
import org.jacodb.taint.configuration.ConditionVisitor
import org.jacodb.taint.configuration.ConstantBooleanValue
import org.jacodb.taint.configuration.ConstantEq
import org.jacodb.taint.configuration.ConstantGt
import org.jacodb.taint.configuration.ConstantIntValue
import org.jacodb.taint.configuration.ConstantLt
import org.jacodb.taint.configuration.ConstantMatches
import org.jacodb.taint.configuration.ConstantStringValue
import org.jacodb.taint.configuration.ConstantTrue
import org.jacodb.taint.configuration.ContainsMark
import org.jacodb.taint.configuration.IsConstant
import org.jacodb.taint.configuration.IsType
import org.jacodb.taint.configuration.Not
import org.jacodb.taint.configuration.Or
import org.jacodb.taint.configuration.PositionResolver
import org.jacodb.taint.configuration.SourceFunctionMatches
import org.jacodb.taint.configuration.TypeMatches

open class BasicConditionEvaluator(
internal val positionResolver: PositionResolver<Maybe<JcValue>>,
) : ConditionVisitor<Boolean> {

override fun visit(condition: ConstantTrue): Boolean {
return true
}

override fun visit(condition: Not): Boolean {
return !condition.arg.accept(this)
}

override fun visit(condition: And): Boolean {
return condition.args.all { it.accept(this) }
}

override fun visit(condition: Or): Boolean {
return condition.args.any { it.accept(this) }
}

override fun visit(condition: IsConstant): Boolean {
positionResolver.resolve(condition.position).onSome { return it is JcConstant }
return false
}

override fun visit(condition: IsType): Boolean {
// Note: TaintConfigurationFeature.ConditionSpecializer is responsible for
// expanding IsType condition upon parsing the taint configuration.
error("Unexpected condition: $condition")
}

override fun visit(condition: AnnotationType): Boolean {
// Note: TaintConfigurationFeature.ConditionSpecializer is responsible for
// expanding AnnotationType condition upon parsing the taint configuration.
error("Unexpected condition: $condition")
}

override fun visit(condition: ConstantEq): Boolean {
positionResolver.resolve(condition.position).onSome { value ->
return when (val constant = condition.value) {
is ConstantBooleanValue -> {
value is JcBool && value.value == constant.value
}

is ConstantIntValue -> {
value is JcInt && value.value == constant.value
}

is ConstantStringValue -> {
// TODO: if 'value' is not string, convert it to string and compare with 'constant.value'
value is JcStringConstant && value.value == constant.value
}
}
}
return false
}

override fun visit(condition: ConstantLt): Boolean {
positionResolver.resolve(condition.position).onSome { value ->
return when (val constant = condition.value) {
is ConstantIntValue -> {
value is JcInt && value.value < constant.value
}

else -> error("Unexpected constant: $constant")
}
}
return false
}

override fun visit(condition: ConstantGt): Boolean {
positionResolver.resolve(condition.position).onSome { value ->
return when (val constant = condition.value) {
is ConstantIntValue -> {
value is JcInt && value.value > constant.value
}

else -> error("Unexpected constant: $constant")
}
}
return false
}

override fun visit(condition: ConstantMatches): Boolean {
positionResolver.resolve(condition.position).onSome { value ->
val re = condition.pattern.toRegex()
return re.matches(value.toString())
}
return false
}

override fun visit(condition: SourceFunctionMatches): Boolean {
TODO("Not implemented yet")
}

override fun visit(condition: ContainsMark): Boolean {
error("This visitor does not support condition $condition. Use FactAwareConditionEvaluator instead")
}

override fun visit(condition: TypeMatches): Boolean {
positionResolver.resolve(condition.position).onSome { value ->
return value.type.isAssignable(condition.type)
}
return false
}
}

class FactAwareConditionEvaluator(
private val fact: Tainted,
positionResolver: PositionResolver<Maybe<JcValue>>,
) : BasicConditionEvaluator(positionResolver) {

override fun visit(condition: ContainsMark): Boolean {
if (fact.mark != condition.mark) return false
positionResolver.resolve(condition.position).onSome { value ->
val variable = value.toPath()

// FIXME: Adhoc for arrays
val variableWithoutStars = variable.removeTrailingElementAccessors()
val factWithoutStars = fact.variable.removeTrailingElementAccessors()
if (variableWithoutStars == factWithoutStars) return true

return variable == fact.variable
}
return false
}

private fun AccessPath.removeTrailingElementAccessors(): AccessPath {
val accesses = accesses.toMutableList()
while (accesses.lastOrNull() is ElementAccessor) {
accesses.removeLast()
}
return AccessPath(value, accesses)
}
}
Loading