Skip to content

Commit cdb5d86

Browse files
author
MayaCrmi
committed
Merge branch 'main' of github.com:redhat-community-ai-tools/UnifAI into GENIE-1148/story/create-ui-testing
2 parents ff7ad81 + d432cbe commit cdb5d86

File tree

213 files changed

+14163
-397
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

213 files changed

+14163
-397
lines changed

.coderabbit.yaml

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
reviews:
2+
auto_review:
3+
enabled: true
4+
auto_incremental_review: true
5+
6+
commit_status: true
7+
high_level_summary: true
8+
9+
path_filters:
10+
- "multi-agents/**"
11+
- "backend/**"
12+
- "ui/**"
13+
14+
instructions: |
15+
This repository represents an agentic AI system with a modular, layered architecture.
16+
17+
Each of the main directories (multi-agents/, backend/, ui/) contains README.md and ARCHITECTURE.md files
18+
that define architectural intent, responsibilities, and boundaries.
19+
20+
These documents should be treated as the source of truth.
21+
22+
During reviews, always cross-check code changes against the architectural guidance in those files.
23+
24+
FOCUS AREAS:
25+
- architecture
26+
- maintainability
27+
- modularity
28+
- separation of concerns
29+
- dependency boundaries
30+
- layering
31+
- testability
32+
33+
Please pay special attention to:
34+
35+
- Whether new components are placed in the correct layer and folder
36+
- Whether responsibilities align with the architecture defined in the directory-level README.md and ARCHITECTURE.md files
37+
- Whether any architectural boundaries are violated
38+
- Whether new dependencies increase coupling in undesirable ways
39+
- Whether changes weaken modularity or future extensibility
40+
- Whether agent, backend, and UI concerns are leaking across layers
41+
42+
Prefer architectural and system-level feedback over stylistic feedback.
43+
When relevant, explicitly reference architectural mismatches (for example: "this component seems to belong under X based on ARCHITECTURE.md").
44+
45+
Act as a senior software engineer deeply familiar with this project's architecture.

.github/CODEOWNERS

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
* @odaiodeh @nirsisr
22

3-
ci/ @sfiresht
4-
helm/ @sfiresht
3+
ci/ @sfiresht @yhabushi
4+
helm/ @sfiresht @yhabushi
5+
.github/ @sfiresht @yhabushi

.github/CODERABBIT.md

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
# 🤖 CodeRabbit – Automated Code Review Agent
2+
3+
This repository integrates **CodeRabbit** as an automated AI-powered code review (CR) agent.
4+
5+
CodeRabbit analyzes every pull request and provides structured, context-aware feedback directly on the PR, helping improve overall code quality, maintainability, and consistency across the project.
6+
7+
---
8+
9+
## 🎯 Purpose
10+
11+
The goal of using CodeRabbit in this project is to serve as a **first-line automated code reviewer** that assists both contributors and maintainers by:
12+
13+
- Summarizing the intent and scope of each pull request
14+
- Highlighting potential bugs, edge cases, and logical issues
15+
- Suggesting improvements related to code quality, readability, and best practices
16+
- Identifying maintainability, performance, and security concerns early
17+
18+
CodeRabbit complements human reviewers and our internal CR agents. It is not intended to replace manual reviews, but to strengthen and accelerate them.
19+
20+
---
21+
22+
## ⚙️ How It Works
23+
24+
CodeRabbit is automatically triggered via **GitHub Actions** on every:
25+
26+
- Newly opened pull request
27+
- Updated pull request (new commits)
28+
- Reopened pull request
29+
30+
Once triggered, CodeRabbit:
31+
32+
1. Analyzes the code changes in the context of the repository
33+
2. Posts a high-level summary explaining what the PR introduces
34+
3. Adds inline review comments on relevant lines in the diff
35+
4. Provides actionable suggestions to improve code quality and structure
36+
37+
All feedback appears directly inside the pull request conversation and diff view.
38+
39+
---
40+
41+
## 🧠 Role in the Application
42+
43+
Within this project’s agentic AI system, CodeRabbit operates as a **general-purpose CR agent**, focusing on:
44+
45+
- Broad software engineering best practices
46+
- Cross-file change awareness
47+
- Early issue detection
48+
- Developer-oriented feedback
49+
50+
It works alongside our internal AI CR assistant, which focuses on project-specific logic, architectural reasoning, and workflow validation. Together, they form a multi-agent automated code review pipeline.
51+
52+
---
53+
54+
## 🧩 Benefits
55+
56+
- Faster review cycles
57+
- Higher baseline code quality
58+
- Reduced reviewer fatigue
59+
- More consistent feedback across pull requests
60+
- Early detection of issues before human review
61+
62+
---
63+
64+
## 🔁 Continuous Improvement
65+
66+
CodeRabbit continuously adapts through interaction in pull requests and repository-level configuration, allowing its reviews to better align over time with the project’s conventions and expectations.
67+
68+
---

.github/README.md

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
# GitHub Actions & CI/CD
2+
3+
This folder contains all CI/automation scripts and workflows for GitHub Actions.
4+
5+
## Overview
6+
7+
Workflows are configured to run automated tasks using GitHub Actions. For complex operations, we use dedicated scripts (in `.github/scripts/`) that are invoked from the workflow files.
8+
9+
## Available Workflows
10+
11+
- **backup-dbs.yaml** - Automated database backups for MongoDB and Qdrant (backups are being uploaded to internal gitlab and have no retention at the moment)
12+
- **verify-agent-deps.yaml** - Dependency verification for agents
13+
14+
15+
16+
## Prerequisites
17+
18+
1. GitHub must be able to access the target cluster **OR** you must have a self-hosted runner that can access both GitHub and the cluster (see [Creating a Runner](#creating-a-runner) below)
19+
2. GitHub Environments must be configured with the appropriate variables and secrets for each cluster (e.g., `PRE-PRODUCTION`, `PRODUCTION`)
20+
21+
### Important Notes
22+
23+
- Since every deployment is a bit different. the existing workflows won't necessarily work out of the box for deployment different from the one currently in use. Users wanting to deploy UnifAI in their own clusters should be aware of the infra and networking to either fit the workflow to their needs or create a new workflow that fits it.
24+
- When using runners, the `runs-on` field refers to **labels**, not runner names. Ensure matching labels exist before running workflows.
25+
- Environment-specific variables (like `QDRANT_URL`, `MONGO_URI`, `API_URL`) must be configured in GitHub repository settings under **Environments**.
26+
27+
## GitHub Environments Setup
28+
29+
The workflows use GitHub Environments to manage cluster-specific configurations:
30+
31+
1. Go to **Settings****Environments** in your repository
32+
2. Create environments matching your cluster names (e.g., `PRE-PRODUCTION`, `PRODUCTION`)
33+
3. Add environment-specific variables:
34+
- `API_URL` - Kubernetes API server URL
35+
- `MONGO_URI` - MongoDB connection string
36+
- `QDRANT_URL` - Qdrant cluster URL
37+
4. Add environment-specific secrets:
38+
- `ACCESS_TOKEN` - Kubernetes access token
39+
- Other sensitive credentials as needed
40+
41+
## Database Backup Details
42+
43+
### MongoDB Backup
44+
45+
MongoDB backups are performed using `mongodump`, which is straightforward:
46+
47+
```bash
48+
mongodump --uri="mongodb://localhost:27017" --out="/tmp/backup"
49+
```
50+
51+
**Parameters:**
52+
- `--uri` - Connection string to the MongoDB instance to backup
53+
- `--out` - Target directory for the backup (creates a new folder)
54+
- `--db` (optional) - Specific database name (default: all databases)
55+
56+
### Qdrant Backup
57+
58+
Qdrant backups require creating snapshots via the API or UI. The workflow uses a Python script (`.github/scripts/qdrant_backup.py`) to:
59+
1. Connect to the Qdrant cluster
60+
2. Create snapshots for all collections
61+
3. Download the snapshots locally
62+
4. Upload them to the backup repository
63+
64+
For more details, see the [Qdrant documentation](https://qdrant.tech/documentation/database-tutorials/create-snapshot/).
65+
66+
## Running Workflows Manually
67+
68+
### Prerequisites
69+
70+
1. The workflow must have the `workflow_dispatch` trigger enabled
71+
2. GitHub CLI must be installed and authenticated
72+
3. The workflow file must exist in the `main` branch (workflows in feature branches cannot be manually triggered)
73+
74+
### Example Command
75+
76+
```bash
77+
gh workflow run backup-dbs.yaml \
78+
-f target_cluster=PRE-PRODUCTION \
79+
-f target_branch=GENIE-1071/backup_dbs \
80+
-f target_namespace=tag-ai--pipeline
81+
```
82+
83+
**Parameters:**
84+
- `-f target_cluster` - The cluster environment to backup (must match a configured GitHub Environment)
85+
- `-f target_branch` - The branch to checkout for the workflow
86+
- `-f target_namespace` - The Kubernetes namespace to backup
87+
88+
## Appendix
89+
90+
### Creating a Runner
91+
92+
To create a new self-hosted runner:
93+
94+
1. Go to your repository's **Settings** tab
95+
2. In the left sidebar, select **Actions****Runners**
96+
3. Click **New self-hosted runner**
97+
4. Follow the setup instructions (the authentication tokens are unique to your repository)
98+
99+
For more details, see the [GitHub documentation on self-hosted runners](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/adding-self-hosted-runners).
100+
101+
102+
### Connecting to gitlab
103+
104+
Since the GitHub runners can't reach gitlab we ha to use a VM running on CNV.
105+
To make gitlab "accessible" to this runner, we need to set a deploy token at the target repo (go to repository > deploy keys and set the VM public key as the deploy key). This allows the VM perform actions on the target repo without needing to specify credentials.
106+
107+
### UnifAI team infra structure
108+
109+
In the case if the Unifai team the lab structure is a bit "special" the code resides in a public GitHub repo whereas all the deployment resources reside inside the company intra-net. to overcome this we have a self-hosted runner with access to both domains so the code is downloaded from github (for example in order to run a workflow) and then all actions are being run against the intra resources.
110+
111+

.github/requirements.txt

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
qdrant-client==1.16.1
2+
requests>=2.32.5
3+
kubernetes==35.0.0
4+
GitPython==3.1.46

.github/scripts/backup_mongo.py

Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
import os
2+
import subprocess
3+
from datetime import datetime
4+
from kubernetes import client, config
5+
from kubernetes.stream import stream
6+
7+
8+
9+
# Environment variables
10+
MONGO_POD = os.getenv("MONGO_POD")
11+
NAMESPACE = os.getenv("NAMESPACE")
12+
CLUSTER = os.getenv("CLUSTER")
13+
ACCESS_TOKEN = os.getenv("ACCESS_TOKEN")
14+
API_URL = os.getenv("API_URL")
15+
MONGO_URI = os.getenv("MONGO_URI")
16+
VERIFY_TLS = bool(os.getenv("SKIP_VERIFY_TLS"))
17+
18+
def setup_k8s_connection():
19+
"""
20+
Set up Kubernetes connection using environment variables
21+
"""
22+
kube_config = {
23+
"apiVersion": "v1",
24+
"kind": "Config",
25+
"clusters": [{
26+
"name": CLUSTER,
27+
"cluster": {
28+
"server": API_URL,
29+
"insecure-skip-tls-verify": VERIFY_TLS
30+
}
31+
}],
32+
"users": [{
33+
"name": CLUSTER,
34+
"user": {"token": ACCESS_TOKEN}
35+
}],
36+
"contexts": [{
37+
"name": CLUSTER,
38+
"context": {
39+
"cluster": CLUSTER,
40+
"user": CLUSTER,
41+
"namespace": NAMESPACE
42+
}
43+
}],
44+
"current-context": CLUSTER
45+
}
46+
47+
config.load_kube_config_from_dict(kube_config)
48+
print(f"✓ Connected to {CLUSTER}")
49+
return client.CoreV1Api()
50+
51+
def check_k8s_connection():
52+
"""Verify connection by listing resources"""
53+
v1 = client.CoreV1Api()
54+
apps_v1 = client.AppsV1Api()
55+
56+
print("Checking pods and deployments...")
57+
pods = v1.list_namespaced_pod(namespace=NAMESPACE)
58+
deployments = apps_v1.list_namespaced_deployment(namespace=NAMESPACE)
59+
60+
print(f"Found {len(pods.items)} pods and {len(deployments.items)} deployments")
61+
return True
62+
63+
def run_cmd_on_pod(pod_name: str, namespace: str, command: list[str]):
64+
v1 = client.CoreV1Api()
65+
result = stream(
66+
v1.connect_get_namespaced_pod_exec,
67+
pod_name,
68+
namespace,
69+
command=command,
70+
stderr=True,
71+
stdin=False,
72+
stdout=True,
73+
tty=False
74+
)
75+
return result
76+
77+
def copy_backup_from_pod(local_path: str = None):
78+
"""
79+
Copy the compressed backup file from the pod to local filesystem
80+
81+
Args:
82+
local_path: Local path to save the backup. If None, generates a timestamped filename
83+
84+
"""
85+
if local_path is None:
86+
timestamp = datetime.now().strftime('%Y%m%d_%H%M%S')
87+
local_path = f"/tmp/mongo_backup_{timestamp}.tar.gz"
88+
89+
print(f"Downloading backup from pod to {local_path}")
90+
91+
# Using kubectl cp command
92+
pod_spec = f"{NAMESPACE}/{MONGO_POD}:/tmp/backup.tar.gz"
93+
cmd = ['kubectl', 'cp', pod_spec, local_path, '-n', NAMESPACE, '--retries', '10']
94+
95+
result = subprocess.run(cmd, capture_output=True, text=True)
96+
97+
if result.returncode != 0:
98+
raise Exception(f"Failed to copy backup from pod: {result.stderr}")
99+
100+
print(f"✓ Backup downloaded to {local_path}")
101+
102+
103+
def remove_old_backup():
104+
print("Removing old backup if they exist")
105+
run_cmd_on_pod(MONGO_POD, NAMESPACE, ["rm", "-rf", "/tmp/backup"])
106+
run_cmd_on_pod(MONGO_POD, NAMESPACE, ["rm", "-rf", "/tmp/backup.tar.gz"])
107+
print("Old backup removed")
108+
109+
def test_mongodb_connection():
110+
print("Testing MongoDB connection")
111+
run_cmd_on_pod(MONGO_POD, NAMESPACE, ["mongosh", "--eval", "db.version()", "--uri", MONGO_URI])
112+
print("MongoDB connection test completed")
113+
114+
def backup_mongodb():
115+
print("Running MongoDB backup")
116+
run_cmd_on_pod(MONGO_POD, NAMESPACE, ["mongodump", "--uri", MONGO_URI, "--out", "/tmp/backup"])
117+
print("MongoDB backup completed")
118+
119+
def compress_backup():
120+
print("Compressing MongoDB backup")
121+
run_cmd_on_pod(MONGO_POD, NAMESPACE, ["tar", "-czf", "/tmp/backup.tar.gz", "/tmp/backup"])
122+
print("MongoDB backup compressed")
123+
124+
if __name__ == "__main__":
125+
# Setup connection
126+
setup_k8s_connection()
127+
check_k8s_connection()
128+
129+
# Run backup
130+
print("Starting MongoDB backup...")
131+
remove_old_backup()
132+
test_mongodb_connection()
133+
backup_mongodb()
134+
compress_backup()
135+
copy_backup_from_pod()
136+
print("✓ Backup complete!")

0 commit comments

Comments
 (0)