refactor: fully extend dynamic mat tables

[abdimo101]

Description

Dynamic mat tables are now fully extended and pagination system has also been added above the tables.

Motivation

Background on use case, changes needed

Fixes:

• Items added

Changes:

• Replaced cdk-virtual-scroll-viewport with a standard table
• Created a datasource that extends MatTableDataSource

Tests included

• Included for each change/fix?
• Passing? (Merge will not be approved unless this is checked)

Documentation

• swagger documentation updated [required]
• official documentation updated [nice-to-have]

official documentation info

If you have updated the official documentation, please provide PR # and URL of the pages where the updates are included

Backend version

• Does it require a specific version of the backend
• which version of the backend is required:

Summary by Sourcery

Refactor dynamic-mat-table to fully migrate from virtual scrolling to a standard Material table configuration with built-in pagination and a custom TableDataSource, clean up related code and templates, update styles, and adjust end-to-end tests accordingly.

New Features:

• Introduce TableDataSource extending MatTableDataSource to power sorting, filtering, and pagination
• Add MatPaginator above the table and wire it into the new data source

Enhancements:

• Replace CDK virtual scroll viewport with a standard MatTable setup and clean up virtualization logic throughout the component and core directive
• Remove inline height styles and update elementRefs for scrolling instead of viewport APIs
• Adjust dynamic-mat-table templates, directives, and styles to align with the standard table and pagination
• Add ChangeDetectorRef calls in dataset component to ensure view updates after data load

Tests:

• Update Cypress e2e selectors to target the new MatPaginator structure

[sourcery-ai]

Hey there - I've reviewed your changes - here's some feedback:

• Remove the stray console.log in DynamicMatTableComponent to avoid leaking debug output into production logs.
• Ensure any manual RxJS subscriptions (e.g. dataSource.subscribe or sort.sortChange) are properly cleaned up in OnDestroy to prevent memory leaks.
• Clean up leftover virtual-scroll and tvsDataSource references in the directive/component now that only the standardDataSource is used.

Prompt for AI Agents

Please address the comments from this code review:
## Overall Comments
- Remove the stray console.log in DynamicMatTableComponent to avoid leaking debug output into production logs.
- Ensure any manual RxJS subscriptions (e.g. dataSource.subscribe or sort.sortChange) are properly cleaned up in OnDestroy to prevent memory leaks.
- Clean up leftover virtual-scroll and tvsDataSource references in the directive/component now that only the standardDataSource is used.

## Individual Comments

### Comment 1
<location> `src/app/shared/modules/dynamic-material-table/table/dynamic-mat-table.component.ts:292` </location>
<code_context>
           const visibleColumns = this.columns.filter(
             (c) => c.display !== "hidden" && c.index < data.e.columnIndex,
           );
+          console.log("DEBUG visibleColumns:", visibleColumns);
           i = visibleColumns[visibleColumns.length - 1].index;
         }
</code_context>

<issue_to_address>
Leftover debug logging should be removed for production code.

Console.log statements may expose sensitive data and clutter production logs. Please remove or use an appropriate logging solution.
</issue_to_address>

### Comment 2
<location> `src/app/shared/modules/dynamic-material-table/cores/table.core.directive.ts:356` </location>
<code_context>
-      if (this.viewport) {
-        this.viewport.scrollTo({ top: 0, behavior: "auto" });
+      if (this.tableContainer) {
+        this.tableContainer.nativeElement.scrollTop = 0;
       }
       this.tvsDataSource.clearData();
</code_context>

<issue_to_address>
Direct DOM manipulation for scrolling may not be cross-platform or framework-idiomatic.

Using nativeElement.scrollTop may cause compatibility issues in platforms where direct DOM access is discouraged. Prefer Angular's Renderer2 for safer, framework-consistent DOM manipulation.

Suggested implementation:

```typescript
  public clear() {
    if (!isNullorUndefined(this.tvsDataSource)) {
      if (this.tableContainer) {
        this.renderer.setProperty(this.tableContainer.nativeElement, 'scrollTop', 0);
      }
      this.tvsDataSource.clearData();
      this.expandedElement = null;
      this.cdr.detectChanges();
      this.refreshColumn(this.tableColumns);
      this.table.renderRows();
    }
  }

```

```typescript
    this.tableSetting.scrollStrategy = value;
  }

```

You must inject `Renderer2` in your directive/class constructor. For example:

```typescript
constructor(
  private renderer: Renderer2,
  // ...other injections
) { }
```

If your class does not already have Renderer2 injected, add it to the constructor and update all usages accordingly.
</issue_to_address>

### Comment 3
<location> `src/app/shared/modules/dynamic-material-table/cores/table-data-source.ts:21` </location>
<code_context>
-  T extends TableRow,
-> extends MatTableDataSource<T> {
-  private streamsReady: boolean;
+export class TableDataSource<T extends TableRow> extends MatTableDataSource<T> {
   private filterMap: HashMap<AbstractFilter[]> = {};
-  public dataToRender$: Subject<T[]>;
</code_context>

<issue_to_address>
Consider maintaining the separation of filtering logic and virtual scroll features to keep responsibilities clear and avoid code duplication.

No actionable changes—this split cleanly separates core filtering (in TableDataSource) from virtual‐scroll concerns (in TableVirtualScrollDataSource) without duplication.
</issue_to_address>

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[sourcery-ai]

suggestion: Direct DOM manipulation for scrolling may not be cross-platform or framework-idiomatic.

Using nativeElement.scrollTop may cause compatibility issues in platforms where direct DOM access is discouraged. Prefer Angular's Renderer2 for safer, framework-consistent DOM manipulation.

Suggested implementation:

  public clear() {
    if (!isNullorUndefined(this.tvsDataSource)) {
      if (this.tableContainer) {
        this.renderer.setProperty(this.tableContainer.nativeElement, 'scrollTop', 0);
      }
      this.tvsDataSource.clearData();
      this.expandedElement = null;
      this.cdr.detectChanges();
      this.refreshColumn(this.tableColumns);
      this.table.renderRows();
    }
  }

    this.tableSetting.scrollStrategy = value;
  }

You must inject Renderer2 in your directive/class constructor. For example:

constructor(
  private renderer: Renderer2,
  // ...other injections
) { }

If your class does not already have Renderer2 injected, add it to the constructor and update all usages accordingly.

[sourcery-ai]

issue (complexity): Consider maintaining the separation of filtering logic and virtual scroll features to keep responsibilities clear and avoid code duplication.

No actionable changes—this split cleanly separates core filtering (in TableDataSource) from virtual‐scroll concerns (in TableVirtualScrollDataSource) without duplication.

[martin-trajanovski]

I have tested this version on my end and I have few concerns. First it is very hard to see the data in the table as there is no horizontal scroll like it used to be and if I expand some columns some others get invisible and the the one can not see anything from the data. I think the previous version worked much better in that sense. Another one is that in the proposals table we have a double horizontal scroll for some reason. Here is a recording that can explain the problems better:

Screenshare.-.2025-08-21.1_58_28.PM.mp4

[abdimo101]

I have tested this version on my end and I have few concerns. First it is very hard to see the data in the table as there is no horizontal scroll like it used to be and if I expand some columns some others get invisible and the the one can not see anything from the data. I think the previous version worked much better in that sense. Another one is that in the proposals table we have a double horizontal scroll for some reason. Here is a recording that can explain the problems better:

Screenshare.-.2025-08-21.1_58_28.PM.mp4

Fixed in 00dbe62 & a2fa0f2

[martin-trajanovski]

Looks good to me now