Skip to content

Conversation

HayenNico
Copy link
Member

@HayenNico HayenNico commented May 30, 2025

Description

Add new controllers OrigDatablocksV4Controller and OrigDatablocksPublicV4Controller, mirroring the structure for Datasets. Add required functionality in the OrigDatablocksService and refactor support functions from DatasetsV4 to the common module where possible

Motivation

The improvements from API v4 currently only cover the Dataset subsystem, other collections are not yet available under /api/v4/ endpoints. Instead of copying large parts of the v3 API, v4 should use the new querying standards developed for Datasets.
This PR is a trial run to transfer the same functionality to the OrigDatablocks system.

Changes:

  • add OrigDatablocksV4Controller and OrigDatablocksPublicV4Controller with full v3 functionality

  • merge GET /api/v3/origdatablocks and /api/v3/fullquery endpoints to /api/v4/origdatablocks

  • move GET /api/v3/origdatablocks/fullquery/files endpoint to /api/v4/origdatablocks/files

  • copy querying machinery from Datasets v4 implementation

  • update OrigDatablocksService with new functions to accept queries in the v4 format

  • add isPublished as optional field in UpdateOrigDatablockDto

  • add unit and api tests for new systems

  • add pid as required field in OutputDatasetDto (was optional as inherited from CreateDatasetDto)

  • Refactor of support functions:

  • FilterValidationPipe and IncludeValidationPipe moved to common module

  • Needed constants moved into collection/types/collection-lookup.ts files for import

  • All API v4 controllers refactored to use common module pipes, removed individual versions

Tests included

  • Included for each change/fix?
  • Passing?

Documentation

  • swagger documentation updated (required for API changes)
  • official documentation updated

official documentation info

@HayenNico HayenNico self-assigned this May 30, 2025
@HayenNico HayenNico added enhancement New feature or request DCS DAPHNE Contribution to SciCat labels May 30, 2025
@HayenNico
Copy link
Member Author

One additional proposed change:
Right now during OrigDatablock creation any ownerGroup, accessGroups or instrumentGroup specified in the CreateOrigDatablockDto will always be overwritten by the fields of the Dataset referenced by datasetId. I propose changing that behavior to only use the Dataset values if the fields are not set in the DTO. Opinions?

@HayenNico HayenNico marked this pull request as ready for review June 12, 2025 11:42
@HayenNico

This comment was marked as resolved.

@Junjiequan
Copy link
Member

Plus one minor oddity: Swagger does not add own tabs for the two OrigDatablock controllers like for Dataset, instead the public endpoints are listed under origdatablocks v4. Any idea why?

The API tag for origdatablock public controller needs to be changed

@HayenNico
Copy link
Member Author

HayenNico commented Jun 12, 2025

The API tag for origdatablock public controller needs to be changed

Okay, will be fixed. Not sure how I managed to miss that...

Copy link
Collaborator

@martin-trajanovski martin-trajanovski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR in general looks great. I have left few minor comments but otherwise I think you did a great job.

@@ -1364,7 +1372,7 @@ export class CaslAbilityFactory {
accessGroups: { $in: user.currentGroups },
});
can(Action.OrigdatablockReadOneAccess, OrigDatablock, {
ownerGroup: { $in: user.currentGroups },
isPublished: true,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So as I understand with this change the CREATE_DATASET_PRIVILEGED_GROUPS can only read published Origdatablock and not all that are in their access group

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The removed check for ownerGroup here was a duplicate, the exact same check is still in the code a few lines above. This one was a copy-paste error I assume, where there should always have been the isPublished check..

})
@IsOptional()
@IsBoolean()
readonly isPublished?: boolean;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess the idea behind adding the field in the update dto is to be able to control it from outside by allowing a user with proper access to update the isPublished flag

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is specifically in the update DTO because the base class OwnableDto does not include isPublished, so each subclass needs to add it manually. From #1757 I got the impression that long-term isPublished may not stay in the Ownable schema, so i opted to put it here

@@ -1,40 +1,12 @@
import { PipeTransform, Injectable } from "@nestjs/common";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea that you made this reusable and moved it to a common module

@fpotier fpotier linked an issue Jul 2, 2025 that may be closed by this pull request
@Junjiequan Junjiequan force-pushed the origdatablock-v4-controllers branch from 6f237a6 to aa02269 Compare July 17, 2025 12:44
@nitrosx nitrosx merged commit 776025b into master Aug 1, 2025
13 checks passed
@nitrosx nitrosx deleted the origdatablock-v4-controllers branch August 1, 2025 12:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
DCS DAPHNE Contribution to SciCat enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Anonymous user can read non-public OrigDatablock
4 participants