Make objectId lexographically sortable

[JasCodes]

New Feature / Enhancement Checklist

• I am not disclosing a vulnerability.
• I am not just asking a question.
• I have searched through existing issues.

Current Limitation

objectID is not sortable by creation time.

Feature / Enhancement Description

Implement ULID spec based on https://www.npmjs.com/package/ulid-workers
One also don't need to store createdAt separately anymore and objectId can be sorted based on creation time without conflicts that might arise when you clone multiple objects from dashboard.

Example Use Case

I wanted to implement cursor based pagination. I wanted to say less than objectID with x limit to load next batch of items.

[parse-github-assistant]

Thanks for opening this issue!

• ❌ Please edit your post and use the provided template when creating a new issue. This helps everyone to understand your post better and asks for essential information to quicker review the issue.

[JasCodes]

@flovilmart

[davimacedo]

The sortable object id by creation time is something that can be discussed but it is not something necessary for your use-case. You can implement cursor based pagination with the current objectId.

@mtrezza thoughts about this?

[JasCodes]

@davimacedo Thanks for coming back on this man. You can implement cursor based pagination with the current objectId. - Can you explain or refer me to doc/article for this? The column needs to be uniquely sortable no?

And is there way to overide objectId generation function on server setup?

[davimacedo]

The objectId is uniquely sortable. It does not represent the creation time, but it is sortable and it is unique. If you order the objects by objectId then you can use the objectId as the cursor key for the pagination. If you want to paginate according to the creation date, it is possible as well. You can combine (createdAt, objectId) and sort by the two columns, in this order, then use the combination of the two columns as the key for the cursor pagination.

Parse Server also supports custom object id: parse-community/Parse-SDK-JS#1309

[JasCodes]

You can combine (createdAt, objectId) and sort by the two columns not very performant unless I put a compound index on those in MongoDB yeah?
Parse Server also supports custom object id: https://github.com/parse-community/Parse-SDK-JS/pull/1309 But can I do it from server-side and not allow clients to set object id?
@davimacedo

[JasCodes]

You can combine (createdAt, objectId) and sort by the two columns, in this order, then use the combination of the two columns as the key for the cursor pagination. Do you mean to create another column for this?

[davimacedo]

You can combine (createdAt, objectId) and sort by the two columns not very performant unless I put a compound index on those in MongoDB yeah?

Yes. An index is welcome for this case.

Parse Server also supports custom object id: parse-community/Parse-SDK-JS#1309 But can I do it from server-side and not allow clients to set object id?

I've never tried that but I believe you can do that on a beforeSave trigger.

You can combine (createdAt, objectId) and sort by the two columns, in this order, then use the combination of the two columns as the key for the cursor pagination. Do you mean to create another column for this?

No. You don't need a new column. Your cursor will be the createdAt and objectId of the last object of the last page. You can pack them: concatenate them (and then optionally encode), or stringify an object with both of them (and then optionally encode), or any other mechanism. When retrieving the next page, you will unpack the last createdAt and last objectId. Then your query will sort the objects by createdAt and objectId at the same time in this order, and then filter by createdAt > lastCreatedAt or createdAt = lastCreatedAt && objectId > lastObjectId.

[JasCodes]

Thanks very much for getting back bro!! @davimacedo
createdAt > lastCreatedAt or createdAt = lastCreatedAt && objectId > lastObjectId I already tried that. Will compound index will be useful in this case? I was sorting them first, that's why I thought compound index. But this sounds like hit to performance for no reason.

Anyways it's far easier and I think performant to create a new column and generate ULID using beforeSave. And custom object id doesn't work on server-side btw as id and createdAt is assigned after beforeSave is run, I think.

But these customizations of setting objectKey or createdAt should be exposed on server configurations in triggers or just passing function in options. What do you think Mr. @mtrezza ?

[JasCodes]

@mtrezza I would even recommend using ULID as default id and createdAt can be just read from it. Infact cursor based pagination should be first-class citizen.

[mtrezza]

Yes, I think allowing to set a custom object ID or overriding createdAt and any of the auto-generated fields server-side would be a useful feature, if that is not possible yet. Please feel free to look into that and submit a PR. There may be some hidden complexity depending on the field, as a fields may be used - in an opaque way - to infer object states.

ULID sounds like a good idea, but it may be a significant change for existing deployments, given the difference in characteristics to the currently used ID. Making ULID a default would certainly be possible, but I think only in the context of a Parse Server option that allows to determine the object ID server side. That new option could easily allow some pre-set ID generators and also allow to set a custom generator function.

You could go about it like this: find out where the object ID is currently being generated in Parse Server and expose that as a Parse Server option with some easy to configure pre-sets.

[JasCodes]

@mtrezza For sure, but I am really new to parse and unfamiliar to the codebase. Though I will love to be a regular code contributor as started using it with two commercial projects. And its always good to give back. Can somebody from the community who knows the codebase do a quick call with me? Is that possible? Are there any web meetups etc?

[mtrezza]

I suggest you look at the respective code path when saving an object (POST route). Then you can see where the object ID is assigned. That is the functionality you'd want to extend. In the contribution guide you can see how to introduce a new Parse Server option. Any questions, feel free to ask.