-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Sideloading Wrapper for the Attributes/JSON Adapters #1198
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
_flattened[key] << data | ||
end | ||
|
||
def ids_name_for(name) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
are there built in helpers for these?
@@ -8,6 +8,7 @@ module Configuration | |||
base.config.array_serializer = ActiveModel::Serializer::ArraySerializer | |||
base.config.adapter = :attributes | |||
base.config.jsonapi_resource_type = :plural | |||
base.config.sideload_associations = false |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This config should be namespaced, as it is specific to this adapter.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah yes, this was left over from the last implementation. thanks
What's the purpose of this adapter and where / how is it used? I can't tell. Also, what does the name mean? |
@bf4 the purpose is for those who for some reason can't use json-api, and need sideloaded data (for example, in an ember app - which doesn't like nested relationships) I called it flat json, because there should be no nested json objects representing models in the result -- the originally nested json is _flat_tened. idk. I'm not in love with the name. |
@byteg glad you find this helpful! Unfortunately, I don't know the best way to go about only including the ids for an association. And then, I don't know how nested relationships would work when only using ids. But maybe there are some things from json api that we could borrow from for this? |
@NullVoxPopuli i could solve the problem with that simple workaround: class PostSerializer < ActiveModel::Serializer def comment_ids undefined method `each_with_object' for 1:Fixnum If it's possible to fix that, all the problems are solved for now :) |
I am pretty sure Ember supports nested relationships. Which ember adapter/serializer do you use ? In any case it's good to have more adapters given that they are maintained of course. |
@vasilakisfil I use the ActiveModelAdapter. |
@byteg I just pushed something that fixes your issue. However, as I continue to work on this, I begin to be more and more in favor of just having people switch to json-api. I'll be making this change myself in the coming months, whenever I can find time for it. |
Is this still valid? Is the core team interested to merge it? Would be nice cause it would provide compatibility with 0.8.x and 0.9.x adapters. |
Any updates on how 0.10.x handles sideloading the association ids with the json adapter? I tried the config.sideload_associations but that didn't work? And I'm getting undefined method embed because it seems this has been an ongoing debate for a while now. Just curious about the latest? |
@erlandsona does using this branch in your gemfile help your problem?
|
@erlandsona unfortunately the core team is not interested to interoperate the 0.8.x/0.9.x adapters in 0.10.x so the only way is to build your own adapter :) @NullVoxPopuli PR is a good start. @NullVoxPopuli this branch embeds the whole associations not just the ids which I guess @erlandsona wants |
Gimme a second to try @NullVoxPopuli and see if that works... |
@vasilakisfil we'd just like everyone to upgrade, but we understand that - that isn't a possibility with some people, and that's why everything is (or is trying to be) architected to be pretty flexible. .8 and .9 compatibility can be added to this adapter. -- I'm no expert on the older versions, though -- so I'm not sure how much hackery would need to be done to achieve full 0.8 / 0.9 compatibility. |
Sure I can work on fixining the compatibility, but is the core interested to merge the new adapter in the master or will it be yet another pull request hanging around ? |
Because saying to the user use |
@NullVoxPopuli I just tried you're version with the config.sideload_associations option set to true but it's still giving me the entire record. Here's the output and setup... Output (json) localhost:3000/brands (shortened)
localhost:3000/models (shortened)
Here's some more info on the setup...
brand_serializer.rb
model_serializer.rb
sub_model_serializer.rb
brands_controller.rb
A few things I'm trying to get to work are... I'd like only to include the id's of the associations because I'm getting the infinite loop with the HABTM circular reference that happens when I try to get brands from models and brands gets models which gets brands etc... I also want to utilize if possible Rails include to get rid of the N+1 issue. I'm also trying to figure out how to get the Oj gem to play nicely with AMS so when I ask for per_page=2500 it doesn't take 5+ seconds to load the json response??? Can't quite figure that out either. If anyone has anytime to set up a google hangout to figure this out I would greatly appreciate the help! |
are you setting the adapter to :flat_json? |
@erlandsona what you ask for (including only ids) is not possible in this adapter. But yea you should set it to :flat_json :) |
:O did you embed the ids? cause I was trying that last week and wasn't I could say it's almost 0.9.x compliant then
|
@erlandsona in your controller, you could specify the basically, using you can also manually specify which associations you want, like: |
@NullVoxPopuli I've come across that reading through all these issues and there's a couple places where people define the includes piece and I just tried to add it alone and with the includes calls in the models but without the includes calls in the models I'm still getting N+1 and with them I get infinite loop. I'm probably doing it wrong though... here's the server output with the includes calls in the models
|
@vasilakisfil You're right that 0.9.x is set up this way but with most issues I've read about the main contributors basing the design of 0.10.x on 0.8.x I didn't want to commit myself to a version that wasn't going to be well supported in the future? I don't know maybe that's wise or not, I'm just trying to get all this stuff to just work. |
@erlandsona it's possible then that this adapter will need re-doing a bit, and actually won't be able to inherit from the attributes adapter. cause it's the attributes adapter that's causing the infinite loop. if the sideloading only adapter didn't inherit from attributes, it could just skip over models that have already been prepped for serialization. |
Hmm... little over my head there. I just want to be sure you're clear that the infinite loop only happens when I specify the includes calls in the models. I'm not actually sure it's an AMS issue or whether it's just a straight up Rails issue? But I'm not finding anything useful on SO about HABTM with includes going both directions? That said, your inclination is that the adapter is the issue here? What I'm really asking is whether or not I screwed up somewhere? I'm just trying to get json where...
and for /models
I really don't want to have to look into other Serializer gems because I keep reading about how jBuilder is too slow or too complex and AMS just has this really nice PORO interface that I enjoy but I'm just trying to get it to do what I want? |
I think the problem is happening because of the circular association definition in the models. However, in an adapter, it's possible to evade infinite loops by keeping track of which models have already been prepped for serialization (and this would be the fastest way to side load). Currently, the whole json object is 'rendered' in the attributes adapter, and then the FlatJson adapter parses the output, and rearranges everything in to the nifty sideloadedness. The solution here would be for me to change the implementation of this adapter. Which, I can do tomorrow morning. :-) |
Hmmm! Alrighty then. I'll look forward to seeing the update! Is this issue the best place to keep in touch or where else should I check in? |
yeah, this is a good spot. :-) |
Alright, sounds good... Not to be buggy but should I follow for adding pagination links with the :flat_json adapter too? Or do you have any other thoughts? |
most of the json-api-specific features aren't planned for the other adapters, as far as I know. I'd first perfer a re-architecting all adapter are to share features. |
Yeah that sounds nice! |
possibly a cleaner way to do this would be to just unwrap the json-api produced hash - that way all the features with json api (the most actively developed adapter) would be present in the sideloaded json adapter |
removed pry rubocop! minor cleanup, 100% coverage remove html formatter that was inserted for debugging coverage forgot to remove some things -- improved call to super fixed issue where ids array is manually specified resurrect some old code that sideloads, and should have infinite recursion protection
…izers into sideloaded-json
…ed include parsing in deserialization
cf55dc8
to
746eae1
Compare
@erlandsona I just made some changes to this branch which implement @beauby 's deserialization (with some modifications). Can you see if that meets your needs? |
# the key in data is changed to be plural, | ||
# so we need to check for the existence of both | ||
# singular and plural keys | ||
exists = data.keys.include?(key) || data.keys.include?(plural_key) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@beauby these are the types of inline comments I talk about when I ask you to add comments.
just real informal explanation behind the decision of the implementation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't like those, they clutter my screen and I can't see the code :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it's very helpful for people seeing the code for the first time though. and I think it's worth it. It reduces the required amount of retained control flow in a person's short term memory, which leads to quicker changes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As someone who has looked a lot in the code of this project and monkey patched some of it to adapt in my needs I have to admit such comments would help me a lot + would make me confident to brag about my changes and send pull requests back. Now I feel like I have found a way to "make it work" how I want it but there might be a better way because I don't have a clue what the original maintainer meant/wanted to achieve on some parts. Ruby is awesome but sometimes it's not self explained like other languages :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, exactly. :-)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To be honest, I'm all up for comments stating intents, but still against comments demystifying bad code (which should instead be rewritten into good code).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I totally agree with your intention there. No amount of comments can compensate for bad code.
But even good code needs narration.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@beauby I'm pro @NullVoxPopuli and @vasilakisfil. If the code isn't self-documenting, it needs documentation. Comments help tremendously. Ideal could is self documenting, real code, whether good or bad, may benefit from comments explaining why the code is the way it is, what decisions were made, and why. When I read code, I don't want to have to pull my hair out trying to figure out what it does. So, major props to @NullVoxPopuli for pointing out this example
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OTOH, classic tweet:
@NullVoxPopuli Sorry I've been absent from this thread... I've since moved onto just overriding as_json in the model layer to make things compatible with the Oj gem... That said, this project is also currently undergoing an overhaul... Given the current state of my project... I'm foreseeing the eventual need to come back to AMS. That said I fixed the N+1 issue I was having with the old structure and realized I didn't actually need the nested association reference id's in the JSON output. My current issue is trying to get AMS to play nicely with the Oj gem... basically the main issue is that there's like four or five different definitions of the as_json / to_json methods going on between the two gems and rails... so I can't get a handle on what's happing where and when to make sure that I'm properly integrating both gems... Any thoughts from the AMS community? |
do you have a link to the Oj gem? I haven't heard of it. |
|
||
puts '--------------------------' | ||
puts data | ||
puts '----------------------------' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
obviously the puts has to go :)... maybe use the logger here for your purposes?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
:-)
yeah, I don't think I'm going to make any updates to this until the deserialization PR gets merged.
Then I can apply the suggestions @beauby had
I think I'm going to close this, as I want to support the push to JSON API, and not try to mimic its functionality in less-strict adapters. |
Related: #1084 #1107 #1088
So, in order to preserve functionality implemented in the attributes/json adapters, I decided to still use the attributes adapter to handle the rendering of the hash/json (so we get all the features (include, etc)).
It is a little more overhead than sideloading while iterating over the associations, but I think this could be the least likely to produce bugs, as it relies on pre-existing, thoroughly tested code.
the purpose is for those who for some reason can't use json-api, and need sideloaded data (for example, in an ember app - which doesn't like nested relationships)