-
Notifications
You must be signed in to change notification settings - Fork 40
feat: Image embeddings #397
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
packages/react-native-executorch/common/rnexecutorch/host_objects/JsiConversions.h
Outdated
Show resolved
Hide resolved
packages/react-native-executorch/common/rnexecutorch/host_objects/JsiConversions.h
Outdated
Show resolved
Hide resolved
...ages/react-native-executorch/common/rnexecutorch/models/image_embeddings/ImageEmbeddings.cpp
Outdated
Show resolved
Hide resolved
...ages/react-native-executorch/common/rnexecutorch/models/image_embeddings/ImageEmbeddings.cpp
Show resolved
Hide resolved
...ages/react-native-executorch/common/rnexecutorch/models/image_embeddings/ImageEmbeddings.cpp
Show resolved
Hide resolved
packages/react-native-executorch/common/rnexecutorch/models/image_embeddings/ImageEmbeddings.h
Outdated
Show resolved
Hide resolved
...ages/react-native-executorch/common/rnexecutorch/models/image_embeddings/ImageEmbeddings.cpp
Show resolved
Hide resolved
69456e3
to
2157bec
Compare
packages/react-native-executorch/common/rnexecutorch/data_processing/Numerical.cpp
Outdated
Show resolved
Hide resolved
packages/react-native-executorch/common/rnexecutorch/host_objects/JsiConversions.h
Outdated
Show resolved
Hide resolved
aacc266
to
4cbe3a2
Compare
packages/react-native-executorch/common/rnexecutorch/data_processing/Numerical.cpp
Outdated
Show resolved
Hide resolved
**`modelSource`** | ||
A string that specifies the location of the model binary. For more information, take a look at [loading models](../fundamentals/loading-models.md) page. | ||
|
||
**`preventLoad?`** - Boolean that can prevent automatic model loading (and downloading the data if you load it for the first time) after running the hook. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
**`preventLoad?`** - Boolean that can prevent automatic model loading (and downloading the data if you load it for the first time) after running the hook. | |
**`preventLoad?`** - Boolean that can prevent automatic model loading (and downloading the model if you load it for the first time) after running the hook. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i copied it over from other pages. I'd leave it like that - I think we actually mean it. Model loading (which mean downloading a lot of data
)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, but it might be misleading what data we mean
|
||
// Running the model | ||
const embedding = await ImageEmbeddingsModule.forward( | ||
'https://url-to-image.jpg' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
in general i don't like things that would compile without error here, I saw too many peoples just copying it and then being upset about the url not working. Just a thought tho
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's good point, but on the other hand it shows that any existing URL works - '...' doesn't transfer this knowledge
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was thinking something more like ... which just wouldn't run at all and point to this as an error in code syntax itself
|
||
## Loading the model | ||
|
||
To load the model, use the `load` method. It accepts the `modelSource` which is a string that specifies the location of the model binary. For more information, take a look at [loading models](../fundamentals/loading-models.md) page. This method returns a promise, which can resolve to an error or void. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To load the model, use the `load` method. It accepts the `modelSource` which is a string that specifies the location of the model binary. For more information, take a look at [loading models](../fundamentals/loading-models.md) page. This method returns a promise, which can resolve to an error or void. | |
To load the model, use the `load` method. It accepts `modelSource` which is a string that specifies the location of the model binary. For more information, take a look at [loading models](../fundamentals/loading-models.md) page. This method returns a promise which can resolve to an error or void. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Again - it should be fixed in several pages. I think we should utilize .mdx more
Description
Introducing image embeddings feature.
TODO:
benchmarks
Everything else is good to review.
Type of change
Tested on
Related issues
#353
Checklist
Additional notes