Add PDF support to OpenAI AsIChatClient #6344
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
RussKie
approved these changes
Apr 29, 2025
eiriktsarpalis
approved these changes
Apr 29, 2025
|
|
||
| case DataContent dataContent when dataContent.MediaType.StartsWith("application/pdf", StringComparison.OrdinalIgnoreCase): | ||
| parts.Add(ChatMessageContentPart.CreateFilePart(BinaryData.FromBytes(dataContent.Data), dataContent.MediaType, $"{Guid.NewGuid():N}.pdf")); | ||
| break; |
There was a problem hiding this comment.
Would there be a drawback to just mapping all other DataContent objects, not limiting it to PDF? Are there cases where someone would add a DataContent but not want it to be sent in the request?
There was a problem hiding this comment.
Just that, at least today, anything else will result in the request failing.
jeffhandley
pushed a commit
that referenced
this pull request
Apr 30, 2025
* Add PDF support to OpenAI AsIChatClient * Add missing reference
jeffhandley
added a commit
that referenced
this pull request
Apr 30, 2025
* Merged PR 49569: Cherry pick Evaluation changes for 9.4.3 release * Update package-lock.json for Evaluations * Reset package-lock.json to same as main branch * Merged PR 49585: [9.4.3] [cherry pick] A couple of minor fixes A couple of minor fixes 1. Adds back tool tip for metrics which are useful in cases where the metric name is long and does not fit entirely in the card 2. Makes the size of the selection buttons on the left of cards consistent with the size of status icons on the right to make the alignment cleaner 3. Implement IEquatable for UrlCacheKey (per Copilot's suggestion) 4. Throw ArgumentException when the number of messages passed to ContentSafetyChatClient does not match expectations (also per Copilot's suggestion) * Add test for optional parameters being required with RequireAllProperties (#6265) * Add test for optional parameters being required with RequireAllProperties * Test both requireAllProperties values * Use AssertDeepEquals that logs the difference to other tests * Adding reference to an unsupported built-in tool on OpenAI Chat API no longer throws (#6276) * Update M.E.AI changelogs (#6269) * Augment UseDistributedCache XML docs (#6256) * Augment AIFunctionFactory.Create XML docs (#6255) * Rename EmbeddingGeneratorExtensions.GenerateEmbedding extension methods (#6295) To align with the the base method on IEmbeddingGenerator * Augment FunctionInvokingChatClient's span with token counts (#6296) * Rename ChatThreadId to ConversationId (#6300) * In OpenAI responses client, use response ID as ChatThreadId * Rename ChatThreadId -> ConversationId * Related renames * Restore deleted members as obsolete (#6304) * Restore EmbeddingGeneratorExtensions members as obsolete * Restore ChatThreadId as obsolete * Restore ChatResponse.ChatThreadId and ChatResponseUpdate.ChatThreadId as obsolete. * Remove the tests for obsolete members * Remove the Embeddings tests for obsolete members * Support [FromKeyedServices] in AIFunctionFactory (#6310) * Utilize IServiceProviderIsService in AIFunctionFactory (#6317) Add AIFunctionFactoryOptions.Services, and use it when examining function parameters to determine whether they should be resolved by default from DI. * Remove AsChatClient/AsEmbeddingGenerator that were obsoleted in 9.4.0-preview.1.25207.5 (#6327) * Add ChatOptions.AllowMultipleToolCalls (#6326) * Add ChatOptions.AllowMultipleToolCalls * Use it in OpenAI adapter * Disable default required property schema generation and OpenAI strict mode. (#6285) * Disable default required property schema generation and OpenAI strict mode. * Default strictness to OpenAI client defaults. * Fix a few failing tests. * Undo a number of additional changes made by #6064 * Revert change to structured output defaults. * Incorporate more test fixes. * Address feedback. * Tweak strict mode signature, fix failing tests. * Roll back schemaIsStrict flag from ChatResponseFormat. * Merged PR 49624: [9.4.3] [cherry-pick] Update readmes (#6345) Includes name of new metric that was added to Quality package. Also updates some doc comments. * Bump version to 9.4.3 * Merged PR 49636: [9.4.3] [cherry-pick] Skip messages that have no text when rendering conversations as part of evaluation prompts (#6349) * Add test for optional parameters being required with RequireAllProperties (#6265) * Add test for optional parameters being required with RequireAllProperties * Test both requireAllProperties values * Use AssertDeepEquals that logs the difference to other tests * Adding reference to an unsupported built-in tool on OpenAI Chat API no longer throws (#6276) * Update M.E.AI changelogs (#6269) * Augment UseDistributedCache XML docs (#6256) * Augment AIFunctionFactory.Create XML docs (#6255) * Rename EmbeddingGeneratorExtensions.GenerateEmbedding extension methods (#6295) To align with the the base method on IEmbeddingGenerator * Augment FunctionInvokingChatClient's span with token counts (#6296) * Rename ChatThreadId to ConversationId (#6300) * In OpenAI responses client, use response ID as ChatThreadId * Rename ChatThreadId -> ConversationId * Related renames * Restore deleted members as obsolete (#6304) * Restore EmbeddingGeneratorExtensions members as obsolete * Restore ChatThreadId as obsolete * Restore ChatResponse.ChatThreadId and ChatResponseUpdate.ChatThreadId as obsolete. * Remove the tests for obsolete members * Remove the Embeddings tests for obsolete members * Support [FromKeyedServices] in AIFunctionFactory (#6310) * Utilize IServiceProviderIsService in AIFunctionFactory (#6317) Add AIFunctionFactoryOptions.Services, and use it when examining function parameters to determine whether they should be resolved by default from DI. * Remove AsChatClient/AsEmbeddingGenerator that were obsoleted in 9.4.0-preview.1.25207.5 (#6327) * Add ChatOptions.AllowMultipleToolCalls (#6326) * Add ChatOptions.AllowMultipleToolCalls * Use it in OpenAI adapter * Disable default required property schema generation and OpenAI strict mode. (#6285) * Disable default required property schema generation and OpenAI strict mode. * Default strictness to OpenAI client defaults. * Fix a few failing tests. * Undo a number of additional changes made by #6064 * Revert change to structured output defaults. * Incorporate more test fixes. * Address feedback. * Tweak strict mode signature, fix failing tests. * Roll back schemaIsStrict flag from ChatResponseFormat. * Bump version to 9.4.3 * Merged PR 49636: [9.4.3] [cherry-pick] Skip messages that have no text when rendering conversations as part of evaluation prompts (#6349) * Update chat template dependencies, fix OpenAI/Aspire config, and address build warnings (#6280) * Update chat template dependencies * Update test snapshots * Update Aspire version * Revert Aspire + update CommunityToolkit.Aspire * Open README in VS after project creation * Add Known Issue to Aspire README for Qdrant * Update survey template URL * Set the Project Template package version to -preview.2 * Update template baseline * Update template pinned versions * Do not append template args to snapshot names * Fix vector store index in the README. Add an AzureAISearch template test. * Add a note to the Aspire README for trusting the development certificate * Use AddOpenAIClient for OpenAI and AddAzureOpenAIClient for Azure OpenAI * Remove duplicated using System.ClientModel * Update Aspire README to specify the exception thrown for the known issue * Augment the Aspire README for more Docker notes for Ollama and Qdrant * Fix Microsoft.Extensions.Http.Resilience warnings with separate pinned versions * Update test baseline for: Rename EmbeddingGeneratorExtensions.GenerateEmbedding extension methods (#6295) * Add --managed-identity to the template developer README * Apply the Ollama timeout recommendation in the template code * Call http.RemoveAllResilienceHandlers before adding the handler * Update template test baseline --------- Co-authored-by: Jeff Handley <[email protected]> Co-authored-by: David Cantu <[email protected]> * Expose AIContent constructor (#6346) * Add PDF support to OpenAI AsIChatClient (#6344) * Add PDF support to OpenAI AsIChatClient * Add missing reference * Make CreateJsonSchema tolerate JSO inputs that don't have a resolver set. (#6348) * Update MEAI.Templates to use the just-built version of the libraries * Enhance Function Invocation Extensibility for Microsoft.Extensions.AI (#6325) --------- Co-authored-by: Peter Waldschmidt <[email protected]> Co-authored-by: Peter Waldschmidt <[email protected]> Co-authored-by: Shyam Namboodiripad <[email protected]> Co-authored-by: David Cantú <[email protected]> Co-authored-by: Art Leonard <[email protected]> Co-authored-by: Stephen Toub <[email protected]> Co-authored-by: Shay Rojansky <[email protected]> Co-authored-by: Steve Sanderson <[email protected]> Co-authored-by: Eirik Tsarpalis <[email protected]> Co-authored-by: Mackinnon Buck <[email protected]> Co-authored-by: Roger Barreto <[email protected]>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Microsoft Reviewers: Open in CodeFlow