(EAI-993): deprecate framework #752

mongodben · 2025-05-27T20:23:21Z

Jira: https://jira.mongodb.org/browse/EAI-993

Changes

Notes

nlarew

Few nits but otherwise LGTM

nlarew · 2025-06-04T20:47:09Z

README.md


-You can also check out the following articles and videos about the framework:
+The MongoDB Chatbot Framework in now deprecated. We will no longer be maintaining it.


Maybe add a little blurb as to why?

Third party frameworks have gotten a lot better

Recommended alternative?: Vercel AI SDK

docs/docs/contact.md

nlarew · 2025-06-04T20:49:47Z

docs/docs/framework.md

@@ -0,0 +1,12 @@
+# MongoDB Chatbot Framework (deprecated)


If you update the README, make sure to copy here too

nlarew · 2025-06-04T20:52:36Z

docs/docs/server/openapi.yaml

+      description: |
+        Custom header required for authentication with one of the pre-configured values:
+        - "https://mongodb.com"
+        - "https://www.mongodb.com"
+        - "https://learn.mongodb.com"


I don't think this is 100% accurate (though that's probably fine tbh)

We accept any Origin or X-Request-Origin as long as its present. The WAF has additional filters based on User-Agent.

thx will update.

should we document the User-Agent stuff here?

update: i mention the user agent stuff, but kept it vague for now and pointed to the contact page

nlarew · 2025-06-04T20:55:48Z

docs/docusaurus.config.ts

-  title: "MongoDB Chatbot Framework",
-  tagline: "Build RAG applications with MongoDB and Atlas Vector Search",
+  title: "MongoDB Knowledge Service",
+  tagline: "Serving MongoDB knowledge",


Over 500k served!

nlarew · 2025-06-04T20:56:21Z

packages/chatbot-server-mongodb-public/src/processors/makeMongoDbReferences.test.ts

+    const result = makeMongoDbReferences(
+      chunks.map((c) => ({
+        ...c,
+        title: c.metadata?.pageTitle,
+      }))
+    );


Per your comment on the other PR - I don't think we need these maps?

yea def not. should be removed. git funk

nlarew · 2025-06-04T20:58:08Z

packages/mongodb-chatbot-server/src/processors/makeDefaultReferenceLinks.test.ts

@@ -59,7 +59,6 @@ describe("makeDefaultReferenceLinks()", () => {
        title: "https://mongodb.com/docs/realm/sdk/node/",
        url: "https://mongodb.com/docs/realm/sdk/node/",
        metadata: {
-          sourceName: "realm",


These tests are failing in CI because the sourceName value is still present

think some gitfunk. will 2x check and revert

* (EAI-988): Refactor `GenerateResponse` for tool call support (#687) * refactor GenerateRespose * Clean up imports * consolidate generate user prompt to the legacy file * update test config imports * Fix broken tests * (EAI-989): Refactor verified answers to wrap `GenerateResponse` (#688) verified answer generate response Co-authored-by: Ben Perlmutter <[email protected]> * handle streaming * separate generateresponse * typo fix --------- Co-authored-by: Ben Perlmutter <[email protected]> * (EAI-990): Refactor search as a tool (#705) * refactor GenerateRespose * Clean up imports * consolidate generate user prompt to the legacy file * update test config imports * Fix broken tests * get started * nominally working generate res w/ search * small refactors * aint pretty but fully functional * hacky if more functional * more hack * tools * functional if not pretty * Add processing * working tool calling * making progress * keepin on * Clean config * working e2e * update model version * Remove no longer used stuff * decouple search results for references and whats shown to model * fix scripts build errs * fix broken tests * update default ref links * fix broken tests * Apply suggestions from code review Co-authored-by: Nick Larew <[email protected]> * revert default reference links * adding missing test --------- Co-authored-by: Ben Perlmutter <[email protected]> Co-authored-by: Nick Larew <[email protected]> * (EAI-992): Remove `ChatLlm` (#751) * refactor GenerateRespose * Clean up imports * consolidate generate user prompt to the legacy file * update test config imports * Fix broken tests * get started * nominally working generate res w/ search * small refactors * aint pretty but fully functional * hacky if more functional * more hack * tools * functional if not pretty * Add processing * working tool calling * making progress * keepin on * Clean config * working e2e * update model version * Remove no longer used stuff * decouple search results for references and whats shown to model * fix scripts build errs * remove ChatLlm * lite fixes * Remove stub --------- Co-authored-by: Ben Perlmutter <[email protected]> * (EAI-993): deprecate framework (#752) * refactor GenerateRespose * Clean up imports * consolidate generate user prompt to the legacy file * update test config imports * Fix broken tests * get started * nominally working generate res w/ search * small refactors * aint pretty but fully functional * hacky if more functional * more hack * tools * functional if not pretty * Add processing * working tool calling * making progress * keepin on * Clean config * working e2e * update model version * Remove no longer used stuff * decouple search results for references and whats shown to model * fix scripts build errs * fix broken tests * deprecation * build out docs following last week convo * clean up spec + contact * fix merge funk * docs updates --------- Co-authored-by: Ben Perlmutter <[email protected]> * (EAI-1071): Fix broken Atlas OpenAPI ingest (#765) * update to fix broken test * Update packages/ingest-mongodb-public/src/sources/snooty/snootyAstToOpenApiSpec.ts --------- Co-authored-by: Ben Perlmutter <[email protected]> * (EAI-995): add guardrail (#755) * refactor GenerateRespose * Clean up imports * consolidate generate user prompt to the legacy file * update test config imports * Fix broken tests * get started * nominally working generate res w/ search * small refactors * aint pretty but fully functional * hacky if more functional * more hack * tools * functional if not pretty * Add processing * working tool calling * making progress * keepin on * Clean config * working e2e * update model version * Remove no longer used stuff * decouple search results for references and whats shown to model * fix scripts build errs * fix broken tests * update default ref links * fix broken tests * input guardrail refactor * guardrail works well * simpler validity metric * add guardrail to server * add next step todo * llm refusal msg * remove TODO comment * merge fix * fix unnec changes * NL feedback --------- Co-authored-by: Ben Perlmutter <[email protected]> * fix type in text * (EAI-991 & EAI-1050): Evaluate and clean up retrieval as a tool (#757) * refactor GenerateRespose * Clean up imports * consolidate generate user prompt to the legacy file * update test config imports * Fix broken tests * get started * nominally working generate res w/ search * small refactors * aint pretty but fully functional * hacky if more functional * more hack * tools * functional if not pretty * Add processing * working tool calling * making progress * keepin on * Clean config * working e2e * update model version * Remove no longer used stuff * decouple search results for references and whats shown to model * fix scripts build errs * fix broken tests * update default ref links * fix broken tests * input guardrail refactor * guardrail works well * simpler validity metric * add guardrail to server * add next step todo * llm refusal msg * remove TODO comment * evals on new architecture * Get urls in a way that supports verified answers * dont eval on retrieved elems if no context * Cleaner handling * update trace handling * update trace handling * undo git funk * handle undefined case * Fix tracing test --------- Co-authored-by: Ben Perlmutter <[email protected]> * remove console logs + redunancies --------- Co-authored-by: Ben Perlmutter <[email protected]> Co-authored-by: Nick Larew <[email protected]>

Ben Perlmutter added 29 commits April 28, 2025 12:08

refactor GenerateRespose

2e286c3

Clean up imports

0be9fe2

consolidate generate user prompt to the legacy file

2556bc2

update test config imports

feee66a

Fix broken tests

f6fe862

get started

0fc58bd

nominally working generate res w/ search

ea40c6d

small refactors

c3e69e3

aint pretty but fully functional

0345453

hacky if more functional

a4144db

more hack

04076d2

tools

8372bd3

functional if not pretty

24d1cf7

Add processing

40fa9d1

working tool calling

31bd0a8

making progress

653aa59

keepin on

9d68d50

Clean config

57e65a2

working e2e

aefa9ca

update model version

c33f64e

Merge remote-tracking branch 'upstream/retrieval_tool_call' into EAI-990

c446756

Remove no longer used stuff

728416f

decouple search results for references and whats shown to model

479ccb8

fix scripts build errs

cc8dd45

fix broken tests

e768dd6

deprecation

6be3e7b

build out docs following last week convo

b6f9c7e

clean up spec + contact

6f66f9e

Merge remote-tracking branch 'upstream/retrieval_tool_call' into EAI-993

eb31415

mongodben marked this pull request as ready for review June 4, 2025 15:40

nlarew approved these changes Jun 4, 2025

View reviewed changes

Ben Perlmutter added 2 commits June 5, 2025 10:46

fix merge funk

84bdd7a

docs updates

5581398

mongodben changed the base branch from retrieval_tool_call to main June 5, 2025 14:47

mongodben changed the base branch from main to retrieval_tool_call June 5, 2025 14:51

mongodben merged commit 8eb82bf into retrieval_tool_call Jun 5, 2025
1 check failed

mongodben deleted the EAI-993 branch June 5, 2025 15:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

(EAI-993): deprecate framework #752

(EAI-993): deprecate framework #752

Uh oh!

mongodben commented May 27, 2025 •

edited

Loading

Uh oh!

nlarew left a comment

Uh oh!

nlarew Jun 4, 2025

Uh oh!

Uh oh!

nlarew Jun 4, 2025

Uh oh!

nlarew Jun 4, 2025

Uh oh!

mongodben Jun 4, 2025

Uh oh!

mongodben Jun 5, 2025

Uh oh!

nlarew Jun 4, 2025

Uh oh!

nlarew Jun 4, 2025

Uh oh!

mongodben Jun 4, 2025

Uh oh!

nlarew Jun 4, 2025

Uh oh!

mongodben Jun 4, 2025

Uh oh!

Uh oh!

Uh oh!


		You can also check out the following articles and videos about the framework:
		The MongoDB Chatbot Framework in now deprecated. We will no longer be maintaining it.

(EAI-993): deprecate framework #752

(EAI-993): deprecate framework #752

Uh oh!

Conversation

mongodben commented May 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Notes

Uh oh!

nlarew left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

mongodben commented May 27, 2025 •

edited

Loading