(EAI-930): Retrieval tool call (feature branch)

[mongodben]

Jira: https://jira.mongodb.org/browse/EAI-930

Changes

• Refactor chatbot retrieval system to use tool calling
• Upgrade chatbot to use GPT-4.1
• Deprecate framework npm packages
  • mongodb-chatbot-server
  • mongodb-rag-core
  • mongodb-rag-ingest
• Publish new docs

Notes

[nlarew]

I think something broke with follow up questions - this works on prod but not locally on the feature branch:

PROD

USER: how to insertone nodejs
ASST: <... gives a good answer ...>
USER: show me in python?
ASST: <... gives a good answer in python ...>

FEATURE BRANCH

USER: how to insertone nodejs
ASST: <... gives a good answer ...>
USER: show me in python?
ASST: I'm sorry, I do not know how to answer that question. Please try to rephrase your query.

Conversation ID (in dev): 6849b9bce143f2ae905528e6

[nlarew]

Issue with logging - each line of the response is getting a separate log line.

Also note that this is logging the first message "Final result" after I sent the second message

chatbot-server-mongodb-public: {"level":"info","message":"Request info:\nUser message: show me in python?\nStream: true\nIP: ::1\nConversationId: 6849bad4e143f2ae905528f5","reqId":"6849bae0e143f2ae905528fa"}
chatbot-server-mongodb-public: Name matches found: []
chatbot-server-mongodb-public: Final result: To insert a single document into MongoDB using Node.js, use the insertOne() method from the MongoDB Node.js driver. Here's a simple example:
chatbot-server-mongodb-public: ```javascript
chatbot-server-mongodb-public: import { MongoClient } from "mongodb";
chatbot-server-mongodb-public: // Replace with your MongoDB connection string
chatbot-server-mongodb-public: const uri = "<connection string uri>";
chatbot-server-mongodb-public: const client = new MongoClient(uri);
chatbot-server-mongodb-public: async function run() {
chatbot-server-mongodb-public:   try {
chatbot-server-mongodb-public:     // Connect to the database and collection
chatbot-server-mongodb-public:     const database = client.db("myDB");
chatbot-server-mongodb-public:     const collection = database.collection("pizzaMenu");
chatbot-server-mongodb-public:     // Document to insert
chatbot-server-mongodb-public:     const doc = { name: "Neapolitan pizza", shape: "round" };
chatbot-server-mongodb-public:     // Insert the document
chatbot-server-mongodb-public:     const result = await collection.insertOne(doc);
chatbot-server-mongodb-public:     // Output the inserted document's _id
chatbot-server-mongodb-public:     console.log(`A document was inserted with the _id: ${result.insertedId}`);
chatbot-server-mongodb-public:   } finally {
chatbot-server-mongodb-public:     await client.close();
chatbot-server-mongodb-public:   }
chatbot-server-mongodb-public: }
chatbot-server-mongodb-public: run().catch(console.dir);
chatbot-server-mongodb-public: ```
chatbot-server-mongodb-public: - Replace <connection string uri> with your MongoDB connection string.
chatbot-server-mongodb-public: - The insertOne() method returns an object containing the inserted document's _id.
chatbot-server-mongodb-public: Let me know if you need more details or help with setup!
chatbot-server-mongodb-public: {"level":"info","message":"SSE connection was closed."}