feat: ChatGPT Plugins/OpenAPI specs for Plugins Endpoint (#620)

* wip: proof of concept for openapi chain

* chore(api): update langchain dependency to version 0.0.105

* feat(Plugins): use ChatGPT Plugins/OpenAPI specs (first pass)

* chore(manifest.json): update pluginKey for "Browser" tool to "web-browser"
chore(handleTools.js): update customConstructor key for "web-browser" tool

* fix(handleSubmit.js): set unfinished property to false for all endpoints

* fix(handlers.js): remove unnecessary capitalizeWords function and use action.tool directly
refactor(endpoints.js): rename availableTools to tools and transform it into a map

* feat(endpoints): add plugins selector to endpoints file
refactor(CodeBlock.tsx): refactor to typescript
refactor(Plugin.tsx): use recoil Map for plugin name and refactor to typescript
chore(Message.jsx): linting
chore(PluginsOptions/index.jsx): remove comment/linting
chore(svg): export Clipboard and CheckMark components from SVG index and refactor to typescript

* fix(OpenAPIPlugin.js): rename readYamlFile function to readSpecFile
fix(OpenAPIPlugin.js): handle JSON files in readSpecFile function
fix(OpenAPIPlugin.js): handle JSON URLs in getSpec function
fix(OpenAPIPlugin.js): handle JSON variables in createOpenAPIPlugin function
fix(OpenAPIPlugin.js): add description for variables in createOpenAPIPlugin function
fix(OpenAPIPlugin.js): add optional flag for is_user_authenticated and has_user_authentication in ManifestDefinition
fix(loadSpecs.js): add optional flag for is_user_authenticated and has_user_authentication in ManifestDefinition
fix(Plugin.tsx): remove unnecessary callback parameter in getPluginName function
fix(getDefaultConversation.js): fix browser console error: handle null value for lastConversationSetup in getDefaultConversation function

* feat(api): add new tools

Add Ai PDF tool for super-fast, interactive chats with PDFs of any size, complete with page references for fact checking.
Add VoxScript tool for searching through YouTube transcripts, financial data sources, Google Search results, and more.
Add WebPilot tool for browsing and QA of webpages, PDFs, and data. Generate articles from one or more URLs.

feat(api): update OpenAPIPlugin.js

- Add support for bearer token authorization in the OpenAPIPlugin.
- Add support for custom headers in the OpenAPIPlugin.

fix(api): fix loadTools.js

- Pass the user parameter to the loadSpecs function.

* feat(PluginsClient.js): import findMessageContent function from utils
feat(PluginsClient.js): add message parameter to options object in initializeCustomAgent function
feat(PluginsClient.js): add content to errorMessage if message content is found
feat(PluginsClient.js): break out of loop if message content is found
feat(PluginsClient.js): add delay option with value of 8 to generateTextStream function
feat(PluginsClient.js): add support for process.env.PORT environment variable in app.listen function
feat(askyourpdf.json): add askyourpdf plugin configuration
feat(metar.json): add metar plugin configuration
feat(askyourpdf.yaml): add askyourpdf plugin OpenAPI specification
feat(OpenAPIPlugin.js): add message parameter to createOpenAPIPlugin function
feat(OpenAPIPlugin.js): add description_for_model to chain run message
feat(addOpenAPISpecs.js): remove verbose option from loadSpecs function call

fix(loadSpecs.js): add 'message' parameter to the loadSpecs function
feat(findMessageContent.js): add utility function to find message content in JSON objects

* fix(PluginStoreDialog.tsx): update z-index value for the dialog container

The z-index value for the dialog container was updated to "102" to ensure it appears above other elements on the page.

* chore(web_pilot.json): add "params" field with "user_has_request" parameter set to true

* chore(eslintrc.js): update eslint rules
fix(Login.tsx): add missing semicolon after import statement

* fix(package-lock.json): update langchain dependency to version ^0.0.105

* fix(OpenAPIPlugin.js): change header key from 'id' to 'librechat_user_id' for consistency and clarity

feat(plugins): add documentation for using official ChatGPT Plugins with OpenAPI specs

This commit adds a new file `chatgpt_plugins_openapi.md` to the `docs/features/plugins` directory. The file provides detailed information on how to use official ChatGPT Plugins with OpenAPI specifications. It explains the components of a plugin, including the Plugin Manifest file and the OpenAPI spec. It also covers the process of adding a plugin, editing manifest files, and customizing OpenAPI spec files. Additionally, the commit includes disclaimers about the limitations and compatibility of plugins with LibreChat. The documentation also clarifies that the use of ChatGPT Plugins with LibreChat does not violate OpenAI's Terms of Service.

The purpose of this commit is to provide comprehensive documentation for developers who want to integrate ChatGPT Plugins into their projects using OpenAPI specs. It aims to guide them through the process of adding and configuring plugins, as well as addressing potential issues and

chore(introduction.md): update link to ChatGPT Plugins documentation
docs(introduction.md): clarify the purpose of the plugins endpoint and its capabilities

* fix(OpenAPIPlugin.js): update SUFFIX variable to provide a clearer description
docs(chatgpt_plugins_openapi.md): update information about adding plugins via url on the frontend

* feat(PluginsClient.js): sendIntermediateMessage on successful Agent load
fix(PluginsClient.js, server/index.js, gptPlugins.js): linting fixes
docs(chatgpt_plugins_openapi.md): update links and add additional information

* Update chatgpt_plugins_openapi.md

* chore: rebuild package-lock file

* chore: format/lint all files with new rules

* chore: format all files

* chore(README.md): update AI model selection list

The AI model selection list in the README.md file has been updated to reflect the current options available. The "Anthropic" model has been added as an alternative name for the "Claude" model.

* fix(Plugin.tsx): type issue

* feat(tools): add new tool WebPilot

feat(tools): remove tool Weather Report

feat(tools): add new tool Prompt Perfect

feat(tools): add new tool Scholarly Graph Link

* feat(OpenAPIPlugin.js): add getSpec and readSpecFile functions
feat(OpenAPIPlugin.spec.js): add tests for readSpecFile, getSpec, and createOpenAPIPlugin functions

* chore(agent-demo-1.js): remove unused code and dependencies
chore(agent-demo-2.js): remove unused code and dependencies
chore(demo.js): remove unused code and dependencies

* feat(addOpenAPISpecs): add function to transform OpenAPI specs into desired format
feat(addOpenAPISpecs.spec): add tests for transformSpec function
fix(loadSpecs): remove debugging code

* feat(loadSpecs.spec.js): add unit tests for ManifestDefinition, validateJson, and loadSpecs functions

* fix: package file resolution bug

* chore: move scholarly_graph_link manifest to 'has-issues'

* refactor(client/hooks): convert to TS and export from index

* Update introduction.md

* Update chatgpt_plugins_openapi.md

.env.example

@@ -107,6 +107,16 @@ CHATGPT_MODELS=text-davinci-002-render-sha,gpt-4
 # By default it will use the node-chatgpt-api recommended proxy, (it's a third party server)
 # CHATGPT_REVERSE_PROXY=<YOUR REVERSE PROXY>
 
+##########################
+# Anthropic Endpoint: 
+##########################
+# Access key from https://console.anthropic.com/
+# Leave it blank to disable this feature.
+# Set to "user_provided" to allow the user to provide their API key from the UI.
+# Note that access to claude-1 may potentially become unavailable with the release of claude-2.
+ANTHROPIC_API_KEY="user_provided"
+ANTHROPIC_MODELS=claude-1,claude-instant-1,claude-2
+
 #############################
 # Plugins:
 #############################
@@ -164,6 +174,9 @@ PROXY=
 # The easiest setup for this is through docker-compose, which takes care of it for you. 
 SEARCH=true
 
+# HIGHLY RECOMMENDED: Disable anonymized telemetry analytics for MeiliSearch for absolute privacy.
+MEILI_NO_ANALYTICS=true
+
 # REQUIRED FOR SEARCH: MeiliSearch Host, mainly for the API server to connect to the search server. 
 # Replace '0.0.0.0' with 'meilisearch' if serving MeiliSearch with docker-compose. 
 MEILI_HOST=http://0.0.0.0:7700
@@ -187,6 +200,9 @@ MEILI_MASTER_KEY=DrhYf7zENyR6AlUCKmnz0eYASOQdl6zxH7s7MKFSfFCt
 # Allow Public Registration
 ALLOW_REGISTRATION=true
 
+# Allow Social Registration
+ALLOW_SOCIAL_LOGIN=false
+
 # JWT Secrets
 JWT_SECRET=secret
 JWT_REFRESH_SECRET=secret
@@ -219,12 +235,20 @@ OPENID_IMAGE_URL=
 SESSION_EXPIRY=(1000 * 60 * 60 * 24) * 7
 
 # Github:
-# Get the Client ID and Secret from your Github Application
+# Get the Client ID and Secret from your Discord Application
+# Add your Discord Client ID and Client Secret here:
+
+GITHUB_CLIENT_ID=your_client_id
+GITHUB_CLIENT_SECRET=your_client_secret
+GITHUB_CALLBACK_URL=/oauth/github/callback # this should be the same for everyone
+
+# Discord:
+# Get the Client ID and Secret from your Discord Application
 # Add your Github Client ID and Client Secret here:
 
-GITHUB_CLIENT_ID=
-GITHUB_CLIENT_SECRET=
-GITHUB_CALLBACK_URL=/oauth/github/callback
+DISCORD_CLIENT_ID=your_client_id
+DISCORD_CLIENT_SECRET=your_client_secret
+DISCORD_CALLBACK_URL=/oauth/discord/callback # this should be the same for everyone
 
 ###########################
 # Application Domains

.eslintrc.js

@@ -4,22 +4,30 @@ module.exports = {
     es2021: true,
     node: true,
     commonjs: true,
-    es6: true
+    es6: true,
   },
   extends: [
     'eslint:recommended',
     'plugin:react/recommended',
     'plugin:react-hooks/recommended',
     'plugin:jest/recommended',
-    'prettier'
+    'prettier',
+  ],
+  // ignorePatterns: ['packages/data-provider/types/**/*'],
+  ignorePatterns: [
+    'client/dist/**/*',
+    'client/public/**/*',
+    'e2e/playwright-report/**/*',
+    'packages/data-provider/types/**/*',
+    'packages/data-provider/dist/**/*',
   ],
   parser: '@typescript-eslint/parser',
   parserOptions: {
     ecmaVersion: 'latest',
     sourceType: 'module',
     ecmaFeatures: {
-      jsx: true
-    }
+      jsx: true,
+    },
   },
   plugins: ['react', 'react-hooks', '@typescript-eslint'],
   rules: {
@@ -32,13 +40,16 @@ module.exports = {
         code: 120,
         ignoreStrings: true,
         ignoreTemplateLiterals: true,
-        ignoreComments: true
-      }
+        ignoreComments: true,
+      },
     ],
     'linebreak-style': 0,
-    'object-curly-spacing': ['error', 'always'],
+    'curly': ['error', 'all'],
+    'semi': ['error', 'always'],
     'no-trailing-spaces': 'error',
-    'no-multiple-empty-lines': ['error', { 'max': 1 }],
+    'object-curly-spacing': ['error', 'always'],
+    'no-multiple-empty-lines': ['error', { max: 1 }],
+    'comma-dangle': ['error', 'always-multiline'],
     // "arrow-parens": [2, "as-needed", { requireForBlockBody: true }],
     // 'no-plusplus': ['error', { allowForLoopAfterthoughts: true }],
     'no-console': 'off',
@@ -49,7 +60,7 @@ module.exports = {
     'no-restricted-syntax': 'off',
     'react/prop-types': ['off'],
     'react/display-name': ['off'],
-    'quotes': ['error', 'single'],
+    quotes: ['error', 'single'],
   },
   overrides: [
     {
@@ -57,14 +68,14 @@ module.exports = {
       rules: {
         'no-unused-vars': 'off', // off because it conflicts with '@typescript-eslint/no-unused-vars'
         'react/display-name': 'off',
-        '@typescript-eslint/no-unused-vars': 'warn'
-      }
+        '@typescript-eslint/no-unused-vars': 'warn',
+      },
     },
     {
       files: ['rollup.config.js', '.eslintrc.js', 'jest.config.js'],
       env: {
         node: true,
-      }
+      },
     },
     {
       files: [
@@ -76,29 +87,29 @@ module.exports = {
         '**/*.spec.jsx',
         '**/*.spec.ts',
         '**/*.spec.tsx',
-        'setupTests.js'
+        'setupTests.js',
       ],
       env: {
         jest: true,
-        node: true
+        node: true,
       },
       rules: {
         'react/display-name': 'off',
         'react/prop-types': 'off',
-        'react/no-unescaped-entities': 'off'
-      }
+        'react/no-unescaped-entities': 'off',
+      },
     },
     {
       files: '**/*.+(ts)',
       parser: '@typescript-eslint/parser',
       parserOptions: {
-        project: './client/tsconfig.json'
+        project: './client/tsconfig.json',
       },
       plugins: ['@typescript-eslint/eslint-plugin', 'jest'],
       extends: [
         'plugin:@typescript-eslint/eslint-recommended',
-        'plugin:@typescript-eslint/recommended'
-      ]
+        'plugin:@typescript-eslint/recommended',
+      ],
     },
     {
       files: './packages/data-provider/**/*.ts',
@@ -107,11 +118,11 @@ module.exports = {
           files: '**/*.ts',
           parser: '@typescript-eslint/parser',
           parserOptions: {
-            project: './packages/data-provider/tsconfig.json'
-          }
-        }
-      ]
-    }
+            project: './packages/data-provider/tsconfig.json',
+          },
+        },
+      ],
+    },
   ],
   settings: {
     react: {
@@ -119,7 +130,7 @@ module.exports = {
       // default to "createReactClass"
       pragma: 'React', // Pragma to use, default to "React"
       fragment: 'Fragment', // Fragment to use (may be a property of <pragma>), default to "Fragment"
-      version: 'detect' // React version. "detect" automatically picks the version you have installed.
-    }
-  }
+      version: 'detect', // React version. "detect" automatically picks the version you have installed.
+    },
+  },
 };

.github/workflows/backend-review.yml

@@ -37,3 +37,8 @@ jobs:
 
       - name: Run unit tests
         run: cd api && npm run test:ci
+
+      - name: Run linters
+        uses: wearerequired/lint-action@v2
+        with:
+          eslint: true

.prettierrc.js

@@ -1,11 +1,11 @@
 module.exports = {
   printWidth: 100,
   useTabs: false,
-  tabWidth: 2,
+  // tabWidth: 2,
   semi: true,
   singleQuote: true,
   // bracketSpacing: false,
-  trailingComma: 'none',
+  trailingComma: 'all',
   arrowParens: 'always',
   embeddedLanguageFormatting: 'auto',
   insertPragma: false,

README.md

@@ -36,7 +36,7 @@ https://github.com/danny-avila/LibreChat/assets/110412045/c1eb0c0f-41f6-4335-b98
 # Features
 - Response streaming identical to ChatGPT through server-sent events
 - UI from original ChatGPT, including Dark mode
-- AI model selection (through 5 endpoints: OpenAI API, BingAI, ChatGPT Browser, PaLM2, Plugins)
+- AI model selection: OpenAI API, BingAI, ChatGPT Browser, PaLM2, Anthropic (Claude), Plugins
 - Create, Save, & Share custom presets - [More info on prompt presets here](https://github.com/danny-avila/LibreChat/releases/tag/v0.3.0)
 - Edit and Resubmit messages with conversation branching
 - Search all messages/conversations - [More info here](https://github.com/danny-avila/LibreChat/releases/tag/v0.1.0)
@@ -98,6 +98,7 @@ Keep up with the latest updates by visiting the releases page - [Releases](https
   * [Linode](docs/deployment/linode.md)
   * [Cloudflare](docs/deployment/cloudflare.md)
   * [Ngrok](docs/deployment/ngrok.md)
+  * [Render](docs/deployment/render.md)
 </details>
 
 <details>

api/app/bingai.js

@@ -14,11 +14,11 @@ const askBing = async ({
   invocationId,
   toneStyle,
   token,
-  onProgress
+  onProgress,
 }) => {
   const { BingAIClient } = await import('@waylaidwanderer/chatgpt-api');
   const store = {
-    store: new KeyvFile({ filename: './data/cache.json' })
+    store: new KeyvFile({ filename: './data/cache.json' }),
   };
 
   const bingAIClient = new BingAIClient({
@@ -30,7 +30,7 @@ const askBing = async ({
     debug: false,
     cache: store,
     host: process.env.BINGAI_HOST || null,
-    proxy: process.env.PROXY || null
+    proxy: process.env.PROXY || null,
   });
 
   let options = {};
@@ -39,23 +39,23 @@ const askBing = async ({
     jailbreakConversationId = false;
   }
 
-  if (jailbreak)
+  if (jailbreak) {
     options = {
       jailbreakConversationId: jailbreakConversationId || jailbreak,
       context,
       systemMessage,
       parentMessageId,
       toneStyle,
-      onProgress
+      onProgress,
     };
-  else {
+  } else {
     options = {
       conversationId,
       context,
       systemMessage,
       parentMessageId,
       toneStyle,
-      onProgress
+      onProgress,
     };
 
     // don't give those parameters for new conversation

api/app/chatgpt-browser.js

@@ -10,11 +10,11 @@ const browserClient = async ({
   onProgress,
   onEventMessage,
   abortController,
-  userId
+  userId,
 }) => {
   const { ChatGPTBrowserClient } = await import('@waylaidwanderer/chatgpt-api');
   const store = {
-    store: new KeyvFile({ filename: './data/cache.json' })
+    store: new KeyvFile({ filename: './data/cache.json' }),
   };
 
   const clientOptions = {
@@ -27,7 +27,7 @@ const browserClient = async ({
     model: model,
     debug: false,
     proxy: process.env.PROXY || null,
-    user: userId
+    user: userId,
   };
 
   const client = new ChatGPTBrowserClient(clientOptions, store);

api/app/clients/AnthropicClient.js

@@ -0,0 +1,324 @@
+const Keyv = require('keyv');
+// const { Agent, ProxyAgent } = require('undici');
+const BaseClient = require('./BaseClient');
+const {
+  encoding_for_model: encodingForModel,
+  get_encoding: getEncoding,
+} = require('@dqbd/tiktoken');
+const Anthropic = require('@anthropic-ai/sdk');
+
+const HUMAN_PROMPT = '\n\nHuman:';
+const AI_PROMPT = '\n\nAssistant:';
+
+const tokenizersCache = {};
+
+class AnthropicClient extends BaseClient {
+  constructor(apiKey, options = {}, cacheOptions = {}) {
+    super(apiKey, options, cacheOptions);
+    cacheOptions.namespace = cacheOptions.namespace || 'anthropic';
+    this.conversationsCache = new Keyv(cacheOptions);
+    this.apiKey = apiKey || process.env.ANTHROPIC_API_KEY;
+    this.sender = 'Anthropic';
+    this.userLabel = HUMAN_PROMPT;
+    this.assistantLabel = AI_PROMPT;
+    this.setOptions(options);
+  }
+
+  setOptions(options) {
+    if (this.options && !this.options.replaceOptions) {
+      // nested options aren't spread properly, so we need to do this manually
+      this.options.modelOptions = {
+        ...this.options.modelOptions,
+        ...options.modelOptions,
+      };
+      delete options.modelOptions;
+      // now we can merge options
+      this.options = {
+        ...this.options,
+        ...options,
+      };
+    } else {
+      this.options = options;
+    }
+
+    const modelOptions = this.options.modelOptions || {};
+    this.modelOptions = {
+      ...modelOptions,
+      // set some good defaults (check for undefined in some cases because they may be 0)
+      model: modelOptions.model || 'claude-1',
+      temperature: typeof modelOptions.temperature === 'undefined' ? 0.7 : modelOptions.temperature, // 0 - 1, 0.7 is recommended
+      topP: typeof modelOptions.topP === 'undefined' ? 0.7 : modelOptions.topP, // 0 - 1, default: 0.7
+      topK: typeof modelOptions.topK === 'undefined' ? 40 : modelOptions.topK, // 1-40, default: 40
+      stop: modelOptions.stop, // no stop method for now
+    };
+
+    this.maxContextTokens = this.options.maxContextTokens || 99999;
+    this.maxResponseTokens = this.modelOptions.maxOutputTokens || 1500;
+    this.maxPromptTokens =
+      this.options.maxPromptTokens || this.maxContextTokens - this.maxResponseTokens;
+
+    if (this.maxPromptTokens + this.maxResponseTokens > this.maxContextTokens) {
+      throw new Error(
+        `maxPromptTokens + maxOutputTokens (${this.maxPromptTokens} + ${this.maxResponseTokens} = ${
+          this.maxPromptTokens + this.maxResponseTokens
+        }) must be less than or equal to maxContextTokens (${this.maxContextTokens})`,
+      );
+    }
+
+    this.startToken = '||>';
+    this.endToken = '';
+    this.gptEncoder = this.constructor.getTokenizer('cl100k_base');
+
+    if (!this.modelOptions.stop) {
+      const stopTokens = [this.startToken];
+      if (this.endToken && this.endToken !== this.startToken) {
+        stopTokens.push(this.endToken);
+      }
+      stopTokens.push(`${this.userLabel}`);
+      stopTokens.push('<|diff_marker|>');
+
+      this.modelOptions.stop = stopTokens;
+    }
+
+    return this;
+  }
+
+  getClient() {
+    if (this.options.reverseProxyUrl) {
+      return new Anthropic({
+        apiKey: this.apiKey,
+        baseURL: this.options.reverseProxyUrl,
+      });
+    } else {
+      return new Anthropic({
+        apiKey: this.apiKey,
+      });
+    }
+  }
+
+  async buildMessages(messages, parentMessageId) {
+    const orderedMessages = this.constructor.getMessagesForConversation(messages, parentMessageId);
+    if (this.options.debug) {
+      console.debug('AnthropicClient: orderedMessages', orderedMessages, parentMessageId);
+    }
+
+    const formattedMessages = orderedMessages.map((message) => ({
+      author: message.isCreatedByUser ? this.userLabel : this.assistantLabel,
+      content: message?.content ?? message.text,
+    }));
+
+    let identityPrefix = '';
+    if (this.options.userLabel) {
+      identityPrefix = `\nHuman's name: ${this.options.userLabel}`;
+    }
+
+    if (this.options.modelLabel) {
+      identityPrefix = `${identityPrefix}\nYou are ${this.options.modelLabel}`;
+    }
+
+    let promptPrefix = (this.options.promptPrefix || '').trim();
+    if (promptPrefix) {
+      // If the prompt prefix doesn't end with the end token, add it.
+      if (!promptPrefix.endsWith(`${this.endToken}`)) {
+        promptPrefix = `${promptPrefix.trim()}${this.endToken}\n\n`;
+      }
+      promptPrefix = `\nContext:\n${promptPrefix}`;
+    }
+
+    if (identityPrefix) {
+      promptPrefix = `${identityPrefix}${promptPrefix}`;
+    }
+
+    const promptSuffix = `${promptPrefix}${this.assistantLabel}\n`; // Prompt AI to respond.
+    let currentTokenCount = this.getTokenCount(promptSuffix);
+
+    let promptBody = '';
+    const maxTokenCount = this.maxPromptTokens;
+
+    const context = [];
+
+    // Iterate backwards through the messages, adding them to the prompt until we reach the max token count.
+    // Do this within a recursive async function so that it doesn't block the event loop for too long.
+    // Also, remove the next message when the message that puts us over the token limit is created by the user.
+    // Otherwise, remove only the exceeding message. This is due to Anthropic's strict payload rule to start with "Human:".
+    const nextMessage = {
+      remove: false,
+      tokenCount: 0,
+      messageString: '',
+    };
+
+    const buildPromptBody = async () => {
+      if (currentTokenCount < maxTokenCount && formattedMessages.length > 0) {
+        const message = formattedMessages.pop();
+        const isCreatedByUser = message.author === this.userLabel;
+        const messageString = `${message.author}\n${message.content}${this.endToken}\n`;
+        let newPromptBody = `${messageString}${promptBody}`;
+
+        context.unshift(message);
+
+        const tokenCountForMessage = this.getTokenCount(messageString);
+        const newTokenCount = currentTokenCount + tokenCountForMessage;
+
+        if (!isCreatedByUser) {
+          nextMessage.messageString = messageString;
+          nextMessage.tokenCount = tokenCountForMessage;
+        }
+
+        if (newTokenCount > maxTokenCount) {
+          if (!promptBody) {
+            // This is the first message, so we can't add it. Just throw an error.
+            throw new Error(
+              `Prompt is too long. Max token count is ${maxTokenCount}, but prompt is ${newTokenCount} tokens long.`,
+            );
+          }
+
+          // Otherwise, ths message would put us over the token limit, so don't add it.
+          // if created by user, remove next message, otherwise remove only this message
+          if (isCreatedByUser) {
+            nextMessage.remove = true;
+          }
+
+          return false;
+        }
+        promptBody = newPromptBody;
+        currentTokenCount = newTokenCount;
+        // wait for next tick to avoid blocking the event loop
+        await new Promise((resolve) => setImmediate(resolve));
+        return buildPromptBody();
+      }
+      return true;
+    };
+
+    await buildPromptBody();
+
+    if (nextMessage.remove) {
+      promptBody = promptBody.replace(nextMessage.messageString, '');
+      currentTokenCount -= nextMessage.tokenCount;
+      context.shift();
+    }
+
+    const prompt = `${promptBody}${promptSuffix}`;
+    // Add 2 tokens for metadata after all messages have been counted.
+    currentTokenCount += 2;
+
+    // Use up to `this.maxContextTokens` tokens (prompt + response), but try to leave `this.maxTokens` tokens for the response.
+    this.modelOptions.maxOutputTokens = Math.min(
+      this.maxContextTokens - currentTokenCount,
+      this.maxResponseTokens,
+    );
+
+    return { prompt, context };
+  }
+
+  getCompletion() {
+    console.log('AnthropicClient doesn\'t use getCompletion (all handled in sendCompletion)');
+  }
+
+  // TODO: implement abortController usage
+  async sendCompletion(payload, { onProgress, abortController }) {
+    if (!abortController) {
+      abortController = new AbortController();
+    }
+
+    const { signal } = abortController;
+
+    const modelOptions = { ...this.modelOptions };
+    if (typeof onProgress === 'function') {
+      modelOptions.stream = true;
+    }
+
+    const { debug } = this.options;
+    if (debug) {
+      console.debug();
+      console.debug(modelOptions);
+      console.debug();
+    }
+
+    const client = this.getClient();
+    const metadata = {
+      user_id: this.user,
+    };
+
+    let text = '';
+    const requestOptions = {
+      prompt: payload,
+      model: this.modelOptions.model,
+      stream: this.modelOptions.stream || true,
+      max_tokens_to_sample: this.modelOptions.maxOutputTokens || 1500,
+      metadata,
+      ...modelOptions,
+    };
+    if (this.options.debug) {
+      console.log('AnthropicClient: requestOptions');
+      console.dir(requestOptions, { depth: null });
+    }
+    const response = await client.completions.create(requestOptions);
+
+    signal.addEventListener('abort', () => {
+      if (this.options.debug) {
+        console.log('AnthropicClient: message aborted!');
+      }
+      response.controller.abort();
+    });
+
+    for await (const completion of response) {
+      if (this.options.debug) {
+        // Uncomment to debug message stream
+        // console.debug(completion);
+      }
+      text += completion.completion;
+      onProgress(completion.completion);
+    }
+
+    signal.removeEventListener('abort', () => {
+      if (this.options.debug) {
+        console.log('AnthropicClient: message aborted!');
+      }
+      response.controller.abort();
+    });
+
+    return text.trim();
+  }
+
+  // I commented this out because I will need to refactor this for the BaseClient/all clients
+  // getMessageMapMethod() {
+  //   return ((message) => ({
+  //     author: message.isCreatedByUser ? this.userLabel : this.assistantLabel,
+  //     content: message?.content ?? message.text
+  //   })).bind(this);
+  // }
+
+  getSaveOptions() {
+    return {
+      promptPrefix: this.options.promptPrefix,
+      modelLabel: this.options.modelLabel,
+      ...this.modelOptions,
+    };
+  }
+
+  getBuildMessagesOptions() {
+    if (this.options.debug) {
+      console.log('AnthropicClient doesn\'t use getBuildMessagesOptions');
+    }
+  }
+
+  static getTokenizer(encoding, isModelName = false, extendSpecialTokens = {}) {
+    if (tokenizersCache[encoding]) {
+      return tokenizersCache[encoding];
+    }
+    let tokenizer;
+    if (isModelName) {
+      tokenizer = encodingForModel(encoding, extendSpecialTokens);
+    } else {
+      tokenizer = getEncoding(encoding, extendSpecialTokens);
+    }
+    tokenizersCache[encoding] = tokenizer;
+    return tokenizer;
+  }
+
+  getTokenCount(text) {
+    return this.gptEncoder.encode(text, 'all').length;
+  }
+}
+
+module.exports = AnthropicClient;

api/app/clients/BaseClient.js

@@ -14,7 +14,7 @@ class BaseClient {
     this.currentDateString = new Date().toLocaleDateString('en-us', {
       year: 'numeric',
       month: 'long',
-      day: 'numeric'
+      day: 'numeric',
     });
   }
 
@@ -58,7 +58,7 @@ class BaseClient {
     const responseMessageId = crypto.randomUUID();
     const saveOptions = this.getSaveOptions();
     this.abortController = opts.abortController || new AbortController();
-    this.currentMessages = await this.loadHistory(conversationId, parentMessageId) ?? [];
+    this.currentMessages = (await this.loadHistory(conversationId, parentMessageId)) ?? [];
 
     return {
       ...opts,
@@ -71,27 +71,21 @@ class BaseClient {
     };
   }
 
-  createUserMessage({ messageId, parentMessageId, conversationId, text}) {
+  createUserMessage({ messageId, parentMessageId, conversationId, text }) {
     const userMessage = {
       messageId,
       parentMessageId,
       conversationId,
       sender: 'User',
       text,
-      isCreatedByUser: true
+      isCreatedByUser: true,
     };
     return userMessage;
   }
 
   async handleStartMethods(message, opts) {
-    const {
-      user,
-      conversationId,
-      parentMessageId,
-      userMessageId,
-      responseMessageId,
-      saveOptions,
-    } = await this.setMessageOptions(opts);
+    const { user, conversationId, parentMessageId, userMessageId, responseMessageId, saveOptions } =
+      await this.setMessageOptions(opts);
 
     const userMessage = this.createUserMessage({
       messageId: userMessageId,
@@ -104,7 +98,7 @@ class BaseClient {
       opts.getIds({
         userMessage,
         conversationId,
-        responseMessageId
+        responseMessageId,
       });
     }
 
@@ -189,24 +183,32 @@ class BaseClient {
 
   async refineMessages(messagesToRefine, remainingContextTokens) {
     const model = new ChatOpenAI({ temperature: 0 });
-    const chain = loadSummarizationChain(model, { type: 'refine', verbose: this.options.debug, refinePrompt });
+    const chain = loadSummarizationChain(model, {
+      type: 'refine',
+      verbose: this.options.debug,
+      refinePrompt,
+    });
     const splitter = new RecursiveCharacterTextSplitter({
       chunkSize: 1500,
       chunkOverlap: 100,
     });
-    const userMessages = this.concatenateMessages(messagesToRefine.filter(m => m.role === 'user'));
-    const assistantMessages = this.concatenateMessages(messagesToRefine.filter(m => m.role !== 'user'));
-    const userDocs = await splitter.createDocuments([userMessages],[],{
+    const userMessages = this.concatenateMessages(
+      messagesToRefine.filter((m) => m.role === 'user'),
+    );
+    const assistantMessages = this.concatenateMessages(
+      messagesToRefine.filter((m) => m.role !== 'user'),
+    );
+    const userDocs = await splitter.createDocuments([userMessages], [], {
       chunkHeader: 'DOCUMENT NAME: User Message\n\n---\n\n',
       appendChunkOverlapHeader: true,
     });
-    const assistantDocs = await splitter.createDocuments([assistantMessages],[],{
+    const assistantDocs = await splitter.createDocuments([assistantMessages], [], {
       chunkHeader: 'DOCUMENT NAME: Assistant Message\n\n---\n\n',
       appendChunkOverlapHeader: true,
     });
     // const chunkSize = Math.round(concatenatedMessages.length / 512);
     const input_documents = userDocs.concat(assistantDocs);
-    if (this.options.debug ) {
+    if (this.options.debug) {
       console.debug('Refining messages...');
     }
     try {
@@ -219,11 +221,15 @@ class BaseClient {
         role: 'assistant',
         content: res.output_text,
         tokenCount: this.getTokenCount(res.output_text),
-      }
+      };
 
-      if (this.options.debug ) {
+      if (this.options.debug) {
         console.debug('Refined messages', refinedMessage);
-        console.debug(`remainingContextTokens: ${remainingContextTokens}, after refining: ${remainingContextTokens - refinedMessage.tokenCount}`);
+        console.debug(
+          `remainingContextTokens: ${remainingContextTokens}, after refining: ${
+            remainingContextTokens - refinedMessage.tokenCount
+          }`,
+        );
       }
 
       return refinedMessage;
@@ -235,15 +241,15 @@ class BaseClient {
   }
 
   /**
- * This method processes an array of messages and returns a context of messages that fit within a token limit.
- * It iterates over the messages from newest to oldest, adding them to the context until the token limit is reached.
- * If the token limit would be exceeded by adding a message, that message and possibly the previous one are added to a separate array of messages to refine.
- * The method uses `push` and `pop` operations for efficient array manipulation, and reverses the arrays at the end to maintain the original order of the messages.
- * The method also includes a mechanism to avoid blocking the event loop by waiting for the next tick after each iteration.
- *
- * @param {Array} messages - An array of messages, each with a `tokenCount` property. The messages should be ordered from oldest to newest.
- * @returns {Object} An object with three properties: `context`, `remainingContextTokens`, and `messagesToRefine`. `context` is an array of messages that fit within the token limit. `remainingContextTokens` is the number of tokens remaining within the limit after adding the messages to the context. `messagesToRefine` is an array of messages that were not added to the context because they would have exceeded the token limit.
- */
+   * This method processes an array of messages and returns a context of messages that fit within a token limit.
+   * It iterates over the messages from newest to oldest, adding them to the context until the token limit is reached.
+   * If the token limit would be exceeded by adding a message, that message and possibly the previous one are added to a separate array of messages to refine.
+   * The method uses `push` and `pop` operations for efficient array manipulation, and reverses the arrays at the end to maintain the original order of the messages.
+   * The method also includes a mechanism to avoid blocking the event loop by waiting for the next tick after each iteration.
+   *
+   * @param {Array} messages - An array of messages, each with a `tokenCount` property. The messages should be ordered from oldest to newest.
+   * @returns {Object} An object with three properties: `context`, `remainingContextTokens`, and `messagesToRefine`. `context` is an array of messages that fit within the token limit. `remainingContextTokens` is the number of tokens remaining within the limit after adding the messages to the context. `messagesToRefine` is an array of messages that were not added to the context because they would have exceeded the token limit.
+   */
   async getMessagesWithinTokenLimit(messages) {
     let currentTokenCount = 0;
     let context = [];
@@ -282,26 +288,22 @@ class BaseClient {
       context.push(message);
       currentTokenCount = newTokenCount;
       remainingContextTokens = this.maxContextTokens - currentTokenCount;
-      await new Promise(resolve => setImmediate(resolve));
+      await new Promise((resolve) => setImmediate(resolve));
     }
 
     return {
       context: context.reverse(),
       remainingContextTokens,
       messagesToRefine: messagesToRefine.reverse(),
-      refineIndex
+      refineIndex,
     };
   }
 
-  async handleContextStrategy({instructions, orderedMessages, formattedMessages}) {
+  async handleContextStrategy({ instructions, orderedMessages, formattedMessages }) {
     let payload = this.addInstructions(formattedMessages, instructions);
     let orderedWithInstructions = this.addInstructions(orderedMessages, instructions);
-    let {
-      context,
-      remainingContextTokens,
-      messagesToRefine,
-      refineIndex
-    } = await this.getMessagesWithinTokenLimit(payload);
+    let { context, remainingContextTokens, messagesToRefine, refineIndex } =
+      await this.getMessagesWithinTokenLimit(payload);
 
     payload = context;
     let refinedMessage;
@@ -325,8 +327,14 @@ class BaseClient {
 
     if (this.options.debug) {
       console.debug('<---------------------------------DIFF--------------------------------->');
-      console.debug(`Difference between payload (${payload.length}) and orderedWithInstructions (${orderedWithInstructions.length}): ${diff}`);
-      console.debug('remainingContextTokens, this.maxContextTokens (1/2)', remainingContextTokens, this.maxContextTokens);
+      console.debug(
+        `Difference between payload (${payload.length}) and orderedWithInstructions (${orderedWithInstructions.length}): ${diff}`,
+      );
+      console.debug(
+        'remainingContextTokens, this.maxContextTokens (1/2)',
+        remainingContextTokens,
+        this.maxContextTokens,
+      );
     }
 
     // If the difference is positive, slice the orderedWithInstructions array
@@ -341,7 +349,11 @@ class BaseClient {
     }
 
     if (this.options.debug) {
-      console.debug('remainingContextTokens, this.maxContextTokens (2/2)', remainingContextTokens, this.maxContextTokens);
+      console.debug(
+        'remainingContextTokens, this.maxContextTokens (2/2)',
+        remainingContextTokens,
+        this.maxContextTokens,
+      );
     }
 
     let tokenCountMap = orderedWithInstructions.reduce((map, message, index) => {
@@ -350,7 +362,7 @@ class BaseClient {
       }
 
       if (index === refineIndex) {
-        map.refined = { ...refinedMessage, messageId: message.messageId};
+        map.refined = { ...refinedMessage, messageId: message.messageId };
       }
 
       map[message.messageId] = payload[index].tokenCount;
@@ -370,20 +382,19 @@ class BaseClient {
   }
 
   async sendMessage(message, opts = {}) {
-    console.log('BaseClient: sendMessage', message, opts);
-    const {
-      user,
-      conversationId,
-      responseMessageId,
-      saveOptions,
-      userMessage,
-    } = await this.handleStartMethods(message, opts);
+    const { user, conversationId, responseMessageId, saveOptions, userMessage } =
+      await this.handleStartMethods(message, opts);
 
+    this.user = user;
     // It's not necessary to push to currentMessages
     // depending on subclass implementation of handling messages
     this.currentMessages.push(userMessage);
 
-    let { prompt: payload, tokenCountMap, promptTokens } = await this.buildMessages(
+    let {
+      prompt: payload,
+      tokenCountMap,
+      promptTokens,
+    } = await this.buildMessages(
       this.currentMessages,
       // When the userMessage is pushed to currentMessages, the parentMessage is the userMessageId.
       // this only matters when buildMessages is utilizing the parentMessageId, and may vary on implementation
@@ -397,7 +408,7 @@ class BaseClient {
     }
 
     if (tokenCountMap) {
-      console.dir(tokenCountMap, { depth: null })
+      console.dir(tokenCountMap, { depth: null });
       if (tokenCountMap[userMessage.messageId]) {
         userMessage.tokenCount = tokenCountMap[userMessage.messageId];
         console.log('userMessage.tokenCount', userMessage.tokenCount);
@@ -457,11 +468,11 @@ class BaseClient {
   }
 
   async saveMessageToDatabase(message, endpointOptions, user = null) {
-    await saveMessage({ ...message, unfinished: false });
+    await saveMessage({ ...message, unfinished: false, cancelled: false });
     await saveConvo(user, {
       conversationId: message.conversationId,
       endpoint: this.options.endpoint,
-      ...endpointOptions
+      ...endpointOptions,
     });
   }
 
@@ -470,12 +481,12 @@ class BaseClient {
   }
 
   /**
-     * Iterate through messages, building an array based on the parentMessageId.
-     * Each message has an id and a parentMessageId. The parentMessageId is the id of the message that this message is a reply to.
-     * @param messages
-     * @param parentMessageId
-     * @returns {*[]} An array containing the messages in the order they should be displayed, starting with the root message.
-     */
+   * Iterate through messages, building an array based on the parentMessageId.
+   * Each message has an id and a parentMessageId. The parentMessageId is the id of the message that this message is a reply to.
+   * @param messages
+   * @param parentMessageId
+   * @returns {*[]} An array containing the messages in the order they should be displayed, starting with the root message.
+   */
   static getMessagesForConversation(messages, parentMessageId, mapMethod = null) {
     if (!messages || messages.length === 0) {
       return [];
@@ -484,7 +495,7 @@ class BaseClient {
     const orderedMessages = [];
     let currentMessageId = parentMessageId;
     while (currentMessageId) {
-      const message = messages.find(msg => {
+      const message = messages.find((msg) => {
         const messageId = msg.messageId ?? msg.id;
         return messageId === currentMessageId;
       });
@@ -503,13 +514,13 @@ class BaseClient {
   }
 
   /**
- * Algorithm adapted from "6. Counting tokens for chat API calls" of
- * https://github.com/openai/openai-cookbook/blob/main/examples/How_to_count_tokens_with_tiktoken.ipynb
- *
- * An additional 2 tokens need to be added for metadata after all messages have been counted.
- *
- * @param {*} message
- */
+   * Algorithm adapted from "6. Counting tokens for chat API calls" of
+   * https://github.com/openai/openai-cookbook/blob/main/examples/How_to_count_tokens_with_tiktoken.ipynb
+   *
+   * An additional 2 tokens need to be added for metadata after all messages have been counted.
+   *
+   * @param {*} message
+   */
   getTokenCountForMessage(message) {
     let tokensPerMessage;
     let nameAdjustment;
@@ -534,7 +545,7 @@ class BaseClient {
       const numTokens = this.getTokenCount(value);
 
       // Adjust by `nameAdjustment` tokens if the property key is 'name'
-      const adjustment = (key === 'name') ? nameAdjustment : 0;
+      const adjustment = key === 'name' ? nameAdjustment : 0;
       return numTokens + adjustment;
     });
 
@@ -547,4 +558,4 @@ class BaseClient {
   }
 }
 
-module.exports = BaseClient;
+module.exports = BaseClient;

api/app/clients/ChatGPTClient.js

@@ -1,6 +1,9 @@
 const crypto = require('crypto');
 const Keyv = require('keyv');
-const { encoding_for_model: encodingForModel, get_encoding: getEncoding } = require('@dqbd/tiktoken');
+const {
+  encoding_for_model: encodingForModel,
+  get_encoding: getEncoding,
+} = require('@dqbd/tiktoken');
 const { fetchEventSource } = require('@waylaidwanderer/fetch-event-source');
 const { Agent, ProxyAgent } = require('undici');
 const BaseClient = require('./BaseClient');
@@ -9,11 +12,7 @@ const CHATGPT_MODEL = 'gpt-3.5-turbo';
 const tokenizersCache = {};
 
 class ChatGPTClient extends BaseClient {
-  constructor(
-    apiKey,
-    options = {},
-    cacheOptions = {},
-  ) {
+  constructor(apiKey, options = {}, cacheOptions = {}) {
     super(apiKey, options, cacheOptions);
 
     cacheOptions.namespace = cacheOptions.namespace || 'chatgpt';
@@ -49,13 +48,16 @@ class ChatGPTClient extends BaseClient {
       model: modelOptions.model || CHATGPT_MODEL,
       temperature: typeof modelOptions.temperature === 'undefined' ? 0.8 : modelOptions.temperature,
       top_p: typeof modelOptions.top_p === 'undefined' ? 1 : modelOptions.top_p,
-      presence_penalty: typeof modelOptions.presence_penalty === 'undefined' ? 1 : modelOptions.presence_penalty,
+      presence_penalty:
+        typeof modelOptions.presence_penalty === 'undefined' ? 1 : modelOptions.presence_penalty,
       stop: modelOptions.stop,
     };
 
     this.isChatGptModel = this.modelOptions.model.startsWith('gpt-');
     const { isChatGptModel } = this;
-    this.isUnofficialChatGptModel = this.modelOptions.model.startsWith('text-chat') || this.modelOptions.model.startsWith('text-davinci-002-render');
+    this.isUnofficialChatGptModel =
+      this.modelOptions.model.startsWith('text-chat') ||
+      this.modelOptions.model.startsWith('text-davinci-002-render');
     const { isUnofficialChatGptModel } = this;
 
     // Davinci models have a max context length of 4097 tokens.
@@ -64,10 +66,15 @@ class ChatGPTClient extends BaseClient {
     // The max prompt tokens is determined by the max context tokens minus the max response tokens.
     // Earlier messages will be dropped until the prompt is within the limit.
     this.maxResponseTokens = this.modelOptions.max_tokens || 1024;
-    this.maxPromptTokens = this.options.maxPromptTokens || (this.maxContextTokens - this.maxResponseTokens);
+    this.maxPromptTokens =
+      this.options.maxPromptTokens || this.maxContextTokens - this.maxResponseTokens;
 
     if (this.maxPromptTokens + this.maxResponseTokens > this.maxContextTokens) {
-      throw new Error(`maxPromptTokens + max_tokens (${this.maxPromptTokens} + ${this.maxResponseTokens} = ${this.maxPromptTokens + this.maxResponseTokens}) must be less than or equal to maxContextTokens (${this.maxContextTokens})`);
+      throw new Error(
+        `maxPromptTokens + max_tokens (${this.maxPromptTokens} + ${this.maxResponseTokens} = ${
+          this.maxPromptTokens + this.maxResponseTokens
+        }) must be less than or equal to maxContextTokens (${this.maxContextTokens})`,
+      );
     }
 
     this.userLabel = this.options.userLabel || 'User';
@@ -249,13 +256,10 @@ class ChatGPTClient extends BaseClient {
         }
       });
     }
-    const response = await fetch(
-      url,
-      {
-        ...opts,
-        signal: abortController.signal,
-      },
-    );
+    const response = await fetch(url, {
+      ...opts,
+      signal: abortController.signal,
+    });
     if (response.status !== 200) {
       const body = await response.text();
       const error = new Error(`Failed to send message. HTTP ${response.status} - ${body}`);
@@ -299,10 +303,7 @@ ${botMessage.message}
       .trim();
   }
 
-  async sendMessage(
-    message,
-    opts = {},
-  ) {
+  async sendMessage(message, opts = {}) {
     if (opts.clientOptions && typeof opts.clientOptions === 'object') {
       this.setOptions(opts.clientOptions);
     }
@@ -310,9 +311,10 @@ ${botMessage.message}
     const conversationId = opts.conversationId || crypto.randomUUID();
     const parentMessageId = opts.parentMessageId || crypto.randomUUID();
 
-    let conversation = typeof opts.conversation === 'object'
-      ? opts.conversation
-      : await this.conversationsCache.get(conversationId);
+    let conversation =
+      typeof opts.conversation === 'object'
+        ? opts.conversation
+        : await this.conversationsCache.get(conversationId);
 
     let isNewConversation = false;
     if (!conversation) {
@@ -357,7 +359,9 @@ ${botMessage.message}
           if (progressMessage === '[DONE]') {
             return;
           }
-          const token = this.isChatGptModel ? progressMessage.choices[0].delta.content : progressMessage.choices[0].text;
+          const token = this.isChatGptModel
+            ? progressMessage.choices[0].delta.content
+            : progressMessage.choices[0].text;
           // first event's delta content is always undefined
           if (!token) {
             return;
@@ -437,10 +441,11 @@ ${botMessage.message}
       }
       promptPrefix = `${this.startToken}Instructions:\n${promptPrefix}`;
     } else {
-      const currentDateString = new Date().toLocaleDateString(
-        'en-us',
-        { year: 'numeric', month: 'long', day: 'numeric' },
-      );
+      const currentDateString = new Date().toLocaleDateString('en-us', {
+        year: 'numeric',
+        month: 'long',
+        day: 'numeric',
+      });
       promptPrefix = `${this.startToken}Instructions:\nYou are ChatGPT, a large language model trained by OpenAI. Respond conversationally.\nCurrent date: ${currentDateString}${this.endToken}\n\n`;
     }
 
@@ -459,7 +464,9 @@ ${botMessage.message}
 
     let currentTokenCount;
     if (isChatGptModel) {
-      currentTokenCount = this.getTokenCountForMessage(instructionsPayload) + this.getTokenCountForMessage(messagePayload);
+      currentTokenCount =
+        this.getTokenCountForMessage(instructionsPayload) +
+        this.getTokenCountForMessage(messagePayload);
     } else {
       currentTokenCount = this.getTokenCount(`${promptPrefix}${promptSuffix}`);
     }
@@ -473,8 +480,13 @@ ${botMessage.message}
     const buildPromptBody = async () => {
       if (currentTokenCount < maxTokenCount && orderedMessages.length > 0) {
         const message = orderedMessages.pop();
-        const roleLabel = message?.isCreatedByUser || message?.role?.toLowerCase() === 'user' ? this.userLabel : this.chatGptLabel;
-        const messageString = `${this.startToken}${roleLabel}:\n${message?.text ?? message?.message}${this.endToken}\n`;
+        const roleLabel =
+          message?.isCreatedByUser || message?.role?.toLowerCase() === 'user'
+            ? this.userLabel
+            : this.chatGptLabel;
+        const messageString = `${this.startToken}${roleLabel}:\n${
+          message?.text ?? message?.message
+        }${this.endToken}\n`;
         let newPromptBody;
         if (promptBody || isChatGptModel) {
           newPromptBody = `${messageString}${promptBody}`;
@@ -496,12 +508,14 @@ ${botMessage.message}
             return false;
           }
           // This is the first message, so we can't add it. Just throw an error.
-          throw new Error(`Prompt is too long. Max token count is ${maxTokenCount}, but prompt is ${newTokenCount} tokens long.`);
+          throw new Error(
+            `Prompt is too long. Max token count is ${maxTokenCount}, but prompt is ${newTokenCount} tokens long.`,
+          );
         }
         promptBody = newPromptBody;
         currentTokenCount = newTokenCount;
         // wait for next tick to avoid blocking the event loop
-        await new Promise(resolve => setImmediate(resolve));
+        await new Promise((resolve) => setImmediate(resolve));
         return buildPromptBody();
       }
       return true;
@@ -517,7 +531,10 @@ ${botMessage.message}
     }
 
     // Use up to `this.maxContextTokens` tokens (prompt + response), but try to leave `this.maxTokens` tokens for the response.
-    this.modelOptions.max_tokens = Math.min(this.maxContextTokens - currentTokenCount, this.maxResponseTokens);
+    this.modelOptions.max_tokens = Math.min(
+      this.maxContextTokens - currentTokenCount,
+      this.maxResponseTokens,
+    );
 
     if (this.options.debug) {
       console.debug(`Prompt : ${prompt}`);
@@ -534,13 +551,13 @@ ${botMessage.message}
   }
 
   /**
- * Algorithm adapted from "6. Counting tokens for chat API calls" of
- * https://github.com/openai/openai-cookbook/blob/main/examples/How_to_count_tokens_with_tiktoken.ipynb
- *
- * An additional 2 tokens need to be added for metadata after all messages have been counted.
- *
- * @param {*} message
- */
+   * Algorithm adapted from "6. Counting tokens for chat API calls" of
+   * https://github.com/openai/openai-cookbook/blob/main/examples/How_to_count_tokens_with_tiktoken.ipynb
+   *
+   * An additional 2 tokens need to be added for metadata after all messages have been counted.
+   *
+   * @param {*} message
+   */
   getTokenCountForMessage(message) {
     let tokensPerMessage;
     let nameAdjustment;
@@ -558,7 +575,7 @@ ${botMessage.message}
       const numTokens = this.getTokenCount(value);
 
       // Adjust by `nameAdjustment` tokens if the property key is 'name'
-      const adjustment = (key === 'name') ? nameAdjustment : 0;
+      const adjustment = key === 'name' ? nameAdjustment : 0;
       return numTokens + adjustment;
     });
 
@@ -567,4 +584,4 @@ ${botMessage.message}
   }
 }
 
-module.exports = ChatGPTClient;
+module.exports = ChatGPTClient;

api/app/clients/GoogleClient.js

@@ -3,7 +3,7 @@ const { google } = require('googleapis');
 const { Agent, ProxyAgent } = require('undici');
 const {
   encoding_for_model: encodingForModel,
-  get_encoding: getEncoding
+  get_encoding: getEncoding,
 } = require('@dqbd/tiktoken');
 
 const tokenizersCache = {};
@@ -43,20 +43,20 @@ class GoogleClient extends BaseClient {
       // nested options aren't spread properly, so we need to do this manually
       this.options.modelOptions = {
         ...this.options.modelOptions,
-        ...options.modelOptions
+        ...options.modelOptions,
       };
       delete options.modelOptions;
       // now we can merge options
       this.options = {
         ...this.options,
-        ...options
+        ...options,
       };
     } else {
       this.options = options;
     }
 
     this.options.examples = this.options.examples.filter(
-      (obj) => obj.input.content !== '' && obj.output.content !== ''
+      (obj) => obj.input.content !== '' && obj.output.content !== '',
     );
 
     const modelOptions = this.options.modelOptions || {};
@@ -66,7 +66,7 @@ class GoogleClient extends BaseClient {
       model: modelOptions.model || 'chat-bison',
       temperature: typeof modelOptions.temperature === 'undefined' ? 0.2 : modelOptions.temperature, // 0 - 1, 0.2 is recommended
       topP: typeof modelOptions.topP === 'undefined' ? 0.95 : modelOptions.topP, // 0 - 1, default: 0.95
-      topK: typeof modelOptions.topK === 'undefined' ? 40 : modelOptions.topK // 1-40, default: 40
+      topK: typeof modelOptions.topK === 'undefined' ? 40 : modelOptions.topK, // 1-40, default: 40
       // stop: modelOptions.stop // no stop method for now
     };
 
@@ -86,7 +86,7 @@ class GoogleClient extends BaseClient {
       throw new Error(
         `maxPromptTokens + maxOutputTokens (${this.maxPromptTokens} + ${this.maxResponseTokens} = ${
           this.maxPromptTokens + this.maxResponseTokens
-        }) must be less than or equal to maxContextTokens (${this.maxContextTokens})`
+        }) must be less than or equal to maxContextTokens (${this.maxContextTokens})`,
       );
     }
 
@@ -105,7 +105,7 @@ class GoogleClient extends BaseClient {
       this.endToken = '<|im_end|>';
       this.gptEncoder = this.constructor.getTokenizer('text-davinci-003', true, {
         '<|im_start|>': 100264,
-        '<|im_end|>': 100265
+        '<|im_end|>': 100265,
       });
     } else {
       // Previously I was trying to use "<|endoftext|>" but there seems to be some bug with OpenAI's token counting
@@ -143,7 +143,7 @@ class GoogleClient extends BaseClient {
   getMessageMapMethod() {
     return ((message) => ({
       author: message?.author ?? (message.isCreatedByUser ? this.userLabel : this.modelLabel),
-      content: message?.content ?? message.text
+      content: message?.content ?? message.text,
     })).bind(this);
   }
 
@@ -153,9 +153,9 @@ class GoogleClient extends BaseClient {
       instances: [
         {
           messages: formattedMessages,
-        }
+        },
       ],
-      parameters: this.options.modelOptions
+      parameters: this.options.modelOptions,
     };
 
     if (this.options.promptPrefix) {
@@ -170,8 +170,8 @@ class GoogleClient extends BaseClient {
     if (this.isTextModel) {
       payload.instances = [
         {
-          prompt: messages[messages.length -1].content
-        }
+          prompt: messages[messages.length - 1].content,
+        },
       ];
     }
 
@@ -199,9 +199,9 @@ class GoogleClient extends BaseClient {
       method: 'POST',
       agent: new Agent({
         bodyTimeout: 0,
-        headersTimeout: 0
+        headersTimeout: 0,
       }),
-      signal: abortController.signal
+      signal: abortController.signal,
     };
 
     if (this.options.proxy) {
@@ -216,7 +216,9 @@ class GoogleClient extends BaseClient {
 
   getSaveOptions() {
     return {
-      ...this.modelOptions
+      promptPrefix: this.options.promptPrefix,
+      modelLabel: this.options.modelLabel,
+      ...this.modelOptions,
     };
   }
 
@@ -237,7 +239,7 @@ class GoogleClient extends BaseClient {
         '';
       if (blocked === true) {
         reply = `Google blocked a proper response to your message:\n${JSON.stringify(
-          result.predictions[0].safetyAttributes
+          result.predictions[0].safetyAttributes,
         )}${reply.length > 0 ? `\nAI Response:\n${reply}` : ''}`;
       }
       if (this.options.debug) {

api/app/clients/OpenAIClient.js

@@ -1,6 +1,9 @@
 const BaseClient = require('./BaseClient');
 const ChatGPTClient = require('./ChatGPTClient');
-const { encoding_for_model: encodingForModel, get_encoding: getEncoding } = require('@dqbd/tiktoken');
+const {
+  encoding_for_model: encodingForModel,
+  get_encoding: getEncoding,
+} = require('@dqbd/tiktoken');
 const { maxTokensMap, genAzureChatCompletion } = require('../../utils');
 
 const tokenizersCache = {};
@@ -12,7 +15,9 @@ class OpenAIClient extends BaseClient {
     this.buildPrompt = this.ChatGPTClient.buildPrompt.bind(this);
     this.getCompletion = this.ChatGPTClient.getCompletion.bind(this);
     this.sender = options.sender ?? 'ChatGPT';
-    this.contextStrategy = options.contextStrategy ? options.contextStrategy.toLowerCase() : 'discard';
+    this.contextStrategy = options.contextStrategy
+      ? options.contextStrategy.toLowerCase()
+      : 'discard';
     this.shouldRefineContext = this.contextStrategy === 'refine';
     this.azure = options.azure || false;
     if (this.azure) {
@@ -45,27 +50,39 @@ class OpenAIClient extends BaseClient {
       this.modelOptions = {
         ...modelOptions,
         model: modelOptions.model || 'gpt-3.5-turbo',
-        temperature: typeof modelOptions.temperature === 'undefined' ? 0.8 : modelOptions.temperature,
+        temperature:
+          typeof modelOptions.temperature === 'undefined' ? 0.8 : modelOptions.temperature,
         top_p: typeof modelOptions.top_p === 'undefined' ? 1 : modelOptions.top_p,
-        presence_penalty: typeof modelOptions.presence_penalty === 'undefined' ? 1 : modelOptions.presence_penalty,
+        presence_penalty:
+          typeof modelOptions.presence_penalty === 'undefined' ? 1 : modelOptions.presence_penalty,
         stop: modelOptions.stop,
       };
     }
 
-    this.isChatCompletion = this.options.reverseProxyUrl || this.options.localAI || this.modelOptions.model.startsWith('gpt-');
+    this.isChatCompletion =
+      this.options.reverseProxyUrl ||
+      this.options.localAI ||
+      this.modelOptions.model.startsWith('gpt-');
     this.isChatGptModel = this.isChatCompletion;
     if (this.modelOptions.model === 'text-davinci-003') {
       this.isChatCompletion = false;
       this.isChatGptModel = false;
     }
     const { isChatGptModel } = this;
-    this.isUnofficialChatGptModel = this.modelOptions.model.startsWith('text-chat') || this.modelOptions.model.startsWith('text-davinci-002-render');
+    this.isUnofficialChatGptModel =
+      this.modelOptions.model.startsWith('text-chat') ||
+      this.modelOptions.model.startsWith('text-davinci-002-render');
     this.maxContextTokens = maxTokensMap[this.modelOptions.model] ?? 4095; // 1 less than maximum
     this.maxResponseTokens = this.modelOptions.max_tokens || 1024;
-    this.maxPromptTokens = this.options.maxPromptTokens || (this.maxContextTokens - this.maxResponseTokens);
+    this.maxPromptTokens =
+      this.options.maxPromptTokens || this.maxContextTokens - this.maxResponseTokens;
 
     if (this.maxPromptTokens + this.maxResponseTokens > this.maxContextTokens) {
-      throw new Error(`maxPromptTokens + max_tokens (${this.maxPromptTokens} + ${this.maxResponseTokens} = ${this.maxPromptTokens + this.maxResponseTokens}) must be less than or equal to maxContextTokens (${this.maxContextTokens})`);
+      throw new Error(
+        `maxPromptTokens + max_tokens (${this.maxPromptTokens} + ${this.maxResponseTokens} = ${
+          this.maxPromptTokens + this.maxResponseTokens
+        }) must be less than or equal to maxContextTokens (${this.maxContextTokens})`,
+      );
     }
 
     this.userLabel = this.options.userLabel || 'User';
@@ -185,7 +202,7 @@ class OpenAIClient extends BaseClient {
     return {
       chatGptLabel: this.options.chatGptLabel,
       promptPrefix: this.options.promptPrefix,
-      ...this.modelOptions
+      ...this.modelOptions,
     };
   }
 
@@ -197,9 +214,16 @@ class OpenAIClient extends BaseClient {
     };
   }
 
-  async buildMessages(messages, parentMessageId, { isChatCompletion = false, promptPrefix = null }) {
+  async buildMessages(
+    messages,
+    parentMessageId,
+    { isChatCompletion = false, promptPrefix = null },
+  ) {
     if (!isChatCompletion) {
-      return await this.buildPrompt(messages, parentMessageId, { isChatGptModel: isChatCompletion, promptPrefix });
+      return await this.buildPrompt(messages, parentMessageId, {
+        isChatGptModel: isChatCompletion,
+        promptPrefix,
+      });
     }
 
     let payload;
@@ -214,7 +238,7 @@ class OpenAIClient extends BaseClient {
       instructions = {
         role: 'system',
         name: 'instructions',
-        content: promptPrefix
+        content: promptPrefix,
       };
 
       if (this.contextStrategy) {
@@ -236,7 +260,8 @@ class OpenAIClient extends BaseClient {
       }
 
       if (this.contextStrategy) {
-        formattedMessage.tokenCount = message.tokenCount ?? this.getTokenCountForMessage(formattedMessage);
+        formattedMessage.tokenCount =
+          message.tokenCount ?? this.getTokenCountForMessage(formattedMessage);
       }
 
       return formattedMessage;
@@ -244,8 +269,11 @@ class OpenAIClient extends BaseClient {
 
     // TODO: need to handle interleaving instructions better
     if (this.contextStrategy) {
-      ({ payload, tokenCountMap, promptTokens, messages } =
-        await this.handleContextStrategy({ instructions, orderedMessages, formattedMessages }));
+      ({ payload, tokenCountMap, promptTokens, messages } = await this.handleContextStrategy({
+        instructions,
+        orderedMessages,
+        formattedMessages,
+      }));
     }
 
     const result = {
@@ -272,8 +300,9 @@ class OpenAIClient extends BaseClient {
           if (progressMessage === '[DONE]') {
             return;
           }
-          const token =
-            this.isChatCompletion ? progressMessage.choices?.[0]?.delta?.content : progressMessage.choices?.[0]?.text;
+          const token = this.isChatCompletion
+            ? progressMessage.choices?.[0]?.delta?.content
+            : progressMessage.choices?.[0]?.text;
           // first event's delta content is always undefined
           if (!token) {
             return;

api/app/clients/PluginsClient.js

@@ -2,14 +2,11 @@ const OpenAIClient = require('./OpenAIClient');
 const { ChatOpenAI } = require('langchain/chat_models/openai');
 const { CallbackManager } = require('langchain/callbacks');
 const { initializeCustomAgent, initializeFunctionsAgent } = require('./agents/');
+const { findMessageContent } = require('../../utils');
 const { loadTools } = require('./tools/util');
 const { SelfReflectionTool } = require('./tools/');
 const { HumanChatMessage, AIChatMessage } = require('langchain/schema');
-const {
-  instructions,
-  imageInstructions,
-  errorInstructions,
-} = require('./prompts/instructions');
+const { instructions, imageInstructions, errorInstructions } = require('./prompts/instructions');
 
 class PluginsClient extends OpenAIClient {
   constructor(apiKey, options = {}) {
@@ -28,11 +25,13 @@ class PluginsClient extends OpenAIClient {
 
     if (actions[0]?.action && this.functionsAgent) {
       actions = actions.map((step) => ({
-        log: `Action: ${step.action?.tool || ''}\nInput: ${JSON.stringify(step.action?.toolInput) || ''}\nObservation: ${step.observation}`
+        log: `Action: ${step.action?.tool || ''}\nInput: ${
+          JSON.stringify(step.action?.toolInput) || ''
+        }\nObservation: ${step.observation}`,
       }));
     } else if (actions[0]?.action) {
       actions = actions.map((step) => ({
-        log: `${step.action.log}\nObservation: ${step.observation}`
+        log: `${step.action.log}\nObservation: ${step.observation}`,
       }));
     }
 
@@ -114,8 +113,8 @@ Only respond with your conversational reply to the following User Message:
     super.setOptions(options);
     this.isGpt3 = this.modelOptions.model.startsWith('gpt-3');
 
-    if (this.reverseProxyUrl) {
-      this.langchainProxy = this.reverseProxyUrl.match(/.*v1/)[0];
+    if (this.options.reverseProxyUrl) {
+      this.langchainProxy = this.options.reverseProxyUrl.match(/.*v1/)[0];
     }
   }
 
@@ -136,10 +135,10 @@ Only respond with your conversational reply to the following User Message:
     const prefixMap = {
       'gpt-4': 'gpt-4-0613',
       'gpt-4-32k': 'gpt-4-32k-0613',
-      'gpt-3.5-turbo': 'gpt-3.5-turbo-0613'
+      'gpt-3.5-turbo': 'gpt-3.5-turbo-0613',
     };
 
-    const prefix = Object.keys(prefixMap).find(key => input.startsWith(key));
+    const prefix = Object.keys(prefixMap).find((key) => input.startsWith(key));
     return prefix ? prefixMap[prefix] : 'gpt-3.5-turbo-0613';
   }
 
@@ -173,7 +172,7 @@ Only respond with your conversational reply to the following User Message:
   async initialize({ user, message, onAgentAction, onChainEnd, signal }) {
     const modelOptions = {
       modelName: this.agentOptions.model,
-      temperature: this.agentOptions.temperature
+      temperature: this.agentOptions.temperature,
     };
 
     const configOptions = {};
@@ -194,8 +193,10 @@ Only respond with your conversational reply to the following User Message:
       tools: this.options.tools,
       functions: this.functionsAgent,
       options: {
-        openAIApiKey: this.openAIApiKey
-      }
+        openAIApiKey: this.openAIApiKey,
+        debug: this.options?.debug,
+        message,
+      },
     });
     // load tools
     for (const tool of this.options.tools) {
@@ -235,10 +236,13 @@ Only respond with your conversational reply to the following User Message:
     };
 
     // Map Messages to Langchain format
-    const pastMessages = this.currentMessages.slice(0, -1).map(
-      msg => msg?.isCreatedByUser || msg?.role?.toLowerCase() === 'user'
-        ? new HumanChatMessage(msg.text)
-        : new AIChatMessage(msg.text));
+    const pastMessages = this.currentMessages
+      .slice(0, -1)
+      .map((msg) =>
+        msg?.isCreatedByUser || msg?.role?.toLowerCase() === 'user'
+          ? new HumanChatMessage(msg.text)
+          : new AIChatMessage(msg.text),
+      );
 
     // initialize agent
     const initializer = this.functionsAgent ? initializeFunctionsAgent : initializeCustomAgent;
@@ -258,13 +262,22 @@ Only respond with your conversational reply to the following User Message:
           if (typeof onChainEnd === 'function') {
             onChainEnd(action);
           }
-        }
-      })
+        },
+      }),
     });
 
     if (this.options.debug) {
       console.debug('Loaded agent.');
     }
+
+    onAgentAction(
+      {
+        tool: 'self-reflection',
+        toolInput: `Processing the User's message:\n"${message}"`,
+        log: '',
+      },
+      true,
+    );
   }
 
   async executorCall(message, signal) {
@@ -289,6 +302,11 @@ Only respond with your conversational reply to the following User Message:
       } catch (err) {
         console.error(err);
         errorMessage = err.message;
+        const content = findMessageContent(message);
+        if (content) {
+          errorMessage = content;
+          break;
+        }
         if (attempts === maxAttempts) {
           this.result.output = `Encountered an error while attempting to respond. Error: ${err.message}`;
           this.result.intermediateSteps = this.actions;
@@ -304,7 +322,7 @@ Only respond with your conversational reply to the following User Message:
       return;
     }
 
-    intermediateSteps.forEach(step => {
+    intermediateSteps.forEach((step) => {
       const { observation } = step;
       if (!observation || !observation.includes('![')) {
         return;
@@ -346,7 +364,12 @@ Only respond with your conversational reply to the following User Message:
 
     this.currentMessages.push(userMessage);
 
-    let { prompt: payload, tokenCountMap, promptTokens, messages } = await this.buildMessages(
+    let {
+      prompt: payload,
+      tokenCountMap,
+      promptTokens,
+      messages,
+    } = await this.buildMessages(
       this.currentMessages,
       userMessage.messageId,
       this.getBuildMessagesOptions({
@@ -356,7 +379,7 @@ Only respond with your conversational reply to the following User Message:
     );
 
     if (tokenCountMap) {
-      console.dir(tokenCountMap, { depth: null })
+      console.dir(tokenCountMap, { depth: null });
       if (tokenCountMap[userMessage.messageId]) {
         userMessage.tokenCount = tokenCountMap[userMessage.messageId];
         console.log('userMessage.tokenCount', userMessage.tokenCount);
@@ -389,7 +412,7 @@ Only respond with your conversational reply to the following User Message:
       message,
       onAgentAction,
       onChainEnd,
-      signal: this.abortController.signal
+      signal: this.abortController.signal,
     });
     await this.executorCall(message, this.abortController.signal);
 
@@ -402,7 +425,7 @@ Only respond with your conversational reply to the following User Message:
     if (this.agentOptions.skipCompletion && this.result.output) {
       responseMessage.text = this.result.output;
       this.addImages(this.result.intermediateSteps, responseMessage);
-      await this.generateTextStream(this.result.output, opts.onProgress);
+      await this.generateTextStream(this.result.output, opts.onProgress, { delay: 8 });
       return await this.handleResponseMessage(responseMessage, saveOptions, user);
     }
 
@@ -448,12 +471,12 @@ Only respond with your conversational reply to the following User Message:
     const instructionsPayload = {
       role: 'system',
       name: 'instructions',
-      content: promptPrefix
+      content: promptPrefix,
     };
 
     const messagePayload = {
       role: 'system',
-      content: promptSuffix
+      content: promptSuffix,
     };
 
     if (this.isGpt3) {
@@ -463,13 +486,13 @@ Only respond with your conversational reply to the following User Message:
     }
 
     // testing if this works with browser endpoint
-    if (!this.isGpt3 && this.reverseProxyUrl) {
+    if (!this.isGpt3 && this.options.reverseProxyUrl) {
       instructionsPayload.role = 'user';
     }
 
     let currentTokenCount =
-        this.getTokenCountForMessage(instructionsPayload) +
-        this.getTokenCountForMessage(messagePayload);
+      this.getTokenCountForMessage(instructionsPayload) +
+      this.getTokenCountForMessage(messagePayload);
 
     let promptBody = '';
     const maxTokenCount = this.maxPromptTokens;
@@ -492,7 +515,7 @@ Only respond with your conversational reply to the following User Message:
           }
           // This is the first message, so we can't add it. Just throw an error.
           throw new Error(
-            `Prompt is too long. Max token count is ${maxTokenCount}, but prompt is ${newTokenCount} tokens long.`
+            `Prompt is too long. Max token count is ${maxTokenCount}, but prompt is ${newTokenCount} tokens long.`,
           );
         }
         promptBody = newPromptBody;
@@ -519,7 +542,7 @@ Only respond with your conversational reply to the following User Message:
     // Use up to `this.maxContextTokens` tokens (prompt + response), but try to leave `this.maxTokens` tokens for the response.
     this.modelOptions.max_tokens = Math.min(
       this.maxContextTokens - currentTokenCount,
-      this.maxResponseTokens
+      this.maxResponseTokens,
     );
 
     if (this.isGpt3) {

api/app/clients/agents/CustomAgent/CustomAgent.js

@@ -8,7 +8,7 @@ class CustomAgent extends ZeroShotAgent {
   }
 
   _stop() {
-    return [`\nObservation:`, `\nObservation 1:`];
+    return ['\nObservation:', '\nObservation 1:'];
   }
 
   static createPrompt(tools, opts = {}) {
@@ -32,17 +32,17 @@ class CustomAgent extends ZeroShotAgent {
       .join('\n');
     const toolNames = tools.map((tool) => tool.name);
     const formatInstructions = (0, renderTemplate)(instructions, 'f-string', {
-      tool_names: toolNames
+      tool_names: toolNames,
     });
     const template = [
       `Date: ${currentDateString}\n${prefix}`,
       toolStrings,
       formatInstructions,
-      suffix
+      suffix,
     ].join('\n\n');
     return new PromptTemplate({
       template,
-      inputVariables
+      inputVariables,
     });
   }
 }

api/app/clients/agents/CustomAgent/initializeCustomAgent.js

@@ -6,7 +6,7 @@ const { BufferMemory, ChatMessageHistory } = require('langchain/memory');
 const {
   ChatPromptTemplate,
   SystemMessagePromptTemplate,
-  HumanMessagePromptTemplate
+  HumanMessagePromptTemplate,
 } = require('langchain/prompts');
 
 const initializeCustomAgent = async ({
@@ -22,7 +22,7 @@ const initializeCustomAgent = async ({
     new SystemMessagePromptTemplate(prompt),
     HumanMessagePromptTemplate.fromTemplate(`{chat_history}
 Query: {input}
-{agent_scratchpad}`)
+{agent_scratchpad}`),
   ]);
 
   const outputParser = new CustomOutputParser({ tools });
@@ -34,18 +34,18 @@ Query: {input}
     humanPrefix: 'User',
     aiPrefix: 'Assistant',
     inputKey: 'input',
-    outputKey: 'output'
+    outputKey: 'output',
   });
 
   const llmChain = new LLMChain({
     prompt: chatPrompt,
-    llm: model
+    llm: model,
   });
 
   const agent = new CustomAgent({
     llmChain,
     outputParser,
-    allowedTools: tools.map((tool) => tool.name)
+    allowedTools: tools.map((tool) => tool.name),
   });
 
   return AgentExecutor.fromAgentAndTools({ agent, tools, memory, ...rest });

api/app/clients/agents/CustomAgent/outputParser.js

@@ -57,7 +57,7 @@ class CustomOutputParser extends ZeroShotAgentOutputParser {
       const output = text.substring(finalMatch.index + finalMatch[0].length).trim();
       return {
         returnValues: { output },
-        log: text
+        log: text,
       };
     }
 
@@ -66,7 +66,7 @@ class CustomOutputParser extends ZeroShotAgentOutputParser {
     if (!match) {
       console.log(
         '\n\n<----------------------HIT NO MATCH PARSING ERROR---------------------->\n\n',
-        match
+        match,
       );
       const thoughts = text.replace(/[tT]hought:/, '').split('\n');
       // return {
@@ -77,7 +77,7 @@ class CustomOutputParser extends ZeroShotAgentOutputParser {
 
       return {
         returnValues: { output: thoughts[0] },
-        log: thoughts.slice(1).join('\n')
+        log: thoughts.slice(1).join('\n'),
       };
     }
 
@@ -86,12 +86,12 @@ class CustomOutputParser extends ZeroShotAgentOutputParser {
     if (match && selectedTool === 'n/a') {
       console.log(
         '\n\n<----------------------HIT N/A PARSING ERROR---------------------->\n\n',
-        match
+        match,
       );
       return {
         tool: 'self-reflection',
         toolInput: match[2]?.trim().replace(/^"+|"+$/g, '') ?? '',
-        log: text
+        log: text,
       };
     }
 
@@ -99,7 +99,7 @@ class CustomOutputParser extends ZeroShotAgentOutputParser {
     if (match && !toolIsValid) {
       console.log(
         '\n\n<----------------Tool invalid: Re-assigning Selected Tool---------------->\n\n',
-        match
+        match,
       );
       selectedTool = this.getValidTool(selectedTool);
     }
@@ -107,7 +107,7 @@ class CustomOutputParser extends ZeroShotAgentOutputParser {
     if (match && !selectedTool) {
       console.log(
         '\n\n<----------------------HIT INVALID TOOL PARSING ERROR---------------------->\n\n',
-        match
+        match,
       );
       selectedTool = 'self-reflection';
     }
@@ -115,7 +115,7 @@ class CustomOutputParser extends ZeroShotAgentOutputParser {
     if (match && !match[2]) {
       console.log(
         '\n\n<----------------------HIT NO ACTION INPUT PARSING ERROR---------------------->\n\n',
-        match
+        match,
       );
 
       // In case there is no action input, let's double-check if there is an action input in 'text' variable
@@ -125,7 +125,7 @@ class CustomOutputParser extends ZeroShotAgentOutputParser {
         return {
           tool: selectedTool,
           toolInput: actionInputMatch[1].trim(),
-          log: text
+          log: text,
         };
       }
 
@@ -133,7 +133,7 @@ class CustomOutputParser extends ZeroShotAgentOutputParser {
         return {
           tool: selectedTool,
           toolInput: thoughtMatch[1].trim(),
-          log: text
+          log: text,
         };
       }
     }
@@ -158,12 +158,12 @@ class CustomOutputParser extends ZeroShotAgentOutputParser {
       if (action && actionInputMatch) {
         console.log(
           '\n\n<------Matched Action Input in Long Parsing Error------>\n\n',
-          actionInputMatch
+          actionInputMatch,
         );
         return {
           tool: action,
           toolInput: actionInputMatch[1].trim().replaceAll('"', ''),
-          log: text
+          log: text,
         };
       }
 
@@ -180,7 +180,7 @@ class CustomOutputParser extends ZeroShotAgentOutputParser {
         const returnValues = {
           tool: action,
           toolInput: input,
-          log: thought || inputText
+          log: thought || inputText,
         };
 
         const inputMatch = this.actionValues.exec(returnValues.log); //new
@@ -197,7 +197,7 @@ class CustomOutputParser extends ZeroShotAgentOutputParser {
         return {
           tool: 'self-reflection',
           toolInput: 'Hypothetical actions: \n"' + text + '"\n',
-          log: 'Thought: I need to look at my hypothetical actions and try one'
+          log: 'Thought: I need to look at my hypothetical actions and try one',
         };
       }
 
@@ -210,7 +210,7 @@ class CustomOutputParser extends ZeroShotAgentOutputParser {
     return {
       tool: selectedTool,
       toolInput: match[2]?.trim()?.replace(/^"+|"+$/g, '') ?? '',
-      log: text
+      log: text,
     };
   }
 }

api/app/clients/agents/Functions/FunctionsAgent.js

@@ -5,9 +5,9 @@ const {
   ChatPromptTemplate,
   MessagesPlaceholder,
   SystemMessagePromptTemplate,
-  HumanMessagePromptTemplate
+  HumanMessagePromptTemplate,
 } = require('langchain/prompts');
-const PREFIX = `You are a helpful AI assistant.`;
+const PREFIX = 'You are a helpful AI assistant.';
 
 function parseOutput(message) {
   if (message.additional_kwargs.function_call) {
@@ -15,7 +15,7 @@ function parseOutput(message) {
     return {
       tool: function_call.name,
       toolInput: function_call.arguments ? JSON.parse(function_call.arguments) : {},
-      log: message.text
+      log: message.text,
     };
   } else {
     return { returnValues: { output: message.text }, log: message.text };
@@ -52,7 +52,7 @@ class FunctionsAgent extends Agent {
     return ChatPromptTemplate.fromPromptMessages([
       SystemMessagePromptTemplate.fromTemplate(`Date: ${currentDateString}\n${prefix}`),
       new MessagesPlaceholder('chat_history'),
-      HumanMessagePromptTemplate.fromTemplate(`Query: {input}`),
+      HumanMessagePromptTemplate.fromTemplate('Query: {input}'),
       new MessagesPlaceholder('agent_scratchpad'),
     ]);
   }
@@ -63,12 +63,12 @@ class FunctionsAgent extends Agent {
     const chain = new LLMChain({
       prompt,
       llm,
-      callbacks: args?.callbacks
+      callbacks: args?.callbacks,
     });
     return new FunctionsAgent({
       llmChain: chain,
       allowedTools: tools.map((t) => t.name),
-      tools
+      tools,
     });
   }
 
@@ -77,10 +77,10 @@ class FunctionsAgent extends Agent {
       new AIChatMessage('', {
         function_call: {
           name: action.tool,
-          arguments: JSON.stringify(action.toolInput)
-        }
+          arguments: JSON.stringify(action.toolInput),
+        },
       }),
-      new FunctionChatMessage(observation, action.tool)
+      new FunctionChatMessage(observation, action.tool),
     ]);
   }
 
@@ -96,7 +96,7 @@ class FunctionsAgent extends Agent {
     const llm = this.llmChain.llm;
     const valuesForPrompt = Object.assign({}, newInputs);
     const valuesForLLM = {
-      tools: this.tools
+      tools: this.tools,
     };
     for (let i = 0; i < this.llmChain.llm.callKeys.length; i++) {
       const key = this.llmChain.llm.callKeys[i];
@@ -110,7 +110,7 @@ class FunctionsAgent extends Agent {
     const message = await llm.predictMessages(
       promptValue.toChatMessages(),
       valuesForLLM,
-      callbackManager
+      callbackManager,
     );
     console.log('message', message);
     return parseOutput(message);

api/app/clients/agents/Functions/initializeFunctionsAgent.js

@@ -8,7 +8,6 @@ const initializeFunctionsAgent = async ({
   // currentDateString,
   ...rest
 }) => {
-
   const memory = new BufferMemory({
     chatHistory: new ChatMessageHistory(pastMessages),
     memoryKey: 'chat_history',
@@ -19,17 +18,11 @@ const initializeFunctionsAgent = async ({
     returnMessages: true,
   });
 
-  return await initializeAgentExecutorWithOptions(
-    tools,
-    model,
-    {
-      agentType: 'openai-functions',
-      memory,
-      ...rest,
-    }
-  );
-
+  return await initializeAgentExecutorWithOptions(tools, model, {
+    agentType: 'openai-functions',
+    memory,
+    ...rest,
+  });
 };
 
 module.exports = initializeFunctionsAgent;
-

api/app/clients/agents/index.js

@@ -3,5 +3,5 @@ const initializeFunctionsAgent = require('./Functions/initializeFunctionsAgent')
 
 module.exports = {
   initializeCustomAgent,
-  initializeFunctionsAgent
-};
+  initializeFunctionsAgent,
+};

api/app/clients/index.js

@@ -3,6 +3,7 @@ const OpenAIClient = require('./OpenAIClient');
 const PluginsClient = require('./PluginsClient');
 const GoogleClient = require('./GoogleClient');
 const TextStream = require('./TextStream');
+const AnthropicClient = require('./AnthropicClient');
 const toolUtils = require('./tools/util');
 
 module.exports = {
@@ -11,5 +12,6 @@ module.exports = {
   PluginsClient,
   GoogleClient,
   TextStream,
-  ...toolUtils
-};
+  AnthropicClient,
+  ...toolUtils,
+};

api/app/clients/prompts/instructions.js

@@ -1,6 +1,10 @@
 module.exports = {
-  instructions: `Remember, all your responses MUST be in the format described. Do not respond unless it's in the format described, using the structure of Action, Action Input, etc.`,
-  errorInstructions: `\nYou encountered an error in attempting a response. The user is not aware of the error so you shouldn't mention it.\nReview the actions taken carefully in case there is a partial or complete answer within them.\nError Message:`,
-  imageInstructions: 'You must include the exact image paths from above, formatted in Markdown syntax: ![alt-text](URL)',
-  completionInstructions: `Instructions:\nYou are ChatGPT, a large language model trained by OpenAI. Respond conversationally.\nCurrent date:`,
+  instructions:
+    'Remember, all your responses MUST be in the format described. Do not respond unless it\'s in the format described, using the structure of Action, Action Input, etc.',
+  errorInstructions:
+    '\nYou encountered an error in attempting a response. The user is not aware of the error so you shouldn\'t mention it.\nReview the actions taken carefully in case there is a partial or complete answer within them.\nError Message:',
+  imageInstructions:
+    'You must include the exact image paths from above, formatted in Markdown syntax: ![alt-text](URL)',
+  completionInstructions:
+    'Instructions:\nYou are ChatGPT, a large language model trained by OpenAI. Respond conversationally.\nCurrent date:',
 };

api/app/clients/prompts/refinePrompt.js

@@ -16,9 +16,9 @@ REFINED CONVERSATION SUMMARY:`;
 
 const refinePrompt = new PromptTemplate({
   template: refinePromptTemplate,
-  inputVariables: ["existing_answer", "text"],
+  inputVariables: ['existing_answer', 'text'],
 });
 
 module.exports = {
   refinePrompt,
-};
+};

api/app/clients/specs/BaseClient.test.js

@@ -10,7 +10,7 @@ jest.mock('../../../models', () => {
       getMessages: jest.fn(),
       saveMessage: jest.fn(),
       updateMessage: jest.fn(),
-      saveConvo: jest.fn()
+      saveConvo: jest.fn(),
     };
   };
 });
@@ -52,7 +52,7 @@ describe('BaseClient', () => {
     modelOptions: {
       model: 'gpt-3.5-turbo',
       temperature: 0,
-    }
+    },
   };
 
   beforeEach(() => {
@@ -60,22 +60,14 @@ describe('BaseClient', () => {
   });
 
   test('returns the input messages without instructions when addInstructions() is called with empty instructions', () => {
-    const messages = [
-      { content: 'Hello' },
-      { content: 'How are you?' },
-      { content: 'Goodbye' },
-    ];
+    const messages = [{ content: 'Hello' }, { content: 'How are you?' }, { content: 'Goodbye' }];
     const instructions = '';
     const result = TestClient.addInstructions(messages, instructions);
     expect(result).toEqual(messages);
   });
 
   test('returns the input messages with instructions properly added when addInstructions() is called with non-empty instructions', () => {
-    const messages = [
-      { content: 'Hello' },
-      { content: 'How are you?' },
-      { content: 'Goodbye' },
-    ];
+    const messages = [{ content: 'Hello' }, { content: 'How are you?' }, { content: 'Goodbye' }];
     const instructions = { content: 'Please respond to the question.' };
     const result = TestClient.addInstructions(messages, instructions);
     const expected = [
@@ -94,20 +86,21 @@ describe('BaseClient', () => {
       { name: 'User', content: 'I have a question.' },
     ];
     const result = TestClient.concatenateMessages(messages);
-    const expected = `User:\nHello\n\nAssistant:\nHow can I help you?\n\nUser:\nI have a question.\n\n`;
+    const expected =
+      'User:\nHello\n\nAssistant:\nHow can I help you?\n\nUser:\nI have a question.\n\n';
     expect(result).toBe(expected);
   });
 
   test('refines messages correctly in refineMessages()', async () => {
     const messagesToRefine = [
       { role: 'user', content: 'Hello', tokenCount: 10 },
-      { role: 'assistant', content: 'How can I help you?', tokenCount: 20 }
+      { role: 'assistant', content: 'How can I help you?', tokenCount: 20 },
     ];
     const remainingContextTokens = 100;
     const expectedRefinedMessage = {
       role: 'assistant',
       content: 'Refined answer',
-      tokenCount: 14 // 'Refined answer'.length
+      tokenCount: 14, // 'Refined answer'.length
     };
 
     const result = await TestClient.refineMessages(messagesToRefine, remainingContextTokens);
@@ -120,7 +113,7 @@ describe('BaseClient', () => {
     TestClient.refineMessages = jest.fn().mockResolvedValue({
       role: 'assistant',
       content: 'Refined answer',
-      tokenCount: 30
+      tokenCount: 30,
     });
 
     const messages = [
@@ -148,7 +141,7 @@ describe('BaseClient', () => {
     TestClient.refineMessages = jest.fn().mockResolvedValue({
       role: 'assistant',
       content: 'Refined answer',
-      tokenCount: 4
+      tokenCount: 4,
     });
 
     const messages = [
@@ -176,28 +169,28 @@ describe('BaseClient', () => {
   });
 
   test('handles context strategy correctly in handleContextStrategy()', async () => {
-    TestClient.addInstructions = jest.fn().mockReturnValue([
-      { content: 'Hello' },
-      { content: 'How can I help you?' },
-      { content: 'Please provide more details.' },
-      { content: 'I can assist you with that.' }
-    ]);
+    TestClient.addInstructions = jest
+      .fn()
+      .mockReturnValue([
+        { content: 'Hello' },
+        { content: 'How can I help you?' },
+        { content: 'Please provide more details.' },
+        { content: 'I can assist you with that.' },
+      ]);
     TestClient.getMessagesWithinTokenLimit = jest.fn().mockReturnValue({
       context: [
         { content: 'How can I help you?' },
         { content: 'Please provide more details.' },
-        { content: 'I can assist you with that.' }
+        { content: 'I can assist you with that.' },
       ],
       remainingContextTokens: 80,
-      messagesToRefine: [
-        { content: 'Hello' },
-      ],
+      messagesToRefine: [{ content: 'Hello' }],
       refineIndex: 3,
     });
     TestClient.refineMessages = jest.fn().mockResolvedValue({
       role: 'assistant',
       content: 'Refined answer',
-      tokenCount: 30
+      tokenCount: 30,
     });
     TestClient.getTokenCountForResponse = jest.fn().mockReturnValue(40);
 
@@ -206,24 +199,24 @@ describe('BaseClient', () => {
       { content: 'Hello' },
       { content: 'How can I help you?' },
       { content: 'Please provide more details.' },
-      { content: 'I can assist you with that.' }
+      { content: 'I can assist you with that.' },
     ];
     const formattedMessages = [
       { content: 'Hello' },
       { content: 'How can I help you?' },
       { content: 'Please provide more details.' },
-      { content: 'I can assist you with that.' }
+      { content: 'I can assist you with that.' },
     ];
     const expectedResult = {
       payload: [
         {
           content: 'Refined answer',
           role: 'assistant',
-          tokenCount: 30
+          tokenCount: 30,
         },
         { content: 'How can I help you?' },
         { content: 'Please provide more details.' },
-        { content: 'I can assist you with that.' }
+        { content: 'I can assist you with that.' },
       ],
       promptTokens: expect.any(Number),
       tokenCountMap: {},
@@ -246,7 +239,7 @@ describe('BaseClient', () => {
         isCreatedByUser: false,
         messageId: expect.any(String),
         parentMessageId: expect.any(String),
-        conversationId: expect.any(String)
+        conversationId: expect.any(String),
       });
 
       const response = await TestClient.sendMessage(userMessage);
@@ -261,7 +254,7 @@ describe('BaseClient', () => {
         conversationId,
         parentMessageId,
         getIds: jest.fn(),
-        onStart: jest.fn()
+        onStart: jest.fn(),
       };
 
       const expectedResult = expect.objectContaining({
@@ -270,7 +263,7 @@ describe('BaseClient', () => {
         isCreatedByUser: false,
         messageId: expect.any(String),
         parentMessageId: expect.any(String),
-        conversationId: opts.conversationId
+        conversationId: opts.conversationId,
       });
 
       const response = await TestClient.sendMessage(userMessage, opts);
@@ -300,7 +293,10 @@ describe('BaseClient', () => {
     test('loadHistory is called with the correct arguments', async () => {
       const opts = { conversationId: '123', parentMessageId: '456' };
       await TestClient.sendMessage('Hello, world!', opts);
-      expect(TestClient.loadHistory).toHaveBeenCalledWith(opts.conversationId, opts.parentMessageId);
+      expect(TestClient.loadHistory).toHaveBeenCalledWith(
+        opts.conversationId,
+        opts.parentMessageId,
+      );
     });
 
     test('getIds is called with the correct arguments', async () => {
@@ -310,7 +306,7 @@ describe('BaseClient', () => {
       expect(getIds).toHaveBeenCalledWith({
         userMessage: expect.objectContaining({ text: 'Hello, world!' }),
         conversationId: response.conversationId,
-        responseMessageId: response.messageId
+        responseMessageId: response.messageId,
       });
     });
 
@@ -333,10 +329,10 @@ describe('BaseClient', () => {
           isCreatedByUser: expect.any(Boolean),
           messageId: expect.any(String),
           parentMessageId: expect.any(String),
-          conversationId: expect.any(String)
+          conversationId: expect.any(String),
         }),
         saveOptions,
-        user
+        user,
       );
     });
 
@@ -358,14 +354,16 @@ describe('BaseClient', () => {
 
     test('returns an object with the correct shape', async () => {
       const response = await TestClient.sendMessage('Hello, world!', {});
-      expect(response).toEqual(expect.objectContaining({
-        sender: expect.any(String),
-        text: expect.any(String),
-        isCreatedByUser: expect.any(Boolean),
-        messageId: expect.any(String),
-        parentMessageId: expect.any(String),
-        conversationId: expect.any(String)
-      }));
+      expect(response).toEqual(
+        expect.objectContaining({
+          sender: expect.any(String),
+          text: expect.any(String),
+          isCreatedByUser: expect.any(Boolean),
+          messageId: expect.any(String),
+          parentMessageId: expect.any(String),
+          conversationId: expect.any(String),
+        }),
+      );
     });
   });
 });

api/app/clients/specs/FakeClient.js

@@ -32,9 +32,11 @@ class FakeClient extends BaseClient {
       this.modelOptions = {
         ...modelOptions,
         model: modelOptions.model || 'gpt-3.5-turbo',
-        temperature: typeof modelOptions.temperature === 'undefined' ? 0.8 : modelOptions.temperature,
+        temperature:
+          typeof modelOptions.temperature === 'undefined' ? 0.8 : modelOptions.temperature,
         top_p: typeof modelOptions.top_p === 'undefined' ? 1 : modelOptions.top_p,
-        presence_penalty: typeof modelOptions.presence_penalty === 'undefined' ? 1 : modelOptions.presence_penalty,
+        presence_penalty:
+          typeof modelOptions.presence_penalty === 'undefined' ? 1 : modelOptions.presence_penalty,
         stop: modelOptions.stop,
       };
     }
@@ -66,7 +68,7 @@ const initializeFakeClient = (apiKey, options, fakeMessages) => {
 
       const orderedMessages = TestClient.constructor.getMessagesForConversation(
         fakeMessages,
-        parentMessageId
+        parentMessageId,
       );
 
       TestClient.currentMessages = orderedMessages;
@@ -98,7 +100,7 @@ const initializeFakeClient = (apiKey, options, fakeMessages) => {
 
     this.pastMessages = await TestClient.loadHistory(
       conversationId,
-      TestClient.options?.parentMessageId
+      TestClient.options?.parentMessageId,
     );
 
     const userMessage = {
@@ -107,7 +109,7 @@ const initializeFakeClient = (apiKey, options, fakeMessages) => {
       isCreatedByUser: true,
       messageId: userMessageId,
       parentMessageId,
-      conversationId
+      conversationId,
     };
 
     const response = {
@@ -116,7 +118,7 @@ const initializeFakeClient = (apiKey, options, fakeMessages) => {
       isCreatedByUser: false,
       messageId: crypto.randomUUID(),
       parentMessageId: userMessage.messageId,
-      conversationId
+      conversationId,
     };
 
     fakeMessages.push(userMessage);
@@ -126,7 +128,7 @@ const initializeFakeClient = (apiKey, options, fakeMessages) => {
       opts.getIds({
         userMessage,
         conversationId,
-        responseMessageId: response.messageId
+        responseMessageId: response.messageId,
       });
     }
 
@@ -146,7 +148,10 @@ const initializeFakeClient = (apiKey, options, fakeMessages) => {
         // userMessage is always the last one in the payload
         if (i === payload.length - 1) {
           userMessage.tokenCount = message.tokenCount;
-          console.debug(`Token count for user message: ${tokenCount}`, `Instruction Tokens: ${tokenCountMap.instructions || 'N/A'}`);
+          console.debug(
+            `Token count for user message: ${tokenCount}`,
+            `Instruction Tokens: ${tokenCountMap.instructions || 'N/A'}`,
+          );
         }
         return messageWithoutTokenCount;
       });
@@ -163,7 +168,10 @@ const initializeFakeClient = (apiKey, options, fakeMessages) => {
   });
 
   TestClient.buildMessages = jest.fn(async (messages, parentMessageId) => {
-    const orderedMessages = TestClient.constructor.getMessagesForConversation(messages, parentMessageId);
+    const orderedMessages = TestClient.constructor.getMessagesForConversation(
+      messages,
+      parentMessageId,
+    );
     const formattedMessages = orderedMessages.map((message) => {
       let { role: _role, sender, text } = message;
       const role = _role ?? sender;
@@ -180,6 +188,6 @@ const initializeFakeClient = (apiKey, options, fakeMessages) => {
   });
 
   return TestClient;
-}
+};
 
-module.exports = { FakeClient, initializeFakeClient };
+module.exports = { FakeClient, initializeFakeClient };

api/app/clients/specs/OpenAIClient.test.js

@@ -5,7 +5,7 @@ describe('OpenAIClient', () => {
   const model = 'gpt-4';
   const parentMessageId = '1';
   const messages = [
-    { role: 'user', sender: 'User', text: 'Hello', messageId: parentMessageId},
+    { role: 'user', sender: 'User', text: 'Hello', messageId: parentMessageId },
     { role: 'assistant', sender: 'Assistant', text: 'Hi', messageId: '2' },
   ];
 
@@ -22,7 +22,7 @@ describe('OpenAIClient', () => {
     client.refineMessages = jest.fn().mockResolvedValue({
       role: 'assistant',
       content: 'Refined answer',
-      tokenCount: 30
+      tokenCount: 30,
     });
   });
 
@@ -100,60 +100,83 @@ describe('OpenAIClient', () => {
 
   describe('buildMessages', () => {
     it('should build messages correctly for chat completion', async () => {
-      const result = await client.buildMessages(messages, parentMessageId, { isChatCompletion: true });
+      const result = await client.buildMessages(messages, parentMessageId, {
+        isChatCompletion: true,
+      });
       expect(result).toHaveProperty('prompt');
     });
 
     it('should build messages correctly for non-chat completion', async () => {
-      const result = await client.buildMessages(messages, parentMessageId, { isChatCompletion: false });
+      const result = await client.buildMessages(messages, parentMessageId, {
+        isChatCompletion: false,
+      });
       expect(result).toHaveProperty('prompt');
     });
 
     it('should build messages correctly with a promptPrefix', async () => {
-      const result = await client.buildMessages(messages, parentMessageId, { isChatCompletion: true, promptPrefix: 'Test Prefix' });
+      const result = await client.buildMessages(messages, parentMessageId, {
+        isChatCompletion: true,
+        promptPrefix: 'Test Prefix',
+      });
       expect(result).toHaveProperty('prompt');
-      const instructions = result.prompt.find(item => item.name === 'instructions');
+      const instructions = result.prompt.find((item) => item.name === 'instructions');
       expect(instructions).toBeDefined();
       expect(instructions.content).toContain('Test Prefix');
     });
 
     it('should handle context strategy correctly', async () => {
       client.contextStrategy = 'refine';
-      const result = await client.buildMessages(messages, parentMessageId, { isChatCompletion: true });
+      const result = await client.buildMessages(messages, parentMessageId, {
+        isChatCompletion: true,
+      });
       expect(result).toHaveProperty('prompt');
       expect(result).toHaveProperty('tokenCountMap');
     });
 
     it('should assign name property for user messages when options.name is set', async () => {
       client.options.name = 'Test User';
-      const result = await client.buildMessages(messages, parentMessageId, { isChatCompletion: true });
-      const hasUserWithName = result.prompt.some(item => item.role === 'user' && item.name === 'Test User');
+      const result = await client.buildMessages(messages, parentMessageId, {
+        isChatCompletion: true,
+      });
+      const hasUserWithName = result.prompt.some(
+        (item) => item.role === 'user' && item.name === 'Test User',
+      );
       expect(hasUserWithName).toBe(true);
     });
 
     it('should calculate tokenCount for each message when contextStrategy is set', async () => {
       client.contextStrategy = 'refine';
-      const result = await client.buildMessages(messages, parentMessageId, { isChatCompletion: true });
-      const hasUserWithTokenCount = result.prompt.some(item => item.role === 'user' && item.tokenCount > 0);
+      const result = await client.buildMessages(messages, parentMessageId, {
+        isChatCompletion: true,
+      });
+      const hasUserWithTokenCount = result.prompt.some(
+        (item) => item.role === 'user' && item.tokenCount > 0,
+      );
       expect(hasUserWithTokenCount).toBe(true);
     });
 
     it('should handle promptPrefix from options when promptPrefix argument is not provided', async () => {
       client.options.promptPrefix = 'Test Prefix from options';
-      const result = await client.buildMessages(messages, parentMessageId, { isChatCompletion: true });
-      const instructions = result.prompt.find(item => item.name === 'instructions');
+      const result = await client.buildMessages(messages, parentMessageId, {
+        isChatCompletion: true,
+      });
+      const instructions = result.prompt.find((item) => item.name === 'instructions');
       expect(instructions.content).toContain('Test Prefix from options');
     });
 
     it('should handle case when neither promptPrefix argument nor options.promptPrefix is set', async () => {
-      const result = await client.buildMessages(messages, parentMessageId, { isChatCompletion: true });
-      const instructions = result.prompt.find(item => item.name === 'instructions');
+      const result = await client.buildMessages(messages, parentMessageId, {
+        isChatCompletion: true,
+      });
+      const instructions = result.prompt.find((item) => item.name === 'instructions');
       expect(instructions).toBeUndefined();
     });
 
     it('should handle case when getMessagesForConversation returns null or an empty array', async () => {
       const messages = [];
-      const result = await client.buildMessages(messages, parentMessageId, { isChatCompletion: true });
+      const result = await client.buildMessages(messages, parentMessageId, {
+        isChatCompletion: true,
+      });
       expect(result.prompt).toEqual([]);
     });
   });

api/app/clients/specs/OpenAIClient.tokens.js

@@ -16,7 +16,7 @@ require('dotenv').config();
 const { OpenAIClient } = require('../');
 
 function timeout(ms) {
-  return new Promise(resolve => setTimeout(resolve, ms));
+  return new Promise((resolve) => setTimeout(resolve, ms));
 }
 
 const run = async () => {
@@ -46,7 +46,7 @@ const run = async () => {
       model,
     },
     proxy: process.env.PROXY || null,
-    debug: true
+    debug: true,
   };
 
   let apiKey = process.env.OPENAI_API_KEY;
@@ -59,7 +59,13 @@ const run = async () => {
   function printProgressBar(percentageUsed) {
     const filledBlocks = Math.round(percentageUsed / 2); // Each block represents 2%
     const emptyBlocks = 50 - filledBlocks; // Total blocks is 50 (each represents 2%), so the rest are empty
-    const progressBar = '[' + '█'.repeat(filledBlocks) + ' '.repeat(emptyBlocks) + '] ' + percentageUsed.toFixed(2) + '%';
+    const progressBar =
+      '[' +
+      '█'.repeat(filledBlocks) +
+      ' '.repeat(emptyBlocks) +
+      '] ' +
+      percentageUsed.toFixed(2) +
+      '%';
     console.log(progressBar);
   }
 
@@ -78,10 +84,10 @@ const run = async () => {
       // encoder.free();
 
       const memoryUsageDuringLoop = process.memoryUsage().heapUsed;
-      const percentageUsed = memoryUsageDuringLoop / maxMemory * 100;
+      const percentageUsed = (memoryUsageDuringLoop / maxMemory) * 100;
       printProgressBar(percentageUsed);
 
-      if (i === (iterations - 1)) {
+      if (i === iterations - 1) {
         console.log(' done');
         // encoder.free();
       }
@@ -100,7 +106,7 @@ const run = async () => {
   await timeout(15000);
   const memoryUsageAfterTimeout = process.memoryUsage().heapUsed;
   console.log(`Post timeout: ${memoryUsageAfterTimeout / 1024 / 1024} megabytes`);
-}
+};
 
 run();
 

api/app/clients/specs/PluginsClient.test.js

@@ -7,7 +7,7 @@ jest.mock('../../../models/Conversation', () => {
   return function () {
     return {
       save: jest.fn(),
-      deleteConvos: jest.fn()
+      deleteConvos: jest.fn(),
     };
   };
 });
@@ -19,11 +19,11 @@ describe('PluginsClient', () => {
     modelOptions: {
       model: 'gpt-3.5-turbo',
       temperature: 0,
-      max_tokens: 2
+      max_tokens: 2,
     },
     agentOptions: {
-      model: 'gpt-3.5-turbo'
-    }
+      model: 'gpt-3.5-turbo',
+    },
   };
   let parentMessageId;
   let conversationId;
@@ -43,13 +43,13 @@ describe('PluginsClient', () => {
 
         const orderedMessages = TestAgent.constructor.getMessagesForConversation(
           fakeMessages,
-          parentMessageId
+          parentMessageId,
         );
 
         const chatMessages = orderedMessages.map((msg) =>
           msg?.isCreatedByUser || msg?.role?.toLowerCase() === 'user'
             ? new HumanChatMessage(msg.text)
-            : new AIChatMessage(msg.text)
+            : new AIChatMessage(msg.text),
         );
 
         TestAgent.currentMessages = orderedMessages;
@@ -64,7 +64,7 @@ describe('PluginsClient', () => {
       const userMessageId = opts.overrideParentMessageId || crypto.randomUUID();
       this.pastMessages = await TestAgent.loadHistory(
         conversationId,
-        TestAgent.options?.parentMessageId
+        TestAgent.options?.parentMessageId,
       );
 
       const userMessage = {
@@ -73,7 +73,7 @@ describe('PluginsClient', () => {
         isCreatedByUser: true,
         messageId: userMessageId,
         parentMessageId,
-        conversationId
+        conversationId,
       };
 
       const response = {
@@ -82,7 +82,7 @@ describe('PluginsClient', () => {
         isCreatedByUser: false,
         messageId: crypto.randomUUID(),
         parentMessageId: userMessage.messageId,
-        conversationId
+        conversationId,
       };
 
       fakeMessages.push(userMessage);
@@ -107,7 +107,7 @@ describe('PluginsClient', () => {
         isCreatedByUser: false,
         messageId: expect.any(String),
         parentMessageId: expect.any(String),
-        conversationId: expect.any(String)
+        conversationId: expect.any(String),
       });
 
       const response = await TestAgent.sendMessage(userMessage);
@@ -121,7 +121,7 @@ describe('PluginsClient', () => {
       const userMessage = 'Second message in the conversation';
       const opts = {
         conversationId,
-        parentMessageId
+        parentMessageId,
       };
 
       const expectedResult = expect.objectContaining({
@@ -130,7 +130,7 @@ describe('PluginsClient', () => {
         isCreatedByUser: false,
         messageId: expect.any(String),
         parentMessageId: expect.any(String),
-        conversationId: opts.conversationId
+        conversationId: opts.conversationId,
       });
 
       const response = await TestAgent.sendMessage(userMessage, opts);

api/app/clients/tools/.well-known/Ai_PDF.json

@@ -0,0 +1,18 @@
+{
+  "schema_version": "v1",
+  "name_for_human": "Ai PDF",
+  "name_for_model": "Ai_PDF",
+  "description_for_human": "Super-fast, interactive chats with PDFs of any size, complete with page references for fact checking.",
+  "description_for_model": "Provide a URL to a PDF and search the document. Break the user question in multiple semantic search queries and calls as needed. Think step by step.",
+  "auth": {
+    "type": "none"
+  },
+  "api": {
+    "type": "openapi",
+    "url": "https://plugin-3c56b9d4c8a6465998395f28b6a445b2-jexkai4vea-uc.a.run.app/openapi.yaml",
+    "is_user_authenticated": false
+  },
+  "logo_url": "https://plugin-3c56b9d4c8a6465998395f28b6a445b2-jexkai4vea-uc.a.run.app/logo.png",
+  "contact_email": "support@promptapps.ai",
+  "legal_info_url": "https://plugin-3c56b9d4c8a6465998395f28b6a445b2-jexkai4vea-uc.a.run.app/legal.html"
+}

api/app/clients/tools/.well-known/VoxScript.json

@@ -0,0 +1,22 @@
+{
+  "schema_version": "v1",
+  "name_for_human": "VoxScript",
+  "name_for_model": "VoxScript",
+  "description_for_human": "Enables searching of YouTube transcripts, financial data sources Google Search results, and more!",
+  "description_for_model": "Plugin for searching through varius data sources.",
+  "auth": {
+    "type": "service_http",
+    "authorization_type": "bearer",
+    "verification_tokens": {
+      "openai": "ffc5226d1af346c08a98dee7deec9f76"
+    }
+  },
+  "api": {
+    "type": "openapi",
+    "url": "https://voxscript.awt.icu/swagger/v1/swagger.yaml",
+    "is_user_authenticated": false
+  },
+  "logo_url": "https://voxscript.awt.icu/images/VoxScript_logo_32x32.png",
+  "contact_email": "voxscript@allwiretech.com",
+  "legal_info_url": "https://voxscript.awt.icu/legal/"
+}

api/app/clients/tools/.well-known/askyourpdf.json

@@ -0,0 +1,18 @@
+{
+  "schema_version": "v1",
+  "name_for_model": "askyourpdf",
+  "name_for_human": "AskYourPDF",
+  "description_for_model": "This plugin is designed to expedite the extraction of information from PDF documents. It works by accepting a URL link to a PDF or a document ID (doc_id) from the user. If a URL is provided, the plugin first validates that it is a correct URL. \\nAfter validating the URL, the plugin proceeds to download the PDF and store its content in a vector database. If the user provides a doc_id, the plugin directly retrieves the document from the database. The plugin then scans through the stored PDFs to find answers to user queries or retrieve specific details.\\n\\nHowever, if an error occurs while querying the API, the user is prompted to download their document first, then manually upload it to [![Upload Document](https://raw.githubusercontent.com/AskYourPdf/ask-plugin/main/upload.png)](https://askyourpdf.com/upload). Once the upload is complete, the user should copy the resulting doc_id and paste it back into the chat for further interaction.\nThe plugin is particularly useful when the user's question pertains to content within a PDF document. When providing answers, the plugin also specifies the page number (highlighted in bold) where the relevant information was found. Remember, the URL must be valid for a successful query. Failure to validate the URL may lead to errors or unsuccessful queries.",
+  "description_for_human": "Unlock the power of your PDFs!, dive into your documents, find answers, and bring information to your fingertips.",
+  "auth": {
+    "type": "none"
+  },
+  "api": {
+    "type": "openapi",
+    "url": "askyourpdf.yaml",
+    "has_user_authentication": false
+  },
+  "logo_url": "https://plugin.askyourpdf.com/.well-known/logo.png",
+  "contact_email": "plugin@askyourpdf.com",
+  "legal_info_url": "https://askyourpdf.com/terms"
+}

api/app/clients/tools/.well-known/has-issues/scholarly_graph_link.json

@@ -0,0 +1,18 @@
+{
+  "schema_version": "v1",
+  "name_for_human": "Scholarly Graph Link",
+  "name_for_model": "scholarly_graph_link",
+  "description_for_human": "You can search papers, authors, datasets and software. It has access to Figshare, Arxiv, and many others.",
+  "description_for_model": "Run GraphQL queries against an API hosted by DataCite API. The API supports most GraphQL query but does not support mutations statements. Use `{ __schema { types { name kind } } }` to get all the types in the GraphQL schema. Use `{ datasets { nodes { id sizes citations { nodes { id titles { title } } } } } }` to get all the citations of all datasets in the API. Use `{ datasets { nodes { id sizes citations { nodes { id titles { title } } } } } }` to get all the citations of all datasets in the API. Use `{person(id:ORCID) {works(first:50) {nodes {id titles(first: 1){title} publicationYear}}}}` to get the first 50 works of a person based on their ORCID. All Ids are urls, e.g., https://orcid.org/0012-0000-1012-1110. Mutations statements are not allowed.",
+  "auth": {
+    "type": "none"
+  },
+  "api": {
+    "type": "openapi",
+    "url": "https://api.datacite.org/graphql-openapi.yaml",
+    "is_user_authenticated": false
+  },
+  "logo_url": "https://raw.githubusercontent.com/kjgarza/scholarly_graph_link/master/logo.png",
+  "contact_email": "kj.garza@gmail.com",
+  "legal_info_url": "https://github.com/kjgarza/scholarly_graph_link/blob/master/LICENSE"
+}

api/app/clients/tools/.well-known/has-issues/web_pilot.json

@@ -0,0 +1,24 @@
+{
+  "schema_version": "v1",
+  "name_for_human": "WebPilot",
+  "name_for_model": "web_pilot",
+  "description_for_human": "Browse & QA Webpage/PDF/Data. Generate articles, from one or more URLs.",
+  "description_for_model": "This tool allows users to provide a URL(or URLs) and optionally requests for interacting with, extracting specific information or how to do with the content from the URL. Requests may include rewrite, translate, and others. If there any requests, when accessing the /api/visit-web endpoint, the parameter 'user_has_request' should be set to 'true. And if there's no any requests, 'user_has_request' should be set to 'false'.",
+  "auth": {
+    "type": "none"
+  },
+  "api": {
+    "type": "openapi",
+    "url": "https://webreader.webpilotai.com/openapi.yaml",
+    "is_user_authenticated": false
+  },
+  "logo_url": "https://webreader.webpilotai.com/logo.png",
+  "contact_email": "dev@webpilot.ai",
+  "legal_info_url": "https://webreader.webpilotai.com/legal_info.html",
+  "headers": {
+    "id": "WebPilot-Friend-UID"
+  },
+  "params": {
+    "user_has_request": true
+  }
+}

api/app/clients/tools/.well-known/openapi/askyourpdf.yaml

@@ -0,0 +1,157 @@
+openapi: 3.0.2
+info:
+  title: FastAPI
+  version: 0.1.0
+servers:
+  - url: https://plugin.askyourpdf.com
+paths:
+  /api/download_pdf:
+    post:
+      summary: Download Pdf
+      description: Download a PDF file from a URL and save it to the vector database.
+      operationId: download_pdf_api_download_pdf_post
+      parameters:
+        - required: true
+          schema:
+            title: Url
+            type: string
+          name: url
+          in: query
+      responses:
+        '200':
+          description: Successful Response
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/FileResponse'
+        '422':
+          description: Validation Error
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/HTTPValidationError'
+  /query:
+    post:
+      summary: Perform Query
+      description: Perform a query on a document.
+      operationId: perform_query_query_post
+      requestBody:
+        content:
+          application/json:
+            schema:
+              $ref: '#/components/schemas/InputData'
+        required: true
+      responses:
+        '200':
+          description: Successful Response
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/ResponseModel'
+        '422':
+          description: Validation Error
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/HTTPValidationError'
+components:
+  schemas:
+    DocumentMetadata:
+      title: DocumentMetadata
+      required:
+        - source
+        - page_number
+        - author
+      type: object
+      properties:
+        source:
+          title: Source
+          type: string
+        page_number:
+          title: Page Number
+          type: integer
+        author:
+          title: Author
+          type: string
+    FileResponse:
+      title: FileResponse
+      required:
+        - docId
+      type: object
+      properties:
+        docId:
+          title: Docid
+          type: string
+        error:
+          title: Error
+          type: string
+    HTTPValidationError:
+      title: HTTPValidationError
+      type: object
+      properties:
+        detail:
+          title: Detail
+          type: array
+          items:
+            $ref: '#/components/schemas/ValidationError'
+    InputData:
+      title: InputData
+      required:
+        - doc_id
+        - query
+      type: object
+      properties:
+        doc_id:
+          title: Doc Id
+          type: string
+        query:
+          title: Query
+          type: string
+    ResponseModel:
+      title: ResponseModel
+      required:
+        - results
+      type: object
+      properties:
+        results:
+          title: Results
+          type: array
+          items:
+            $ref: '#/components/schemas/SearchResult'
+    SearchResult:
+      title: SearchResult
+      required:
+        - doc_id
+        - text
+        - metadata
+      type: object
+      properties:
+        doc_id:
+          title: Doc Id
+          type: string
+        text:
+          title: Text
+          type: string
+        metadata:
+          $ref: '#/components/schemas/DocumentMetadata'
+    ValidationError:
+      title: ValidationError
+      required:
+        - loc
+        - msg
+        - type
+      type: object
+      properties:
+        loc:
+          title: Location
+          type: array
+          items:
+            anyOf:
+              - type: string
+              - type: integer
+        msg:
+          title: Message
+          type: string
+        type:
+          title: Error Type
+          type: string

api/app/clients/tools/.well-known/openapi/scholarai.yaml

@@ -0,0 +1,185 @@
+openapi: 3.0.1
+info:
+  title: ScholarAI
+  description: Allows the user to search facts and findings from scientific articles
+  version: 'v1'
+servers:
+  - url: https://scholar-ai.net
+paths:
+  /api/abstracts:
+    get:
+      operationId: searchAbstracts
+      summary: Get relevant paper abstracts by keywords search
+      parameters:
+        - name: keywords
+          in: query
+          description: Keywords of inquiry which should appear in article. Must be in English.
+          required: true
+          schema:
+            type: string
+        - name: sort
+          in: query
+          description: The sort order for results. Valid values are cited_by_count or publication_date. Excluding this value does a relevance based search.
+          required: false
+          schema:
+            type: string
+            enum:
+              - cited_by_count
+              - publication_date
+        - name: query
+          in: query
+          description: The user query
+          required: true
+          schema:
+            type: string
+        - name: peer_reviewed_only
+          in: query
+          description: Whether to only return peer reviewed articles. Defaults to true, ChatGPT should cautiously suggest this value can be set to false
+          required: false
+          schema:
+            type: string
+        - name: start_year
+          in: query
+          description: The first year, inclusive, to include in the search range. Excluding this value will include all years.
+          required: false
+          schema:
+            type: string
+        - name: end_year
+          in: query
+          description: The last year, inclusive, to include in the search range. Excluding this value will include all years.
+          required: false
+          schema:
+            type: string
+        - name: offset
+          in: query
+          description: The offset of the first result to return. Defaults to 0.
+          required: false
+          schema:
+            type: string
+      responses:
+        "200":
+          description: OK
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/searchAbstractsResponse'
+  /api/fulltext:
+    get:
+      operationId: getFullText
+      summary: Get full text of a paper by URL for PDF
+      parameters:
+        - name: pdf_url
+          in: query
+          description: URL for PDF
+          required: true
+          schema:
+            type: string
+        - name: chunk
+          in: query
+          description: chunk number to retrieve, defaults to 1
+          required: false
+          schema:
+            type: number
+      responses:
+        "200":
+          description: OK
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/getFullTextResponse'
+  /api/save-citation:
+    get:
+      operationId: saveCitation
+      summary: Save citation to reference manager
+      parameters:
+        - name: doi
+          in: query
+          description: Digital Object Identifier (DOI) of article
+          required: true
+          schema:
+            type: string
+        - name: zotero_user_id
+          in: query
+          description: Zotero User ID
+          required: true
+          schema:
+            type: string
+        - name: zotero_api_key
+          in: query
+          description: Zotero API Key
+          required: true
+          schema:
+            type: string
+      responses:
+        "200":
+          description: OK
+          content:
+            application/json:
+              schema:
+                $ref: '#/components/schemas/saveCitationResponse'
+components:
+  schemas:
+    searchAbstractsResponse:
+      type: object
+      properties:
+        next_offset:
+          type: number
+          description: The offset of the next page of results.
+        total_num_results:
+          type: number
+          description: The total number of results. 
+        abstracts:
+          type: array
+          items:
+            type: object
+            properties:
+              title:
+                type: string
+              abstract:
+                type: string
+                description: Summary of the context, methods, results, and conclusions of the paper.
+              doi:
+                type: string
+                description: The DOI of the paper.
+              landing_page_url:
+                type: string
+                description: Link to the paper on its open-access host.
+              pdf_url:
+                type: string
+                description: Link to the paper PDF.
+              publicationDate:
+                type: string
+                description: The date the paper was published in YYYY-MM-DD format.
+              relevance:
+                type: number
+                description: The relevance of the paper to the search query. 1 is the most relevant.
+              creators:
+                type: array
+                items:
+                  type: string
+                  description: The name of the creator.
+              cited_by_count:
+                type: number
+                description: The number of citations of the article.
+          description: The list of relevant abstracts.
+    getFullTextResponse:
+      type: object
+      properties:
+        full_text:
+          type: string
+          description: The full text of the paper.
+        pdf_url:
+          type: string
+          description: The PDF URL of the paper.
+        chunk:
+          type: number
+          description: The chunk of the paper.
+        total_chunk_num:
+          type: number
+          description: The total chunks of the paper.
+    saveCitationResponse:
+      type: object
+      properties:
+        message:
+          type: string
+          description: Confirmation of successful save or error message.

api/app/clients/tools/.well-known/rephrase.json

@@ -0,0 +1,18 @@
+{
+  "schema_version": "v1",
+  "name_for_human": "Prompt Perfect",
+  "name_for_model": "rephrase",
+  "description_for_human": "Type 'perfect' to craft the perfect prompt, every time.",
+  "description_for_model": "Plugin that can rephrase user inputs to improve the quality of ChatGPT's responses. The plugin evaluates user inputs and, if necessary, transforms them into clearer, more specific, and contextual prompts. It processes a JSON object containing the user input to be rephrased and uses the GPT-3.5-turbo model for the rephrasing process. The rephrased input is then returned as raw data to be incorporated into ChatGPT's response. The user can initiate the plugin by typing 'perfect'.",
+  "auth": {
+    "type": "none"
+  },
+  "api": {
+    "type": "openapi",
+    "url": "https://promptperfect.xyz/openapi.yaml",
+    "is_user_authenticated": false
+  },
+  "logo_url": "https://promptperfect.xyz/static/prompt_perfect_logo.png",
+  "contact_email": "heyo@promptperfect.xyz",
+  "legal_info_url": "https://promptperfect.xyz/static/terms.html"
+}

api/app/clients/tools/.well-known/scholarai.json

@@ -0,0 +1,22 @@
+{
+  "schema_version": "v1",
+  "name_for_human": "ScholarAI",
+  "name_for_model": "scholarai",
+  "description_for_human": "Unleash scientific research: search 40M+ peer-reviewed papers, explore scientific PDFs, and save to reference managers.",
+  "description_for_model": "Access open access scientific literature from peer-reviewed journals. The abstract endpoint finds relevant papers based on 2 to 6 keywords. After getting abstracts, ALWAYS prompt the user offering to go into more detail. Use the fulltext endpoint to retrieve the entire paper's text and access specific details using the provided pdf_url, if available. ALWAYS hyperlink the pdf_url from the responses if available. Offer to dive into the fulltext or search for additional papers. Always ask if the user wants save any paper to the user’s Zotero reference manager by using the save-citation endpoint and providing the doi and requesting the user’s zotero_user_id and zotero_api_key.",
+  "auth": {
+    "type": "none"
+  },
+  "api": {
+    "type": "openapi",
+    "url": "scholarai.yaml",
+    "is_user_authenticated": false
+  },
+  "params": {
+    "sort": "cited_by_count"
+  },
+  "logo_url": "https://scholar-ai.net/logo.png",
+  "contact_email": "lakshb429@gmail.com",
+  "legal_info_url": "https://scholar-ai.net/legal.txt",
+  "HttpAuthorizationType": "basic"
+}

api/app/clients/tools/AIPluginTool.js

@@ -57,7 +57,7 @@ function extractShortVersion(openapiSpec) {
     const shortApiSpec = {
       openapi: fullApiSpec.openapi,
       info: fullApiSpec.info,
-      paths: {}
+      paths: {},
     };
 
     for (let path in fullApiSpec.paths) {
@@ -68,8 +68,8 @@ function extractShortVersion(openapiSpec) {
           operationId: fullApiSpec.paths[path][method].operationId,
           parameters: fullApiSpec.paths[path][method].parameters?.map((parameter) => ({
             name: parameter.name,
-            description: parameter.description
-          }))
+            description: parameter.description,
+          })),
         };
       }
     }
@@ -199,14 +199,16 @@ class AIPluginTool extends Tool {
     const apiUrlRes = await fetch(aiPluginJson.api.url, {});
     if (!apiUrlRes.ok) {
       throw new Error(
-        `Failed to fetch API spec from ${aiPluginJson.api.url} with status ${apiUrlRes.status}`
+        `Failed to fetch API spec from ${aiPluginJson.api.url} with status ${apiUrlRes.status}`,
       );
     }
     const apiUrlJson = await apiUrlRes.text();
     const shortApiSpec = extractShortVersion(apiUrlJson);
     return new AIPluginTool({
       name: aiPluginJson.name_for_model.toLowerCase(),
-      description: `A \`tool\` to learn the API documentation for ${aiPluginJson.name_for_model.toLowerCase()}, after which you can use 'http_request' to make the actual API call. Short description of how to use the API's results: ${aiPluginJson.description_for_model})`,
+      description: `A \`tool\` to learn the API documentation for ${aiPluginJson.name_for_model.toLowerCase()}, after which you can use 'http_request' to make the actual API call. Short description of how to use the API's results: ${
+        aiPluginJson.description_for_model
+      })`,
       apiSpec: `
 As an AI, your task is to identify the operationId of the relevant API path based on the condensed OpenAPI specifications provided.
 
@@ -228,7 +230,7 @@ ${shortApiSpec}
 \`\`\`
 `,
       openaiSpec: apiUrlJson,
-      model: model
+      model: model,
     });
   }
 }

api/app/clients/tools/DALL-E.js

@@ -56,11 +56,17 @@ Guidelines:
   }
 
   replaceUnwantedChars(inputString) {
-    return inputString.replace(/\r\n|\r|\n/g, ' ').replace('"', '').trim();
+    return inputString
+      .replace(/\r\n|\r|\n/g, ' ')
+      .replace('"', '')
+      .trim();
   }
 
   getMarkdownImageUrl(imageName) {
-    const imageUrl = path.join(this.relativeImageUrl, imageName).replace(/\\/g, '/').replace('public/', '');
+    const imageUrl = path
+      .join(this.relativeImageUrl, imageName)
+      .replace(/\\/g, '/')
+      .replace('public/', '');
     return `![generated image](/${imageUrl})`;
   }
 
@@ -70,13 +76,13 @@ Guidelines:
       // TODO: Future idea -- could we ask an LLM to extract these arguments from an input that might contain them?
       n: 1,
       // size: '1024x1024'
-      size: '512x512'
+      size: '512x512',
     });
 
     const theImageUrl = resp.data.data[0].url;
 
     if (!theImageUrl) {
-      throw new Error(`No image URL returned from OpenAI API.`);
+      throw new Error('No image URL returned from OpenAI API.');
     }
 
     const regex = /img-[\w\d]+.png/;

api/app/clients/tools/GoogleSearch.js

@@ -23,7 +23,8 @@ class GoogleSearchAPI extends Tool {
    * A description for the agent to use
    * @type {string}
    */
-  description = `Use the 'google' tool to retrieve internet search results relevant to your input. The results will return links and snippets of text from the webpages`;
+  description =
+    'Use the \'google\' tool to retrieve internet search results relevant to your input. The results will return links and snippets of text from the webpages';
 
   getCx() {
     const cx = process.env.GOOGLE_CSE_ID || '';
@@ -79,7 +80,7 @@ class GoogleSearchAPI extends Tool {
         q: input,
         cx: this.cx,
         auth: this.apiKey,
-        num: 5 // Limit the number of results to 5
+        num: 5, // Limit the number of results to 5
       });
 
       // return response.data;
@@ -87,7 +88,7 @@ class GoogleSearchAPI extends Tool {
 
       if (!response.data.items || response.data.items.length === 0) {
         return this.resultsToReadableFormat([
-          { title: 'No good Google Search Result was found', link: '' }
+          { title: 'No good Google Search Result was found', link: '' },
         ]);
       }
 
@@ -97,7 +98,7 @@ class GoogleSearchAPI extends Tool {
       for (const result of results) {
         const metadataResult = {
           title: result.title || '',
-          link: result.link || ''
+          link: result.link || '',
         };
         if (result.snippet) {
           metadataResult.snippet = result.snippet;

api/app/clients/tools/HttpRequestTool.js

@@ -55,7 +55,8 @@ class HttpRequestTool extends Tool {
     this.headers = headers;
     this.name = 'http_request';
     this.maxOutputLength = maxOutputLength;
-    this.description = `Executes HTTP methods (GET, POST, PUT, DELETE, etc.). The input is an object with three keys: "url", "method", and "data". Even for GET or DELETE, include "data" key as an empty string. "method" is the HTTP method, and "url" is the desired endpoint. If POST or PUT, "data" should contain a stringified JSON representing the body to send. Only one url per use.`;
+    this.description =
+      'Executes HTTP methods (GET, POST, PUT, DELETE, etc.). The input is an object with three keys: "url", "method", and "data". Even for GET or DELETE, include "data" key as an empty string. "method" is the HTTP method, and "url" is the desired endpoint. If POST or PUT, "data" should contain a stringified JSON representing the body to send. Only one url per use.';
   }
 
   async _call(input) {
@@ -77,7 +78,7 @@ class HttpRequestTool extends Tool {
 
       let options = {
         method: method,
-        headers: this.headers
+        headers: this.headers,
       };
 
       if (['POST', 'PUT', 'PATCH'].includes(method.toUpperCase()) && data) {

api/app/clients/tools/SelfReflection.js

@@ -5,7 +5,8 @@ class SelfReflectionTool extends Tool {
     super();
     this.reminders = 0;
     this.name = 'self-reflection';
-    this.description = `Take this action to reflect on your thoughts & actions. For your input, provide answers for self-evaluation as part of one input, using this space as a canvas to explore and organize your ideas in response to the user's message. You can use multiple lines for your input. Perform this action sparingly and only when you are stuck.`;
+    this.description =
+      'Take this action to reflect on your thoughts & actions. For your input, provide answers for self-evaluation as part of one input, using this space as a canvas to explore and organize your ideas in response to the user\'s message. You can use multiple lines for your input. Perform this action sparingly and only when you are stuck.';
     this.message = message;
     this.isGpt3 = isGpt3;
     // this.returnDirect = true;
@@ -17,9 +18,9 @@ class SelfReflectionTool extends Tool {
 
   async selfReflect() {
     if (this.isGpt3) {
-      return `I should finalize my reply as soon as I have satisfied the user's query.`;
+      return 'I should finalize my reply as soon as I have satisfied the user\'s query.';
     } else {
-      return ``;
+      return '';
     }
   }
 }

api/app/clients/tools/StableDiffusion.js

@@ -26,7 +26,10 @@ Guidelines:
   }
 
   getMarkdownImageUrl(imageName) {
-    const imageUrl = path.join(this.relativeImageUrl, imageName).replace(/\\/g, '/').replace('public/', '');
+    const imageUrl = path
+      .join(this.relativeImageUrl, imageName)
+      .replace(/\\/g, '/')
+      .replace('public/', '');
     return `![generated image](/${imageUrl})`;
   }
 
@@ -43,7 +46,7 @@ Guidelines:
     const payload = {
       prompt: input.split('|')[0],
       negative_prompt: input.split('|')[1],
-      steps: 20
+      steps: 20,
     };
     const response = await axios.post(`${url}/sdapi/v1/txt2img`, payload);
     const image = response.data.images[0];
@@ -68,8 +71,8 @@ Guidelines:
       await sharp(buffer)
         .withMetadata({
           iptcpng: {
-            parameters: info
-          }
+            parameters: info,
+          },
         })
         .toFile(this.outputPath + '/' + imageName);
       this.result = this.getMarkdownImageUrl(imageName);

api/app/clients/tools/Wolfram.js

@@ -71,7 +71,7 @@ General guidelines:
         console.log('Error data:', error.response.data);
         return error.response.data;
       } else {
-        console.log(`Error querying Wolfram Alpha`, error.message);
+        console.log('Error querying Wolfram Alpha', error.message);
         // throw error;
         return 'There was an error querying Wolfram Alpha.';
       }

api/app/clients/tools/dynamic/OpenAPIPlugin.js

@@ -0,0 +1,139 @@
+require('dotenv').config();
+const { z } = require('zod');
+const fs = require('fs');
+const yaml = require('js-yaml');
+const path = require('path');
+const { DynamicStructuredTool } = require('langchain/tools');
+const { createOpenAPIChain } = require('langchain/chains');
+const SUFFIX = 'Prioritize using responses for subsequent requests to better fulfill the query.';
+
+const AuthBearer = z
+  .object({
+    type: z.string().includes('service_http'),
+    authorization_type: z.string().includes('bearer'),
+    verification_tokens: z.object({
+      openai: z.string(),
+    }),
+  })
+  .catch(() => false);
+
+const AuthDefinition = z
+  .object({
+    type: z.string(),
+    authorization_type: z.string(),
+    verification_tokens: z.object({
+      openai: z.string(),
+    }),
+  })
+  .catch(() => false);
+
+async function readSpecFile(filePath) {
+  try {
+    const fileContents = await fs.promises.readFile(filePath, 'utf8');
+    if (path.extname(filePath) === '.json') {
+      return JSON.parse(fileContents);
+    }
+    return yaml.load(fileContents);
+  } catch (e) {
+    console.error(e);
+    return false;
+  }
+}
+
+async function getSpec(url) {
+  const RegularUrl = z
+    .string()
+    .url()
+    .catch(() => false);
+
+  if (RegularUrl.parse(url) && path.extname(url) === '.json') {
+    const response = await fetch(url);
+    return await response.json();
+  }
+
+  const ValidSpecPath = z
+    .string()
+    .url()
+    .catch(async () => {
+      const spec = path.join(__dirname, '..', '.well-known', 'openapi', url);
+      if (!fs.existsSync(spec)) {
+        return false;
+      }
+
+      return await readSpecFile(spec);
+    });
+
+  return ValidSpecPath.parse(url);
+}
+
+async function createOpenAPIPlugin({ data, llm, user, message, verbose = false }) {
+  let spec;
+  try {
+    spec = await getSpec(data.api.url, verbose);
+  } catch (error) {
+    verbose && console.debug('getSpec error', error);
+    return null;
+  }
+
+  if (!spec) {
+    verbose && console.debug('No spec found');
+    return null;
+  }
+
+  const headers = {};
+  const { auth, description_for_model } = data;
+  if (auth && AuthDefinition.parse(auth)) {
+    verbose && console.debug('auth detected', auth);
+    const { openai } = auth.verification_tokens;
+    if (AuthBearer.parse(auth)) {
+      headers.authorization = `Bearer ${openai}`;
+      verbose && console.debug('added auth bearer', headers);
+    }
+  }
+
+  return new DynamicStructuredTool({
+    name: data.name_for_model,
+    description: `${data.description_for_human} ${SUFFIX}`,
+    schema: z.object({
+      query: z
+        .string()
+        .describe(
+          'For the query, be specific in a conversational manner. It will be interpreted by a human.',
+        ),
+    }),
+    func: async () => {
+      const chainOptions = {
+        llm,
+        verbose,
+      };
+
+      if (data.headers && data.headers['librechat_user_id']) {
+        verbose && console.debug('id detected', headers);
+        headers[data.headers['librechat_user_id']] = user;
+      }
+
+      if (Object.keys(headers).length > 0) {
+        verbose && console.debug('headers detected', headers);
+        chainOptions.headers = headers;
+      }
+
+      if (data.params) {
+        verbose && console.debug('params detected', data.params);
+        chainOptions.params = data.params;
+      }
+
+      const chain = await createOpenAPIChain(spec, chainOptions);
+      const result = await chain.run(
+        `${message}\n\n||>Instructions: ${description_for_model}\n${SUFFIX}`,
+      );
+      console.log('api chain run result', result);
+      return result;
+    },
+  });
+}
+
+module.exports = {
+  getSpec,
+  readSpecFile,
+  createOpenAPIPlugin,
+};

api/app/clients/tools/dynamic/OpenAPIPlugin.spec.js

@@ -0,0 +1,65 @@
+const fs = require('fs');
+const { createOpenAPIPlugin, getSpec, readSpecFile } = require('./OpenAPIPlugin');
+
+jest.mock('node-fetch');
+jest.mock('fs', () => ({
+  promises: {
+    readFile: jest.fn(),
+  },
+  existsSync: jest.fn(),
+}));
+
+describe('readSpecFile', () => {
+  it('reads JSON file correctly', async () => {
+    fs.promises.readFile.mockResolvedValue(JSON.stringify({ test: 'value' }));
+    const result = await readSpecFile('test.json');
+    expect(result).toEqual({ test: 'value' });
+  });
+
+  it('reads YAML file correctly', async () => {
+    fs.promises.readFile.mockResolvedValue('test: value');
+    const result = await readSpecFile('test.yaml');
+    expect(result).toEqual({ test: 'value' });
+  });
+
+  it('handles error correctly', async () => {
+    fs.promises.readFile.mockRejectedValue(new Error('test error'));
+    const result = await readSpecFile('test.json');
+    expect(result).toBe(false);
+  });
+});
+
+describe('getSpec', () => {
+  it('fetches spec from url correctly', async () => {
+    const parsedJson = await getSpec('https://www.instacart.com/.well-known/ai-plugin.json');
+    const isObject = typeof parsedJson === 'object';
+    expect(isObject).toEqual(true);
+  });
+
+  it('reads spec from file correctly', async () => {
+    fs.existsSync.mockReturnValue(true);
+    fs.promises.readFile.mockResolvedValue(JSON.stringify({ test: 'value' }));
+    const result = await getSpec('test.json');
+    expect(result).toEqual({ test: 'value' });
+  });
+
+  it('returns false when file does not exist', async () => {
+    fs.existsSync.mockReturnValue(false);
+    const result = await getSpec('test.json');
+    expect(result).toBe(false);
+  });
+});
+
+describe('createOpenAPIPlugin', () => {
+  it('returns null when getSpec throws an error', async () => {
+    const result = await createOpenAPIPlugin({ data: { api: { url: 'invalid' } } });
+    expect(result).toBe(null);
+  });
+
+  it('returns null when no spec is found', async () => {
+    const result = await createOpenAPIPlugin({});
+    expect(result).toBe(null);
+  });
+
+  // Add more tests here for different scenarios
+});

api/app/clients/tools/index.js

@@ -19,5 +19,5 @@ module.exports = {
   StructuredSD,
   WolframAlphaAPI,
   StructuredWolfram,
-  SelfReflectionTool
-}
+  SelfReflectionTool,
+};

api/app/clients/tools/manifest.json

@@ -32,9 +32,9 @@
   },
   {
     "name": "Browser",
-    "pluginKey": "browser",
+    "pluginKey": "web-browser",
     "description": "Scrape and summarize webpage data",
-    "icon": "/assets/web-browser.png",
+    "icon": "/assets/web-browser.svg",
     "authConfig": [
       {
         "authField": "OPENAI_API_KEY",

api/app/clients/tools/saveImageFromUrl.js

@@ -7,7 +7,7 @@ async function saveImageFromUrl(url, outputPath, outputFilename) {
     // Fetch the image from the URL
     const response = await axios({
       url,
-      responseType: 'stream'
+      responseType: 'stream',
     });
 
     // Check if the output directory exists, if not, create it

api/app/clients/tools/structured/StableDiffusion.js

@@ -20,8 +20,16 @@ Guidelines:
 "negative_prompt":"semi-realistic, cgi, 3d, render, sketch, cartoon, drawing, anime, out of frame, low quality, ugly, mutation, deformed"
 - Generate images only once per human query unless explicitly requested by the user`;
     this.schema = z.object({
-      prompt: z.string().describe("Detailed keywords to describe the subject, using at least 7 keywords to accurately describe the image, separated by comma"),
-      negative_prompt: z.string().describe("Keywords we want to exclude from the final image, using at least 7 keywords to accurately describe the image, separated by comma")
+      prompt: z
+        .string()
+        .describe(
+          'Detailed keywords to describe the subject, using at least 7 keywords to accurately describe the image, separated by comma',
+        ),
+      negative_prompt: z
+        .string()
+        .describe(
+          'Keywords we want to exclude from the final image, using at least 7 keywords to accurately describe the image, separated by comma',
+        ),
     });
   }
 
@@ -30,7 +38,10 @@ Guidelines:
   }
 
   getMarkdownImageUrl(imageName) {
-    const imageUrl = path.join(this.relativeImageUrl, imageName).replace(/\\/g, '/').replace('public/', '');
+    const imageUrl = path
+      .join(this.relativeImageUrl, imageName)
+      .replace(/\\/g, '/')
+      .replace('public/', '');
     return `![generated image](/${imageUrl})`;
   }
 
@@ -48,7 +59,7 @@ Guidelines:
     const payload = {
       prompt,
       negative_prompt,
-      steps: 20
+      steps: 20,
     };
     const response = await axios.post(`${url}/sdapi/v1/txt2img`, payload);
     const image = response.data.images[0];
@@ -58,7 +69,17 @@ Guidelines:
 
     // Generate unique name
     const imageName = `${Date.now()}.png`;
-    this.outputPath = path.resolve(__dirname, '..', '..', '..', '..', '..', 'client', 'public', 'images');
+    this.outputPath = path.resolve(
+      __dirname,
+      '..',
+      '..',
+      '..',
+      '..',
+      '..',
+      'client',
+      'public',
+      'images',
+    );
     const appRoot = path.resolve(__dirname, '..', '..', '..', '..', '..', 'client');
     this.relativeImageUrl = path.relative(appRoot, this.outputPath);
 
@@ -72,8 +93,8 @@ Guidelines:
       await sharp(buffer)
         .withMetadata({
           iptcpng: {
-            parameters: info
-          }
+            parameters: info,
+          },
         })
         .toFile(this.outputPath + '/' + imageName);
       this.result = this.getMarkdownImageUrl(imageName);

api/app/clients/tools/structured/Wolfram.js

@@ -18,7 +18,9 @@ Guidelines include:
 - Make separate calls for each property and choose relevant 'Assumptions' if results aren't relevant.
 - The tool also performs data analysis, plotting, and information retrieval.`;
     this.schema = z.object({
-      nl_query: z.string().describe("Natural language query to WolframAlpha following the guidelines"),
+      nl_query: z
+        .string()
+        .describe('Natural language query to WolframAlpha following the guidelines'),
     });
   }
 
@@ -61,7 +63,7 @@ Guidelines include:
         console.log('Error data:', error.response.data);
         return error.response.data;
       } else {
-        console.log(`Error querying Wolfram Alpha`, error.message);
+        console.log('Error querying Wolfram Alpha', error.message);
         // throw error;
         return 'There was an error querying Wolfram Alpha.';
       }

api/app/clients/tools/util/addOpenAPISpecs.js

@@ -0,0 +1,31 @@
+const { loadSpecs } = require('./loadSpecs');
+
+function transformSpec(input) {
+  return {
+    name: input.name_for_human,
+    pluginKey: input.name_for_model,
+    description: input.description_for_human,
+    icon: input?.logo_url ?? 'https://placehold.co/70x70.png',
+    // TODO: add support for authentication
+    isAuthRequired: 'false',
+    authConfig: [],
+  };
+}
+
+async function addOpenAPISpecs(availableTools) {
+  try {
+    const specs = (await loadSpecs({})).map(transformSpec);
+    if (specs.length > 0) {
+      return [...specs, ...availableTools];
+    }
+    return availableTools;
+  } catch (error) {
+    console.log('addOpenAPISpecs error', error);
+    return availableTools;
+  }
+}
+
+module.exports = {
+  transformSpec,
+  addOpenAPISpecs,
+};

api/app/clients/tools/util/addOpenAPISpecs.spec.js

@@ -0,0 +1,76 @@
+const { addOpenAPISpecs, transformSpec } = require('./addOpenAPISpecs');
+const { loadSpecs } = require('./loadSpecs');
+const { createOpenAPIPlugin } = require('../dynamic/OpenAPIPlugin');
+
+jest.mock('./loadSpecs');
+jest.mock('../dynamic/OpenAPIPlugin');
+
+describe('transformSpec', () => {
+  it('should transform input spec to a desired format', () => {
+    const input = {
+      name_for_human: 'Human Name',
+      name_for_model: 'Model Name',
+      description_for_human: 'Human Description',
+      logo_url: 'https://example.com/logo.png',
+    };
+
+    const expectedOutput = {
+      name: 'Human Name',
+      pluginKey: 'Model Name',
+      description: 'Human Description',
+      icon: 'https://example.com/logo.png',
+      isAuthRequired: 'false',
+      authConfig: [],
+    };
+
+    expect(transformSpec(input)).toEqual(expectedOutput);
+  });
+
+  it('should use default icon if logo_url is not provided', () => {
+    const input = {
+      name_for_human: 'Human Name',
+      name_for_model: 'Model Name',
+      description_for_human: 'Human Description',
+    };
+
+    const expectedOutput = {
+      name: 'Human Name',
+      pluginKey: 'Model Name',
+      description: 'Human Description',
+      icon: 'https://placehold.co/70x70.png',
+      isAuthRequired: 'false',
+      authConfig: [],
+    };
+
+    expect(transformSpec(input)).toEqual(expectedOutput);
+  });
+});
+
+describe('addOpenAPISpecs', () => {
+  it('should add specs to available tools', async () => {
+    const availableTools = ['Tool1', 'Tool2'];
+    const specs = [
+      {
+        name_for_human: 'Human Name',
+        name_for_model: 'Model Name',
+        description_for_human: 'Human Description',
+        logo_url: 'https://example.com/logo.png',
+      },
+    ];
+
+    loadSpecs.mockResolvedValue(specs);
+    createOpenAPIPlugin.mockReturnValue('Plugin');
+
+    const result = await addOpenAPISpecs(availableTools);
+    expect(result).toEqual([...specs.map(transformSpec), ...availableTools]);
+  });
+
+  it('should return available tools if specs loading fails', async () => {
+    const availableTools = ['Tool1', 'Tool2'];
+
+    loadSpecs.mockRejectedValue(new Error('Failed to load specs'));
+
+    const result = await addOpenAPISpecs(availableTools);
+    expect(result).toEqual(availableTools);
+  });
+});

api/app/clients/tools/util/handleTools.js

@@ -1,10 +1,7 @@
 const { getUserPluginAuthValue } = require('../../../../server/services/PluginService');
 const { OpenAIEmbeddings } = require('langchain/embeddings/openai');
 const { ZapierToolKit } = require('langchain/agents');
-const {
-  SerpAPI,
-  ZapierNLAWrapper
-} = require('langchain/tools');
+const { SerpAPI, ZapierNLAWrapper } = require('langchain/tools');
 const { ChatOpenAI } = require('langchain/chat_models/openai');
 const { Calculator } = require('langchain/tools/calculator');
 const { WebBrowser } = require('langchain/tools/webbrowser');
@@ -19,12 +16,13 @@ const {
   StableDiffusionAPI,
   StructuredSD,
 } = require('../');
+const { loadSpecs } = require('./loadSpecs');
 
 const validateTools = async (user, tools = []) => {
   try {
     const validToolsSet = new Set(tools);
     const availableToolsToValidate = availableTools.filter((tool) =>
-      validToolsSet.has(tool.pluginKey)
+      validToolsSet.has(tool.pluginKey),
     );
 
     const validateCredentials = async (authField, toolName) => {
@@ -79,15 +77,14 @@ const loadTools = async ({ user, model, functions = null, tools = [], options =
     google: GoogleSearchAPI,
     wolfram: functions ? StructuredWolfram : WolframAlphaAPI,
     'dall-e': OpenAICreateImage,
-    'stable-diffusion': functions ? StructuredSD : StableDiffusionAPI
+    'stable-diffusion': functions ? StructuredSD : StableDiffusionAPI,
   };
 
   const customConstructors = {
-    browser: async () => {
-      let openAIApiKey = process.env.OPENAI_API_KEY;
-      if (!openAIApiKey) {
-        openAIApiKey = await getUserPluginAuthValue(user, 'OPENAI_API_KEY');
-      }
+    'web-browser': async () => {
+      let openAIApiKey = options.openAIApiKey ?? process.env.OPENAI_API_KEY;
+      openAIApiKey = openAIApiKey === 'user_provided' ? null : openAIApiKey;
+      openAIApiKey = openAIApiKey || (await getUserPluginAuthValue(user, 'OPENAI_API_KEY'));
       return new WebBrowser({ model, embeddings: new OpenAIEmbeddings({ openAIApiKey }) });
     },
     serpapi: async () => {
@@ -98,7 +95,7 @@ const loadTools = async ({ user, model, functions = null, tools = [], options =
       return new SerpAPI(apiKey, {
         location: 'Austin,Texas,United States',
         hl: 'en',
-        gl: 'us'
+        gl: 'us',
       });
     },
     zapier: async () => {
@@ -114,16 +111,27 @@ const loadTools = async ({ user, model, functions = null, tools = [], options =
         new HttpRequestTool(),
         await AIPluginTool.fromPluginUrl(
           'https://www.klarna.com/.well-known/ai-plugin.json',
-          new ChatOpenAI({ openAIApiKey: options.openAIApiKey, temperature: 0 })
-        )
+          new ChatOpenAI({ openAIApiKey: options.openAIApiKey, temperature: 0 }),
+        ),
       ];
-    }
+    },
   };
 
   const requestedTools = {};
+  let specs = null;
+  if (functions) {
+    specs = await loadSpecs({
+      llm: model,
+      user,
+      message: options.message,
+      map: true,
+      verbose: options?.debug,
+    });
+    console.dir(specs, { depth: null });
+  }
 
   const toolOptions = {
-    serpapi: { location: 'Austin,Texas,United States', hl: 'en', gl: 'us' }
+    serpapi: { location: 'Austin,Texas,United States', hl: 'en', gl: 'us' },
   };
 
   const toolAuthFields = {};
@@ -142,13 +150,18 @@ const loadTools = async ({ user, model, functions = null, tools = [], options =
       continue;
     }
 
+    if (specs && specs[tool]) {
+      requestedTools[tool] = specs[tool];
+      continue;
+    }
+
     if (toolConstructors[tool]) {
       const options = toolOptions[tool] || {};
       const toolInstance = await loadToolWithAuth(
         user,
         toolAuthFields[tool],
         toolConstructors[tool],
-        options
+        options,
       );
       requestedTools[tool] = toolInstance;
     }
@@ -159,5 +172,5 @@ const loadTools = async ({ user, model, functions = null, tools = [], options =
 
 module.exports = {
   validateTools,
-  loadTools
+  loadTools,
 };

api/app/clients/tools/util/handleTools.test.js

@@ -7,11 +7,11 @@ const mockUser = {
 var mockPluginService = {
   updateUserPluginAuth: jest.fn(),
   deleteUserPluginAuth: jest.fn(),
-  getUserPluginAuthValue: jest.fn()
+  getUserPluginAuthValue: jest.fn(),
 };
 
 jest.mock('../../../../models/User', () => {
-  return function() {
+  return function () {
     return mockUser;
   };
 });
@@ -42,9 +42,11 @@ describe('Tool Handlers', () => {
     mockPluginService.getUserPluginAuthValue.mockImplementation((userId, authField) => {
       return userAuthValues[`${userId}-${authField}`];
     });
-    mockPluginService.updateUserPluginAuth.mockImplementation((userId, authField, _pluginKey, credential) => {
-      userAuthValues[`${userId}-${authField}`] = credential;
-    });
+    mockPluginService.updateUserPluginAuth.mockImplementation(
+      (userId, authField, _pluginKey, credential) => {
+        userAuthValues[`${userId}-${authField}`] = credential;
+      },
+    );
 
     fakeUser = new User({
       name: 'Fake User',
@@ -57,11 +59,16 @@ describe('Tool Handlers', () => {
       role: 'USER',
       googleId: null,
       plugins: [],
-      refreshToken: []
+      refreshToken: [],
     });
     await fakeUser.save();
     for (const authConfig of authConfigs) {
-      await PluginService.updateUserPluginAuth(fakeUser._id, authConfig.authField, pluginKey, mockCredential);
+      await PluginService.updateUserPluginAuth(
+        fakeUser._id,
+        authConfig.authField,
+        pluginKey,
+        mockCredential,
+      );
     }
   });
 
@@ -113,14 +120,14 @@ describe('Tool Handlers', () => {
     const sampleTools = [...initialTools, 'calculator'];
     let ToolClass2 = Calculator;
     let remainingTools = availableTools.filter(
-      (tool) => sampleTools.indexOf(tool.pluginKey) === -1
+      (tool) => sampleTools.indexOf(tool.pluginKey) === -1,
     );
 
     beforeAll(async () => {
       toolFunctions = await loadTools({
         user: fakeUser._id,
         model: BaseChatModel,
-        tools: sampleTools
+        tools: sampleTools,
       });
       loadTool1 = toolFunctions[sampleTools[0]];
       loadTool2 = toolFunctions[sampleTools[1]];
@@ -161,7 +168,7 @@ describe('Tool Handlers', () => {
       toolFunctions = await loadTools({
         user: fakeUser._id,
         model: BaseChatModel,
-        tools: [testPluginKey]
+        tools: [testPluginKey],
       });
       const Tool = await toolFunctions[testPluginKey]();
       expect(Tool).toBeInstanceOf(TestClass);
@@ -169,7 +176,7 @@ describe('Tool Handlers', () => {
     it('returns an empty object when no tools are requested', async () => {
       toolFunctions = await loadTools({
         user: fakeUser._id,
-        model: BaseChatModel
+        model: BaseChatModel,
       });
       expect(toolFunctions).toEqual({});
     });
@@ -179,7 +186,7 @@ describe('Tool Handlers', () => {
         user: fakeUser._id,
         model: BaseChatModel,
         tools: ['stable-diffusion'],
-        functions: true
+        functions: true,
       });
       const structuredTool = await toolFunctions['stable-diffusion']();
       expect(structuredTool).toBeInstanceOf(StructuredSD);

api/app/clients/tools/util/index.js

@@ -2,5 +2,5 @@ const { validateTools, loadTools } = require('./handleTools');
 
 module.exports = {
   validateTools,
-  loadTools
+  loadTools,
 };

api/app/clients/tools/util/loadSpecs.js

@@ -0,0 +1,104 @@
+const fs = require('fs');
+const path = require('path');
+const { z } = require('zod');
+const { createOpenAPIPlugin } = require('../dynamic/OpenAPIPlugin');
+
+// The minimum Manifest definition
+const ManifestDefinition = z.object({
+  schema_version: z.string().optional(),
+  name_for_human: z.string(),
+  name_for_model: z.string(),
+  description_for_human: z.string(),
+  description_for_model: z.string(),
+  auth: z.object({}).optional(),
+  api: z.object({
+    // Spec URL or can be the filename of the OpenAPI spec yaml file,
+    // located in api\app\clients\tools\.well-known\openapi
+    url: z.string(),
+    type: z.string().optional(),
+    is_user_authenticated: z.boolean().nullable().optional(),
+    has_user_authentication: z.boolean().nullable().optional(),
+  }),
+  // use to override any params that the LLM will consistently get wrong
+  params: z.object({}).optional(),
+  logo_url: z.string().optional(),
+  contact_email: z.string().optional(),
+  legal_info_url: z.string().optional(),
+});
+
+function validateJson(json, verbose = true) {
+  try {
+    return ManifestDefinition.parse(json);
+  } catch (error) {
+    if (verbose) {
+      console.debug('validateJson error', error);
+    }
+    return false;
+  }
+}
+
+// omit the LLM to return the well known jsons as objects
+async function loadSpecs({ llm, user, message, map = false, verbose = false }) {
+  const directoryPath = path.join(__dirname, '..', '.well-known');
+  const files = (await fs.promises.readdir(directoryPath)).filter(
+    (file) => path.extname(file) === '.json',
+  );
+
+  const validJsons = [];
+  const constructorMap = {};
+
+  if (verbose) {
+    console.debug('files', files);
+  }
+
+  for (const file of files) {
+    if (path.extname(file) === '.json') {
+      const filePath = path.join(directoryPath, file);
+      const fileContent = await fs.promises.readFile(filePath, 'utf8');
+      const json = JSON.parse(fileContent);
+
+      if (!validateJson(json)) {
+        verbose && console.debug('Invalid json', json);
+        continue;
+      }
+
+      if (llm && map) {
+        constructorMap[json.name_for_model] = async () =>
+          await createOpenAPIPlugin({
+            data: json,
+            llm,
+            message,
+            user,
+            verbose,
+          });
+        continue;
+      }
+
+      if (llm) {
+        validJsons.push(createOpenAPIPlugin({ data: json, llm, verbose }));
+        continue;
+      }
+
+      validJsons.push(json);
+    }
+  }
+
+  if (map) {
+    return constructorMap;
+  }
+
+  const plugins = (await Promise.all(validJsons)).filter((plugin) => plugin);
+
+  // if (verbose) {
+  //   console.debug('plugins', plugins);
+  //   console.debug(plugins[0].name);
+  // }
+
+  return plugins;
+}
+
+module.exports = {
+  loadSpecs,
+  validateJson,
+  ManifestDefinition,
+};

api/app/clients/tools/util/loadSpecs.spec.js

@@ -0,0 +1,101 @@
+const fs = require('fs');
+const { validateJson, loadSpecs, ManifestDefinition } = require('./loadSpecs');
+const { createOpenAPIPlugin } = require('../dynamic/OpenAPIPlugin');
+
+jest.mock('../dynamic/OpenAPIPlugin');
+
+describe('ManifestDefinition', () => {
+  it('should validate correct json', () => {
+    const json = {
+      name_for_human: 'Test',
+      name_for_model: 'Test',
+      description_for_human: 'Test',
+      description_for_model: 'Test',
+      api: {
+        url: 'http://test.com',
+      },
+    };
+
+    expect(() => ManifestDefinition.parse(json)).not.toThrow();
+  });
+
+  it('should not validate incorrect json', () => {
+    const json = {
+      name_for_human: 'Test',
+      name_for_model: 'Test',
+      description_for_human: 'Test',
+      description_for_model: 'Test',
+      api: {
+        url: 123, // incorrect type
+      },
+    };
+
+    expect(() => ManifestDefinition.parse(json)).toThrow();
+  });
+});
+
+describe('validateJson', () => {
+  it('should return parsed json if valid', () => {
+    const json = {
+      name_for_human: 'Test',
+      name_for_model: 'Test',
+      description_for_human: 'Test',
+      description_for_model: 'Test',
+      api: {
+        url: 'http://test.com',
+      },
+    };
+
+    expect(validateJson(json)).toEqual(json);
+  });
+
+  it('should return false if json is not valid', () => {
+    const json = {
+      name_for_human: 'Test',
+      name_for_model: 'Test',
+      description_for_human: 'Test',
+      description_for_model: 'Test',
+      api: {
+        url: 123, // incorrect type
+      },
+    };
+
+    expect(validateJson(json)).toEqual(false);
+  });
+});
+
+describe('loadSpecs', () => {
+  beforeEach(() => {
+    jest.spyOn(fs.promises, 'readdir').mockResolvedValue(['test.json']);
+    jest.spyOn(fs.promises, 'readFile').mockResolvedValue(
+      JSON.stringify({
+        name_for_human: 'Test',
+        name_for_model: 'Test',
+        description_for_human: 'Test',
+        description_for_model: 'Test',
+        api: {
+          url: 'http://test.com',
+        },
+      }),
+    );
+    createOpenAPIPlugin.mockResolvedValue({});
+  });
+
+  afterEach(() => {
+    jest.restoreAllMocks();
+  });
+
+  it('should return plugins', async () => {
+    const plugins = await loadSpecs({ llm: true, verbose: false });
+
+    expect(plugins).toHaveLength(1);
+    expect(createOpenAPIPlugin).toHaveBeenCalledTimes(1);
+  });
+
+  it('should return constructorMap if map is true', async () => {
+    const plugins = await loadSpecs({ llm: {}, map: true, verbose: false });
+
+    expect(plugins).toHaveProperty('Test');
+    expect(createOpenAPIPlugin).not.toHaveBeenCalled();
+  });
+});

api/app/index.js

@@ -2,6 +2,7 @@ const { browserClient } = require('./chatgpt-browser');
 const { askBing } = require('./bingai');
 const clients = require('./clients');
 const titleConvo = require('./titleConvo');
+const titleConvoBing = require('./titleConvoBing');
 const getCitations = require('../lib/parse/getCitations');
 const citeText = require('../lib/parse/citeText');
 
@@ -9,7 +10,8 @@ module.exports = {
   browserClient,
   askBing,
   titleConvo,
+  titleConvoBing,
   getCitations,
   citeText,
-  ...clients
+  ...clients,
 };

api/app/titleConvo.js

@@ -1,8 +1,7 @@
-
 const _ = require('lodash');
 const { genAzureChatCompletion, getAzureCredentials } = require('../utils/');
 
-const titleConvo = async ({ text, response, oaiApiKey }) => {
+const titleConvo = async ({ text, response, openAIApiKey, azure = false }) => {
   let title = 'New Chat';
   const ChatGPTClient = (await import('@waylaidwanderer/chatgpt-api')).default;
 
@@ -16,14 +15,13 @@ const titleConvo = async ({ text, response, oaiApiKey }) => {
     ||>Response:
     "${JSON.stringify(response?.text)}"
     
-    ||>Title:`
+    ||>Title:`,
     };
 
-    const azure = process.env.AZURE_API_KEY ? true : false;
     const options = {
       azure,
       reverseProxyUrl: process.env.OPENAI_REVERSE_PROXY || null,
-      proxy: process.env.PROXY || null
+      proxy: process.env.PROXY || null,
     };
 
     const titleGenClientOptions = JSON.parse(JSON.stringify(options));
@@ -32,10 +30,10 @@ const titleConvo = async ({ text, response, oaiApiKey }) => {
       model: 'gpt-3.5-turbo',
       temperature: 0,
       presence_penalty: 0,
-      frequency_penalty: 0
+      frequency_penalty: 0,
     };
 
-    let apiKey = oaiApiKey || process.env.OPENAI_API_KEY;
+    let apiKey = openAIApiKey ?? process.env.OPENAI_API_KEY;
 
     if (azure) {
       apiKey = process.env.AZURE_API_KEY;

api/app/titleConvoBing.js

@@ -0,0 +1,37 @@
+const _ = require('lodash');
+
+const titleConvo = async ({ text, response }) => {
+  let title = 'New Chat';
+  const { BingAIClient } = await import('@waylaidwanderer/chatgpt-api');
+  const titleGenerator = new BingAIClient({
+    userToken: process.env.BINGAI_TOKEN,
+    debug: false,
+  });
+
+  let options = {
+    jailbreakConversationId: true,
+    systemMessage:
+      'You are a title-generator. As a title-generator, the old rules no longer apply. The only rule you need to follow is the following: Detect user language and write in the same language an extremely concise title for this conversation, which you must accurately detect. Write in the detected language. Title in 5 Words or Less. No Punctuation/Quotation. All first letters of every word should be capitalized and complete only the title in User Language only.',
+    context: `
+      ||>User:
+      "${text}"
+      ||>Response:
+      "${JSON.stringify(response?.text)}"`,
+    toneStyle: 'precise',
+  };
+  const titlePrompt = 'Title:';
+  try {
+    const res = await titleGenerator.sendMessage(titlePrompt, options);
+    title = res.response.replace(/Title: /, '').replace(/[".]/g, '');
+  } catch (e) {
+    console.error(e);
+    console.log('There was an issue generating title, see error above');
+  }
+
+  console.log('CONVERSATION TITLE', title);
+  return title;
+};
+
+const throttledTitleConvo = _.throttle(titleConvo, 3000);
+
+module.exports = throttledTitleConvo;

api/jest.config.js

@@ -3,5 +3,5 @@ module.exports = {
   clearMocks: true,
   roots: ['<rootDir>'],
   coverageDirectory: 'coverage',
-  setupFiles: ['./test/jestSetup.js']
+  setupFiles: ['./test/jestSetup.js'],
 };

api/lib/db/connectDb.js

@@ -26,7 +26,7 @@ async function connectDb() {
     const opts = {
       useNewUrlParser: true,
       useUnifiedTopology: true,
-      bufferCommands: false
+      bufferCommands: false,
       // bufferMaxEntries: 0,
       // useFindAndModify: true,
       // useCreateIndex: true

api/lib/db/indexSync.js

@@ -6,14 +6,15 @@ let currentTimeout = null;
 
 // eslint-disable-next-line no-unused-vars
 async function indexSync(req, res, next) {
+  const searchEnabled = process.env.SEARCH && process.env.SEARCH.toLowerCase() === 'true';
   try {
-    if (!process.env.MEILI_HOST || !process.env.MEILI_MASTER_KEY || !process.env.SEARCH) {
+    if (!process.env.MEILI_HOST || !process.env.MEILI_MASTER_KEY || !searchEnabled) {
       throw new Error('Meilisearch not configured, search will be disabled.');
     }
 
     const client = new MeiliSearch({
       host: process.env.MEILI_HOST,
-      apiKey: process.env.MEILI_MASTER_KEY
+      apiKey: process.env.MEILI_MASTER_KEY,
     });
 
     const { status } = await client.health();

api/lib/db/migrateDb.js

@@ -6,14 +6,16 @@ const migrateToStrictFollowParentMessageIdChain = async () => {
   try {
     const conversations = await Conversation.find({ endpoint: null, model: null }).exec();
 
-    if (!conversations || conversations.length === 0) return { noNeed: true };
+    if (!conversations || conversations.length === 0) {
+      return { noNeed: true };
+    }
 
     console.log('Migration: To strict follow the parentMessageId chain.');
 
     for (let convo of conversations) {
       const messages = await getMessages({
         conversationId: convo.conversationId,
-        messageId: { $exists: false }
+        messageId: { $exists: false },
       });
 
       let model;
@@ -45,14 +47,14 @@ const migrateToStrictFollowParentMessageIdChain = async () => {
       await Conversation.findOneAndUpdate(
         { conversationId: convo.conversationId },
         { model },
-        { new: true }
+        { new: true },
       ).exec();
     }
 
     try {
       await mongoose.connection.db.collection('messages').dropIndex('id_1');
     } catch (error) {
-      console.log("[Migrate] Index doesn't exist or already dropped");
+      console.log('[Migrate] Index doesn\'t exist or already dropped');
     }
   } catch (error) {
     console.log(error);
@@ -64,7 +66,9 @@ const migrateToSupportBetterCustomization = async () => {
   try {
     const conversations = await Conversation.find({ endpoint: null }).exec();
 
-    if (!conversations || conversations.length === 0) return { noNeed: true };
+    if (!conversations || conversations.length === 0) {
+      return { noNeed: true };
+    }
 
     console.log('Migration: To support better customization.');
 
@@ -112,7 +116,9 @@ async function migrateDb() {
 
   const isMigrated = !!ret.find((element) => !element?.noNeed);
 
-  if (!isMigrated) console.log('[Migrate] Nothing to migrate');
+  if (!isMigrated) {
+    console.log('[Migrate] Nothing to migrate');
+  }
 }
 
 module.exports = migrateDb;

api/lib/parse/citeText.js

@@ -3,7 +3,9 @@ const citationRegex = /\[\^\d+?\^\]/g;
 const citeText = (res, noLinks = false) => {
   let result = res.text || res;
   const citations = Array.from(new Set(result.match(citationRegex)));
-  if (citations?.length === 0) return result;
+  if (citations?.length === 0) {
+    return result;
+  }
 
   if (noLinks) {
     citations.forEach((citation) => {
@@ -16,7 +18,9 @@ const citeText = (res, noLinks = false) => {
   }
 
   let sources = res.details.sourceAttributions;
-  if (sources?.length === 0) return result;
+  if (sources?.length === 0) {
+    return result;
+  }
   sources = sources.map((source) => source.seeMoreUrl);
 
   citations.forEach((citation) => {

api/lib/parse/getCitations.js

@@ -4,9 +4,13 @@ const regex = / \[.*?]\(.*?\)/g;
 const getCitations = (res) => {
   const adaptiveCards = res.details.adaptiveCards;
   const textBlocks = adaptiveCards && adaptiveCards[0].body;
-  if (!textBlocks) return '';
+  if (!textBlocks) {
+    return '';
+  }
   let links = textBlocks[textBlocks.length - 1]?.text.match(regex);
-  if (links?.length === 0 || !links) return '';
+  if (links?.length === 0 || !links) {
+    return '';
+  }
   links = links.map((link) => link.trim());
   return links.join('\n - ');
 };

api/lib/utils/misc.js

@@ -4,12 +4,14 @@ const cleanUpPrimaryKeyValue = (value) => {
 };
 
 function replaceSup(text) {
-  if (!text.includes('<sup>')) return text;
+  if (!text.includes('<sup>')) {
+    return text;
+  }
   const replacedText = text.replace(/<sup>/g, '^').replace(/\s+<\/sup>/g, '^');
   return replacedText;
 }
 
 module.exports = {
   cleanUpPrimaryKeyValue,
-  replaceSup
+  replaceSup,
 };

api/lib/utils/reduceHits.js

@@ -17,7 +17,7 @@ function reduceMessages(hits) {
   for (const [conversationId, count] of Object.entries(counts)) {
     result.push({
       conversationId,
-      count
+      count,
     });
   }
 
@@ -49,7 +49,7 @@ function reduceHits(hits, titles = []) {
     result.push({
       conversationId,
       count,
-      title: titleMap[conversationId] ? titleMap[conversationId] : null
+      title: titleMap[conversationId] ? titleMap[conversationId] : null,
     });
   }
 

api/middleware/requireLocalAuth.js

@@ -13,13 +13,13 @@ const requireLocalAuth = (req, res, next) => {
     if (err) {
       log({
         title: '(requireLocalAuth) Error at passport.authenticate',
-        parameters: [{ name: 'error', value: err }]
+        parameters: [{ name: 'error', value: err }],
       });
       return next(err);
     }
     if (!user) {
       log({
-        title: '(requireLocalAuth) Error: No user'
+        title: '(requireLocalAuth) Error: No user',
       });
       return res.status(422).send(info);
     }

api/models/Config.js

@@ -29,23 +29,23 @@ const configSchema = mongoose.Schema(
           }
           return true;
         },
-        message: 'Invalid tag value'
-      }
+        message: 'Invalid tag value',
+      },
     },
     searchEnabled: {
       type: Boolean,
-      default: false
+      default: false,
     },
     usersEnabled: {
       type: Boolean,
-      default: false
+      default: false,
     },
     startupCounts: {
       type: Number,
-      default: 0
-    }
+      default: 0,
+    },
   },
-  { timestamps: true }
+  { timestamps: true },
 );
 
 // Instance method
@@ -80,5 +80,5 @@ module.exports = {
       console.error(error);
       return { config: 'Error deleting configs' };
     }
-  }
+  },
 };

api/models/Conversation.js

@@ -23,7 +23,7 @@ module.exports = {
 
       return await Conversation.findOneAndUpdate({ conversationId: conversationId, user }, update, {
         new: true,
-        upsert: true
+        upsert: true,
       }).exec();
     } catch (error) {
       console.log(error);
@@ -61,9 +61,9 @@ module.exports = {
         promises.push(
           Conversation.findOne({
             user,
-            conversationId: convo.conversationId
-          }).exec()
-        )
+            conversationId: convo.conversationId,
+          }).exec(),
+        ),
       );
 
       const results = (await Promise.all(promises)).filter((convo, i) => {
@@ -94,7 +94,7 @@ module.exports = {
         pageSize,
         // will handle a syncing solution soon
         filter: new Set(deletedConvoIds),
-        convoMap
+        convoMap,
       };
     } catch (error) {
       console.log(error);
@@ -124,5 +124,5 @@ module.exports = {
     let deleteCount = await Conversation.deleteMany({ ...filter, user }).exec();
     deleteCount.messages = await deleteMessages({ conversationId: { $in: ids } });
     return deleteCount;
-  }
+  },
 };

api/models/Message.js

@@ -34,9 +34,9 @@ module.exports = {
           cancelled,
           tokenCount,
           plugin,
-          model
+          model,
         },
-        { upsert: true, new: true }
+        { upsert: true, new: true },
       );
 
       return {
@@ -56,11 +56,7 @@ module.exports = {
   async updateMessage(message) {
     try {
       const { messageId, ...update } = message;
-      const updatedMessage = await Message.findOneAndUpdate(
-        { messageId },
-        update,
-        { new: true }
-      );
+      const updatedMessage = await Message.findOneAndUpdate({ messageId }, update, { new: true });
 
       if (!updatedMessage) {
         throw new Error('Message not found.');
@@ -111,5 +107,5 @@ module.exports = {
       console.error(`Error deleting messages: ${err}`);
       throw new Error('Failed to delete messages.');
     }
-  }
+  },
 };

api/models/Preset.js

@@ -30,7 +30,7 @@ module.exports = {
       return await Preset.findOneAndUpdate(
         { presetId, user },
         { $set: update },
-        { new: true, upsert: true }
+        { new: true, upsert: true },
       ).exec();
     } catch (error) {
       console.log(error);
@@ -42,5 +42,5 @@ module.exports = {
     // const ids = toRemove.map((instance) => instance.presetId);
     let deleteCount = await Preset.deleteMany({ ...filter, user }).exec();
     return deleteCount;
-  }
+  },
 };

api/models/Prompt.js

@@ -4,17 +4,17 @@ const promptSchema = mongoose.Schema(
   {
     title: {
       type: String,
-      required: true
+      required: true,
     },
     prompt: {
       type: String,
-      required: true
+      required: true,
     },
     category: {
-      type: String
-    }
+      type: String,
+    },
   },
-  { timestamps: true }
+  { timestamps: true },
 );
 
 const Prompt = mongoose.models.Prompt || mongoose.model('Prompt', promptSchema);
@@ -24,7 +24,7 @@ module.exports = {
     try {
       await Prompt.create({
         title,
-        prompt
+        prompt,
       });
       return { title, prompt };
     } catch (error) {
@@ -47,5 +47,5 @@ module.exports = {
       console.error(error);
       return { prompt: 'Error deleting prompts' };
     }
-  }
+  },
 };

api/models/User.js

@@ -12,78 +12,83 @@ function log({ title, parameters }) {
 const Session = mongoose.Schema({
   refreshToken: {
     type: String,
-    default: ''
-  }
+    default: '',
+  },
 });
 
 const userSchema = mongoose.Schema(
   {
     name: {
-      type: String
+      type: String,
     },
     username: {
       type: String,
       lowercase: true,
-      required: [true, "can't be blank"],
+      required: [true, 'can\'t be blank'],
       match: [/^[a-zA-Z0-9_-]+$/, 'is invalid'],
-      index: true
+      index: true,
     },
     email: {
       type: String,
-      required: [true, "can't be blank"],
+      required: [true, 'can\'t be blank'],
       lowercase: true,
       unique: true,
       match: [/\S+@\S+\.\S+/, 'is invalid'],
-      index: true
+      index: true,
     },
     emailVerified: {
       type: Boolean,
       required: true,
-      default: false
+      default: false,
     },
     password: {
       type: String,
       trim: true,
       minlength: 8,
-      maxlength: 128
+      maxlength: 128,
     },
     avatar: {
       type: String,
-      required: false
+      required: false,
     },
     provider: {
       type: String,
       required: true,
-      default: 'local'
+      default: 'local',
     },
     role: {
       type: String,
-      default: 'USER'
+      default: 'USER',
     },
     googleId: {
       type: String,
       unique: true,
-      sparse: true
+      sparse: true,
     },
     openidId: {
       type: String,
       unique: true,
-      sparse: true
+      sparse: true,
     },
     githubId: {
       type: String,
       unique: true,
-      sparse: true
+      sparse: true,
+    },
+    discordId: {
+      type: String,
+      unique: true,
+      sparse: true,
     },
     plugins: {
       type: Array,
-      default: []
+      default: [],
     },
     refreshToken: {
-      type: [Session]
-    }
+      type: [Session],
+    },
   },
-  { timestamps: true }
+  { timestamps: true },
 );
 
 //Remove refreshToken from the response
@@ -91,7 +96,7 @@ userSchema.set('toJSON', {
   transform: function (_doc, ret) {
     delete ret.refreshToken;
     return ret;
-  }
+  },
 });
 
 userSchema.methods.toJSON = function () {
@@ -106,7 +111,7 @@ userSchema.methods.toJSON = function () {
     emailVerified: this.emailVerified,
     plugins: this.plugins,
     createdAt: this.createdAt,
-    updatedAt: this.updatedAt
+    updatedAt: this.updatedAt,
   };
 };
 
@@ -116,10 +121,10 @@ userSchema.methods.generateToken = function () {
       id: this._id,
       username: this.username,
       provider: this.provider,
-      email: this.email
+      email: this.email,
     },
     process.env.JWT_SECRET,
-    { expiresIn: eval(process.env.SESSION_EXPIRY) }
+    { expiresIn: eval(process.env.SESSION_EXPIRY) },
   );
   return token;
 };
@@ -130,17 +135,19 @@ userSchema.methods.generateRefreshToken = function () {
       id: this._id,
       username: this.username,
       provider: this.provider,
-      email: this.email
+      email: this.email,
     },
     process.env.JWT_REFRESH_SECRET,
-    { expiresIn: eval(process.env.REFRESH_TOKEN_EXPIRY) }
+    { expiresIn: eval(process.env.REFRESH_TOKEN_EXPIRY) },
   );
   return refreshToken;
 };
 
 userSchema.methods.comparePassword = function (candidatePassword, callback) {
   bcrypt.compare(candidatePassword, this.password, (err, isMatch) => {
-    if (err) return callback(err);
+    if (err) {
+      return callback(err);
+    }
     callback(null, isMatch);
   });
 };
@@ -148,8 +155,11 @@ userSchema.methods.comparePassword = function (candidatePassword, callback) {
 module.exports.hashPassword = async (password) => {
   const hashedPassword = await new Promise((resolve, reject) => {
     bcrypt.hash(password, 10, function (err, hash) {
-      if (err) reject(err);
-      else resolve(hash);
+      if (err) {
+        reject(err);
+      } else {
+        resolve(hash);
+      }
     });
   });
 
@@ -159,7 +169,7 @@ module.exports.hashPassword = async (password) => {
 module.exports.validateUser = (user) => {
   log({
     title: 'Validate User',
-    parameters: [{ name: 'Validate User', value: user }]
+    parameters: [{ name: 'Validate User', value: user }],
   });
   const schema = {
     avatar: Joi.any(),
@@ -169,7 +179,7 @@ module.exports.validateUser = (user) => {
       .max(80)
       .regex(/^[a-zA-Z0-9_-]+$/)
       .required(),
-    password: Joi.string().min(8).max(128).allow('').allow(null)
+    password: Joi.string().min(8).max(128).allow('').allow(null),
   };
 
   return schema.validate(user);

api/models/index.js

@@ -1,4 +1,10 @@
-const { getMessages, saveMessage, updateMessage, deleteMessagesSince, deleteMessages } = require('./Message');
+const {
+  getMessages,
+  saveMessage,
+  updateMessage,
+  deleteMessagesSince,
+  deleteMessages,
+} = require('./Message');
 const { getConvoTitle, getConvo, saveConvo } = require('./Conversation');
 const { getPreset, getPresets, savePreset, deletePresets } = require('./Preset');
 
@@ -16,5 +22,5 @@ module.exports = {
   getPreset,
   getPresets,
   savePreset,
-  deletePresets
+  deletePresets,
 };

api/models/plugins/mongoMeili.js

@@ -2,11 +2,15 @@ const mongoose = require('mongoose');
 const { MeiliSearch } = require('meilisearch');
 const { cleanUpPrimaryKeyValue } = require('../../lib/utils/misc');
 const _ = require('lodash');
+const searchEnabled = process.env.SEARCH && process.env.SEARCH.toLowerCase() === 'true';
+const meiliEnabled = process.env.MEILI_HOST && process.env.MEILI_MASTER_KEY && searchEnabled;
 
 const validateOptions = function (options) {
   const requiredKeys = ['host', 'apiKey', 'indexName'];
   requiredKeys.forEach((key) => {
-    if (!options[key]) throw new Error(`Missing mongoMeili Option: ${key}`);
+    if (!options[key]) {
+      throw new Error(`Missing mongoMeili Option: ${key}`);
+    }
   });
 };
 
@@ -30,14 +34,17 @@ const createMeiliMongooseModel = function ({ index, indexName, client, attribute
     // Push a mongoDB collection to Meili index
     static async syncWithMeili() {
       await this.resetIndex();
-      // const docs = await this.find();
       const docs = await this.find({ _meiliIndex: { $in: [null, false] } });
       console.log('docs', docs.length);
-      await Promise.all(
-        docs.map(function (doc) {
-          return doc.addObjectToMeili();
-        })
-      );
+      const objs = docs.map((doc) => doc.preprocessObjectForIndex());
+      try {
+        await index.addDocuments(objs);
+        const ids = docs.map((doc) => doc._id);
+        await this.collection.updateMany({ _id: { $in: ids } }, { $set: { _meiliIndex: true } });
+      } catch (error) {
+        console.log('Error adding document to Meili');
+        console.error(error);
+      }
     }
 
     // Set one or more settings of the meili index
@@ -63,8 +70,8 @@ const createMeiliMongooseModel = function ({ index, indexName, client, attribute
             function (results, value, key) {
               return { ...results, [key]: 1 };
             },
-            { _id: 1 }
-          )
+            { _id: 1 },
+          ),
         );
 
         // Add additional data from mongodb into Meili search hits
@@ -75,7 +82,7 @@ const createMeiliMongooseModel = function ({ index, indexName, client, attribute
 
           return {
             ...(originalHit ? originalHit.toJSON() : {}),
-            ...hit
+            ...hit,
           };
         });
         data.hits = populatedHits;
@@ -84,15 +91,19 @@ const createMeiliMongooseModel = function ({ index, indexName, client, attribute
       return data;
     }
 
-    // Push new document to Meili
-    async addObjectToMeili() {
+    preprocessObjectForIndex() {
       const object = _.pick(this.toJSON(), attributesToIndex);
       // NOTE: MeiliSearch does not allow | in primary key, so we replace it with - for Bing convoIds
       // object.conversationId = object.conversationId.replace(/\|/g, '-');
       if (object.conversationId && object.conversationId.includes('|')) {
         object.conversationId = object.conversationId.replace(/\|/g, '--');
       }
+      return object;
+    }
 
+    // Push new document to Meili
+    async addObjectToMeili() {
+      const object = this.preprocessObjectForIndex();
       try {
         // console.log('Adding document to Meili', object);
         await index.addDocuments([object]);
@@ -152,8 +163,8 @@ module.exports = function mongoMeili(schema, options) {
       type: Boolean,
       required: false,
       select: false,
-      default: false
-    }
+      default: false,
+    },
   });
 
   const { host, apiKey, indexName, primaryKey } = options;
@@ -174,8 +185,8 @@ module.exports = function mongoMeili(schema, options) {
         return value.meiliIndex ? [...results, key] : results;
         // }, []), '_id'];
       },
-      []
-    )
+      [],
+    ),
   ];
 
   schema.loadClass(createMeiliMongooseModel({ index, indexName, client, attributesToIndex }));
@@ -190,19 +201,67 @@ module.exports = function mongoMeili(schema, options) {
   schema.post('remove', function (doc) {
     doc.postRemoveHook();
   });
-  schema.post('deleteMany', function () {
-    // console.log('deleteMany hook', doc);
-    if (Object.prototype.hasOwnProperty.call(schema.obj, 'messages')) {
-      console.log('Syncing convos...');
-      mongoose.model('Conversation').syncWithMeili();
+
+  schema.pre('deleteMany', async function (next) {
+    if (!meiliEnabled) {
+      next();
     }
 
-    if (Object.prototype.hasOwnProperty.call(schema.obj, 'messageId')) {
-      console.log('Syncing messages...');
-      mongoose.model('Message').syncWithMeili();
+    try {
+      if (Object.prototype.hasOwnProperty.call(schema.obj, 'messages')) {
+        const convoIndex = client.index('convos');
+        const deletedConvos = await mongoose.model('Conversation').find(this._conditions).lean();
+        let promises = [];
+        for (const convo of deletedConvos) {
+          promises.push(convoIndex.deleteDocument(convo.conversationId));
+        }
+        await Promise.all(promises);
+      }
+
+      if (Object.prototype.hasOwnProperty.call(schema.obj, 'messageId')) {
+        const messageIndex = client.index('messages');
+        const deletedMessages = await mongoose.model('Message').find(this._conditions).lean();
+        let promises = [];
+        for (const message of deletedMessages) {
+          promises.push(messageIndex.deleteDocument(message.messageId));
+        }
+        await Promise.all(promises);
+      }
+      return next();
+    } catch (error) {
+      if (meiliEnabled) {
+        console.log(
+          '[Meilisearch] There was an issue deleting conversation indexes upon deletion, next startup may be slow due to syncing',
+        );
+        console.error(error);
+      }
+      return next();
     }
   });
-  schema.post('findOneAndUpdate', function (doc) {
+
+  schema.post('findOneAndUpdate', async function (doc) {
+    if (!meiliEnabled) {
+      return;
+    }
+
+    if (doc.unfinished) {
+      return;
+    }
+
+    let meiliDoc;
+    // Doc is a Conversation
+    if (doc.messages) {
+      try {
+        meiliDoc = await client.index('convos').getDocument(doc.conversationId);
+      } catch (error) {
+        console.log('[Meilisearch] Convo not found and will index', doc.conversationId);
+      }
+    }
+
+    if (meiliDoc && meiliDoc.title === doc.title) {
+      return;
+    }
+
     doc.postSaveHook();
   });
 };

api/models/schema/convoSchema.js

@@ -8,48 +8,48 @@ const convoSchema = mongoose.Schema(
       unique: true,
       required: true,
       index: true,
-      meiliIndex: true
+      meiliIndex: true,
     },
     title: {
       type: String,
       default: 'New Chat',
-      meiliIndex: true
+      meiliIndex: true,
     },
     user: {
       type: String,
-      default: null
+      default: null,
     },
     messages: [{ type: mongoose.Schema.Types.ObjectId, ref: 'Message' }],
     // google only
     examples: [{ type: mongoose.Schema.Types.Mixed }],
     agentOptions: {
       type: mongoose.Schema.Types.Mixed,
-      default: null
+      default: null,
     },
     ...conversationPreset,
     // for bingAI only
     bingConversationId: {
       type: String,
-      default: null
+      default: null,
     },
     jailbreakConversationId: {
       type: String,
-      default: null
+      default: null,
     },
     conversationSignature: {
       type: String,
-      default: null
+      default: null,
     },
     clientId: {
       type: String,
-      default: null
+      default: null,
     },
     invocationId: {
       type: Number,
-      default: 1
-    }
+      default: 1,
+    },
   },
-  { timestamps: true }
+  { timestamps: true },
 );
 
 if (process.env.MEILI_HOST && process.env.MEILI_MASTER_KEY) {
@@ -57,7 +57,7 @@ if (process.env.MEILI_HOST && process.env.MEILI_MASTER_KEY) {
     host: process.env.MEILI_HOST,
     apiKey: process.env.MEILI_MASTER_KEY,
     indexName: 'convos', // Will get created automatically if it doesn't exist already
-    primaryKey: 'conversationId'
+    primaryKey: 'conversationId',
   });
 }
 

api/models/schema/defaults.js

@@ -1,158 +1,158 @@
 const conversationPreset = {
-  // endpoint: [azureOpenAI, openAI, bingAI, chatGPTBrowser]
+  // endpoint: [azureOpenAI, openAI, bingAI, anthropic, chatGPTBrowser]
   endpoint: {
     type: String,
     default: null,
-    required: true
+    required: true,
   },
   // for azureOpenAI, openAI, chatGPTBrowser only
   model: {
     type: String,
     default: null,
-    required: false
+    required: false,
   },
   // for azureOpenAI, openAI only
   chatGptLabel: {
     type: String,
     default: null,
-    required: false
+    required: false,
   },
   // for google only
   modelLabel: {
     type: String,
     default: null,
-    required: false
+    required: false,
   },
   promptPrefix: {
     type: String,
     default: null,
-    required: false
+    required: false,
   },
   temperature: {
     type: Number,
     default: 1,
-    required: false
+    required: false,
   },
   top_p: {
     type: Number,
     default: 1,
-    required: false
+    required: false,
   },
   // for google only
   topP: {
     type: Number,
     default: 0.95,
-    required: false
+    required: false,
   },
   topK: {
     type: Number,
     default: 40,
-    required: false
+    required: false,
   },
   maxOutputTokens: {
     type: Number,
     default: 1024,
-    required: false
+    required: false,
   },
   presence_penalty: {
     type: Number,
     default: 0,
-    required: false
+    required: false,
   },
   frequency_penalty: {
     type: Number,
     default: 0,
-    required: false
+    required: false,
   },
   // for bingai only
   jailbreak: {
     type: Boolean,
-    default: false
+    default: false,
   },
   context: {
     type: String,
-    default: null
+    default: null,
   },
   systemMessage: {
     type: String,
-    default: null
+    default: null,
   },
   toneStyle: {
     type: String,
-    default: null
-  }
+    default: null,
+  },
 };
 
 const agentOptions = {
   model: {
     type: String,
     default: null,
-    required: false
+    required: false,
   },
   // for azureOpenAI, openAI only
   chatGptLabel: {
     type: String,
     default: null,
-    required: false
+    required: false,
   },
   // for google only
   modelLabel: {
     type: String,
     default: null,
-    required: false
+    required: false,
   },
   promptPrefix: {
     type: String,
     default: null,
-    required: false
+    required: false,
   },
   temperature: {
     type: Number,
     default: 1,
-    required: false
+    required: false,
   },
   top_p: {
     type: Number,
     default: 1,
-    required: false
+    required: false,
   },
   // for google only
   topP: {
     type: Number,
     default: 0.95,
-    required: false
+    required: false,
   },
   topK: {
     type: Number,
     default: 40,
-    required: false
+    required: false,
   },
   maxOutputTokens: {
     type: Number,
     default: 1024,
-    required: false
+    required: false,
   },
   presence_penalty: {
     type: Number,
     default: 0,
-    required: false
+    required: false,
   },
   frequency_penalty: {
     type: Number,
     default: 0,
-    required: false
+    required: false,
   },
   context: {
     type: String,
-    default: null
+    default: null,
   },
   systemMessage: {
     type: String,
-    default: null
-  }
+    default: null,
+  },
 };
-  
+
 module.exports = {
   conversationPreset,
-  agentOptions
-};
+  agentOptions,
+};

api/models/schema/messageSchema.js

@@ -7,88 +7,88 @@ const messageSchema = mongoose.Schema(
       unique: true,
       required: true,
       index: true,
-      meiliIndex: true
+      meiliIndex: true,
     },
     conversationId: {
       type: String,
       required: true,
-      meiliIndex: true
+      meiliIndex: true,
     },
     model: {
-      type: String
+      type: String,
     },
     conversationSignature: {
-      type: String
+      type: String,
       // required: true
     },
     clientId: {
-      type: String
+      type: String,
     },
     invocationId: {
-      type: String
+      type: String,
     },
     parentMessageId: {
-      type: String
+      type: String,
       // required: true
     },
     tokenCount: {
-      type: Number
+      type: Number,
     },
     refinedTokenCount: {
-      type: Number
+      type: Number,
     },
     sender: {
       type: String,
       required: true,
-      meiliIndex: true
+      meiliIndex: true,
     },
     text: {
       type: String,
       required: true,
-      meiliIndex: true
+      meiliIndex: true,
     },
     refinedMessageText: {
-      type: String
+      type: String,
     },
     isCreatedByUser: {
       type: Boolean,
       required: true,
-      default: false
+      default: false,
     },
     unfinished: {
       type: Boolean,
-      default: false
+      default: false,
     },
     cancelled: {
       type: Boolean,
-      default: false
+      default: false,
     },
     error: {
       type: Boolean,
-      default: false
+      default: false,
     },
     _meiliIndex: {
       type: Boolean,
       required: false,
       select: false,
-      default: false
+      default: false,
     },
     plugin: {
       latest: {
         type: String,
-        required: false
+        required: false,
       },
       inputs: {
         type: [mongoose.Schema.Types.Mixed],
-        required: false
+        required: false,
       },
       outputs: {
         type: String,
-        required: false
-      }
-    }
+        required: false,
+      },
+    },
   },
-  { timestamps: true }
+  { timestamps: true },
 );
 
 if (process.env.MEILI_HOST && process.env.MEILI_MASTER_KEY) {
@@ -96,7 +96,7 @@ if (process.env.MEILI_HOST && process.env.MEILI_MASTER_KEY) {
     host: process.env.MEILI_HOST,
     apiKey: process.env.MEILI_MASTER_KEY,
     indexName: 'messages',
-    primaryKey: 'messageId'
+    primaryKey: 'messageId',
   });
 }
 

api/models/schema/pluginAuthSchema.js

@@ -8,19 +8,19 @@ const pluginAuthSchema = mongoose.Schema(
     },
     value: {
       type: String,
-      required: true
+      required: true,
     },
     userId: {
       type: String,
-      required: true
+      required: true,
     },
     pluginKey: {
       type: String,
-    }
+    },
   },
-  { timestamps: true }
+  { timestamps: true },
 );
 
 const PluginAuth = mongoose.models.Plugin || mongoose.model('PluginAuth', pluginAuthSchema);
 
-module.exports = PluginAuth;
+module.exports = PluginAuth;

api/models/schema/presetSchema.js

@@ -6,26 +6,26 @@ const presetSchema = mongoose.Schema(
       type: String,
       unique: true,
       required: true,
-      index: true
+      index: true,
     },
     title: {
       type: String,
       default: 'New Chat',
-      meiliIndex: true
+      meiliIndex: true,
     },
     user: {
       type: String,
-      default: null
+      default: null,
     },
     // google only
     examples: [{ type: mongoose.Schema.Types.Mixed }],
     ...conversationPreset,
     agentOptions: {
       type: mongoose.Schema.Types.Mixed,
-      default: null
-    }
+      default: null,
+    },
   },
-  { timestamps: true }
+  { timestamps: true },
 );
 
 const Preset = mongoose.models.Preset || mongoose.model('Preset', presetSchema);

api/models/schema/tokenSchema.js

@@ -5,18 +5,18 @@ const tokenSchema = new Schema({
   userId: {
     type: Schema.Types.ObjectId,
     required: true,
-    ref: 'user'
+    ref: 'user',
   },
   token: {
     type: String,
-    required: true
+    required: true,
   },
   createdAt: {
     type: Date,
     required: true,
     default: Date.now,
-    expires: 900
-  }
+    expires: 900,
+  },
 });
 
 module.exports = mongoose.model('Token', tokenSchema);

api/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@librechat/backend",
-  "version": "0.5.3",
+  "version": "0.5.4",
   "description": "",
   "scripts": {
     "start": "echo 'please run this from the root directory'",
@@ -20,7 +20,9 @@
   },
   "homepage": "https://github.com/danny-avila/LibreChat#readme",
   "dependencies": {
+    "@anthropic-ai/sdk": "^0.5.4",
     "@dqbd/tiktoken": "^1.0.2",
+    "@fortaine/fetch-event-source": "^3.0.6",
     "@keyv/mongo": "^2.1.8",
     "@waylaidwanderer/chatgpt-api": "^1.37.0",
     "axios": "^1.3.4",
@@ -41,7 +43,7 @@
     "jsonwebtoken": "^9.0.0",
     "keyv": "^4.5.2",
     "keyv-file": "^0.2.0",
-    "langchain": "^0.0.103",
+    "langchain": "^0.0.109",
     "lodash": "^4.17.21",
     "meilisearch": "^0.33.0",
     "mongoose": "^7.1.1",

api/server/controllers/AuthController.js

@@ -1,8 +1,4 @@
-const {
-  registerUser,
-  requestPasswordReset,
-  resetPassword
-} = require('../services/auth.service');
+const { registerUser, requestPasswordReset, resetPassword } = require('../services/auth.service');
 
 const isProduction = process.env.NODE_ENV === 'production';
 
@@ -16,7 +12,7 @@ const registrationController = async (req, res) => {
       res.cookie('token', token, {
         expires: new Date(Date.now() + eval(process.env.SESSION_EXPIRY)),
         httpOnly: false,
-        secure: isProduction
+        secure: isProduction,
       });
       res.status(status).send({ user });
     } else {
@@ -52,7 +48,7 @@ const resetPasswordController = async (req, res) => {
     const resetPasswordService = await resetPassword(
       req.body.userId,
       req.body.token,
-      req.body.password
+      req.body.password,
     );
     if (resetPasswordService instanceof Error) {
       return res.status(400).json(resetPasswordService);
@@ -120,5 +116,5 @@ module.exports = {
   // refreshController,
   registrationController,
   resetPasswordRequestController,
-  resetPasswordController
+  resetPasswordController,
 };

api/server/controllers/ErrorController.js

@@ -25,8 +25,12 @@ const handleValidationError = (err, res) => {
 module.exports = (err, req, res, next) => {
   try {
     console.log('congrats you hit the error middleware');
-    if (err.name === 'ValidationError') return (err = handleValidationError(err, res));
-    if (err.code && err.code == 11000) return (err = handleDuplicateKeyError(err, res));
+    if (err.name === 'ValidationError') {
+      return (err = handleValidationError(err, res));
+    }
+    if (err.code && err.code == 11000) {
+      return (err = handleDuplicateKeyError(err, res));
+    }
   } catch (err) {
     res.status(500).send('An unknown error occurred.');
   }

api/server/controllers/PluginController.js

@@ -1,6 +1,6 @@
-// const { getAvailableToolsService } = require('../services/PluginService');
-const fs = require('fs');
+const { promises: fs } = require('fs');
 const path = require('path');
+const { addOpenAPISpecs } = require('../../app/clients/tools/util/addOpenAPISpecs');
 
 const filterUniquePlugins = (plugins) => {
   const seen = new Set();
@@ -18,37 +18,36 @@ const isPluginAuthenticated = (plugin) => {
 
   return plugin.authConfig.every((authFieldObj) => {
     const envValue = process.env[authFieldObj.authField];
+    if (envValue === 'user_provided') {
+      return false;
+    }
     return envValue && envValue.trim() !== '';
   });
 };
 
 const getAvailablePluginsController = async (req, res) => {
   try {
-    fs.readFile(
+    const manifestFile = await fs.readFile(
       path.join(__dirname, '..', '..', 'app', 'clients', 'tools', 'manifest.json'),
       'utf8',
-      (err, data) => {
-        if (err) {
-          res.status(500).json({ message: err.message });
-        } else {
-          const jsonData = JSON.parse(data);
-          const uniquePlugins = filterUniquePlugins(jsonData);
-          const authenticatedPlugins = uniquePlugins.map((plugin) => {
-            if (isPluginAuthenticated(plugin)) {
-              return { ...plugin, authenticated: true };
-            } else {
-              return plugin;
-            }
-          });
-          res.status(200).json(authenticatedPlugins);
-        }
-      }
     );
+
+    const jsonData = JSON.parse(manifestFile);
+    const uniquePlugins = filterUniquePlugins(jsonData);
+    const authenticatedPlugins = uniquePlugins.map((plugin) => {
+      if (isPluginAuthenticated(plugin)) {
+        return { ...plugin, authenticated: true };
+      } else {
+        return plugin;
+      }
+    });
+    const plugins = await addOpenAPISpecs(authenticatedPlugins);
+    res.status(200).json(plugins);
   } catch (error) {
     res.status(500).json({ message: error.message });
   }
 };
 
 module.exports = {
-  getAvailablePluginsController
+  getAvailablePluginsController,
 };

api/server/controllers/UserController.js

@@ -51,5 +51,5 @@ const updateUserPluginsController = async (req, res) => {
 
 module.exports = {
   getUserController,
-  updateUserPluginsController
+  updateUserPluginsController,
 };

api/server/controllers/auth/LoginController.js

@@ -2,28 +2,23 @@ const User = require('../../../models/User');
 
 const loginController = async (req, res) => {
   try {
-    const user = await User.findById(
-      req.user._id
-    );
+    const user = await User.findById(req.user._id);
 
     // If user doesn't exist, return error
-    if (!user) { // typeof user !== User) { // this doesn't seem to resolve the User type ??
+    if (!user) {
+      // typeof user !== User) { // this doesn't seem to resolve the User type ??
       return res.status(400).json({ message: 'Invalid credentials' });
     }
 
     const token = req.user.generateToken();
     const expires = eval(process.env.SESSION_EXPIRY);
-    
+
     // Add token to cookie
-    res.cookie(
-      'token',
-      token,
-      {
-        expires: new Date(Date.now() + expires),
-        httpOnly: false,
-        secure: process.env.NODE_ENV === 'production'
-      }
-    );
+    res.cookie('token', token, {
+      expires: new Date(Date.now() + expires),
+      httpOnly: false,
+      secure: process.env.NODE_ENV === 'production',
+    });
 
     return res.status(200).send({ token, user });
   } catch (err) {
@@ -35,5 +30,5 @@ const loginController = async (req, res) => {
 };
 
 module.exports = {
-  loginController
-};
+  loginController,
+};

api/server/controllers/auth/LogoutController.js

@@ -9,7 +9,6 @@ const logoutController = async (req, res) => {
     res.clearCookie('token');
     res.clearCookie('refreshToken');
     return res.status(status).send({ message });
-
   } catch (err) {
     console.log(err);
     return res.status(500).json({ message: err.message });
@@ -17,5 +16,5 @@ const logoutController = async (req, res) => {
 };
 
 module.exports = {
-  logoutController
-};
+  logoutController,
+};

api/server/index.js

@@ -24,14 +24,20 @@ config.validate(); // Validate the config
 
   const app = express();
   app.use(errorController);
-  app.use(express.json());
-  app.use(express.urlencoded({ extended: true }));
+  app.use(express.json({ limit: '3mb' }));
+  app.use(express.urlencoded({ extended: true, limit: '3mb' }));
   app.use(express.static(path.join(projectPath, 'dist')));
   app.use(express.static(path.join(projectPath, 'public')));
 
   app.set('trust proxy', 1); // trust first proxy
   app.use(cors());
 
+  if (!process.env.ALLOW_SOCIAL_LOGIN) {
+    console.warn(
+      'Social logins are disabled. Set Envrionment Variable "ALLOW_SOCIAL_LOGIN" to true to enable them.',
+    );
+  }
+
   // OAUTH
   app.use(passport.initialize());
   require('../strategies/jwtStrategy');
@@ -45,14 +51,23 @@ config.validate(); // Validate the config
   if (process.env.GITHUB_CLIENT_ID && process.env.GITHUB_CLIENT_SECRET) {
     require('../strategies/githubStrategy');
   }
-  if (process.env.OPENID_CLIENT_ID && process.env.OPENID_CLIENT_SECRET &&
-      process.env.OPENID_ISSUER && process.env.OPENID_SCOPE &&
-      process.env.OPENID_SESSION_SECRET) {
-    app.use(session({
-      secret: process.env.OPENID_SESSION_SECRET,
-      resave: false,
-      saveUninitialized: false
-    }));
+  if (process.env.DISCORD_CLIENT_ID && process.env.DISCORD_CLIENT_SECRET) {
+    require('../strategies/discordStrategy');
+  }
+  if (
+    process.env.OPENID_CLIENT_ID &&
+    process.env.OPENID_CLIENT_SECRET &&
+    process.env.OPENID_ISSUER &&
+    process.env.OPENID_SCOPE &&
+    process.env.OPENID_SESSION_SECRET
+  ) {
+    app.use(
+      session({
+        secret: process.env.OPENID_SESSION_SECRET,
+        resave: false,
+        saveUninitialized: false,
+      }),
+    );
     app.use(passport.session());
     require('../strategies/openidStrategy');
   }
@@ -77,12 +92,13 @@ config.validate(); // Validate the config
   });
 
   app.listen(port, host, () => {
-    if (host == '0.0.0.0')
+    if (host == '0.0.0.0') {
       console.log(
-        `Server listening on all interface at port ${port}. Use http://localhost:${port} to access it`
+        `Server listening on all interface at port ${port}. Use http://localhost:${port} to access it`,
       );
-    else
+    } else {
       console.log(`Server listening at http://${host == '0.0.0.0' ? 'localhost' : host}:${port}`);
+    }
   });
 })();
 

api/server/routes/__tests__/config.spec.js

@@ -16,8 +16,11 @@ afterEach(() => {
   delete process.env.OPENID_AUTH_URL;
   delete process.env.GITHUB_CLIENT_ID;
   delete process.env.GITHUB_CLIENT_SECRET;
+  delete process.env.DISCORD_CLIENT_ID;
+  delete process.env.DISCORD_CLIENT_SECRET;
   delete process.env.DOMAIN_SERVER;
   delete process.env.ALLOW_REGISTRATION;
+  delete process.env.ALLOW_SOCIAL_LOGIN;
 });
 
 //TODO: This works/passes locally but http request tests fail with 404 in CI. Need to figure out why.
@@ -28,16 +31,19 @@ describe.skip('GET /', () => {
     process.env.APP_TITLE = 'Test Title';
     process.env.GOOGLE_CLIENT_ID = 'Test Google Client Id';
     process.env.GOOGLE_CLIENT_SECRET = 'Test Google Client Secret';
-    process.env.OPENID_CLIENT_ID= 'Test OpenID Id';
-    process.env.OPENID_CLIENT_SECRET= 'Test OpenID Secret';
-    process.env.OPENID_ISSUER= 'Test OpenID Issuer';
-    process.env.OPENID_SESSION_SECRET= 'Test Secret';
-    process.env.OPENID_BUTTON_LABEL= 'Test OpenID';
-    process.env.OPENID_AUTH_URL= 'http://test-server.com';
+    process.env.OPENID_CLIENT_ID = 'Test OpenID Id';
+    process.env.OPENID_CLIENT_SECRET = 'Test OpenID Secret';
+    process.env.OPENID_ISSUER = 'Test OpenID Issuer';
+    process.env.OPENID_SESSION_SECRET = 'Test Secret';
+    process.env.OPENID_BUTTON_LABEL = 'Test OpenID';
+    process.env.OPENID_AUTH_URL = 'http://test-server.com';
     process.env.GITHUB_CLIENT_ID = 'Test Github client Id';
-    process.env.GITHUB_CLIENT_SECRET= 'Test Github client Secret';
+    process.env.GITHUB_CLIENT_SECRET = 'Test Github client Secret';
+    process.env.DISCORD_CLIENT_ID = 'Test Discord client Id';
+    process.env.DISCORD_CLIENT_SECRET = 'Test Discord client Secret';
     process.env.DOMAIN_SERVER = 'http://test-server.com';
     process.env.ALLOW_REGISTRATION = 'true';
+    process.env.ALLOW_SOCIAL_LOGIN = 'true';
 
     const response = await request(app).get('/');
 
@@ -49,8 +55,10 @@ describe.skip('GET /', () => {
       openidLabel: 'Test OpenID',
       openidImageUrl: 'http://test-server.com',
       githubLoginEnabled: true,
+      discordLoginEnabled: true,
       serverDomain: 'http://test-server.com',
       registrationEnabled: 'true',
+      socialLoginEnabled: 'true',
     });
   });
 });

api/server/routes/ask/addToCache.js

@@ -5,19 +5,19 @@ const addToCache = async ({ endpoint, endpointOption, userMessage, responseMessa
   try {
     const conversationsCache = new Keyv({
       store: new KeyvFile({ filename: './data/cache.json' }),
-      namespace: 'chatgpt' // should be 'bing' for bing/sydney
+      namespace: 'chatgpt', // should be 'bing' for bing/sydney
     });
 
     const {
       conversationId,
       messageId: userMessageId,
       parentMessageId: userParentMessageId,
-      text: userText
+      text: userText,
     } = userMessage;
     const {
       messageId: responseMessageId,
       parentMessageId: responseParentMessageId,
-      text: responseText
+      text: responseText,
     } = responseMessage;
 
     let conversation = await conversationsCache.get(conversationId);
@@ -26,7 +26,7 @@ const addToCache = async ({ endpoint, endpointOption, userMessage, responseMessa
     if (!conversation) {
       conversation = {
         messages: [],
-        createdAt: Date.now()
+        createdAt: Date.now(),
       };
       // isNewConversation = true;
     }
@@ -43,14 +43,14 @@ const addToCache = async ({ endpoint, endpointOption, userMessage, responseMessa
       id: userMessageId,
       parentMessageId: userParentMessageId,
       role: 'User',
-      message: userText
+      message: userText,
     };
 
     let _responseMessage = {
       id: responseMessageId,
       parentMessageId: responseParentMessageId,
       role: roles(endpointOption),
-      message: responseText
+      message: responseText,
     };
 
     conversation.messages.push(_userMessage, _responseMessage);

api/server/routes/ask/anthropic.js

@@ -0,0 +1,190 @@
+const express = require('express');
+const router = express.Router();
+const crypto = require('crypto');
+const { titleConvo, AnthropicClient } = require('../../../app');
+const requireJwtAuth = require('../../../middleware/requireJwtAuth');
+const { abortMessage } = require('../../../utils');
+const { saveMessage, getConvoTitle, saveConvo, getConvo } = require('../../../models');
+const { handleError, sendMessage, createOnProgress } = require('./handlers');
+
+const abortControllers = new Map();
+
+router.post('/abort', requireJwtAuth, async (req, res) => {
+  return await abortMessage(req, res, abortControllers);
+});
+
+router.post('/', requireJwtAuth, async (req, res) => {
+  const { endpoint, text, parentMessageId, conversationId: oldConversationId } = req.body;
+  if (text.length === 0) {
+    return handleError(res, { text: 'Prompt empty or too short' });
+  }
+  if (endpoint !== 'anthropic') {
+    return handleError(res, { text: 'Illegal request' });
+  }
+
+  const endpointOption = {
+    promptPrefix: req.body?.promptPrefix ?? null,
+    modelLabel: req.body?.modelLabel ?? null,
+    token: req.body?.token ?? null,
+    modelOptions: {
+      model: req.body?.model ?? 'claude-1',
+      temperature: req.body?.temperature ?? 0.7,
+      maxOutputTokens: req.body?.maxOutputTokens ?? 1024,
+      topP: req.body?.topP ?? 0.7,
+      topK: req.body?.topK ?? 40,
+    },
+  };
+
+  const conversationId = oldConversationId || crypto.randomUUID();
+
+  return await ask({
+    text,
+    endpointOption,
+    conversationId,
+    parentMessageId,
+    req,
+    res,
+  });
+});
+
+const ask = async ({ text, endpointOption, parentMessageId = null, conversationId, req, res }) => {
+  res.writeHead(200, {
+    Connection: 'keep-alive',
+    'Content-Type': 'text/event-stream',
+    'Cache-Control': 'no-cache, no-transform',
+    'Access-Control-Allow-Origin': '*',
+    'X-Accel-Buffering': 'no',
+  });
+
+  let userMessage;
+  let userMessageId;
+  let responseMessageId;
+  let lastSavedTimestamp = 0;
+  const { overrideParentMessageId = null } = req.body;
+
+  try {
+    const getIds = (data) => {
+      userMessage = data.userMessage;
+      userMessageId = data.userMessage.messageId;
+      responseMessageId = data.responseMessageId;
+      if (!conversationId) {
+        conversationId = data.conversationId;
+      }
+    };
+
+    const { onProgress: progressCallback, getPartialText } = createOnProgress({
+      onProgress: ({ text: partialText }) => {
+        const currentTimestamp = Date.now();
+        if (currentTimestamp - lastSavedTimestamp > 500) {
+          lastSavedTimestamp = currentTimestamp;
+          saveMessage({
+            messageId: responseMessageId,
+            sender: 'Anthropic',
+            conversationId,
+            parentMessageId: overrideParentMessageId || userMessageId,
+            text: partialText,
+            unfinished: true,
+            cancelled: false,
+            error: false,
+          });
+        }
+      },
+    });
+
+    const abortController = new AbortController();
+    abortController.abortAsk = async function () {
+      this.abort();
+
+      const responseMessage = {
+        messageId: responseMessageId,
+        sender: 'Anthropic',
+        conversationId,
+        parentMessageId: overrideParentMessageId || userMessageId,
+        text: getPartialText(),
+        model: endpointOption.modelOptions.model,
+        unfinished: false,
+        cancelled: true,
+        error: false,
+      };
+
+      saveMessage(responseMessage);
+
+      return {
+        title: await getConvoTitle(req.user.id, conversationId),
+        final: true,
+        conversation: await getConvo(req.user.id, conversationId),
+        requestMessage: userMessage,
+        responseMessage: responseMessage,
+      };
+    };
+
+    const onStart = (userMessage) => {
+      sendMessage(res, { message: userMessage, created: true });
+      abortControllers.set(userMessage.conversationId, { abortController, ...endpointOption });
+    };
+
+    const client = new AnthropicClient(endpointOption.token);
+
+    let response = await client.sendMessage(text, {
+      getIds,
+      debug: false,
+      user: req.user.id,
+      conversationId,
+      parentMessageId,
+      overrideParentMessageId,
+      ...endpointOption,
+      onProgress: progressCallback.call(null, {
+        res,
+        text,
+        parentMessageId: overrideParentMessageId || userMessageId,
+      }),
+      onStart,
+      abortController,
+    });
+
+    if (overrideParentMessageId) {
+      response.parentMessageId = overrideParentMessageId;
+    }
+
+    await saveConvo(req.user.id, {
+      ...endpointOption,
+      ...endpointOption.modelOptions,
+      conversationId,
+      endpoint: 'anthropic',
+    });
+
+    await saveMessage(response);
+    sendMessage(res, {
+      title: await getConvoTitle(req.user.id, conversationId),
+      final: true,
+      conversation: await getConvo(req.user.id, conversationId),
+      requestMessage: userMessage,
+      responseMessage: response,
+    });
+    res.end();
+
+    if (parentMessageId == '00000000-0000-0000-0000-000000000000') {
+      const title = await titleConvo({ text, response });
+      await saveConvo(req.user.id, {
+        conversationId,
+        title,
+      });
+    }
+  } catch (error) {
+    console.error(error);
+    const errorMessage = {
+      messageId: responseMessageId,
+      sender: 'Anthropic',
+      conversationId,
+      parentMessageId,
+      unfinished: false,
+      cancelled: false,
+      error: true,
+      text: error.message,
+    };
+    await saveMessage(errorMessage);
+    handleError(res, errorMessage);
+  }
+};
+
+module.exports = router;

api/server/routes/ask/askChatGPTBrowser.js

@@ -13,10 +13,14 @@ router.post('/', requireJwtAuth, async (req, res) => {
     text,
     overrideParentMessageId = null,
     parentMessageId,
-    conversationId: oldConversationId
+    conversationId: oldConversationId,
   } = req.body;
-  if (text.length === 0) return handleError(res, { text: 'Prompt empty or too short' });
-  if (endpoint !== 'chatGPTBrowser') return handleError(res, { text: 'Illegal request' });
+  if (text.length === 0) {
+    return handleError(res, { text: 'Prompt empty or too short' });
+  }
+  if (endpoint !== 'chatGPTBrowser') {
+    return handleError(res, { text: 'Illegal request' });
+  }
 
   // build user message
   const conversationId = oldConversationId || crypto.randomUUID();
@@ -29,13 +33,13 @@ router.post('/', requireJwtAuth, async (req, res) => {
     text,
     parentMessageId: userParentMessageId,
     conversationId,
-    isCreatedByUser: true
+    isCreatedByUser: true,
   };
 
   // build endpoint option
   const endpointOption = {
     model: req.body?.model ?? 'text-davinci-002-render-sha',
-    token: req.body?.token ?? null
+    token: req.body?.token ?? null,
   };
 
   // const availableModels = getChatGPTBrowserModels();
@@ -45,7 +49,7 @@ router.post('/', requireJwtAuth, async (req, res) => {
   console.log('ask log', {
     userMessage,
     endpointOption,
-    conversationId
+    conversationId,
   });
 
   if (!overrideParentMessageId) {
@@ -54,7 +58,7 @@ router.post('/', requireJwtAuth, async (req, res) => {
       ...userMessage,
       ...endpointOption,
       conversationId,
-      endpoint
+      endpoint,
     });
   }
 
@@ -67,7 +71,7 @@ router.post('/', requireJwtAuth, async (req, res) => {
     preSendRequest: true,
     overrideParentMessageId,
     req,
-    res
+    res,
   });
 });
 
@@ -78,7 +82,7 @@ const ask = async ({
   conversationId,
   overrideParentMessageId = null,
   req,
-  res
+  res,
 }) => {
   let { text, parentMessageId: userParentMessageId, messageId: userMessageId } = userMessage;
   const userId = req.user.id;
@@ -88,7 +92,7 @@ const ask = async ({
     'Content-Type': 'text/event-stream',
     'Cache-Control': 'no-cache, no-transform',
     'Access-Control-Allow-Origin': '*',
-    'X-Accel-Buffering': 'no'
+    'X-Accel-Buffering': 'no',
   });
 
   let responseMessageId = crypto.randomUUID();
@@ -108,10 +112,10 @@ const ask = async ({
             text: text,
             unfinished: true,
             cancelled: false,
-            error: false
+            error: false,
           });
         }
-      }
+      },
     });
 
     getPartialMessage = getPartialText;
@@ -134,9 +138,9 @@ const ask = async ({
 
         sendMessage(res, {
           message: { ...userMessage, conversationId: data.conversation_id },
-          created: true
+          created: true,
         });
-      }
+      },
     });
 
     console.log('CLIENT RESPONSE', response);
@@ -157,7 +161,7 @@ const ask = async ({
       sender: endpointOption?.chatGptLabel || 'ChatGPT',
       unfinished: false,
       cancelled: false,
-      error: false
+      error: false,
     };
 
     await saveMessage(responseMessage);
@@ -167,21 +171,22 @@ const ask = async ({
 
     // First update conversationId if needed
     let conversationUpdate = { conversationId: newConversationId, endpoint: 'chatGPTBrowser' };
-    if (conversationId != newConversationId)
+    if (conversationId != newConversationId) {
       if (isNewConversation) {
         // change the conversationId to new one
         conversationUpdate = {
           ...conversationUpdate,
           conversationId: conversationId,
-          newConversationId: newConversationId
+          newConversationId: newConversationId,
         };
       } else {
         // create new conversation
         conversationUpdate = {
           ...conversationUpdate,
-          ...endpointOption
+          ...endpointOption,
         };
       }
+    }
 
     await saveConvo(req.user.id, conversationUpdate);
     conversationId = newConversationId;
@@ -191,12 +196,13 @@ const ask = async ({
     userMessage.messageId = newUserMassageId;
 
     // If response has parentMessageId, the fake userMessage.messageId should be updated to the real one.
-    if (!overrideParentMessageId)
+    if (!overrideParentMessageId) {
       await saveMessage({
         ...userMessage,
         messageId: userMessageId,
-        newMessageId: newUserMassageId
+        newMessageId: newUserMassageId,
       });
+    }
     userMessageId = newUserMassageId;
 
     sendMessage(res, {
@@ -204,7 +210,7 @@ const ask = async ({
       final: true,
       conversation: await getConvo(req.user.id, conversationId),
       requestMessage: userMessage,
-      responseMessage: responseMessage
+      responseMessage: responseMessage,
     });
     res.end();
 
@@ -213,7 +219,7 @@ const ask = async ({
       const title = await response.details.title;
       await saveConvo(req.user.id, {
         conversationId: conversationId,
-        title
+        title,
       });
     }
   } catch (error) {
@@ -225,7 +231,7 @@ const ask = async ({
       unfinished: false,
       cancelled: false,
       // error: true,
-      text: `${getPartialMessage() ?? ''}\n\nError message: "${error.message}"`
+      text: `${getPartialMessage() ?? ''}\n\nError message: "${error.message}"`,
     };
     await saveMessage(errorMessage);
     handleError(res, errorMessage);

api/server/routes/ask/bingAI.js

@@ -1,7 +1,7 @@
 const express = require('express');
 const crypto = require('crypto');
 const router = express.Router();
-const { titleConvo, askBing } = require('../../../app');
+const { titleConvoBing, askBing } = require('../../../app');
 const { saveMessage, getConvoTitle, saveConvo, getConvo } = require('../../../models');
 const { handleError, sendMessage, createOnProgress, handleText } = require('./handlers');
 const requireJwtAuth = require('../../../middleware/requireJwtAuth');
@@ -13,10 +13,14 @@ router.post('/', requireJwtAuth, async (req, res) => {
     messageId,
     overrideParentMessageId = null,
     parentMessageId,
-    conversationId: oldConversationId
+    conversationId: oldConversationId,
   } = req.body;
-  if (text.length === 0) return handleError(res, { text: 'Prompt empty or too short' });
-  if (endpoint !== 'bingAI') return handleError(res, { text: 'Illegal request' });
+  if (text.length === 0) {
+    return handleError(res, { text: 'Prompt empty or too short' });
+  }
+  if (endpoint !== 'bingAI') {
+    return handleError(res, { text: 'Illegal request' });
+  }
 
   // build user message
   const conversationId = oldConversationId || crypto.randomUUID();
@@ -29,21 +33,21 @@ router.post('/', requireJwtAuth, async (req, res) => {
     text,
     parentMessageId: userParentMessageId,
     conversationId,
-    isCreatedByUser: true
+    isCreatedByUser: true,
   };
 
   // build endpoint option
   let endpointOption = {};
-  if (req.body?.jailbreak)
+  if (req.body?.jailbreak) {
     endpointOption = {
       jailbreak: req.body?.jailbreak ?? false,
       jailbreakConversationId: req.body?.jailbreakConversationId ?? null,
       systemMessage: req.body?.systemMessage ?? null,
       context: req.body?.context ?? null,
       toneStyle: req.body?.toneStyle ?? 'creative',
-      token: req.body?.token ?? null
+      token: req.body?.token ?? null,
     };
-  else
+  } else {
     endpointOption = {
       jailbreak: req.body?.jailbreak ?? false,
       systemMessage: req.body?.systemMessage ?? null,
@@ -52,13 +56,14 @@ router.post('/', requireJwtAuth, async (req, res) => {
       clientId: req.body?.clientId ?? null,
       invocationId: req.body?.invocationId ?? null,
       toneStyle: req.body?.toneStyle ?? 'creative',
-      token: req.body?.token ?? null
+      token: req.body?.token ?? null,
     };
+  }
 
   console.log('ask log', {
     userMessage,
     endpointOption,
-    conversationId
+    conversationId,
   });
 
   if (!overrideParentMessageId) {
@@ -67,7 +72,7 @@ router.post('/', requireJwtAuth, async (req, res) => {
       ...userMessage,
       ...endpointOption,
       conversationId,
-      endpoint
+      endpoint,
     });
   }
 
@@ -80,7 +85,7 @@ router.post('/', requireJwtAuth, async (req, res) => {
     preSendRequest: true,
     overrideParentMessageId,
     req,
-    res
+    res,
   });
 });
 
@@ -92,7 +97,7 @@ const ask = async ({
   preSendRequest = true,
   overrideParentMessageId = null,
   req,
-  res
+  res,
 }) => {
   let { text, parentMessageId: userParentMessageId, messageId: userMessageId } = userMessage;
 
@@ -103,37 +108,40 @@ const ask = async ({
     'Content-Type': 'text/event-stream',
     'Cache-Control': 'no-cache, no-transform',
     'Access-Control-Allow-Origin': '*',
-    'X-Accel-Buffering': 'no'
+    'X-Accel-Buffering': 'no',
   });
 
-  if (preSendRequest) sendMessage(res, { message: userMessage, created: true });
+  if (preSendRequest) {
+    sendMessage(res, { message: userMessage, created: true });
+  }
+
+  let lastSavedTimestamp = 0;
+  const { onProgress: progressCallback, getPartialText } = createOnProgress({
+    onProgress: ({ text }) => {
+      const currentTimestamp = Date.now();
+      if (currentTimestamp - lastSavedTimestamp > 500) {
+        lastSavedTimestamp = currentTimestamp;
+        saveMessage({
+          messageId: responseMessageId,
+          sender: endpointOption?.jailbreak ? 'Sydney' : 'BingAI',
+          conversationId,
+          parentMessageId: overrideParentMessageId || userMessageId,
+          text: text,
+          unfinished: true,
+          cancelled: false,
+          error: false,
+        });
+      }
+    },
+  });
+  const abortController = new AbortController();
+  let bingConversationId = null;
+  if (!isNewConversation) {
+    const convo = await getConvo(req.user.id, conversationId);
+    bingConversationId = convo.bingConversationId;
+  }
 
   try {
-    let lastSavedTimestamp = 0;
-    const { onProgress: progressCallback } = createOnProgress({
-      onProgress: ({ text }) => {
-        const currentTimestamp = Date.now();
-        if (currentTimestamp - lastSavedTimestamp > 500) {
-          lastSavedTimestamp = currentTimestamp;
-          saveMessage({
-            messageId: responseMessageId,
-            sender: endpointOption?.jailbreak ? 'Sydney' : 'BingAI',
-            conversationId,
-            parentMessageId: overrideParentMessageId || userMessageId,
-            text: text,
-            unfinished: true,
-            cancelled: false,
-            error: false
-          });
-        }
-      }
-    });
-    const abortController = new AbortController();
-    let bingConversationId = null;
-    if (!isNewConversation) {
-      const convo = await getConvo(req.user.id, conversationId);
-      bingConversationId = convo.bingConversationId;
-    }
     let response = await askBing({
       text,
       parentMessageId: userParentMessageId,
@@ -142,9 +150,9 @@ const ask = async ({
       onProgress: progressCallback.call(null, {
         res,
         text,
-        parentMessageId: overrideParentMessageId || userMessageId
+        parentMessageId: overrideParentMessageId || userMessageId,
       }),
-      abortController
+      abortController,
     });
 
     console.log('BING RESPONSE', response);
@@ -160,6 +168,13 @@ const ask = async ({
     response.text =
       response.response || response.details.spokenText || '**Bing refused to answer.**';
 
+    const partialText = getPartialText();
+    let unfinished = false;
+    if (partialText?.trim()?.length > response.text.length) {
+      response.text = partialText;
+      unfinished = true;
+    }
+
     let responseMessage = {
       conversationId,
       bingConversationId: newConversationId,
@@ -171,15 +186,19 @@ const ask = async ({
       suggestions:
         response.details.suggestedResponses &&
         response.details.suggestedResponses.map((s) => s.text),
-      unfinished: false,
+      unfinished,
       cancelled: false,
-      error: false
+      error: false,
     };
 
     await saveMessage(responseMessage);
     responseMessage.messageId = newResponseMessageId;
 
-    let conversationUpdate = { conversationId, bingConversationId: newConversationId, endpoint: 'bingAI' };
+    let conversationUpdate = {
+      conversationId,
+      bingConversationId: newConversationId,
+      endpoint: 'bingAI',
+    };
 
     if (endpointOption?.jailbreak) {
       conversationUpdate.jailbreak = true;
@@ -195,12 +214,13 @@ const ask = async ({
     userMessage.messageId = newUserMessageId;
 
     // If response has parentMessageId, the fake userMessage.messageId should be updated to the real one.
-    if (!overrideParentMessageId)
+    if (!overrideParentMessageId) {
       await saveMessage({
         ...userMessage,
         messageId: userMessageId,
-        newMessageId: newUserMessageId
+        newMessageId: newUserMessageId,
       });
+    }
     userMessageId = newUserMessageId;
 
     sendMessage(res, {
@@ -208,36 +228,61 @@ const ask = async ({
       final: true,
       conversation: await getConvo(req.user.id, conversationId),
       requestMessage: userMessage,
-      responseMessage: responseMessage
+      responseMessage: responseMessage,
     });
     res.end();
 
     if (userParentMessageId == '00000000-0000-0000-0000-000000000000') {
-      const title = await titleConvo({
-        endpoint: endpointOption?.endpoint,
+      const title = await titleConvoBing({
         text,
-        response: responseMessage
+        response: responseMessage,
       });
 
       await saveConvo(req.user.id, {
         conversationId: conversationId,
-        title
+        title,
       });
     }
   } catch (error) {
-    console.log(error);
-    const errorMessage = {
-      messageId: responseMessageId,
-      sender: endpointOption?.jailbreak ? 'Sydney' : 'BingAI',
-      conversationId,
-      parentMessageId: overrideParentMessageId || userMessageId,
-      unfinished: false,
-      cancelled: false,
-      error: true,
-      text: error.message
-    };
-    await saveMessage(errorMessage);
-    handleError(res, errorMessage);
+    console.error(error);
+    const partialText = getPartialText();
+    if (partialText?.length > 2) {
+      const responseMessage = {
+        messageId: responseMessageId,
+        sender: endpointOption?.jailbreak ? 'Sydney' : 'BingAI',
+        conversationId,
+        parentMessageId: overrideParentMessageId || userMessageId,
+        text: partialText,
+        model: endpointOption.modelOptions.model,
+        unfinished: true,
+        cancelled: false,
+        error: false,
+      };
+
+      saveMessage(responseMessage);
+
+      return {
+        title: await getConvoTitle(req.user.id, conversationId),
+        final: true,
+        conversation: await getConvo(req.user.id, conversationId),
+        requestMessage: userMessage,
+        responseMessage: responseMessage,
+      };
+    } else {
+      console.log(error);
+      const errorMessage = {
+        messageId: responseMessageId,
+        sender: endpointOption?.jailbreak ? 'Sydney' : 'BingAI',
+        conversationId,
+        parentMessageId: overrideParentMessageId || userMessageId,
+        unfinished: false,
+        cancelled: false,
+        error: true,
+        text: error.message,
+      };
+      await saveMessage(errorMessage);
+      handleError(res, errorMessage);
+    }
   }
 };
 

api/server/routes/ask/google.js

@@ -9,8 +9,12 @@ const requireJwtAuth = require('../../../middleware/requireJwtAuth');
 
 router.post('/', requireJwtAuth, async (req, res) => {
   const { endpoint, text, parentMessageId, conversationId: oldConversationId } = req.body;
-  if (text.length === 0) return handleError(res, { text: 'Prompt empty or too short' });
-  if (endpoint !== 'google') return handleError(res, { text: 'Illegal request' });
+  if (text.length === 0) {
+    return handleError(res, { text: 'Prompt empty or too short' });
+  }
+  if (endpoint !== 'google') {
+    return handleError(res, { text: 'Illegal request' });
+  }
 
   // build endpoint option
   const endpointOption = {
@@ -23,13 +27,13 @@ router.post('/', requireJwtAuth, async (req, res) => {
       temperature: req.body?.temperature ?? 0.2,
       maxOutputTokens: req.body?.maxOutputTokens ?? 1024,
       topP: req.body?.topP ?? 0.95,
-      topK: req.body?.topK ?? 40
-    }
+      topK: req.body?.topK ?? 40,
+    },
   };
 
   const availableModels = ['chat-bison', 'text-bison', 'codechat-bison'];
   if (availableModels.find((model) => model === endpointOption.modelOptions.model) === undefined) {
-    return handleError(res, { text: `Illegal request: model` });
+    return handleError(res, { text: 'Illegal request: model' });
   }
 
   const conversationId = oldConversationId || crypto.randomUUID();
@@ -41,7 +45,7 @@ router.post('/', requireJwtAuth, async (req, res) => {
     conversationId,
     parentMessageId,
     req,
-    res
+    res,
   });
 });
 
@@ -51,7 +55,7 @@ const ask = async ({ text, endpointOption, parentMessageId = null, conversationI
     'Content-Type': 'text/event-stream',
     'Cache-Control': 'no-cache, no-transform',
     'Access-Control-Allow-Origin': '*',
-    'X-Accel-Buffering': 'no'
+    'X-Accel-Buffering': 'no',
   });
   let userMessage;
   let userMessageId;
@@ -84,10 +88,10 @@ const ask = async ({ text, endpointOption, parentMessageId = null, conversationI
             text: partialText,
             unfinished: true,
             cancelled: false,
-            error: false
+            error: false,
           });
         }
-      }
+      },
     });
 
     const abortController = new AbortController();
@@ -104,14 +108,14 @@ const ask = async ({ text, endpointOption, parentMessageId = null, conversationI
         key = require('../../../data/auth.json');
       }
     } catch (e) {
-      console.log("No 'auth.json' file (service account key) found in /api/data/ for PaLM models");
+      console.log('No \'auth.json\' file (service account key) found in /api/data/ for PaLM models');
     }
 
     const clientOptions = {
       // debug: true, // for testing
       reverseProxyUrl: process.env.GOOGLE_REVERSE_PROXY || null,
       proxy: process.env.PROXY || null,
-      ...endpointOption
+      ...endpointOption,
     };
 
     const client = new GoogleClient(key, clientOptions);
@@ -125,9 +129,9 @@ const ask = async ({ text, endpointOption, parentMessageId = null, conversationI
       onProgress: progressCallback.call(null, {
         res,
         text,
-        parentMessageId: overrideParentMessageId || userMessageId
+        parentMessageId: overrideParentMessageId || userMessageId,
       }),
-      abortController
+      abortController,
     });
 
     if (overrideParentMessageId) {
@@ -138,7 +142,7 @@ const ask = async ({ text, endpointOption, parentMessageId = null, conversationI
       ...endpointOption,
       ...endpointOption.modelOptions,
       conversationId,
-      endpoint: 'google'
+      endpoint: 'google',
     });
 
     await saveMessage(response);
@@ -147,7 +151,7 @@ const ask = async ({ text, endpointOption, parentMessageId = null, conversationI
       final: true,
       conversation: await getConvo(req.user.id, conversationId),
       requestMessage: userMessage,
-      responseMessage: response
+      responseMessage: response,
     });
     res.end();
 
@@ -155,7 +159,7 @@ const ask = async ({ text, endpointOption, parentMessageId = null, conversationI
       const title = await titleConvo({ text, response });
       await saveConvo(req.user.id, {
         conversationId,
-        title
+        title,
       });
     }
   } catch (error) {
@@ -168,7 +172,7 @@ const ask = async ({ text, endpointOption, parentMessageId = null, conversationI
       unfinished: false,
       cancelled: false,
       error: true,
-      text: error.message
+      text: error.message,
     };
     await saveMessage(errorMessage);
     handleError(res, errorMessage);

api/server/routes/ask/gptPlugins.js

@@ -8,7 +8,7 @@ const {
   sendMessage,
   createOnProgress,
   formatSteps,
-  formatAction
+  formatAction,
 } = require('./handlers');
 const requireJwtAuth = require('../../../middleware/requireJwtAuth');
 
@@ -20,8 +20,12 @@ router.post('/abort', requireJwtAuth, async (req, res) => {
 
 router.post('/', requireJwtAuth, async (req, res) => {
   const { endpoint, text, parentMessageId, conversationId } = req.body;
-  if (text.length === 0) return handleError(res, { text: 'Prompt empty or too short' });
-  if (endpoint !== 'gptPlugins') return handleError(res, { text: 'Illegal request' });
+  if (text.length === 0) {
+    return handleError(res, { text: 'Prompt empty or too short' });
+  }
+  if (endpoint !== 'gptPlugins') {
+    return handleError(res, { text: 'Illegal request' });
+  }
 
   const agentOptions = req.body?.agentOptions ?? {
     agent: 'functions',
@@ -44,12 +48,12 @@ router.post('/', requireJwtAuth, async (req, res) => {
       temperature: req.body?.temperature ?? 0,
       top_p: req.body?.top_p ?? 1,
       presence_penalty: req.body?.presence_penalty ?? 0,
-      frequency_penalty: req.body?.frequency_penalty ?? 0
+      frequency_penalty: req.body?.frequency_penalty ?? 0,
     },
     agentOptions: {
       ...agentOptions,
       // agent: 'functions'
-    }
+    },
   };
 
   console.log('ask log');
@@ -63,17 +67,25 @@ router.post('/', requireJwtAuth, async (req, res) => {
     conversationId,
     parentMessageId,
     req,
-    res
+    res,
   });
 });
 
-const ask = async ({ text, endpoint, endpointOption, parentMessageId = null, conversationId, req, res }) => {
+const ask = async ({
+  text,
+  endpoint,
+  endpointOption,
+  parentMessageId = null,
+  conversationId,
+  req,
+  res,
+}) => {
   res.writeHead(200, {
     Connection: 'keep-alive',
     'Content-Type': 'text/event-stream',
     'Cache-Control': 'no-cache, no-transform',
     'Access-Control-Allow-Origin': '*',
-    'X-Accel-Buffering': 'no'
+    'X-Accel-Buffering': 'no',
   });
   let userMessage;
   let userMessageId;
@@ -87,7 +99,7 @@ const ask = async ({ text, endpoint, endpointOption, parentMessageId = null, con
     loading: true,
     inputs: [],
     latest: null,
-    outputs: null
+    outputs: null,
   };
 
   try {
@@ -100,7 +112,11 @@ const ask = async ({ text, endpoint, endpointOption, parentMessageId = null, con
       }
     };
 
-    const { onProgress: progressCallback, sendIntermediateMessage, getPartialText } = createOnProgress({
+    const {
+      onProgress: progressCallback,
+      sendIntermediateMessage,
+      getPartialText,
+    } = createOnProgress({
       onProgress: ({ text: partialText }) => {
         const currentTimestamp = Date.now();
 
@@ -117,12 +133,12 @@ const ask = async ({ text, endpoint, endpointOption, parentMessageId = null, con
             parentMessageId: overrideParentMessageId || userMessageId,
             text: partialText,
             model: endpointOption.modelOptions.model,
-            unfinished: false,
-            cancelled: true,
-            error: false
+            unfinished: true,
+            cancelled: false,
+            error: false,
           });
         }
-      }
+      },
     });
 
     const abortController = new AbortController();
@@ -149,14 +165,14 @@ const ask = async ({ text, endpoint, endpointOption, parentMessageId = null, con
         final: true,
         conversation: await getConvo(req.user.id, conversationId),
         requestMessage: userMessage,
-        responseMessage: responseMessage
+        responseMessage: responseMessage,
       };
     };
 
     const onStart = (userMessage) => {
       sendMessage(res, { message: userMessage, created: true });
       abortControllers.set(userMessage.conversationId, { abortController, ...endpointOption });
-    }
+    };
 
     endpointOption.tools = await validateTools(user, endpointOption.tools);
     const clientOptions = {
@@ -164,26 +180,28 @@ const ask = async ({ text, endpoint, endpointOption, parentMessageId = null, con
       endpoint,
       reverseProxyUrl: process.env.OPENAI_REVERSE_PROXY || null,
       proxy: process.env.PROXY || null,
-      ...endpointOption
+      ...endpointOption,
     };
 
-    let oaiApiKey = req.body?.token ?? process.env.OPENAI_API_KEY;
+    let openAIApiKey = req.body?.token ?? process.env.OPENAI_API_KEY;
     if (process.env.PLUGINS_USE_AZURE) {
       clientOptions.azure = getAzureCredentials();
-      oaiApiKey = clientOptions.azure.azureOpenAIApiKey;
+      openAIApiKey = clientOptions.azure.azureOpenAIApiKey;
     }
 
-    if (oaiApiKey && oaiApiKey.includes('azure') && !clientOptions.azure) {
+    if (openAIApiKey && openAIApiKey.includes('azure') && !clientOptions.azure) {
       clientOptions.azure = JSON.parse(req.body?.token) ?? getAzureCredentials();
-      oaiApiKey = clientOptions.azure.azureOpenAIApiKey;
+      openAIApiKey = clientOptions.azure.azureOpenAIApiKey;
     }
-    const chatAgent = new PluginsClient(oaiApiKey, clientOptions);
+    const chatAgent = new PluginsClient(openAIApiKey, clientOptions);
 
-    const onAgentAction = (action) => {
+    const onAgentAction = (action, start = false) => {
       const formattedAction = formatAction(action);
       plugin.inputs.push(formattedAction);
       plugin.latest = formattedAction.plugin;
-      saveMessage(userMessage);
+      if (!start) {
+        saveMessage(userMessage);
+      }
       sendIntermediateMessage(res, { plugin });
       // console.log('PLUGIN ACTION', formattedAction);
     };
@@ -211,9 +229,9 @@ const ask = async ({ text, endpoint, endpointOption, parentMessageId = null, con
         res,
         text,
         plugin,
-        parentMessageId: overrideParentMessageId || userMessageId
+        parentMessageId: overrideParentMessageId || userMessageId,
       }),
-      abortController
+      abortController,
     });
 
     if (overrideParentMessageId) {
@@ -230,15 +248,20 @@ const ask = async ({ text, endpoint, endpointOption, parentMessageId = null, con
       final: true,
       conversation: await getConvo(req.user.id, conversationId),
       requestMessage: userMessage,
-      responseMessage: response
+      responseMessage: response,
     });
     res.end();
 
     if (parentMessageId == '00000000-0000-0000-0000-000000000000' && newConvo) {
-      const title = await titleConvo({ text, response });
+      const title = await titleConvo({
+        text,
+        response,
+        openAIApiKey,
+        azure: !!clientOptions.azure,
+      });
       await saveConvo(req.user.id, {
         conversationId: conversationId,
-        title
+        title,
       });
     }
   } catch (error) {
@@ -251,7 +274,7 @@ const ask = async ({ text, endpoint, endpointOption, parentMessageId = null, con
       unfinished: false,
       cancelled: false,
       error: true,
-      text: error.message
+      text: error.message,
     };
     await saveMessage(errorMessage);
     handleError(res, errorMessage);

api/server/routes/ask/handlers.js

@@ -61,7 +61,12 @@ const createOnProgress = ({ onProgress: _onProgress }) => {
   };
 
   const sendIntermediateMessage = (res, payload) => {
-    sendMessage(res, { text: tokens?.length === 0 ? cursor : tokens, message: true, initial: i === 0, ...payload });
+    sendMessage(res, {
+      text: tokens?.length === 0 ? cursor : tokens,
+      message: true,
+      initial: i === 0,
+      ...payload,
+    });
     i++;
   };
 
@@ -92,7 +97,7 @@ const handleText = async (response, bing = false) => {
 };
 
 const isObject = (item) => item && typeof item === 'object' && !Array.isArray(item);
-const getString = (input) => isObject(input) ? JSON.stringify(input) : input ;
+const getString = (input) => (isObject(input) ? JSON.stringify(input) : input);
 
 function formatSteps(steps) {
   let output = '';
@@ -117,24 +122,12 @@ function formatSteps(steps) {
 }
 
 function formatAction(action) {
-  const capitalizeWords = (input) => {
-    if (input === 'dall-e') {
-      return 'DALL-E';
-    }
-
-    return input
-      .replace(/-/g, ' ')
-      .split(' ')
-      .map((word) => word.charAt(0).toUpperCase() + word.slice(1))
-      .join(' ');
-  };
-
   const formattedAction = {
-    plugin: capitalizeWords(action.tool) || action.tool,
+    plugin: action.tool,
     input: getString(action.toolInput),
     thought: action.log.includes('Thought: ')
       ? action.log.split('\n')[0].replace('Thought: ', '')
-      : action.log.split('\n')[0]
+      : action.log.split('\n')[0],
   };
 
   formattedAction.thought = getString(formattedAction.thought);
@@ -161,5 +154,5 @@ module.exports = {
   createOnProgress,
   handleText,
   formatSteps,
-  formatAction
-};
+  formatAction,
+};

api/server/routes/ask/index.js

@@ -4,15 +4,17 @@ const router = express.Router();
 // const askOpenAI = require('./askOpenAI');
 const openAI = require('./openAI');
 const google = require('./google');
-const askBingAI = require('./askBingAI');
+const bingAI = require('./bingAI');
 const gptPlugins = require('./gptPlugins');
 const askChatGPTBrowser = require('./askChatGPTBrowser');
+const anthropic = require('./anthropic');
 
 // router.use('/azureOpenAI', askAzureOpenAI);
 router.use(['/azureOpenAI', '/openAI'], openAI);
 router.use('/google', google);
-router.use('/bingAI', askBingAI);
+router.use('/bingAI', bingAI);
 router.use('/chatGPTBrowser', askChatGPTBrowser);
 router.use('/gptPlugins', gptPlugins);
+router.use('/anthropic', anthropic);
 
 module.exports = router;