feat: Update Azure Document Intelligence upload function to return Markdown result

* feat: add azure document intelligence ocr support

* Fix mistral AI OCR

* ⌛ feat: Agent Version History and Management (#7455)

* ✨ feat: Enhance agent update functionality to save current state in versions array

- Updated the `updateAgent` function to push the current agent's state into a new `versions` array when an agent is updated.
- Modified the agent schema to include a `versions` field for storing historical states of agents.

* ✨ feat: Add comprehensive CRUD operations for agents in tests

- Introduced a new test suite for CRUD operations on agents, including create, read, update, and delete functionalities.
- Implemented tests for listing agents by author and updating agent projects.
- Enhanced the agent model to support version history tracking during updates.
- Ensured proper environment variable management during tests.

* ✨ feat: Introduce version tracking for agents and enhance UI components

- Added a `version` property to the agent model to track the number of versions.
- Updated the `getAgentHandler` to include the agent's version in the response.
- Introduced a new `VersionButton` component for navigating to the version panel.
- Created a `VersionPanel` component for displaying version-related information.
- Updated the UI to conditionally render the version button and panel based on the active state.
- Added localization for the new version-related UI elements.

* ✨ i18n: Add "version" translation key across multiple languages

- Introduced the "com_ui_agent_version" translation key in various language files to support version tracking for agents.
- Updated Arabic, Czech, German, English, Spanish, Estonian, Persian, Finnish, French, Hebrew, Hungarian, Indonesian, Italian, Japanese, Korean, Dutch, Polish, Portuguese (Brazil and Portugal), Russian, Swedish, Thai, Turkish, Vietnamese, and Chinese (Simplified and Traditional) translations.

* ✨ feat: Update AgentFooter to conditionally render AdminSettings

- Modified the logic for displaying buttons in the AgentFooter component to only show them when the active panel is the builder.
- Ensured that AdminSettings is displayed only when the user has an admin role and the buttons are visible.

* ✨ feat: Enhance AgentPanelSwitch and VersionPanel for improved agent capabilities

- Updated AgentPanelSwitch to include a new VersionPanel for displaying version-related information.
- Enhanced agentsConfig logic to properly handle agent capabilities.
- Modified VersionPanel to improve structure and localization support.
- Integrated createAgent mutation for future agent creation functionality.

* ✨ feat: Enhance VersionPanel to display agent version history and loading states

- Integrated version fetching logic in VersionPanel to retrieve and display agent version history.
- Added loading and error handling states to improve user experience.
- Updated agent schema to use mixed types for versions, allowing for more flexible version data structures.
- Introduced localization support for version-related UI elements.

* ✨ feat: Update VersionPanel and AgentPanelSwitch to enhance agent selection and version display

- Modified AgentPanelSwitch to pass selectedAgentId to VersionPanel for improved agent context.
- Enhanced VersionPanel to handle multiple timestamp formats and display appropriate messages when no agent is selected.
- Improved structure and readability of the VersionPanel component by adding a helper function for timestamp retrieval.

* ✨ feat: Refactor VersionPanel to utilize localization and improve timestamp handling

- Replaced hardcoded text constants with localization support for various UI elements in VersionPanel.
- Enhanced the timestamp retrieval function to handle errors gracefully and utilize localized messages for unknown dates.
- Improved user feedback by displaying localized messages for agent selection, version errors, and empty states.

* ✨ refactor: Clean up VersionPanel by removing unused code and improving timestamp handling

* ✨ feat: Implement agent version reverting functionality

- Added `revertAgentVersion` method in the Agent model to allow reverting to a previous version of an agent.
- Introduced `revertAgentVersionHandler` in the agents controller to handle requests for reverting agent versions.
- Updated API routes to include a new endpoint for reverting agent versions.
- Enhanced the VersionPanel component to support version restoration with user confirmation and feedback.
- Added localization support for success and error messages related to version restoration.

* ✨ i18n: Add localization for agent version restoration messages

* Simplify VersionPanel by removing unused parameters and enhancing agent ID handling

* Refactor Agent model and VersionPanel component to streamline version data handling

* Update version handling in Agent model and VersionPanel

- Enhanced the Agent model to include an `updatedAt` timestamp when pushing new versions.
- Improved the VersionPanel component to sort versions by the `updatedAt` timestamp for better display order.
- Added a new localization entry for indicating the active version of an agent.

* ✨ i18n: Add localization for active agent version across multiple languages

* ✨ feat: Introduce version management components for agent history

- Added `isActiveVersion` utility to determine the active version of an agent based on various criteria.
- Implemented `VersionContent` and `VersionItem` components to display agent version history, including loading and error states.
- Enhanced `VersionPanel` to integrate new components and manage version context effectively.
- Added comprehensive tests for version management functionalities to ensure reliability and correctness.

* Add unit tests for AgentFooter component

* cleanup

* Enhance agent version update handling and add unit tests for update operators

- Updated the `updateAgent` function to properly handle various update operators ($push, $pull, $addToSet) while maintaining version history.
- Modified unit tests to validate the correct behavior of agent updates, including versioning and tool management.

* Enhance version comparison logic and update tests for artifacts handling

- Modified the `isActiveVersion` utility to include artifacts in the version comparison criteria.
- Updated the `VersionPanel` component to support artifacts in the agent state.
- Added new unit tests to validate artifacts matching scenarios and edge cases in the `isActiveVersion` function.

* Implement duplicate version detection in agent updates and enhance error handling

- Added `isDuplicateVersion` function to check for identical versions during agent updates, excluding certain fields.
- Updated `updateAgent` function to throw an error if a duplicate version is detected, with detailed error information.
- Enhanced the `updateAgentHandler` to return appropriate responses for duplicate version errors.
- Modified client-side error handling to display user-friendly messages for duplicate version scenarios.
- Added comprehensive unit tests to validate duplicate version detection and error handling across various update scenarios.

* Update version title localization to include version number across multiple languages

- Modified the `com_ui_agent_version_title` translation key to include a placeholder for the version number in various language files.
- Enhanced the `VersionItem` component to utilize the updated localization for displaying version titles dynamically.

* Enhance agent version handling and add revert functionality

- Updated the `isDuplicateVersion` function to improve version comparison logic, including special handling for `projectIds` and arrays of objects.
- Modified the `updateAgent` function to streamline version updates and removed unnecessary checks for test environments.
- Introduced a new `revertAgentVersion` function to allow reverting agents to specific versions, with detailed documentation.
- Enhanced unit tests to validate duplicate version detection and revert functionality, ensuring robust error handling and version management.

* fix CI issues

* cleanup

* Revert all non-English translations

* clean up tests

---------

Co-authored-by: matt burnett <mawburn@users.noreply.github.com>
Co-authored-by: Ruben Talstra <RubenTalstra1211@outlook.com>

api/server/services/Files/AzureDocumentIntelligence/crud.js

@@ -0,0 +1,74 @@
+const axios = require('axios');
+const fs = require('fs');
+const { logger } = require('~/config');
+
+/**
+ * Uploads a document to Azure Document Intelligence API and returns the Markdown result.
+ *
+ * @param {Object} params - The parameters for the Azure Document Intelligence request.
+ * @param {string} params.filePath - The path to the file on disk.
+ * @param {string} params.apiKey - Azure API key.
+ * @param {string} params.endpoint - Azure Document Intelligence endpoint.
+ * @param {string} params.modelId - The model ID to use for analysis.
+ * @returns {Promise<Object>} - The Document Intelligence result.
+ */
+async function uploadAzureDocumentIntelligence({ filePath, apiKey, endpoint, modelId }) {
+  // Read and encode file
+  const fileBuffer = fs.readFileSync(filePath);
+  const base64Source = fileBuffer.toString('base64');
+
+  // Build URL (ensure no trailing slash on endpoint)
+  const url = `${endpoint.replace(/\/+$/, '')}/documentModels/${modelId}:analyze?outputContentFormat=markdown`;
+
+  try {
+    // Kick off the analysis
+    const response = await axios.post(
+      url,
+      { base64Source },
+      {
+        headers: {
+          'Ocp-Apim-Subscription-Key': apiKey,
+          'Content-Type': 'application/json',
+        },
+      },
+    );
+
+    // Axios lower-cases header keys, but allow either form
+    const headers = response.headers || {};
+    const operationLocation = headers['operation-location'] || headers['Operation-Location'];
+    if (!operationLocation) {
+      throw new Error('Missing Operation-Location header in Azure response.');
+    }
+
+    // Poll until done
+    let resultContent;
+    while (true) {
+      const pollResponse = await axios.get(operationLocation, {
+        headers: { 'Ocp-Apim-Subscription-Key': apiKey },
+      });
+
+      const { status, resultUrl } = pollResponse.data;
+      if (status === 'succeeded') {
+        const final = await axios.get(resultUrl, {
+          headers: { 'Ocp-Apim-Subscription-Key': apiKey },
+        });
+        resultContent = final.data.analyzeResult.content;
+        break;
+      }
+      if (status === 'failed') {
+        throw new Error('Azure Document Intelligence processing failed.');
+      }
+      // Wait 2s before retry
+      await new Promise((r) => setTimeout(r, 2000));
+    }
+
+    return resultContent;
+  } catch (error) {
+    logger.error('Error performing Azure Document Intelligence:', error.message);
+    throw error;
+  }
+}
+
+module.exports = {
+  uploadAzureDocumentIntelligence,
+};

api/server/services/Files/AzureDocumentIntelligence/crud.spec.js

@@ -0,0 +1,103 @@
+const fs = require('fs');
+
+const mockAxios = {
+  interceptors: {
+    request: { use: jest.fn(), eject: jest.fn() },
+    response: { use: jest.fn(), eject: jest.fn() },
+  },
+  create: jest.fn().mockReturnValue({
+    defaults: { proxy: null },
+    get: jest.fn().mockResolvedValue({ data: {} }),
+    post: jest.fn().mockResolvedValue({ data: {} }),
+    put: jest.fn().mockResolvedValue({ data: {} }),
+    delete: jest.fn().mockResolvedValue({ data: {} }),
+  }),
+  get: jest.fn().mockResolvedValue({ data: {} }),
+  post: jest.fn().mockResolvedValue({ data: {} }),
+  put: jest.fn().mockResolvedValue({ data: {} }),
+  delete: jest.fn().mockResolvedValue({ data: {} }),
+  reset: jest.fn().mockImplementation(function () {
+    this.get.mockClear();
+    this.post.mockClear();
+    this.put.mockClear();
+    this.delete.mockClear();
+    this.create.mockClear();
+  }),
+};
+
+jest.mock('axios', () => mockAxios);
+jest.mock('fs');
+jest.mock('~/config', () => ({
+  logger: { error: jest.fn() },
+}));
+
+const { uploadAzureDocumentIntelligence } = require('./crud');
+
+describe('AzureDocumentIntelligence Service', () => {
+  beforeEach(() => {
+    mockAxios.reset();
+    fs.readFileSync.mockReset();
+  });
+
+  it('should upload and poll until it gets the Markdown result', async () => {
+    const mockFileBuffer = Buffer.from('test file content');
+    const mockBase64 = mockFileBuffer.toString('base64');
+    const mockOpLocation = 'https://azure-ocr-endpoint.com/operations/123';
+    const mockResultUrl = 'https://azure-ocr-endpoint.com/results/123';
+    const mockFinal = { analyzeResult: { content: 'Final analysis result' } };
+
+    // fs.readFileSync returns our buffer
+    fs.readFileSync.mockReturnValue(mockFileBuffer);
+
+    // First axios.post => returns Operation-Location header
+    mockAxios.post.mockResolvedValueOnce({
+      headers: { 'Operation-Location': mockOpLocation },
+    });
+
+    // First axios.get => poll success, returns status + resultUrl
+    // Second axios.get => fetch final result
+    mockAxios.get
+      .mockResolvedValueOnce({ data: { status: 'succeeded', resultUrl: mockResultUrl } })
+      .mockResolvedValueOnce({ data: mockFinal });
+
+    const result = await uploadAzureDocumentIntelligence({
+      filePath: '/path/to/test.pdf',
+      apiKey: 'azure-api-key',
+      endpoint: 'https://azure-ocr-endpoint.com/',
+      modelId: 'prebuilt-layout',
+    });
+
+    // Validate read
+    expect(fs.readFileSync).toHaveBeenCalledWith('/path/to/test.pdf');
+
+    // Validate initial POST
+    expect(mockAxios.post).toHaveBeenCalledWith(
+      'https://azure-ocr-endpoint.com/documentModels/prebuilt-layout:analyze?outputContentFormat=markdown',
+      { base64Source: mockBase64 },
+      expect.objectContaining({
+        headers: expect.objectContaining({
+          'Ocp-Apim-Subscription-Key': 'azure-api-key',
+          'Content-Type': 'application/json',
+        }),
+      }),
+    );
+
+    // Validate polling GET
+    expect(mockAxios.get).toHaveBeenCalledWith(
+      mockOpLocation,
+      expect.objectContaining({
+        headers: expect.objectContaining({ 'Ocp-Apim-Subscription-Key': 'azure-api-key' }),
+      }),
+    );
+
+    // Validate final fetch GET
+    expect(mockAxios.get).toHaveBeenCalledWith(
+      mockResultUrl,
+      expect.objectContaining({
+        headers: expect.objectContaining({ 'Ocp-Apim-Subscription-Key': 'azure-api-key' }),
+      }),
+    );
+
+    expect(result).toEqual('Final analysis result');
+  });
+});

api/server/services/Files/AzureDocumentIntelligence/index.js

@@ -0,0 +1,5 @@
+const crud = require('./crud');
+
+module.exports = {
+  ...crud,
+};

api/server/services/Files/strategies.js

@@ -47,6 +47,7 @@ const { uploadOpenAIFile, deleteOpenAIFile, getOpenAIFileStream } = require('./O
 const { getCodeOutputDownloadStream, uploadCodeEnvFile } = require('./Code');
 const { uploadVectors, deleteVectors } = require('./VectorDB');
 const { uploadMistralOCR } = require('./MistralOCR');
+const { uploadAzureDocumentIntelligence } = require('./AzureDocumentIntelligence'); // Import the function
 
 /**
  * Firebase Storage Strategy Functions
@@ -202,6 +203,26 @@ const mistralOCRStrategy = () => ({
   handleFileUpload: uploadMistralOCR,
 });
 
+const azureOCRStrategy = () => ({
+  /** @type {typeof saveFileFromURL | null} */
+  saveURL: null,
+  /** @type {typeof saveFileFromURL | null} */
+  getFileURL: null,
+  /** @type {typeof saveFileFromURL | null} */
+  saveBuffer: null,
+  /** @type {typeof saveFileFromURL | null} */
+  processAvatar: null,
+  /** @type {typeof saveFileFromURL | null} */
+  handleImageUpload: null,
+  /** @type {typeof saveFileFromURL | null} */
+  prepareImagePayload: null,
+  /** @type {typeof saveFileFromURL | null} */
+  deleteFile: null,
+  handleFileUpload: uploadAzureDocumentIntelligence,
+  /** @type {typeof saveFileFromURL | null} */
+  getDownloadStream: null,
+});
+
 // Strategy Selector
 const getStrategyFunctions = (fileSource) => {
   if (fileSource === FileSources.firebase) {
@@ -222,6 +243,8 @@ const getStrategyFunctions = (fileSource) => {
     return codeOutputStrategy();
   } else if (fileSource === FileSources.mistral_ocr) {
     return mistralOCRStrategy();
+  } else if (fileSource === FileSources.azure_ocr) {
+    return azureOCRStrategy();
   } else {
     throw new Error('Invalid file source');
   }

client/src/locales/en/translation.json

@@ -926,4 +926,4 @@
   "com_ui_zoom": "Zoom",
   "com_user_message": "You",
   "com_warning_resubmit_unsupported": "Resubmitting the AI message is not supported for this endpoint."
-}
+}

packages/data-provider/src/config.ts

@@ -581,6 +581,7 @@ export type TStartupConfig = {
 export enum OCRStrategy {
   MISTRAL_OCR = 'mistral_ocr',
   CUSTOM_OCR = 'custom_ocr',
+  AZURE_DOCUMENT_INTELLIGENCE = 'azure_document_intelligence',
 }
 
 export enum SearchCategories {
@@ -627,6 +628,7 @@ export type TWebSearchConfig = z.infer<typeof webSearchSchema>;
 
 export const ocrSchema = z.object({
   mistralModel: z.string().optional(),
+  documentIntelligenceModel: z.string().optional(),
   apiKey: z.string().optional().default('${OCR_API_KEY}'),
   baseURL: z.string().optional().default('${OCR_BASEURL}'),
   strategy: z.nativeEnum(OCRStrategy).default(OCRStrategy.MISTRAL_OCR),

packages/data-provider/src/ocr.ts

@@ -4,11 +4,14 @@ import { OCRStrategy } from '../src/config';
 export function loadOCRConfig(config: TCustomConfig['ocr']): TCustomConfig['ocr'] {
   const baseURL = config?.baseURL ?? '';
   const apiKey = config?.apiKey ?? '';
-  const mistralModel = config?.mistralModel ?? '';
+  const mistralModel = config?.mistralModel ?? '';  
+  const documentIntelligenceModel = config?.documentIntelligenceModel ?? '';
+  
   return {
     apiKey,
     baseURL,
     mistralModel,
+    documentIntelligenceModel,
     strategy: config?.strategy ?? OCRStrategy.MISTRAL_OCR,
   };
 }

packages/data-provider/src/types/files.ts

@@ -10,6 +10,7 @@ export enum FileSources {
   vectordb = 'vectordb',
   execute_code = 'execute_code',
   mistral_ocr = 'mistral_ocr',
+  azure_ocr = 'azure_ocr',
   text = 'text',
 }