Assets API

The Assets API provides file management capabilities including upload, storage, conversion, and embedding functionality. It supports various document formats and integrates with vector storage for content processing.

Supported File Types

File Format Support

Format	Max Size	Auto-Convert
PDF	100MB	No
DOCX	100MB	No
XLSX	100MB	To CSV
PPTX	100MB	No
CSV	100MB	No
TXT	100MB	No
MD	100MB	No
HTML	100MB	To Text

Endpoints

Upload File

POST /assets/upload
Content-Type: multipart/form-data

Upload a new file for processing and storage.

Request

// Form data
interface UploadRequest {
  file: File; // File object
}

// File validation
const validations = {
  maxSize: 100 * 1024 * 1024, // 100MB
  fileTypes: [
    'application/pdf',
    'text/plain',
    'text/csv',
    'text/markdown',
    'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
    'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
    'application/vnd.openxmlformats-officedocument.presentationml.presentation',
  ],
};

Response

interface AssetResponse {
  id: string;
  originalFileUrl: string;
  originalFileName: string;
  convertedFileUrl?: string;
  convertedFileName?: string;
  status: 'UPLOADED' | 'PROCESSING' | 'COMPLETED';
  containerName: string;
  organizationId: string;
  dpoUserId: string;
}

Find Asset

GET /assets/:id

Retrieve information about a specific asset.

Parameters

Name	Type	In	Description
id	string	path	The asset ID

Response

interface Asset {
  id: string;
  originalFileUrl: string;
  originalFileName: string;
  convertedFileUrl?: string;
  convertedFileName?: string;
  status: string;
  containerName: string;
}

Delete Asset

DELETE /assets/:id

Delete an asset and its associated files.

File Processing

1. Upload Flow

2. Conversion Process

• XLSX files are automatically converted to CSV
• HTML content is converted to plain text
• Other formats are stored as-is
• All text content is processed for embeddings

3. Vector Embedding

After upload and conversion:

1. Documents are split into chunks
2. Embeddings are created for each chunk
3. Vectors are stored for search functionality

Integration Examples

File Upload

// Using FormData
const formData = new FormData();
formData.append('file', file);

const response = await fetch('/assets/upload', {
  method: 'POST',
  body: formData,
});

const asset = await response.json();

Document Processing

// Upload and process a document
async function processDocument(file: File) {
  // 1. Upload file
  const asset = await uploadFile(file);

  // 2. Wait for processing
  await checkProcessingStatus(asset.id);

  // 3. Get final asset data
  return await getAsset(asset.id);
}

Error Handling

Common Errors

1. File Size Exceeded

{
  "error": {
    "code": "FILE_TOO_LARGE",
    "message": "File size exceeds maximum limit of 100MB"
  }
}

2. Invalid File Type

{
  "error": {
    "code": "INVALID_FILE_TYPE",
    "message": "Unsupported file type. Supported types: pdf, docx, xlsx, etc."
  }
}

3. Conversion Error

{
  "error": {
    "code": "CONVERSION_FAILED",
    "message": "Failed to convert file",
    "details": {
      "originalFormat": "xlsx",
      "targetFormat": "csv"
    }
  }
}

Best Practices

1. File Upload

• Validate file size before upload
• Check file type compatibility
• Implement retry logic
• Monitor upload progress

async function uploadWithRetry(file: File, maxRetries = 3) {
  for (let i = 0; i < maxRetries; i++) {
    try {
      return await uploadFile(file);
    } catch (error) {
      if (i === maxRetries - 1) throw error;
      await delay(Math.pow(2, i) * 1000);
    }
  }
}

2. File Processing

• Handle timeouts gracefully
• Implement progress tracking
• Cache processed results
• Clean up temporary files

// Monitor processing status
async function waitForProcessing(assetId: string, timeout = 300000) {
  const startTime = Date.now();

  while (Date.now() - startTime < timeout) {
    const asset = await getAsset(assetId);
    if (asset.status === 'COMPLETED') return asset;
    await delay(5000);
  }

  throw new Error('Processing timeout');
}

3. Error Recovery

• Implement cleanup on failure
• Log processing errors
• Provide user feedback
• Handle partial success

Security Considerations

1. File Validation

• Scan for malware
• Validate file integrity
• Check file signatures
• Enforce size limits

2. Access Control

• Verify user permissions
• Validate organization access
• Track file ownership
• Audit file access

3. Storage Security

• Encrypt files at rest
• Secure transfer channels
• Implement TTL policies
• Monitor usage patterns

Performance Optimization

1. Upload Strategy

const uploadConfig = {
  maxConcurrent: 3,
  chunkSize: 5 * 1024 * 1024, // 5MB chunks
  retryDelays: [1000, 3000, 5000],
};

2. Processing Pipeline

• Implement batch processing
• Use worker queues
• Cache frequent accesses
• Optimize conversions

3. Resource Management

• Clean up temporary files
• Monitor storage usage
• Implement quotas
• Track processing time

Monitoring

Key Metrics

1. Upload Performance

   • Success rate
   • Processing time
   • Error frequency
   • Queue length

2. Storage Utilization

   • Space used
   • File counts
   • Access patterns
   • Conversion rates

3. Processing Stats

   • Conversion time
   • Embedding creation
   • Error rates
   • Resource usage