File Upload System

Express Base API includes a complete file upload system with validation, tracking, security, and automatic cleanup.

Features

• 📤 Secure Upload - Validation, size limits, type restrictions
• 📊 Upload Tracking - Track all uploads with metadata
• 🗑️ Auto Cleanup - Automatic deletion of orphaned files
• 🔒 Access Control - User-based file ownership
• 🖼️ Multiple Types - Images, documents, videos supported
• ⚡ Fast Storage - Local filesystem with customizable path
• 📝 Metadata - Store filename, size, MIME type, path
• 🛡️ Security - File type validation, malware prevention

Architecture

┌─────────────┐
│   Client    │
└──────┬──────┘
       │ POST /upload (multipart/form-data)
       ▼
┌─────────────┐
│   Multer    │ Validate size, type
│ Middleware  │ Save to disk
└──────┬──────┘
       │
       ▼
┌─────────────┐
│  Upload     │ Create DB record
│ Controller  │ Track metadata
└──────┬──────┘
       │
       ▼
┌─────────────┐
│  Upload     │ Store info
│   Model     │ - filename
│             │ - size
│             │ - path
│             │ - user
└─────────────┘

Quick Start

Upload a File

curl -X POST http://localhost:3000/upload \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "x-api-key: YOUR_API_KEY" \
  -F "file=@/path/to/image.jpg"

Response (201):

{
  "status_code": 201,
  "status": "SUCCESS",
  "message": "File uploaded successfully",
  "data": {
    "_id": "65abc123...",
    "filename": "1705320600000-image.jpg",
    "original_name": "image.jpg",
    "mime_type": "image/jpeg",
    "size": 245678,
    "path": "public/upload/1705320600000-image.jpg",
    "url": "http://localhost:3000/public/upload/1705320600000-image.jpg",
    "user": "65abc456...",
    "created_at": "2024-01-15T10:30:00.000Z"
  }
}

Get All Uploads

curl -X GET http://localhost:3000/upload \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "x-api-key: YOUR_API_KEY"

Get Single Upload

curl -X GET http://localhost:3000/upload/65abc123... \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "x-api-key: YOUR_API_KEY"

Delete Upload

curl -X DELETE http://localhost:3000/upload/65abc123... \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "x-api-key: YOUR_API_KEY"

What Happens:

1. Checks if upload exists and belongs to user
2. Deletes physical file from disk
3. Removes database record

Configuration

File Size Limits

Default: 10MB per file

Edit in routes/app/upload.route.ts:

const upload = multer({
  storage: storage,
  limits: {
    fileSize: 10 * 1024 * 1024, // 10MB in bytes
  },
  fileFilter: fileFilter,
});

Allowed File Types

Default: Images, PDFs, and common documents

Edit in routes/app/upload.route.ts:

const ALLOWED_MIME_TYPES = [
  // Images
  'image/jpeg',
  'image/jpg',
  'image/png',
  'image/gif',
  'image/webp',

  // Documents
  'application/pdf',
  'application/msword',
  'application/vnd.openxmlformats-officedocument.wordprocessingml.document',

  // Spreadsheets
  'application/vnd.ms-excel',
  'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',

  // Text
  'text/plain',
  'text/csv',
];

const fileFilter = (req, file, cb) => {
  if (ALLOWED_MIME_TYPES.includes(file.mimetype)) {
    cb(null, true);
  } else {
    cb(new Error('Invalid file type'), false);
  }
};

Storage Path

Default: public/upload/

Edit in routes/app/upload.route.ts:

const storage = multer.diskStorage({
  destination: function (req, file, cb) {
    cb(null, 'public/upload/');
  },
  filename: function (req, file, cb) {
    const uniqueName = Date.now() + '-' + file.originalname;
    cb(null, uniqueName);
  }
});

Upload Model

Schema

interface IUpload extends BaseDocument {
  filename: string;         // Stored filename (unique)
  original_name: string;    // Original filename
  mime_type: string;        // MIME type (e.g., "image/jpeg")
  size: number;             // File size in bytes
  path: string;             // Full file path
  url?: string;             // Public URL (optional)
  user: ObjectId | IUser;   // Owner reference
}

Database Model

const uploadSchema = {
  filename: {
    type: String,
    required: true,
    unique: true,
  },
  original_name: {
    type: String,
    required: true,
  },
  mime_type: {
    type: String,
    required: true,
  },
  size: {
    type: Number,
    required: true,
  },
  path: {
    type: String,
    required: true,
  },
  url: {
    type: String,
    required: false,
  },
  user: {
    type: Schema.Types.ObjectId,
    ref: 'users',
    required: true,
    index: true,
  },
};

const Upload: Model<IUpload> = BaseSchema<IUpload>(
  'uploads',
  uploadSchema
);

Upload Service

Available Methods

class UploadService extends BaseService<IUpload> {
  // Create upload record
  async createUpload(data: CreateUploadData): Promise<QueryResult<IUpload>>

  // Get user's uploads
  async getUserUploads(userId: string, options?): Promise<QueryResult<IUpload[]>>

  // Get single upload
  async getUpload(id: string): Promise<QueryResult<IUpload>>

  // Delete upload (file + record)
  async deleteUpload(id: string, userId: string): Promise<QueryResult<IUpload>>

  // Get upload statistics
  async getUploadStats(userId: string): Promise<UploadStats>

  // Cleanup orphaned files
  async cleanupOrphanedFiles(): Promise<CleanupResult>
}

Usage Examples

Create Upload Record

import uploadService from './src/services/admin/upload.service';

const uploadData = {
  filename: req.file.filename,
  original_name: req.file.originalname,
  mime_type: req.file.mimetype,
  size: req.file.size,
  path: req.file.path,
  url: `${APP_URL}/${req.file.path}`,
  user: req.user._id,
};

const result = await uploadService.createUpload(uploadData);

Get User Uploads

const uploads = await uploadService.getUserUploads(userId, {
  limit: 20,
  sort: { created_at: -1 }
});

Get Upload Statistics

const stats = await uploadService.getUploadStats(userId);
console.log(stats);
// {
//   total_uploads: 156,
//   total_size: 45678901,  // bytes
//   total_size_mb: 43.56,
//   by_type: {
//     'image/jpeg': 89,
//     'application/pdf': 45,
//     'image/png': 22
//   }
// }

Cleanup Orphaned Files

// Delete files not tracked in database
const result = await uploadService.cleanupOrphanedFiles();
console.log(result);
// {
//   deleted: 12,
//   freed_space: 5678901,  // bytes
//   freed_space_mb: 5.42
// }

Controller Implementation

Upload Endpoint

// routes/app/upload.route.ts
import express from 'express';
import multer from 'multer';
import UploadController from '../../src/controllers/app/upload.controller';

const router = express.Router();
const uploadController = new UploadController();

// Configure multer
const storage = multer.diskStorage({
  destination: 'public/upload/',
  filename: (req, file, cb) => {
    cb(null, Date.now() + '-' + file.originalname);
  }
});

const upload = multer({
  storage: storage,
  limits: { fileSize: 10 * 1024 * 1024 }, // 10MB
  fileFilter: (req, file, cb) => {
    const allowedTypes = ['image/jpeg', 'image/png', 'application/pdf'];
    if (allowedTypes.includes(file.mimetype)) {
      cb(null, true);
    } else {
      cb(new Error('Invalid file type'), false);
    }
  }
});

// Routes
router.post('/', upload.single('file'), uploadController.store);
router.get('/', uploadController.index);
router.get('/:id', uploadController.show);
router.delete('/:id', uploadController.destroy);

export default router;

Controller Methods

// src/controllers/app/upload.controller.ts
import BaseController from '../../../utils/bases/base.controller';
import uploadService from '../../services/admin/upload.service';
import { response } from '../../../configs/app.config';

class UploadController extends BaseController {
  constructor() {
    super(uploadService);
  }

  async store(req, res) {
    try {
      if (!req.file) {
        return response.error(res, 'No file uploaded', 400);
      }

      const uploadData = {
        filename: req.file.filename,
        original_name: req.file.originalname,
        mime_type: req.file.mimetype,
        size: req.file.size,
        path: req.file.path,
        url: `${process.env.APP_URL}/${req.file.path}`,
        user: req.user._id,
      };

      const result = await uploadService.createUpload(uploadData);

      if (result.error) {
        return response.error(res, result.message, 400);
      }

      return response.success(
        res,
        result.data,
        201,
        'File uploaded successfully'
      );
    } catch (error) {
      return response.error(res, error.message, 500);
    }
  }

  async destroy(req, res) {
    try {
      const { id } = req.params;
      const userId = req.user._id;

      const result = await uploadService.deleteUpload(id, userId);

      if (result.error) {
        return response.error(res, result.message, 400);
      }

      return response.success(res, null, 200, 'File deleted successfully');
    } catch (error) {
      return response.error(res, error.message, 500);
    }
  }
}

export default UploadController;

Security Best Practices

1. File Type Validation

Always validate MIME type and file extension:

const fileFilter = (req, file, cb) => {
  // Check MIME type
  const allowedMimeTypes = ['image/jpeg', 'image/png'];
  if (!allowedMimeTypes.includes(file.mimetype)) {
    return cb(new Error('Invalid file type'), false);
  }

  // Check file extension
  const allowedExtensions = ['.jpg', '.jpeg', '.png'];
  const ext = path.extname(file.originalname).toLowerCase();
  if (!allowedExtensions.includes(ext)) {
    return cb(new Error('Invalid file extension'), false);
  }

  cb(null, true);
};

2. File Size Limits

Set appropriate limits based on use case:

// For profile images: 2MB
limits: { fileSize: 2 * 1024 * 1024 }

// For documents: 10MB
limits: { fileSize: 10 * 1024 * 1024 }

// For videos: 100MB
limits: { fileSize: 100 * 1024 * 1024 }

3. Filename Sanitization

Generate unique, safe filenames:

filename: (req, file, cb) => {
  // Use timestamp + random string
  const uniqueName = Date.now() + '-' + crypto.randomBytes(8).toString('hex');
  const ext = path.extname(file.originalname);
  cb(null, uniqueName + ext);
}

4. Access Control

Only allow users to access their own files:

async getUpload(req, res) {
  const upload = await uploadService.getUpload(req.params.id);

  // Check ownership
  if (upload.data.user.toString() !== req.user._id.toString()) {
    return response.error(res, 'Unauthorized', 403);
  }

  return response.success(res, upload.data);
}

5. Malware Scanning

For production, consider integrating antivirus:

import ClamScan from 'clamscan';

const clamscan = await new ClamScan().init();

const fileFilter = async (req, file, cb) => {
  const { isInfected } = await clamscan.isInfected(file.path);
  if (isInfected) {
    fs.unlinkSync(file.path); // Delete infected file
    return cb(new Error('Infected file detected'), false);
  }
  cb(null, true);
};

Advanced Features

Multiple File Upload

// Upload multiple files (max 5)
router.post('/multiple', upload.array('files', 5), async (req, res) => {
  const uploads = [];

  for (const file of req.files) {
    const result = await uploadService.createUpload({
      filename: file.filename,
      original_name: file.originalname,
      mime_type: file.mimetype,
      size: file.size,
      path: file.path,
      url: `${APP_URL}/${file.path}`,
      user: req.user._id,
    });
    uploads.push(result.data);
  }

  return response.success(res, uploads, 201);
});

Image Resizing

Use sharp for image optimization:

import sharp from 'sharp';

const resizeImage = async (filePath: string) => {
  const resizedPath = filePath.replace('.jpg', '-thumb.jpg');

  await sharp(filePath)
    .resize(300, 300, { fit: 'cover' })
    .jpeg({ quality: 80 })
    .toFile(resizedPath);

  return resizedPath;
};

Cloud Storage (S3)

Integrate AWS S3 for scalable storage:

import AWS from 'aws-sdk';

const s3 = new AWS.S3({
  accessKeyId: process.env.AWS_ACCESS_KEY,
  secretAccessKey: process.env.AWS_SECRET_KEY,
});

const uploadToS3 = async (file) => {
  const params = {
    Bucket: process.env.S3_BUCKET,
    Key: file.filename,
    Body: fs.createReadStream(file.path),
    ContentType: file.mimetype,
    ACL: 'public-read',
  };

  const result = await s3.upload(params).promise();
  return result.Location; // S3 URL
};

Validation

Request Validation

// utils/validations/upload.validation.ts
import Joi from 'joi';
import { validation } from '../../configs/app.config';

const uploadValidation = {
  destroy: validation({
    params: Joi.object({
      id: Joi.string().hex().length(24).required(),
    }),
  }),
};

export default uploadValidation;

Apply Validation

import uploadValidation from '../../utils/validations/upload.validation';

router.delete('/:id', uploadValidation.destroy, uploadController.destroy);

Troubleshooting

"No file uploaded"

Cause: File not included in request or wrong form field name

Solution:

• Ensure request uses multipart/form-data encoding
• Use correct field name: file (or as configured)
• Check file size doesn't exceed limit

"Invalid file type"

Cause: File type not in allowed list

Solution:

• Check ALLOWED_MIME_TYPES configuration
• Verify file MIME type matches extension
• Add new types to whitelist if needed

"File too large"

Cause: File exceeds size limit

Solution:

• Increase fileSize limit in multer config
• Compress file before upload
• Consider cloud storage for large files

Files Not Deleted

Cause: Physical file deletion failed

Solution:

• Check file permissions
• Verify path is correct
• Run cleanup script: await uploadService.cleanupOrphanedFiles()

Next Steps

• Authentication - Protect upload endpoints
• Validation - Custom upload validation
• Custom Services - Extend upload service

---

Questions? Check the GitHub Discussions or open an issue.