Resumable File Upload

In this article, we will explore the concept of resumable file uploads in JavaScript, providing a comprehensive guide with practical examples. This technique is essential for enhancing the user experience, especially when dealing with large files or unreliable network connections. By implementing resumable uploads, we can ensure that our users can continue uploading files from where they left off, minimizing data loss and frustration.

Introduction to Resumable File Uploads

Resumable file uploads allow users to upload files in chunks, ensuring that if an upload is interrupted due to a network issue or any other reason, it can be resumed from the last successfully uploaded chunk. This technique is particularly useful for large files and can significantly improve the reliability of file uploads.

Benefits of Resumable File Uploads

Improved User Experience: Users can resume uploads without starting over.
Efficiency: Reduces the amount of data transferred by only uploading missing parts.
Error Handling: Handles network interruptions gracefully.

Implementing Resumable File Uploads in JavaScript

Setting Up the Environment

Before diving into the implementation, ensure you have the following tools and libraries:

A modern web browser with JavaScript support.
A server capable of handling file uploads.
The resumable.js library (or a similar library) to manage the client-side logic.

Install the required Node.js dependencies:

npm install express cors

Server-Side Configuration

First, configure your server to handle file chunks and store metadata about the uploaded files. Here is an example using Node.js and Express. Note that resumable.js sends chunk metadata in the query string by default, so we read from req.query and use a temporary directory per file to safely handle out-of-order chunk arrival.

const express = require('express');
const cors = require('cors');
const fs = require('fs');
const path = require('path');
const app = express();
const port = 3000;

app.use(cors());

// Handle chunk verification for testChunks: true
app.head('/upload', (req, res) => {
  res.set('Access-Control-Allow-Origin', '*');
  const chunkNumber = parseInt(req.query.resumableChunkNumber);
  const identifier = req.query.resumableIdentifier;
  const chunkPath = path.join('uploads', identifier, `chunk-${chunkNumber}.bin`);
  fs.promises.access(chunkPath)
    .then(() => res.status(200).end())
    .catch(() => res.status(404).end());
});

app.post('/upload', async (req, res) => {
  try {
    const chunkNumber = parseInt(req.query.resumableChunkNumber);
    const totalChunks = parseInt(req.query.resumableTotalChunks);
    const identifier = req.query.resumableIdentifier;
    const fileName = req.query.resumableFilename;

    const chunkDir = path.join('uploads', identifier);
    await fs.promises.mkdir(chunkDir, { recursive: true });

    // Read raw body (resumable.js sends chunks as application/octet-stream)
    const buffer = await new Promise((resolve, reject) => {
      const chunks = [];
      req.on('data', chunk => chunks.push(chunk));
      req.on('end', () => resolve(Buffer.concat(chunks)));
      req.on('error', reject);
    });

    const chunkPath = path.join(chunkDir, `chunk-${chunkNumber}.bin`);
    await fs.promises.writeFile(chunkPath, buffer);

    const receivedChunks = (await fs.promises.readdir(chunkDir)).length;
    if (receivedChunks === totalChunks) {
      const finalPath = path.join('uploads', fileName);
      const writeStream = fs.createWriteStream(finalPath);
      const finishPromise = new Promise((resolve, reject) => {
        writeStream.on('finish', resolve);
        writeStream.on('error', reject);
      });

      for (let i = 1; i <= totalChunks; i++) {
        const chunkPath = path.join(chunkDir, `chunk-${i}.bin`);
        fs.createReadStream(chunkPath).pipe(writeStream);
      }
      await finishPromise;
      await fs.promises.rmdir(chunkDir);
      res.status(200).send('File uploaded successfully');
    } else {
      // resumable.js expects a 200 OK for successful chunk uploads
      res.status(200).send('Chunk uploaded successfully');
    }
  } catch (error) {
    console.error('Upload error:', error);
    res.status(500).send('Server error during upload');
  }
});

app.listen(port, () => {
  console.log(`Server running at http://localhost:${port}`);
});

Client-Side Implementation

Now, let's implement the client-side logic using JavaScript and the resumable.js library. Ensure you include the resumable.js library in your project. We use v2.1.0 for modern compatibility. For production environments, consider the standardized tus protocol or native File.slice with fetch for better control and cross-platform support.

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  <title>Resumable File Upload</title>
</head>
<body>
  <input type="file" id="fileInput" />
  <button id="uploadButton">Upload</button>
  <p id="progress">Ready</p>

  <script src="https://unpkg.com/[email protected]/resumable.min.js"></script>
  <script>
    const fileInput = document.getElementById('fileInput');
    const uploadButton = document.getElementById('uploadButton');
    const progressEl = document.getElementById('progress');

    const r = new Resumable({
      target: '/upload',
      chunkSize: 1 * 1024 * 1024, // 1MB chunks
      simultaneousUploads: 1,
      testChunks: true,
      throttleProgressCallbacks: 1,
    });

    r.assignBrowse(fileInput);

    uploadButton.addEventListener('click', () => {
      if (r.files.length > 0) {
        r.upload();
      } else {
        alert('Please select a file to upload.');
      }
    });

    r.on('progress', (file, loaded, total) => {
      const percent = Math.round((loaded / total) * 100);
      progressEl.textContent = `Uploading ${file.fileName}: ${percent}%`;
    });

    r.on('fileSuccess', (file, message) => {
      console.log(`File ${file.fileName} uploaded successfully.`);
      progressEl.textContent = 'Upload complete!';
    });

    r.on('fileError', (file, message) => {
      console.error(`Error uploading file ${file.fileName}: ${message}`);
      progressEl.textContent = 'Upload failed.';
    });
  </script>
</body>
</html>

Native Alternative: `File.slice` + `fetch`

For projects that prefer zero dependencies, you can implement resumable uploads natively using the File.slice API and fetch. This approach gives you full control over headers, retries, and chunk assembly.

async function uploadFileNative(file) {
  const chunkSize = 1 * 1024 * 1024; // 1MB
  const totalChunks = Math.ceil(file.size / chunkSize);
  const identifier = `${file.name}-${Date.now()}`;

  for (let i = 0; i < totalChunks; i++) {
    const start = i * chunkSize;
    const end = Math.min(start + chunkSize, file.size);
    const chunk = file.slice(start, end);

    const url = new URL('/upload', window.location.origin);
    url.searchParams.set('resumableChunkNumber', i + 1);
    url.searchParams.set('resumableTotalChunks', totalChunks);
    url.searchParams.set('resumableIdentifier', identifier);
    url.searchParams.set('resumableFilename', file.name);

    await fetch(url, { method: 'POST', body: chunk });
  }
  console.log('Native upload complete');
}

Managing Metadata

It is crucial to manage metadata about the uploaded file and its chunks. This information helps in resuming the upload from the correct chunk in case of interruptions. The server logic for tracking and assembling chunks is covered in the previous section. For production environments, avoid in-memory or file-system-only storage as it lacks persistence and thread-safety. Instead, use a database or cache (e.g., Redis) to track chunk completion and assemble files reliably.

Example: Uploading Large Files

The client configuration remains identical to the previous example. To optimize for large files, you can increase the chunkSize (e.g., to 5MB) and adjust simultaneousUploads based on your server's capacity and network conditions.

Professional Tips for Resumable File Uploads

Optimize Chunk Size: Adjust the chunk size based on the average network speed and file size to balance between upload speed and reliability.
Error Handling: Implement robust error handling mechanisms to deal with network interruptions and server issues.
User Feedback: Provide real-time feedback to users about the upload progress and any issues encountered.
Security: Ensure that the file upload process is secure by validating file types and implementing proper authentication and authorization.
Modern Alternatives: For production environments, consider standardized protocols like tus or native File.slice with fetch for better control, resumability, and cross-platform compatibility.

By following these guidelines and examples, we can implement a robust and efficient resumable file upload system in JavaScript, enhancing the user experience and ensuring reliable file uploads.

Practice

Which of the following are benefits of using resumable file uploads?

Improved user experience by allowing users to resume uploads without starting over.
Increased data security through encryption.
Efficiency by reducing the amount of data transferred by only uploading missing parts.
Handles network interruptions gracefully.
Automatically compresses files to save storage space.