Upload to AWS S3 By Using Node.js

Table of Contents

1. Introduction
2. Setup IAM User with S3 Permission
3. Require NPM Package for the Sample Code
4. Update AWS Config and Create S3 Instance
5. Sample Code to List All Buckets
6. Sample Code to Upload a Text File to S3
7. Sample Code to Upload JSON File to S3
8. Sample Code to Upload an Image URL to S3
9. Upload Image File from Local
10. Conclusion

Introduction

S3 is a cloud serverless object storage solution provided by AWS. That means you can store any files in S3 without maintaining the backend machine, and managing these servers is AWS's responsibility. It is also very cheap. You can reference the pricing for each region you'll be using for S3. For example, at the time of writing (2020), when I checked US East Ohio, the standard S3 first 50 TB/month costs $0.023 per GB. The price also depends on how frequently you'll access the file. If you just want to store some files in S3 for a long time for archive purposes, you can use S3 Glacier Deep Archive which costs $0.00099 per GB.

So, what I'm doing in this blog is showing some sample code for how to upload files to AWS S3 by using Node.js.

Setup IAM User with S3 Permission

The first thing you need to do before writing the Node.js code is to set up a user with S3 permissions to access your S3 bucket. I assume you already have an AWS account set up. To make it easy, you could create a user and provide the S3FullAccess policy for now, but for production, you should always give the least privileges. Also, if you will be running Node.js on EC2 or Lambda, you should create an IAM role and attach it to the EC2 or Lambda instead of creating a user.

Require NPM Package for the Sample Code

The following are packages that will be used by the code snippets later.

const AWS = require("aws-sdk");
const axios = require("axios");

Update AWS Config and Create S3 Instance

Here, you update the AWS config to tell which region your bucket is in and add the access key ID and secret access key to the S3 instance.

AWS.config.update({
        region: "us-west-2"
    });
    const s3 = new AWS.S3({
        accessKeyId: '{the access key id for the IAM user}',
        secretAccessKey: '{the secret access key for the IAM user}'
    });

Sample Code to List All the Bucket

s3.listBuckets((err, data) => {
        if (err) {
            console.log("Error", err);
        } else {
            console.log("Success", data);
        }
    });

Sample Code to Upload a Text File to S3

var textObjectParams = {Bucket: 'your bucket name', Key: '{the file name you want to create}.txt', Body: 'Yo! My first text!'};
    s3.upload(textObjectParams, (err, data) => {
        if (err) {
            console.log("Error", err);
        } else {
            console.log("Success", data);
        }
    });

Sample Code to Upload JSON File to S3

var sampleData = {
        name: 'sample',
        count: '999'
    }

    var textObjectParams = {Bucket: '{Your bucket folder}', Key: 'jsonFile.json', Body: JSON.stringify(sampleData)};
    s3.upload(textObjectParams, (err, data) => {
        if (err) {
            console.log("Error", err);
        } else {
            console.log("Success", data);
        }
    });

Sample Code to Upload an Image URL to S3

The following example uses Axios to get the image file as a stream, then upload it to S3.

axios({
        method: 'GET',
        url: 'https://picsum.photos/536/354',
        responseType: 'stream'
    }).then(res => {
        var objectParams = {Bucket: '{Your bucket name}', Key: 'imageFile.png', Body: res.data};
        s3.upload(objectParams, (err, data) => {
            if (err) {
                console.log("Error", err);
            } else {
                console.log("Success", data);
            }
        });

    });

Upload Image File from Local

const content = fs.readFileSync('yourImage.png');

    var objectParams = {Bucket: '{Your bucket name}', Key: 'new image file name.png', Body: content};

    s3.upload(objectParams, (err, data) => {
        if (err) {
            console.log("Error", err);
        } else {
            console.log("Success", data);
        }
    });

Conclusion

The above examples show how to list buckets and upload different kinds of files to AWS S3 by using Node.js, including text files, JSON files, images from URLs, and local image files.